Previously answered questions

 

1. On cluster extent correction.
   How should I apply cluster extent inference and what about tools like 3dClustSim (previously AlphaSim) from AFNI?
   
   I will focus on the results output of SPM because it provides the most inference procedures. SPM provides inference at 3 different levels: voxel, peak and cluster. In your regular output table you will find the peak and cluster inference, uncorrected, FDR- or FWE corrected.
   
   Cluster extent thresholding requires two thresholds: (1) un uncorrected cluster-forming threshold and we recommend the default of 0.001 and (2) the minimal size of a cluster required for statistical significance. You should always report both thresholds. The cluster-forming threshold creates the clusters and should not be too lenient, otherwise you will have very large clusters that will always exceed your cluster extent threshold. We recommend the FWE cluster extent correction, because the FDR cluster extent correction will often have lower power. Below your table you will find the relevant thresholds. If you see Inf, this means that no significant voxels survived that inference procedure.
   
   Sometimes researchers turn to AFNI and more specifically to 3dClustSim (previously AlphaSim) to calculate a cluster extent threshold for their SPM results. Is this meaningful? If you use it properly, this algorithm should give you the same results as SPM. More importantly it is often applied wrong. It requires the dimension, voxel size and smoothness of your images. However, make sure you use the dimension and voxel size of your normalized images as these change with normalization! Most users are not aware that the smoothness does not refer to the smoothing you applied during preprocessing. It is estimated on the data. AFNI recommends estimating the smoothness of the time series noise as input for 3dClustSim with a specific algorithm. SPM also provides an estimate of smoothness at the bottom of the results table, but I doubt that this is similar to the smoothness as estimated by AFNI.
    

2. On (subtle) forms of double dipping.
   Is it always considered double dipping if you would for example define your ROI based on a 2nd level effect of one factor and then test the effect of the other factor in this ROI?
   
   The classic paper on Voodoo correlations "Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognitions" has raised awareness on circularity errors and non-independence in fMRI. However there are many subtle forms. Kriegeskorte and colleagues give a very detailed overview in the supplemental material of their paper on circular analysis in Nature Neuroscience.
   
   The above case is also treated. This implies two orthogonal contrast, one for ROI definition and one for testing the other factor. However, orthogonal contrasts do not imply contrast orthogonality. The latter is required to avoid double dipping. There are two problems: (1) the temporal noise and (2) an unbalanced design matrix. The temporal noise is not a problem at the second level analysis when you are pooling contrasts across subjects and conditions. The problems with the design matrix are often underestimated. It requires the same amount of trials, but with omitting errors, ... this is often not the case.
   
   How should one then proceed? The leave-one-subject-out procedure for ROI definition is an elegant approach (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2823971/.) You estimate the ROI for each subject with a model where that specific subject is not included.

 

3. Independence condition in ANOVA within subjects and flexible factorial.

I was playing around with the different 2nd level analyses in SPM and noticed that the "flexible factorial" and "one way anova within subject" don't give me the same results unless I set "independence" to "no" for the flexible factorial and to "yes" for the one-way anova. I can't really see why one would set independence to yes for a within-subject anova so this made me wondering which model is actually correct? My study has a single factor with 4 levels.