Let's talk now about pitfalls of cluster-extent based correction.
And this also applies to a threshold-free clustering enhancement.
So, let's look at what people are doing.
Of the people who are doing cluster-extent based thresholding, that's 75% of
the studies, let's look at which cluster defining thresholds that people are using.
Remember, this is an arbitrary choice.
I might threshold first at p is less than 0.01, or p is less that 0.001, and then,
look at how big the clusters are above that threshold.
And what you see here is among the major software packages.
The threshold that people use is defined by the default in this software package.
So, the big yellow bar you see here is if people use FSL,
they usually set up primary threshold that p is less than 0.01,
and that's the default in that package.
And if are people are using SPM, the other most widely used package,
then the most common threshold is p is less than 0.001.
And that's because that's the default in that package.
Okay, so what people are doing is fairly arbitrary.
Now, let's look at the impact of what happens when you set these primary
thresholds in different locations.
Well, if I threshold, this is a pain map that we've generated from our lab.
If I threshold at p is less than 0.01, and
I look at the size of the clusters sticking up, I get this map.
And this map looks pretty good.
It's got all these pain areas, lots of activation everywhere.
But it turns out that there are only two contiguous blobs.
We color code those blobs here, blue and orange.
So, what can I conclude from this?
Well, it's very tempting to look at that map and
say, all these areas are activated.
But that's not the right inference.
That's wrong.
Really what I can say is, is that in each blob,
there is at least one voxel that is significant.
So, I have to say, well, I've got activity in the blue blob here in the thalamus, or
the insular or the prefrontal cortex, or the sensory motor cortex, but
I don't know where.
So, that is not as appealing of inference, is it?
So, that creates a big problem because that obviates many of the reasons that we
do functional imaging in the first place because we're on localized activity.
Now, these simulations, I'll just give you a few take homes,
they illustrate some of the issues with cluster-extent based thresholding.
On the left, we're looking at the size of significant clusters,
and the idea is if we threshold at p is less than 0.01,
then we often get non-specificity.
The blobs that we get are much bigger than useful anatomical areas.
So, the problem I illustrated in the previous slide,
is a problem that occurs all the time.
And it creates appealing looking maps and
we can't make the imprints from them that we'd like to make.
So, that's one issue.
Secondly, if I do try to interpret each voxel on that map, if I say, look at all
these areas are really probably active, then I will be wrong much of the time.
So, in similar simulations, in simulations that are similar to the case that I
showed you and a range of cases,
45 to 70% of the activated voxels in the map are actually not truly active.
So, there's a very high false discovery rate.
And a third and
final problem I'll highlight here is the problem of false discoveries.
For technical reasons, the familywise error rate is
not properly controlled when you set very low primary thresholds.
And that's due to the mechanics of [INAUDIBLE] field theory and how it works.
It turns out you're not actually getting, if you ask for p is less than 0.05,
familywise error rate corrected, you're not actually getting 0.05 corrected.
And what you can see here is that when the primary threshold is at 0.01,
so relatively liberal, then the false positive rate is high.
At 0.001 in these simulations at least, or below, it's properly controlled.
In that level, 0.01 to 0.001 will depend on the characteristics of your data.
So, it's not a hard and fast rule.
But what it does mean is that thresholding at at least p is less
than 0.001 is a good practice.
Additionally, it makes sense to set a threshold, so you don't get giant blobs
that span multiple anatomical regions because those are not interpretable.
So, let's wrap up.
We've talked about several multiple comparison methods.
We talked about using uncorrected methods.
We talked about familywise error rate, false discovery rate, and
cluster extent-based methods.
And we talked about several important pitfalls with using
uncorrected thresholds and with using cluster-extent based thresholds.
That's the end of this module.
At this point, we've covered many aspects of experimental design,
physics, acquisition, and analysis.
So, you should have a good working model for how to do an fMRI experiment and
interpret the results.
