This is a nice visualization by Schonbrodt and Perugini who show Schonbrodt與Perugini做了不錯的視覺化範例

how the variation in data decreases as the sample size becomes larger. 展示資料的變異下降與增加樣本數的關係

From the left to the right, we see a continuously increasing sample size.
橫軸由左至右的連續數值代表增加的樣本數

We also see a black line. 留意每一條黑線的走勢

And in this case, the black line is actually the effect size estimate from 每條線是從收集的資料，以對應的樣本數

a study they performed as they collected more and more data. 計算得到的真實效果量

So the effect size was calculated after every participants and 也就是收完一位參與者資料

it's plotted here as a black line. 就計算一次效果量的總成

You can see that there's a huge drop in the effect size. 好幾條線的走勢很明顯是下降的

And what they show is that actually such variation is possible, 整體也顯示真實資料的變異程度

because small samples have very large variation. 樣本越少變異越大

I referred to this as sailing the seas of chaos into what Schonbrodt and 我把這樣的景觀比喻為穿越暴風雨

Perugini named the corridor of stability. 抵達寧靜迴廊之旅

On the left you see huge waves. 圖左邊的好似濤天巨浪

If you have small samples, then your data is riding these huge waves. 小樣本的估計值猶如被浪頭頂著的小船

Effects can be very large or very small. 效果量可大可小

The variation is substantial. 全由資料變異決定

Now as you enter the corridor of stability, as Schonbrodt & 進入Schonbrodt與Perugini所稱的寧靜迴廊時

Perugini call this, you see that the variation becomes much less severe. 資料變異變得沒那麼劇烈

There's still some variation around the true effect size estimate, but 雖然真實效果量的估計值還是有些變異

at this point, you're making very accurate estimations. 過這這一點之後，估計值明顯更正確

Now we see that researchers failed to take the sample size into account when they 採用小樣本資料的研究，研究者很難提出

design new studies. 有說服力的解釋

And this is nothing new. 這並不是新聞

We've known this for about 50 years, and only now it is very slowly changing. 50年前就有人提出警告，但學術界改變很慢

In certain literatures where sample sizes are very expensive, so 在某些域收集資料成本極高

if you want to collect a large number of people, you have to have a lot of money, 經費充裕才有可能收集大樣本資料的研究

such as neuroscience, where we put people in fMRI scanners, 像是fMRI這類收集人腦活動的研究

we see that there's really a huge power failure. 這篇論文指出嚴重的檢定力失效問題

Studies are severely underpowered. 也就是多數神經科學研究的檢定力過低

Interestingly, people have been saying this for 有趣的是這問題吵了這麼多年

years and years, and there are even papers that say that the studies of 還有新論文指出設定統計檢定力的研究

statistical power don't have any effect on the power of studies. 對於提昇研究檢定力幫助有限

This is slightly depressing, because here I'm talking about how you should 正在告訴你如何設計良好的研究，看到一篇

design well-powered studies, even though past research has shown that this 論文說這些努力都是徒勞的

does not have an effect on what researchers really do. 不免讓人有些喪氣

But on the other hand, we see the things are now really changing. 另一方面，情勢也正在改變

More and more journals are requiring what's known as a sample size 越來越多期刊要求作者提供樣本數決定策略

justification. 越來越多期刊要求作者提供樣本數決定策略

You need to explicitly write down why you selected this specific sample size. 作者要明確說明決定樣本數的理由

So I seriously think that in the future we'll see a big change in this 我預見一些特定領域將有很大的變化

specific area. 我預見一些特定領域將有很大的變化

We see that studies in psychology often have very low statistical power. 現在已知許多心理學研究的檢定力偏低

Estimates average around 50%, 估計50%左右

which means that, even if you have a true hypothesis, you only have 也就是說，就算有一個明確的假設

a 50% probability of actually observing a statistically significant finding. 只有50%的機會能獲得統計顯著的結果

I think that if people realized this, 如果一般人都能了解這點

they would seriously reconsider doing the study, 會重新考慮當這行研究人員的意義

because they only have 50% probability of finding an effect when it's there. 因為已知的發現只有50%是真的

That doesn't seem to be worth all the effort to collect this data. 那何必要拼死拼活地收集資料