Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Sunday January 28 2018, @08:52PM   Printer-friendly
from the finding-significance dept.

Psychologist Daniël Lakens disagrees with a proposal to redefine statistical significance to require a 0.005 p-value, and has crowdsourced an alternative set of recommendations with 87 co-authors:

Psychologist Daniël Lakens of Eindhoven University of Technology in the Netherlands is known for speaking his mind, and after he read an article titled "Redefine Statistical Significance" on 22 July 2017, Lakens didn't pull any punches: "Very disappointed such a large group of smart people would give such horribly bad advice," he tweeted.

In the paper, posted on the preprint server PsyArXiv, 70 prominent scientists argued in favor of lowering a widely used threshold for statistical significance in experimental studies: The so-called p-value should be below 0.005 instead of the accepted 0.05, as a way to reduce the rate of false positive findings and improve the reproducibility of science. Lakens, 37, thought it was a disastrous idea. A lower α, or significance level, would require much bigger sample sizes, making many studies impossible. Besides. he says, "Why prescribe a single p-value, when science is so diverse?"

Lakens and others will soon publish their own paper to propose an alternative; it was accepted on Monday by Nature Human Behaviour, which published the original paper proposing a lower threshold in September 2017. The content won't come as a big surprise—a preprint has been up on PsyArXiv for 4 months—but the paper is unique for the way it came about: from 100 scientists around the world, from big names to Ph.D. students, and even a few nonacademics writing and editing in a Google document for 2 months.

Lakens says he wanted to make the initiative as democratic as possible: "I just allowed anyone who wanted to join and did not approach any famous scientists."


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 3, Funny) by TheRaven on Monday January 29 2018, @11:47AM

    by TheRaven (270) on Monday January 29 2018, @11:47AM (#629785) Journal
    That's not limited to social sciences. In computer science, our statistical errors are largely irrelevant because our experimental errors are often an order of magnitude higher. Different cache sizes, code layouts, small changes to pipeline structures, and so on can have a ±50% effect on the results, so arguing about a 0.5% error from misapplied statistics is largely irrelevant. And, yes, I do get cranky every time I read a CGO paper that shows a speedup that's well within the margins of experimental error and doesn't even try to apply statistics (we ran the experiment 10 times, discarded the outliers with no explanation, and present the mean with no indication of distribution, and our algorithm gives a 15% speedup on average, computed by subtracting the before and after values from each benchmark in a cherry-picked subset of SPEC and averaging them).
    --
    sudo mod me up
    Starting Score:    1  point
    Moderation   +1  
       Funny=1, Total=1
    Extra 'Funny' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   3