Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Sunday January 28 2018, @08:52PM   Printer-friendly
from the finding-significance dept.

Psychologist Daniël Lakens disagrees with a proposal to redefine statistical significance to require a 0.005 p-value, and has crowdsourced an alternative set of recommendations with 87 co-authors:

Psychologist Daniël Lakens of Eindhoven University of Technology in the Netherlands is known for speaking his mind, and after he read an article titled "Redefine Statistical Significance" on 22 July 2017, Lakens didn't pull any punches: "Very disappointed such a large group of smart people would give such horribly bad advice," he tweeted.

In the paper, posted on the preprint server PsyArXiv, 70 prominent scientists argued in favor of lowering a widely used threshold for statistical significance in experimental studies: The so-called p-value should be below 0.005 instead of the accepted 0.05, as a way to reduce the rate of false positive findings and improve the reproducibility of science. Lakens, 37, thought it was a disastrous idea. A lower α, or significance level, would require much bigger sample sizes, making many studies impossible. Besides. he says, "Why prescribe a single p-value, when science is so diverse?"

Lakens and others will soon publish their own paper to propose an alternative; it was accepted on Monday by Nature Human Behaviour, which published the original paper proposing a lower threshold in September 2017. The content won't come as a big surprise—a preprint has been up on PsyArXiv for 4 months—but the paper is unique for the way it came about: from 100 scientists around the world, from big names to Ph.D. students, and even a few nonacademics writing and editing in a Google document for 2 months.

Lakens says he wanted to make the initiative as democratic as possible: "I just allowed anyone who wanted to join and did not approach any famous scientists."


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by tfried on Monday January 29 2018, @10:27AM

    by tfried (5534) on Monday January 29 2018, @10:27AM (#629762)

    There are disciplines involved here other than probability theory

    I second that, but I think you forgot the most important example: (Quasi-) experimental design. Statistical analysis may tell you that you (probably) have some non-random relation in your data, but it won't tell you why that relation is in your data. Is it what you think it is? Or is it just some source of bias in your data collection, your measurements not reflecting what you think they do, some correlation with a third variable that you forgot about (or chose to ignore), regression to the mean, temporary effects, ...

    Alpha errors are an annoying source of noise in the discourse, but at least it's relatively easy to identify any papers that are in danger, here (admittedly it may be harder to identify alpha error inflation due to excessive comparisons that are not necessarily reported). But I'll bet, even at today's "significance levels", alpha errors are outnumbered three to one by non-numerical screw-ups among published papers.

    Heck, at least today there is a tiny bit of awareness that maybe a published result should be taken with care, until confirmed by several independent people using independent measurements and designs. I'm afraid, all the current discussion will yield is a "solution" (be it stricter p-values or something, anything, else) that will make everybody feel safe and "correct", without actually helping at all (but probably raising the entry barriers to low budget independent studies).

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2