SoylentNews is people
SoylentNews
Psychologist Daniël Lakens disagrees with a proposal to redefine statistical significance to require a 0.005 p-value, and has crowdsourced an alternative set of recommendations with 87 co-authors:
Psychologist Daniël Lakens of Eindhoven University of Technology in the Netherlands is known for speaking his mind, and after he read an article titled "Redefine Statistical Significance" on 22 July 2017, Lakens didn't pull any punches: "Very disappointed such a large group of smart people would give such horribly bad advice," he tweeted.
In the paper, posted on the preprint server PsyArXiv, 70 prominent scientists argued in favor of lowering a widely used threshold for statistical significance in experimental studies: The so-called p-value should be below 0.005 instead of the accepted 0.05, as a way to reduce the rate of false positive findings and improve the reproducibility of science. Lakens, 37, thought it was a disastrous idea. A lower α, or significance level, would require much bigger sample sizes, making many studies impossible. Besides. he says, "Why prescribe a single p-value, when science is so diverse?"
Lakens and others will soon publish their own paper to propose an alternative; it was accepted on Monday by Nature Human Behaviour, which published the original paper proposing a lower threshold in September 2017. The content won't come as a big surprise—a preprint has been up on PsyArXiv for 4 months—but the paper is unique for the way it came about: from 100 scientists around the world, from big names to Ph.D. students, and even a few nonacademics writing and editing in a Google document for 2 months.
Lakens says he wanted to make the initiative as democratic as possible: "I just allowed anyone who wanted to join and did not approach any famous scientists."
(Score: 4, Insightful) by opinionated_science on Sunday January 28 2018, @11:34PM (6 children)
When used properly statistics are a beautiful tool to observer the world around us.
But you must factor in the sample size, and the balance of probabilities that data is correct.
If you don't know intimately what Bayesian or the Central limit theorems describe, quit trying to comment now - that is the ground floor in analysis...
(Score: 2) by deadstick on Monday January 29 2018, @12:12AM
Statistics is like dynamite. Use it properly, and you can move mountains. Use it improperly, and the mountain will come down on you.
(Score: 3, Interesting) by FatPhil on Monday January 29 2018, @12:27AM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 5, Interesting) by requerdanos on Monday January 29 2018, @12:29AM (2 children)
While based on solid information, this isn't necessarily good advice.
There are disciplines involved here other than probability theory, even though probability theory is at the root of what's going on.
Specifically:
Reporters (from "credentialed journalist" all the way to "dude I have this great science blog") form a group that needs desperately to understand how to read a scientific study and interface with, be the recipients of, the information calculated through probability theory.
Random idiots (from "I am pretty smart, and I like to weigh information carefully" all the way to "wow I better forward this clickbait to everyone just in case it's true") form another group, a further step removed, that need to learn how to call bullshit on the Reporters who say "New Study: Green Jelly Beans Linked To Acne [explainxkcd.com]" instead of blindly parroting what they say*.
Also involved are everyone else (all over the spectrum) who might be affected, which includes just about everyone.
I want to hear quality comments** from people who represent them, and from people who have useful advice for them.
----------
*Which has resulted in large numbers of people believing headlines, in sequence, of "New Study: Coffee Bad For You," "New Study: Coffee Not Bad For You After All," "New Study: Coffee Bad For You," "New Study: Coffee Not Bad For You After All," "New Study: Coffee Bad For You," "New Study: Coffee Not Bad For You After All," "New Study: Coffee Bad For You," "New Study: Coffee Not Bad For You After All." Despite it being impossible for two opposite oversimplifications to be true in the same universe.
** No. If you have to ask, then that would not be a quality comment. Thank you.
(Score: 2) by tfried on Monday January 29 2018, @10:27AM
I second that, but I think you forgot the most important example: (Quasi-) experimental design. Statistical analysis may tell you that you (probably) have some non-random relation in your data, but it won't tell you why that relation is in your data. Is it what you think it is? Or is it just some source of bias in your data collection, your measurements not reflecting what you think they do, some correlation with a third variable that you forgot about (or chose to ignore), regression to the mean, temporary effects, ...
Alpha errors are an annoying source of noise in the discourse, but at least it's relatively easy to identify any papers that are in danger, here (admittedly it may be harder to identify alpha error inflation due to excessive comparisons that are not necessarily reported). But I'll bet, even at today's "significance levels", alpha errors are outnumbered three to one by non-numerical screw-ups among published papers.
Heck, at least today there is a tiny bit of awareness that maybe a published result should be taken with care, until confirmed by several independent people using independent measurements and designs. I'm afraid, all the current discussion will yield is a "solution" (be it stricter p-values or something, anything, else) that will make everybody feel safe and "correct", without actually helping at all (but probably raising the entry barriers to low budget independent studies).
(Score: 2) by opinionated_science on Monday January 29 2018, @02:31PM
the media has problem with all maths - they often quote numbers with no reference to the mean and variance of the distribution.
This is middle school level maths, and an interesting proxy to why so much bad stuff goes on,contrary to the data.
The fact that many respectable journals can barely keep the statistics correct, suggests this is quite widespread...
(Score: 0) by Anonymous Coward on Monday January 29 2018, @07:39PM
Your username isn't joking around!!