In science, the success of an experiment is often determined by a measure called "statistical significance." A result is considered to be "significant" if the difference observed in the experiment between groups (of people, plants, animals and so on) would be very unlikely if no difference actually exists. The common cutoff for "very unlikely" is that you'd see a difference as big or bigger only 5 percent of the time if it wasn't really there — a cutoff that might seem, at first blush, very strict.
It sounds esoteric, but statistical significance has been used to draw a bright line between experimental success and failure. Achieving an experimental result with statistical significance often determines if a scientist's paper gets published or if further research gets funded. That makes the measure far too important in deciding research priorities, statisticians say, and so it's time to throw it in the trash.
More than 800 statisticians and scientists are calling for an end to judging studies by statistical significance in a March 20 comment published in Nature. An accompanying March 20 special issue of the American Statistician makes the manifesto crystal clear in its introduction: "'statistically significant' — don't say it and don't use it."
There is good reason to want to scrap statistical significance. But with so much research now built around the concept, it's unclear how — or with what other measures — the scientific community could replace it. The American Statistician offers a full 43 articles exploring what scientific life might look like without this measure in the mix.
Is is time for "P is less than or equal to 0.05" to be abandoned or changed ??
(Score: 0) by Anonymous Coward on Friday April 19 2019, @07:27PM (11 children)
The letter is signed by a bunch of medical scientists. Their problems with statistical significance do not apply to science generally. Physics uses a .0000003 cutoff to determine statistical significance, and it's not a problem. Medical research has the problem because .05 is such a weak result.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @07:36PM (10 children)
http://nautil.us/blog/the-present-phase-of-stagnation-in-the-foundations-of-physics-is-not-normal [nautil.us]
Progress in physics stopped in the 1970s, right when they adopted NHST (testing something besides the predictions of their theory). Since then all theyve done is "verify" what they already "knew".
It destroys every field that adopts it because it is pseudoscience. In science you predict something and hope to *not* observe a "significant" deviation.
(Score: 1, Funny) by Anonymous Coward on Friday April 19 2019, @08:21PM (2 children)
Wow! The Null Hypothesis AC! Didn't see that coming! What were the odds of him posting in this thread, I wonder? Statistically significant, I bet. P=1.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @08:35PM (1 child)
Wow, the AC with nothing of value to say about the topic.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @09:48PM
The Null Hypothesis is the AC! Thereby proving that the Parent AC is not statistically significant.
(Score: 1, Informative) by Anonymous Coward on Friday April 19 2019, @08:38PM (6 children)
Progress in physics slowed because the low-hanging fruits have been picked, and successful fields like electrical engineering get spun off. do you think there is an endless supply of potential discoveries like electromagnetism, lasers, and transistors?
Before gravitational waves were measured, it was not known how many events would be visible. Some people thought none at all. Statistical significance was used to prove that the measurements were meaningful. Now the term is gauche? A Nobel Prize was recently given for work that required it.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @10:05PM (5 children)
Physicists said all the low hanging fruit was picked in 1900 too. It might be true one day but usually its just an excuse.
https://hsm.stackexchange.com/questions/2129/who-said-that-essentially-everything-in-theoretical-physics-had-already-been-dis [stackexchange.com]
(Score: 3, Insightful) by Coward, Anonymous on Friday April 19 2019, @10:28PM (2 children)
Do you have any evidence that physicists now are less intelligent or creative than a century ago? Yet there are many more of them but they are producing fewer fundamental discoveries. That is some pretty strong evidence for the low hanging fruit having been picked.
Null experiments are something to do when you don't know what else to do. I'm not a big fan myself, and on the list of Nobel Prizes in physics [wikipedia.org] I didn't see any for null-hypothesis testing. Physics is not dominated by this approach.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @11:48PM
Yes there are many more of them wasting their time with NHST-based experiments or confused about what evidence needs to be explained by theory or not due to BS generated by NHST.
(Score: 3, Interesting) by RamiK on Saturday April 20 2019, @04:34AM
A century ago you had John von Neumann and far less scientists-per-capita and people alive. In the spirit of the topic, it averages out to smarter overall.
compiling...
(Score: 2) by hendrikboom on Saturday April 20 2019, @02:20AM (1 child)
Gravitational telescopes and particle colliders are sufficiently expensive that their fruit cannot be called low-hanging.
(Score: 0) by Anonymous Coward on Saturday April 20 2019, @11:38AM
The need to rely on brute force could also indicate a lack of cleverness.