SoylentNews is people
SoylentNews
In science, the success of an experiment is often determined by a measure called "statistical significance." A result is considered to be "significant" if the difference observed in the experiment between groups (of people, plants, animals and so on) would be very unlikely if no difference actually exists. The common cutoff for "very unlikely" is that you'd see a difference as big or bigger only 5 percent of the time if it wasn't really there — a cutoff that might seem, at first blush, very strict.
It sounds esoteric, but statistical significance has been used to draw a bright line between experimental success and failure. Achieving an experimental result with statistical significance often determines if a scientist's paper gets published or if further research gets funded. That makes the measure far too important in deciding research priorities, statisticians say, and so it's time to throw it in the trash.
More than 800 statisticians and scientists are calling for an end to judging studies by statistical significance in a March 20 comment published in Nature. An accompanying March 20 special issue of the American Statistician makes the manifesto crystal clear in its introduction: "'statistically significant' — don't say it and don't use it."
There is good reason to want to scrap statistical significance. But with so much research now built around the concept, it's unclear how — or with what other measures — the scientific community could replace it. The American Statistician offers a full 43 articles exploring what scientific life might look like without this measure in the mix.
Is is time for "P is less than or equal to 0.05" to be abandoned or changed ??
(Score: 0) by Anonymous Coward on Friday April 19 2019, @10:00PM (1 child)
That's another of those "requires greater audience understanding" bits. The stuff I listed as mandatory is able to be written into a short, one-sentence summary, whereas effect size, along with correlation and deviation, go in the second or third sentence. Also, as effect size is usually used in studies of studies, it isn't always applicable. The complaint is about reporting of medical findings, most sensational reporting of which are initial findings with no follow-up. (I work in manufacturing QA and product/test method development, so effect size is not valid in all my work either.)
Completely made-up example:
"In 100 studies of 100+ people, on average, 5 people polled actually understand the meaning of the phrase, "relative standard deviation" (p = 0.05); more people should study statistics.
These polls were conducted outside shopping malls, were one-question, long-answer verbal surveys, and included only those people who would answer the question. Those who would not answer were asked why, and their results were recorded and categorized, if possible; over multiple studies, it is noted that as much as 50% of those asked did not answer the target question, which is considered acceptable by the polling industry."
Effect size is also usually buried the full-text paper alongside either methodology or findings, topic-dependent.
The more actual, non-speculative information reported in an article about a finding, the better.
It sure would be nice if reporters understood what they were reporting on such that their news service was more effective in providing accurate information.
(Score: 0) by Anonymous Coward on Friday April 19 2019, @10:09PM
I was making fun of you, since you only need two of the three to get the other. The point of a p-value is to normalize effect size to sample size.