Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Friday April 19 2019, @06:34PM   Printer-friendly
from the significant-change dept.

In science, the success of an experiment is often determined by a measure called "statistical significance." A result is considered to be "significant" if the difference observed in the experiment between groups (of people, plants, animals and so on) would be very unlikely if no difference actually exists. The common cutoff for "very unlikely" is that you'd see a difference as big or bigger only 5 percent of the time if it wasn't really there — a cutoff that might seem, at first blush, very strict.

It sounds esoteric, but statistical significance has been used to draw a bright line between experimental success and failure. Achieving an experimental result with statistical significance often determines if a scientist's paper gets published or if further research gets funded. That makes the measure far too important in deciding research priorities, statisticians say, and so it's time to throw it in the trash.

More than 800 statisticians and scientists are calling for an end to judging studies by statistical significance in a March 20 comment published in Nature. An accompanying March 20 special issue of the American Statistician makes the manifesto crystal clear in its introduction: "'statistically significant' — don't say it and don't use it."

There is good reason to want to scrap statistical significance. But with so much research now built around the concept, it's unclear how — or with what other measures — the scientific community could replace it. The American Statistician offers a full 43 articles exploring what scientific life might look like without this measure in the mix.

Statistical Significance

Is is time for "P is less than or equal to 0.05" to be abandoned or changed ??


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 4, Funny) by aristarchus on Friday April 19 2019, @09:13PM (5 children)

    by aristarchus (2645) on Friday April 19 2019, @09:13PM (#832295) Journal

    Alright,

    More than 800 statisticians and scientists

    How statistically significant is this? Is it like something one might read on Quillette? Inquiring minds want to know.

    According to the United States Bureau of Labor Statistics, as of 2014, 26,970 jobs were classified as statistician in the United States.

    https://en.wikipedia.org/wiki/Statistician [wikipedia.org]

    The current population of the United States of America is 328,621,262 as of Friday, April 19, 2019, based on the latest United Nations estimates.
    the United States population is equivalent to 4.27% of the total world population.

    https://www.worldometers.info/world-population/us-population/ [worldometers.info]

    26,970/328,621,262=.0000820701613640568 or 0.00820701613641%, so,

    As of February 2019, the total population of the world exceeds 7.71 billion people

    http://worldpopulationreview.com/ [worldpopulationreview.com]

    So we are looking at around 2,581,838.1 Statisticians, world wide. We add in Scientists.
    From Unesco [unesco.org]:

    There were 7.8 million full-time equivalent researchers in 2013, representing growth of 21% since 2007. Researchers accounted for 0.1% of the global population.

      2,581,838.1+7,800,000= 10,381,838

    And 800 out of those ten million are calling for the end of the term "statistically significant".

    Starting Score:    1  point
    Moderation   +2  
       Interesting=1, Funny=1, Total=2
    Extra 'Funny' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   4  
  • (Score: 5, Funny) by Bot on Friday April 19 2019, @10:29PM

    by Bot (3902) on Friday April 19 2019, @10:29PM (#832338) Journal

    >And 800 out of those ten million are calling for the end of the term "statistically significant".

    True true, but appeal to rationality didn't work for the default choice of init on linux systems, so, it might not work for science in general too.

    --
    Account abandoned.
  • (Score: 0) by Anonymous Coward on Saturday April 20 2019, @12:55PM (3 children)

    by Anonymous Coward on Saturday April 20 2019, @12:55PM (#832542)

    You are treating this like a random sample, it isn't.

    • (Score: 2) by aristarchus on Saturday April 20 2019, @07:52PM (2 children)

      by aristarchus (2645) on Saturday April 20 2019, @07:52PM (#832674) Journal

      It is a self-selected sample of a rather large population. Kind of like a Fox News Poll.

      • (Score: 0) by Anonymous Coward on Saturday April 20 2019, @10:55PM (1 child)

        by Anonymous Coward on Saturday April 20 2019, @10:55PM (#832751)

        They dont treat themselves as a sample of anything... only you do because you don't get it.

        • (Score: 0) by Anonymous Coward on Saturday April 20 2019, @10:58PM

          by Anonymous Coward on Saturday April 20 2019, @10:58PM (#832754)

          BTW, I say that as someone who would never sign this.