Stories
Slash Boxes
Comments

SoylentNews is people

posted by CoolHand on Sunday August 30 2015, @04:59AM   Printer-friendly
from the lies-damn-lies-and-statistics dept.

Science is a messy, error fraught business, which is why reproducibility is so essential. Unfortunately, that doesn't appear to be one of psychology's strong suits, according to a massive analysis published yesterday in Science.

A years-long effort to reproduce more than 100 psychology studies across three leading journals paints a pretty dismal picture. When re-tested by independent research psychologists, the conclusions of more than 60 studies on personality, relationships, learning, and memory, turned out to be far less whelming. Strongly significant findings often became weaker, while weakly significant findings became non-existent.

http://gizmodo.com/a-lot-of-published-psychology-results-are-bullshit-1727228060

[Source]: The New York Times


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 0) by Anonymous Coward on Sunday August 30 2015, @08:03AM

    by Anonymous Coward on Sunday August 30 2015, @08:03AM (#229744)

    Houston, we have a problem!

    Notice how he said "80%" and not all. Not sure where that number came from, but still.

    Furthermore, generally, the more complex whatever you're studying is, the more difficult it is to study it properly. Reproducing the studies to see if you get the same result is more simple than doing original and good science in the first place, as there is less that can go wrong. That is why this is more believable than the types of studies that reach arbitrary and subjective conclusions about how violent video games make people more violent, how X makes people "callous" towards Y, etc.

  • (Score: 4, Informative) by Phoenix666 on Sunday August 30 2015, @11:38AM

    by Phoenix666 (552) on Sunday August 30 2015, @11:38AM (#229790) Journal

    As someone trained in the social sciences, I have on occasion thought that physical scientists rather take the easy way out in studying natural phenomena that can be pinned down, controlled, and whose behaviors can be reproduced. People are devilishly difficult to study, because they are complex and moving all the time. They're conscious, and consciously or unconsciously mess with your study. As such it's much more complex, much more multi-variate a system than what, say, chemists tackle.

    --
    Washington DC delenda est.
    • (Score: 2) by c0lo on Sunday August 30 2015, @12:58PM

      by c0lo (156) Subscriber Badge on Sunday August 30 2015, @12:58PM (#229809) Journal

      As such it's much more complex, much more multi-variate a system than what, say, chemists tackle.

      If the "social scientists" would deal with populations on the order of 1023, I reckon the social sciences would be as accurate as chemistry (which is not that accurate as many people think, those equilibrium points for the reversible reactions are correct just statistically and, anyway, empirically determined).

      --
      https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
      • (Score: 2) by maxwell demon on Sunday August 30 2015, @02:59PM

        by maxwell demon (1608) on Sunday August 30 2015, @02:59PM (#229849) Journal

        Actually I've once heard that some chemical reactions won't work if the chemist cleans his equipment too thoroughly.

        --
        The Tao of math: The numbers you can count are not the real numbers.
    • (Score: 0) by Anonymous Coward on Sunday August 30 2015, @01:39PM

      by Anonymous Coward on Sunday August 30 2015, @01:39PM (#229825)

      I think Dr. Fancyfree [wikia.com] says it best regarding this entire field: I'm going to do something that goes against all my training as a therapist... I'm going to CURE you!!!

    • (Score: 0) by Anonymous Coward on Sunday August 30 2015, @04:06PM

      by Anonymous Coward on Sunday August 30 2015, @04:06PM (#229873)

      It's complex, but that is no reason to lower our standards. Bad science is bad science. I'm not saying you were necessarily saying that, though.

    • (Score: 2) by Nerdfest on Sunday August 30 2015, @05:17PM

      by Nerdfest (80) on Sunday August 30 2015, @05:17PM (#229896)

      Hah. Try quantum physics.

      • (Score: 2) by Phoenix666 on Sunday August 30 2015, @05:25PM

        by Phoenix666 (552) on Sunday August 30 2015, @05:25PM (#229902) Journal

        Touche, nerdfest.

        --
        Washington DC delenda est.
      • (Score: 2) by hendrikboom on Tuesday September 01 2015, @01:38PM

        by hendrikboom (1125) Subscriber Badge on Tuesday September 01 2015, @01:38PM (#230796) Homepage Journal

        The theoretical calculations of various properties of fundamental particles have often been orders of magnitude more accurate than the observations the theories were originally based on, as evidenced by later, more accurate observations.

        No one knows why.

    • (Score: 0) by Anonymous Coward on Sunday August 30 2015, @06:29PM

      by Anonymous Coward on Sunday August 30 2015, @06:29PM (#229924)

      It is not specific to social sciences. [washingtonpost.com] There is tons of pressure to bias results in the hard sciences too.

      IIRC there was a recent story about how requiring scientists to formally specify their testing regimen up front, before starting the experiment, drastically cut down the 'success rate.' The theory being that the urge to comb through the data after the fact, looking for novel results was just too strong.

  • (Score: 2) by c0lo on Sunday August 30 2015, @12:47PM

    by c0lo (156) Subscriber Badge on Sunday August 30 2015, @12:47PM (#229806) Journal

    Not sure where that number came from, but still.

    He's trying to align more with the classics than with Sturgeon [wikipedia.org].

    "ninety percent of everything is crap."
    ...
    "I repeat Sturgeon's Revelation, which was wrung out of me after twenty years of wearying defense of science fiction against attacks of people who used the worst examples of the field for ammunition, and whose conclusion was that ninety percent of SF is crud. Using the same standards that categorize 90% of science fiction as trash, crud, or crap, it can be argued that 90% of film, literature, consumer goods, etc. is crap. In other words, the claim (or fact) that 90% of science fiction is crap is ultimately uninformative, because science fiction conforms to the same trends of quality as all other artforms

    From the same page and as a word of caution for those disgusted by 80% crap of everything:

    A similar adage with a slightly different percentage appears in Rudyard Kipling's The Light that Failed, published in 1890. "Four–fifths of everybody's work must be bad. But the remnant is worth the trouble for its own sake."

    --
    https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford