Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Thursday July 01 2021, @11:18AM   Printer-friendly
from the good-science-is-boring dept.

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by Socrastotle on Friday July 02 2021, @05:40AM

    by Socrastotle (13446) on Friday July 02 2021, @05:40AM (#1152165) Journal

    In general I'm referring to times much more recent. The media started to rapidly deteriorate with the advent of the internet. There's a phenomenal article about this by Robert Kaiser, The Bad News About the News [brookings.edu]. Kaiser worked at the Washington Post for 50 years, seeing it from its peak as a reputable paper to its more recent collapse. I could not recommend that article more highly, and I don't think I can do it justice with cliff notes.

    And so I suspect ultimately that the hey day of the media, perhaps some time between 1960 - 1990, give or take some years either way, was likely an anomaly in time. And so our modern media has become much more similar to the media Jefferson was familiar with where in an effort to move a rag (or in this case, generate), they will say anything and everything - regardless of its truthfulness. Yet because times prior to the that golden age of media feel like they may as well have been a millennia ago, we perceive the media in terms of its current abysmal state, and only have what we know came before to contrast it against.

    It's quite remarkable that the NYTimes went from, in 1971, publishing the Pentagon Papers [wikipedia.org] to, just 30 years later, publishing outright intelligence agency propaganda such as "Irrefutable and Undeniable [nytimes.com] where they unabashedly tried to sell a war, built on fabrications and lies, to the American people. And it's only been downhill from there. The contrast makes the decline all the more overt.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2