SoylentNews

Unreliable Social Science Research Gets More Attention Than Solid Studies

posted by janrinok on Thursday July 01 2021, @11:18AM   
from the good-science-is-boring dept.

JoeMerchant writes:

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)

---

Original Submission

• Re:Has science already gone the way of the media? Re:Has science already gone the way of the media? (Score: 1, Touché) by Anonymous Coward on Thursday July 01 2021, @06:33PM (1 child)

  by Anonymous Coward on Thursday July 01 2021, @06:33PM (#1151936)

  In the past when the media said something, you could generally take it with some reasonable degree of assurance that it was very likely to be true. In the worst case it was generally just misleading.

  Really, now?

  To your request of my opinion of the manner in which a newspaper should be conducted, so as to be most useful, I should answer, `by restraining it to true facts & sound principles only.' Yet I fear such a paper would find few subscribers. It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle. The real extent of this state of misinformation is known only to those who are in situations to confront facts within their knowlege with the lies of the day. I really look with commiseration over the great body of my fellow citizens, who, reading newspapers, live & die in the belief, that they have known something of what has been passing in the world in their time; whereas the accounts they have read in newspapers are just as true a history of any other period of the world as of the present, except that the real names of the day are affixed to their fables. General facts may indeed be collected from them, such as that Europe is now at war, that Bonaparte has been a successful warrior, that he has subjected a great portion of Europe to his will, &c., &c.; but no details can be relied on. I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false.

  Thomas Jefferson to John Norvell, June 11, 1807

  Just how far into the past shall we find this utopia of honest journalism?

  Parent

  Starting Score:	0		points
  Moderation		+1
  Touché=1, Total=1
  Extra 'Touché' Modifier		0
  Total Score:		1

• Re:Has science already gone the way of the media? (Score: 2) by Socrastotle on Friday July 02 2021, @05:40AM

  by Socrastotle (13446) on Friday July 02 2021, @05:40AM (#1152165) Journal

  In general I'm referring to times much more recent. The media started to rapidly deteriorate with the advent of the internet. There's a phenomenal article about this by Robert Kaiser, The Bad News About the News [brookings.edu]. Kaiser worked at the Washington Post for 50 years, seeing it from its peak as a reputable paper to its more recent collapse. I could not recommend that article more highly, and I don't think I can do it justice with cliff notes.

  And so I suspect ultimately that the hey day of the media, perhaps some time between 1960 - 1990, give or take some years either way, was likely an anomaly in time. And so our modern media has become much more similar to the media Jefferson was familiar with where in an effort to move a rag (or in this case, generate), they will say anything and everything - regardless of its truthfulness. Yet because times prior to the that golden age of media feel like they may as well have been a millennia ago, we perceive the media in terms of its current abysmal state, and only have what we know came before to contrast it against.

  It's quite remarkable that the NYTimes went from, in 1971, publishing the Pentagon Papers [wikipedia.org] to, just 30 years later, publishing outright intelligence agency propaganda such as "Irrefutable and Undeniable [nytimes.com] where they unabashedly tried to sell a war, built on fabrications and lies, to the American people. And it's only been downhill from there. The contrast makes the decline all the more overt.

  Parent