SoylentNews

Unreliable Social Science Research Gets More Attention Than Solid Studies

posted by janrinok on Thursday July 01 2021, @11:18AM   
from the good-science-is-boring dept.

JoeMerchant writes:

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)

---

Original Submission

• Re:Has science already gone the way of the media? Re:Has science already gone the way of the media? (Score: 2) by JoeMerchant on Wednesday July 21 2021, @02:06PM (1 child)

  by JoeMerchant (3937) on Wednesday July 21 2021, @02:06PM (#1158738)

  The Fox News model of bias + coverage has not only become the rule, but has become intensely magnified.

  As compared to the Cronkite days, I agree. But deeper in the past I think bias + coverage + more bias was quite common, particularly in the print media when not everyone could read and they got bias from the people who did the reading for them plus the people who paid for the printing and distribution.

  --
  Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end

  Parent

  Starting Score:	1		point
  Karma-Bonus Modifier		+1
  Total Score:		2

• Re:Has science already gone the way of the media? (Score: 2) by Socrastotle on Wednesday July 21 2021, @03:00PM

  by Socrastotle (13446) on Wednesday July 21 2021, @03:00PM (#1158745) Journal

  I also get the exact same impression:

  It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle. The real extent of this state of misinformation is known only to those who are in situations to confront facts within their knolege with the lies of the day.

  I really look with commiseration over the great body of my fellow citizens, who, reading newspapers, live & die in the belief, that they have known something of what has been passing in the world in their time; whereas the accounts they have read in newspapers are just as true a history of any other period of the world as of the present, except that the real names of the day are affixed to their fables.

  General facts may indeed be collected from them, such as that Europe is now at war, that Bonaparte has been a successful warrior, that he has subjected a great portion of Europe to his will, &c., &c.; but no details can be relied on. I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false.

  That quote [uchicago.edu] was from Thomas Jefferson, 1807. And there were countless other similar sentiments from various founding fathers and other great thinkers of the times. I used to believe these sentiments were part rhetorical, part metaphor. But now we're entering into an era where what he said, even taken completely at face value, is not even especially hyperbolic.

  So in many ways I think our current era, as crazy as seems things is not really new. Rather we "all" (being the majority of humans born in the last 50-60 years in the developed world), basically just grew in a sort of mini-utopia that was little more than a bubble in time. And so as that bubble pops it kind of feels like we're heading towards some unprecedented insanity. But no, it's quite precedented. What isn't precedented is how nice we had things for some time.

  This also even applies if you go much further back. Reading the writings of Aristotle, Plato, Socrates, Aurelius, and so on - it feels very much as if they are describing the world we now live in. "The Republic" in particular is practically a transcription of the happenings of the day.

  Parent