Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Thursday July 01 2021, @11:18AM   Printer-friendly
from the good-science-is-boring dept.

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by DannyB on Thursday July 01 2021, @03:53PM (4 children)

    by DannyB (5839) Subscriber Badge on Thursday July 01 2021, @03:53PM (#1151836) Journal

    In the past when the media said something, you could generally take it with some reasonable degree of assurance that it was very likely to be true. In the worst case it was generally just misleading.

    In the past, the media was motivated to provide real news. Some strange thing called Journalism. Who, what, when, where, why, how.

    And some editorial opinion, which could be misleading.

    Ownership of a news outlet could bias its coverage to be a bit misleading.

    Yet the facts were true. They even had offices of factual verification.

    Now, the media has become entertainment. Or infotainment. And the news outlet's owners have no shame about their news infotainment being heavily biased. To such an extreme that in 2013 when the Snowden story broke, CNN didn't even make a pretense of objectivity, there simply couldn't possibly be any other POV than the government's view.

    --
    If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 3, Interesting) by JoeMerchant on Thursday July 01 2021, @04:21PM (3 children)

    by JoeMerchant (3937) on Thursday July 01 2021, @04:21PM (#1151850)

    What past are you all living in? The past where the Yellow press pushed the U.S. into the Spanish American War? The past where Northern and Southern ideals were so strongly reinforced that a civil war broke out? Or the past where Walter Cronkite read the news to you and because he hadn't made a total self contradictory ass of himself for his employer, you trusted him?

    The past I remember most clearly is the one where our local news programs would read editorials in line with their political views and present them very professionally, very convincingly, and then a week or six later let some local kook on for 1/3 the time to present an "opposing viewpoint" by someone who clearly doesn't do this for a living, probably wrote their own copy, probably dropped out of high school after failing English too many times, stumbles over their words, and has makeup that makes them look like a carnival clown who slept in late before going on camera.

    --
    Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
    • (Score: 2) by Socrastotle on Friday July 02 2021, @03:59PM (2 children)

      by Socrastotle (13446) on Friday July 02 2021, @03:59PM (#1152267) Journal

      One can only speak for themselves, but I am speaking of times I described in a post below. [soylentnews.org] I rarely recall the media in 'the golden age' going so nutters over political stuff. There is always some degree of implicit bias based on what is covered and what isn't covered, but now a days when you look at the front page even of formerly reputable papers like the NYTimes, it's scarcely different from what used to be relegated to things like post-debate 'spin rooms'. The Fox News model of bias + coverage has not only become the rule, but has become intensely magnified. There seems to be little to nothing left in the way of remotely impartial coverage.

      Of course I have to also consider the possibility that it's a product of [relative] youth. I only began to be somewhat politically aware around the Iraq War and, lo and behold, from my perspective the complete deterioration of the media also began at just about the same time. Of course there are many things that make me suspect this is not the case. This [brookings.edu] article from Bob Kaiser (who had a 50 year career during the 'golden era' of the Washington Post working his way up to managing editor), suggests as much from an individual with a rather radically different perspective and age. And the case he make as an insider directly match the observations I have seen from the outside.

      In any case, I've no delusions of the past of our media being ideal. But I do believe it was orders of magnitude better than its state today which seems designed entirely to generate clicks. And the problem with this is that impartial reporting doesn't generate clicks. Hyperbole, sensationalism, exploiting emotions, appealing to biases, even indeed even lying to generate even more sordid tales - those sort of things generate clicks. At least until they don't. When people begin appreciating that most of all modern media is little different than the "National Enquirers" that we grew up with, the show will be over.

      • (Score: 2) by JoeMerchant on Wednesday July 21 2021, @02:06PM (1 child)

        by JoeMerchant (3937) on Wednesday July 21 2021, @02:06PM (#1158738)

        The Fox News model of bias + coverage has not only become the rule, but has become intensely magnified.

        As compared to the Cronkite days, I agree. But deeper in the past I think bias + coverage + more bias was quite common, particularly in the print media when not everyone could read and they got bias from the people who did the reading for them plus the people who paid for the printing and distribution.

        --
        Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
        • (Score: 2) by Socrastotle on Wednesday July 21 2021, @03:00PM

          by Socrastotle (13446) on Wednesday July 21 2021, @03:00PM (#1158745) Journal

          I also get the exact same impression:

          It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle. The real extent of this state of misinformation is known only to those who are in situations to confront facts within their knolege with the lies of the day.

          I really look with commiseration over the great body of my fellow citizens, who, reading newspapers, live & die in the belief, that they have known something of what has been passing in the world in their time; whereas the accounts they have read in newspapers are just as true a history of any other period of the world as of the present, except that the real names of the day are affixed to their fables.

          General facts may indeed be collected from them, such as that Europe is now at war, that Bonaparte has been a successful warrior, that he has subjected a great portion of Europe to his will, &c., &c.; but no details can be relied on. I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false.

          That quote [uchicago.edu] was from Thomas Jefferson, 1807. And there were countless other similar sentiments from various founding fathers and other great thinkers of the times. I used to believe these sentiments were part rhetorical, part metaphor. But now we're entering into an era where what he said, even taken completely at face value, is not even especially hyperbolic.

          So in many ways I think our current era, as crazy as seems things is not really new. Rather we "all" (being the majority of humans born in the last 50-60 years in the developed world), basically just grew in a sort of mini-utopia that was little more than a bubble in time. And so as that bubble pops it kind of feels like we're heading towards some unprecedented insanity. But no, it's quite precedented. What isn't precedented is how nice we had things for some time.

          This also even applies if you go much further back. Reading the writings of Aristotle, Plato, Socrates, Aurelius, and so on - it feels very much as if they are describing the world we now live in. "The Republic" in particular is practically a transcription of the happenings of the day.