Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Thursday July 01 2021, @11:18AM   Printer-friendly
from the good-science-is-boring dept.

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 5, Interesting) by Socrastotle on Thursday July 01 2021, @03:06PM (1 child)

    by Socrastotle (13446) on Thursday July 01 2021, @03:06PM (#1151811) Journal

    Ah! But again I think you're not considering the order of causality here. Somebody does not become right wing because they stumbled onto and started reading Breitbart. Instead people with right wing tendencies seek out Breitbart and then believe what they read because they *want* it to be true. And similarly, somebody does not become left wing because they picked up a copy of the NYTimes and started reading it. Instead people with left wing tendencies seek out the NYTimes and then believe what they are told because they *want* it to be true. If you swapped the ideological narrative of both sites, yet kept the same respective quality (or lack thereof) of truthfulness, their readers would most certainly suddenly stop believing what they were told. Because it would not be what they *want* to be true.

    And so people are thinking for themselves, on at least a basic level. They choose what they want to be true, and then seek out information to confirm that view. If their news sources of choice stop confirming that view? No problem they just move to the next one that does. In the age of the internet, they are limitless.

    Starting Score:    1  point
    Moderation   +3  
       Insightful=1, Interesting=2, Total=3
    Extra 'Interesting' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   5  
  • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @06:42PM

    by Anonymous Coward on Thursday July 01 2021, @06:42PM (#1151942)

    If you swapped the ideological narrative of both sites, yet kept the same respective quality (or lack thereof) of truthfulness, their readers would most certainly suddenly stop believing what they were told.

    I used to believe that, but I'm no longer sure. The Trump presidency stands as empirical evidence of that. There are literally too many instances to count, but the big one which got me was how quickly "Lock Her Up" disappeared when Trump said a word about not wanting to pursue that anymore. And remember how much of the Republican Party was "Never Trump"ers, but how quickly that changed?

    As another example, consider the Democrat's (and the "left") opinion of Russia. It want from "why can't we all get along, this isn't the cold war anymore" to "you don't understand, they are a major threat to our democracy." Likewise, the Republican's (and the "right") quickly went to "you can never trust a Russian" to "ehh, Putin isn't such a bad guy."

    My current theory is that people establish their affiliation with something (be it an idea, a group, or what have you), and then that defines them. If that group changes, then most people go along with that change. If The Party announced "the cacao ration has been raised from 10 grams to 8 grams," sure, some would question it... but I think a lot more would cheer the party for doing such a great job.

    (Which is why it is all the more important for Benjamins of the world to not just be cynical asses and to actually do something to help the situation... to mix fictional metaphores.)