SoylentNews is people
SoylentNews
This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.
[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.
Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.
And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.
Well, nobody likes a Debbie Downer, do they?
Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)
(Score: 3, Interesting) by Anonymous Coward on Thursday July 01 2021, @01:06PM (5 children)
This is a good reminder to researchers (of any sort) that citing a paper means you read it. Well...at least you scanned it (one can hope)...and didn't just crib the your list of cites from some other paper.
A cite doesn't mean that you verified the results of that paper. It usually means that you incorporated ideas (or even words, properly referenced of course) from that paper into your own work and paper. It doesn't always mean that you agree with the results of the cite.
My take is that citation counts are about like mod points here--a popularity contest? I may look at mods, but they don't mean all that much.
(Score: 3, Insightful) by looorg on Thursday July 01 2021, @02:10PM (1 child)
That is the best case scenario. A sad and quite likely other outcome is that my research assistant did a search for some papers that could back up what we are doing and your paper was inline with that or related to it somehow on a keyword level or something similar so we are including you to get out citation number up since if we cite you we are then more likely to be cited ourselves when the next person looks to cite someone and finds you and then also us so it's a gigantic citation-circlejerk.
(Score: 4, Interesting) by JoeMerchant on Thursday July 01 2021, @04:12PM
I remember working with "lab partners" in social sciences in Junior College - and this is the best possible behavior I could imagine coming out of any of them. Whatever the minimum possible effort to meet the requirements, and often less, is what I saw the majority of them doing - even the ones who were pursuing it as their major and potential career.
Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
(Score: 2) by DeathMonkey on Thursday July 01 2021, @03:14PM
And of course we only know it's not reproduceable because somebody tried to reproduce it. And that failed reproduction would also increase the citation count!
(Score: 4, Insightful) by Thexalon on Thursday July 01 2021, @05:28PM
Among other things: "Here's a Detailed Explanation of Why Popular Study X is Wrong" will invariably cite the study that they're presenting an argument attempting to debunk it. So that means that well-hyped-and-wrong research beats obscure-and-right research on citation counts every time. If you're looking for a lot citations, forget trying to get that acceptance at a prominent conference or publication in Nature, what you really want is your study being sensational enough or financially backed enough that it shows up on CNN.
And if you think that leads to bunk research, you're absolutely right.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.
(Score: 2) by Opportunist on Friday July 02 2021, @06:33AM
So far the theory.
In fact, a cite usually means that whoever cited found the paper in a keyword search, did a cursory read to see whether it supports or contradicts him and in the first case, it gets used.