Stories
Slash Boxes
Comments

SoylentNews is people

posted by n1 on Wednesday November 19 2014, @01:33AM   Printer-friendly
from the peer-reviewed-study-confirms-it dept.

Phys.org is running a story on some of the issues with modern peer review:

Once published, the quality of any particular piece of research is often measured by citations, that is, the number of times that a paper is formally mentioned in a later piece of published research. In theory, this aims to highlight how important, useful or interesting a previous piece of work is. More citations are usually better for the author, although that is not always the case.

Take, for instance, Andrew Wakefield's controversial paper on the association between the MMR jab and autism, published in leading medical journal The Lancet. This paper has received nearly two thousand citations – most authors would be thrilled to receive a hundred. However, the quality of Wakefield's research is not at all reflected by this large number. Many of these citations are a product of the storm of controversy surrounding the work, and are contained within papers which are critical of the methods used. Wakefield's research has now been robustly discredited, and the paper was retracted by the Lancet in 2010. Nevertheless, this extreme case highlights serious problems with judging a paper or an academic by number of citations.

Personally, I've been of the opinion that peer review is all but worthless for quite a while. It's nice to know I'm not the only one who has issues with the process.

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 4, Insightful) by Sir Garlon on Wednesday November 19 2014, @02:43AM

    by Sir Garlon (1264) on Wednesday November 19 2014, @02:43AM (#117490)

    There are plenty of places on the internet where every troll and idiot's comments are treated equally. If you actually enjoy discussing things there, then I don't think you need Soylent News.

    Bringing us back toward the topic, moderation should be used more like citation count and less like peer review. When you're first exploring a topic, highly-cited papers are a good place to start because they will define the major principles of the discussion. Likewise, the highly-moderated posts in a discussion are the ones that best capture the fundamental points. Moderation should indicate that a post is worth the time to read, that it is relevant and illuminating. Confirmation bias being what it is, it is a lot easier to see the merit in a post one happens to agree with. In principle, I think we all know that a coherent, well-argued post we disagree with is worthy of positive moderation.

    I agree that moderation seems too often to be used as a substitute for the facebook "like" button, but that is what the "overrated" mod is for.

    Now, if you think a post is especially good or bad, flawed, stimulating, ignorant, whatever, that's what the reply button is for. That is what peer review should be in a scientific paper -- a response to the research.

    --
    [Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
    Starting Score:    1  point
    Moderation   +2  
       Insightful=1, Interesting=1, Total=2
    Extra 'Insightful' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   4  
  • (Score: 0) by Anonymous Coward on Wednesday November 19 2014, @03:50AM

    by Anonymous Coward on Wednesday November 19 2014, @03:50AM (#117512)

    A "citation count"? That sounds a fuck of a lot like "karma". You know, the goddamn numbers that turn places like Hacker News and Stack Overflow into absolute shitholes. It's bad enough seeing people fawn over tptacek and Jon Skeet, and otherwise engage in the community-wide circle-jerks that those places have become. It's totally disgusting to suggest bringing such a flawed concept here. The /. moderation system is total shit, but it's nowhere near as utterly broken and futile as the moderation systems over at HN and SO are.