SoylentNews is people
SoylentNews
Attendees of a Howard Hughes Medical Institute meeting debated whether or not science journals should publish the text of peer reviews, or even require peer reviewers to publicly sign their paper critiques:
Scientific journals should start routinely publishing the text of peer reviews for each paper they accept, said attendees at a meeting last week of scientists, academic publishers, and funding organizations. But there was little consensus on whether reviewers should have to publicly sign their critiques, which traditionally are accessible only to editors and authors.
The meeting—hosted by the Howard Hughes Medical Institute (HHMI) here, and sponsored by HHMI; ASAPbio, a group that promotes the use of life sciences preprints; and the London-based Wellcome Trust—drew more than 100 participants interested in catalyzing efforts to improve the vetting of manuscripts and exploring ways to open up what many called an excessively opaque and slow system of peer review. The crowd heard presentations and held small group discussions on an array of issues. One hot topic: whether journals should publish the analyses of submitted papers written by peer reviewers.
Publishing the reviews would advance training and understanding about how the peer-review system works, many speakers argued. Some noted that the evaluations sometimes contain insights that can prompt scientists to think about their field in new ways. And the reviews can serve as models for early career researchers, demonstrating how to write thorough evaluations. "We saw huge benefits to [publishing reviews] that outweigh the risks," said Sue Biggins, a genetics researcher at the Fred Hutchinson Cancer Research Center in Seattle, Washington, summarizing one discussion.
But attendees also highlighted potential problems. For example, someone could cherry pick critical comments on clinical research studies that are involved in litigation or public controversy, potentially skewing perceptions of the studies. A possible solution? Scientists should work to "make the public understand that [peer review] is a fault-finding process and that criticism is part of and expected in that process," said Veronique Kiermer, executive editor of the PLOS suite of journals, based in San Francisco, California.
Related: Peer Review is Fraught with Problems, and We Need a Fix
Odd Requirement for Journal Author: Name Other Domain Experts
Gambling Can Save Science!
Wellcome Trust Recommends Free Scientific Journals
Medical Research Discovered to Have Been Peer Reviewed by a Dog
Should Scientists Be Posting Their Work Online Before Peer Review?
Judge Orders Unmasking of Anonymous Peer Reviewers in CrossFit Lawsuit
(Score: 2) by bzipitidoo on Wednesday February 14 2018, @07:25PM (1 child)
Going off on a seeming tangent here, but it'll make sense, just bear with me. Once read a critique in praise of email, for eliminating the stilted formality that had become customary in handwritten (or hand typed) letters, stuff such as starting the letter with "Dear". Scientific research is still stuck with a lot of formality. Journals typically require of authors such trivia as using only approved fonts and sizes and other typesetting details that simply are not relevant to research. Some even provide LaTeX templates, which certainly makes it easier, but doesn't satisfy the issue that scientists should not have to spend time on text formatting issues. There is also a strict page limit that can be impossible to check without formatting the text. While the page limit can help authors stay focused, much like the much lauded 140 character text message and tweet limit, it really does not seem necessary any more, not with the incredible amounts of digital storage at our disposal.
Also traditionally excluded but now being welcomed yet still not part of the article, are such things as source code and data sets. Of course data sets can be way too large for presentation in the same manner as an article. However, mathematical formulae are not only wanted, they are darn near required. But source code? The way a journal article should be is not just in a digital format, but in a format that allows easy transferral of math to suitable mathematical software, which could be MatLab except that's proprietary, and data to a database or spreadsheet or just a file, and source code to the relevant language compiler or interpreter. The "finished" product being a pretty PDF file was the wrong direction. They're still thinking of the ultimate destination being a printed book. A format such as EPUB (which is really just HTML in a zip file) is a better direction to go. It would of course be trivial to include the reviews.
So, thank you for your patience, and what does all that have to do with the subject? It's kind of a slippery slope, a good one. Challenge the assumption that reviews should be private, and maybe that will lead to these other issues being challenged. Or the other way around, break this addiction to PDF, typesetting, and printing on dead trees, and maybe the idea that reviews shouldn't be private any more will follow. There's a lot about scientific publication that needs scrutiny and improvement. I suspect one of the motivations for clinging to these outmoded printing considerations is academic publishers who wish to maintain their current stranglehold. Ending the reign of PDF isn't only about freeing scientists from mere text formatting problems, it's about freeing us all from these academic publishers who have degenerated into nothing more than rent seeking parasites.
Another issue is that publishers don't pay reviewers, we, the public, pay for that, and only indirectly. But reviewers not employed at universities may well get nothing at all, other than brownie points or karma or the satisfaction of knowing that they helped advance mankind's knowledge, or some other vague, ephemeral compensation, for doing a review. I've done reviews where I pointed out very specific things that the authors didn't mention and should have, and even cited the relevant papers. And what did the bastards do but copy my review, nearly verbatim, into their paper! I know, because the revised version was kicked back to me for further review. Felt like I should have been a co-author. In that case, yeah, I think I would've liked not only my review to be public, but my name too.
This brings up yet another problem, that of science being too solitary. You toil away on a paper all by yourself, sweat over whether it's correct, and whether it's good enough. The uncertainty is great because no one else has seen any of it. But of course in the world of "publish or perish", many are happy to get a free ride and add their names and not contribute, so collaboration isn't all roses. Then you finally submit it to a journal, and wait months for a review. And then it gets savaged. Maybe you missed some simple little thing. It's like playing chess, thinking and thinking about a move, and all the while you've overlooked an easy checkmate your opponent has. Chess grandmasters have lost games because of bad blunders like that. The entire education system is too biased towards lone wolf science rather than real collaboration.
I think reviews being public is fair, as long as reviewers know that their reviews will not be private. The dirty laundry you speak of, well, if it's "bad" dirt, like an unfair rejection because the reviewer was too busy and gave the submission short shrift, then good riddance. If it's honest and good criticism of problems with the submission, then I am not afraid of the public misunderstanding. There've been many problems where secrecy merely fueled wild conspiracy theories, and that is worse.
(Score: 2) by AthanasiusKircher on Thursday February 15 2018, @02:49AM
This is an interesting reply; thanks for taking the time. And again, I agree with a lot of what you said -- especially in terms of problems in academic publishing -- so I'm not going to really argue against it (since I already made my thoughts clear in another long post).