Earlier this month, when the biotech firm Human Longevity published a controversial paper claiming that it could predict what a person looks like based on only a teeny bit of DNA, it was just a little over a week before a second paper was published discrediting it as flawed and false. The lightening[sic] speed with which the rebuttal was delivered was thanks to bioRxiv, a server where scientists can publish pre-prints of papers before they have gone through the lengthy peer-review process. It took only four more days before a rebuttal to the rebuttal was up on bioRxiv, too.
This tit-for-tat biological warfare was only the latest in a series of scientific kerfuffles that have played out on pre-print servers like bioRxiv. In a piece that examines the boom of biology pre-prints, Science questions their impact on the field. In a time when a scandal can unfold and resolve in a single day's news cycle, pre-prints can lead to science feuds that go viral, unfolding without the oversight of peer-review at a rapid speed.
"Such online squabbles could leave the public bewildered and erode trust in scientists," Science argued. Many within the scientific community agree.
Should Scientists Be Posting Their Work Online Before Peer Review?
[Source Article (PDF)]: THE PREPRINT DILEMMA
What do you think ??
Related Stories
Attendees of a Howard Hughes Medical Institute meeting debated whether or not science journals should publish the text of peer reviews, or even require peer reviewers to publicly sign their paper critiques:
Scientific journals should start routinely publishing the text of peer reviews for each paper they accept, said attendees at a meeting last week of scientists, academic publishers, and funding organizations. But there was little consensus on whether reviewers should have to publicly sign their critiques, which traditionally are accessible only to editors and authors.
The meeting—hosted by the Howard Hughes Medical Institute (HHMI) here, and sponsored by HHMI; ASAPbio, a group that promotes the use of life sciences preprints; and the London-based Wellcome Trust—drew more than 100 participants interested in catalyzing efforts to improve the vetting of manuscripts and exploring ways to open up what many called an excessively opaque and slow system of peer review. The crowd heard presentations and held small group discussions on an array of issues. One hot topic: whether journals should publish the analyses of submitted papers written by peer reviewers.
Publishing the reviews would advance training and understanding about how the peer-review system works, many speakers argued. Some noted that the evaluations sometimes contain insights that can prompt scientists to think about their field in new ways. And the reviews can serve as models for early career researchers, demonstrating how to write thorough evaluations. "We saw huge benefits to [publishing reviews] that outweigh the risks," said Sue Biggins, a genetics researcher at the Fred Hutchinson Cancer Research Center in Seattle, Washington, summarizing one discussion.
But attendees also highlighted potential problems. For example, someone could cherry pick critical comments on clinical research studies that are involved in litigation or public controversy, potentially skewing perceptions of the studies. A possible solution? Scientists should work to "make the public understand that [peer review] is a fault-finding process and that criticism is part of and expected in that process," said Veronique Kiermer, executive editor of the PLOS suite of journals, based in San Francisco, California.
Related: Peer Review is Fraught with Problems, and We Need a Fix
Odd Requirement for Journal Author: Name Other Domain Experts
Gambling Can Save Science!
Wellcome Trust Recommends Free Scientific Journals
Medical Research Discovered to Have Been Peer Reviewed by a Dog
Should Scientists Be Posting Their Work Online Before Peer Review?
Judge Orders Unmasking of Anonymous Peer Reviewers in CrossFit Lawsuit
(Score: 5, Informative) by gringer on Sunday October 01 2017, @11:56PM (10 children)
Yes.
I've had one research collaborator who submitted a paper for review, got it delayed, then was research-sniped by another lab who published curiously-similar work to what they had submitted in their paper. Telling people early about your research makes it better; telling people early and publicly about your research makes it a lot more difficult for other researchers to claim that they were the first to have the idea.
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 3, Interesting) by melikamp on Monday October 02 2017, @12:54AM (1 child)
(Score: 2) by maxwell demon on Monday October 02 2017, @08:19AM
Of course. Nowhere did the OP say otherwise.
I don't see any indication in that post that the OP thinks that way.
To start with, a preprint server is not a wiki. The only one who has control over the paper that's put there is the author himself. Of course someone else can write a rebuttal, but that is a separate paper. And of course that's possible in a peer-reviewed journal, too.
Second, the OP didn't state that the paper should only be put on the preprint server. I don't know about biology journals, but in physics it is quite common that a journal allows to put a preprint on arXiv before submission; some even offer a procedure where you can submit directly from the preprint server. And authors of published papers will usually put a link to the published version to the arXiv abstract page as soon as the paper is actually published.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 4, Interesting) by Anonymous Coward on Monday October 02 2017, @01:36AM (3 children)
I had my stuff stolen. I was taking a class in logic that was taught by a professor whose PhD mentor was one of the authors of the book. We hit an unanswered question, and I came up with the solution. My professor and worked on writing a paper and were just about ready to submit it, when my professor came to our next meeting with a dejected look on his face. The other coauthor of the book had stolen my idea and was going to have it published. Turns out my professor told his mentor who told his coauthor who rushed out a paper. So now, my idea has numerous papers written about it, appears in SEP, and taught in a textbook used by tens of thousands (at least) of students a year, while being named after someone else.
At least I got an "A" in the class.
(Score: 3, Informative) by TheLink on Monday October 02 2017, @08:22AM (1 child)
I suspect I was the one who inspired CoDel. Everyone and dog was barking up the wrong tree about buffers being too large:
https://dl.acm.org/citation.cfm?id=2071893&preflayout=flat#comments [acm.org]
See Dave Taht's post about "byte queue limits" - limiting queues by bytes:
Followed by my comment:
Followed by my other comment:
After that they came up with CoDel and some revisionist history in Wikipedia: https://en.wikipedia.org/wiki/CoDel#The_CoDel_algorithm [wikipedia.org]
Which is disingenuous since they were still fixated about buffer and queue sizes for years till I made that comment and if you read the actual 2006 pdf ( http://www.pollere.net/Pdfdocs/QrantJul06.pdf [pollere.net] ) it was still stuck in that sort of thinking. You can see the suggestions they were talking about in 2006 was different from the method I suggested. Do note the wiki citation for that sentence is to a 2012 paper, which being in 2012 doesn't back up the claim that it was based on a 2006 notion ;).
But I'm more amused than anything. And I'll be happy if Cisco etc started doing things better. But I wouldn't have any money to challenge any patents though, so if anyone patents ( https://www.google.com/patents/US9686201 [google.com] ) and charges for such stuff I'll leave it to others to fight that prior art thing.
See also my comment on Slashdot in Jan 2011: https://tech.slashdot.org/comments.pl?sid=1939940&cid=34793154 [slashdot.org]
That's before the Dec 2011 comments and 2012 paper and the patent application.
From that Dec 2010 article ( https://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/ [wordpress.com] )
to the Dec 2011 article those bunch were still going "OMG buffers are too big!". When that's not really the problem.
(Score: 1, Informative) by Anonymous Coward on Monday October 02 2017, @11:18AM
See, this is exactly the kind of garbage we'll have to filter through when reading everyone's articles.
(Score: 3, Insightful) by tfried on Monday October 02 2017, @07:14PM
Well, that sucks. I feel for you there.
But then, preprints (aka pre-review-publication) wouldn't have helped you at all, as it just speeds up the process for anyone. If some random asshole can beat you to submitting a paper for peer review, they can beat you to non-reviewed publication just the same.
(Score: 5, Insightful) by looorg on Monday October 02 2017, @05:17AM (1 child)
I do see you point. But I'm afraid we might get over-run with half-finished WIP (work in progress) papers if we go down this path. The WIP work is for colleagues and friends to read and critique. Sure I guess if you don't have either of those then it might be what you have to resort to, after all nobody likes to spend years on something only to be told it's crap and someone was there before you. But I suspect that people instead of writing a good paper are going to post/publish a mountain of WIP work in hope that one of them mentions something that someone else then has finished and in turn claim credit for coming up with the idea for. Trying to cover as much academic ground as they possibly can. It's going to be a nightmare for citation, did or didn't you know about this WIP-work X posted and did you read it and stole their idea.
(Score: 0) by Anonymous Coward on Monday October 02 2017, @06:21AM
Totally agree - who can be bothered to read all the tit-for-tat garbage? Save it for your blog, please.
Also, you can imagine the citations (and bloated citation counts) when every cunt wants to cite every turd they ever squeezed out to try to establish priority 5 minutes before someone else did it. Save it for your court case over patent royalties.
(Score: 2) by tfried on Monday October 02 2017, @07:23PM (1 child)
Don't know. That would appear to mean that you require everybody to read all potentially relevant non-reviewed ("preprint") papers.
There will always be annoying race conditions, and a delay is annoying by definition in itself. But any journal you submit to will know when you submitted your paper, and most will even mention the date of submission on the final publication, so it should be easy enough to show you were first. There might be a bit of a gap in case your paper gets refused by one journal, and you re-submit to a second journal. But even then, you can probably at least get the first editor(s) to rubber-stamp your date of initial submission.
(Score: 2) by gringer on Friday October 06 2017, @01:09AM
I'm thinking along the lines of patent infringement defense. People applying for patents don't need to have a knowledge of everything that's in the public space, and neither do the examiners. It'll just save the applicants a bit of money in being awarded a patent for something that is an already-known public invention.
However, when a claim is made for infringement, it helps to be able to say, "I publicly announced X before the priority date of the patent, so the claim for X should be invalidated."
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 0) by Anonymous Coward on Sunday October 01 2017, @11:56PM
Besides, they already put the paper up at the Cornell archive, don't they?
(Score: 1, Troll) by BK on Monday October 02 2017, @12:58AM (2 children)
This presupposes that there is any trust in scientists.
...but you HAVE heard of me.
(Score: 2) by maxwell demon on Monday October 02 2017, @09:10AM (1 child)
You are aware that the very computer you are using to post on SoylentNews would not have been possible if not for quite a few theories developed, tested and published by scientists?
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by PiMuNu on Monday October 02 2017, @11:57AM
What have scientists ever done for us?
(Score: 2) by takyon on Monday October 02 2017, @01:03AM
If it's out in the open, anybody can review it.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 0) by Anonymous Coward on Monday October 02 2017, @01:48AM
It's always been in public.
If your question is, should researchers go to the press and make claims, or should the press look at some preliminary paper and write about it, that answer is usually "no".
In the end, the public needs to learn to vet the information they consume, whether that source is researchers, the press, random web pages, or politicians.
(Score: 0) by Anonymous Coward on Monday October 02 2017, @03:34AM
People look at these papers in a very wrong way. Publishing one is a final project someone did to graduate, along with everything that entails.
(Score: 2, Insightful) by Anonymous Coward on Monday October 02 2017, @04:20AM (7 children)
The point of journals at some time may have been able being able to get your ideas to the public. But they haven't served that role for a very long time. Now their main role is a filtering mechanism that ensures some degree of value and endorsement of what is stated. Many fields become increasingly esoteric. Even those which have not, seem to have suffered a trend of scientists anxious to try to make their papers as impenetrable as humanly possible. I can only assume they, or journals, feel this makes them seem more educated, like literary editors of the 19th century anxious to one-up each other with their word-smithery. The point there in either case is that many papers are increasingly outside the reach of the public, and so then publishing them to that public implicitly soliciting feedback is like asking a grade schooler to review War and Peace.
Publishing straight to the internet gives a paper no inherent trustability. And in the public mind all papers are created equally. And it leads to scenarios like this where poorly considered 'rebuttals' are "published" only to be immediately rebutted by other "published" papers, all in the same medium where the original paper was published. Science may not need public approval, but it implicitly relies on it. Taking the whole process down to the level of the lowest common denominator is not a good idea.
For the notion of 'protecting' ones work, people could "publish" to a location that does not display the work, but archives it. In cases where it's feared somebody may have stolen an idea from somebody (which in my experience is so phenomenally rare as to be mostly irrelevant) they would have a public means of verification/dating and receiving correct attribution.
(Score: 2) by c0lo on Monday October 02 2017, @04:35AM
Except for esotericism, which lately became quite popular.
Along with chia, quinoa, paleo- and low-carb- diets.
(grin)
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 4, Insightful) by takyon on Monday October 02 2017, @04:54AM (4 children)
I am impressed whenever I come across a paper that explains everything (in a more complex field) in plain English. Some papers also have a simpler summary of the contents after the abstract.
It doesn't seem like the public is harmed by preprints. The very few people and scientists interested in the topic can read it and may judge for themselves whether they think the contents are accurate, or even conduct the same experiment/observations. There seems to be a lot of astronomy preprints, for example, and they could cause some people to point their telecopes at certain objects. No harm done.
Once in a while, a preprint or yet-to-be-published article actually leads to media coverage. Particularly controversial ones on CRISPR topics, and NASA's EmDrive which was highly anticipated. In those cases a lot of attention is paid to it and you might see some kind of informal peer review, like the string of rebuttals mentioned in TFS. If normal unverified studies are leading to crap media coverage, I'd guess it's because of some university putting out a press release, leading other outlets to copy and reword it.
Preprints can also serve another purpose: allowing a non-paywalled version to become freely available. The preprint could be identical to the peer-reviewed article.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 1, Interesting) by Anonymous Coward on Monday October 02 2017, @06:29AM (3 children)
The thing with pre-prints is that invariably one (or more) containing mistakes float around the Interwebs forever. Case in point - I submitted an article, it was accepted and pre-print went online before proofing. During proof I found a horrible error in the equations, which I was able to fix. However both versions are floating on the internet.
Also NIH sponsored work must be published via another portal with its own (very limited) proofing tools. I couldn't fix the equations and the site refused to accept my amended document, so after a couple months I had to press "Accept with no changes" under pain of our grants being rescinded. Another version on the internet :) Enjoy!
(Score: 3, Interesting) by moondrake on Monday October 02 2017, @08:32AM (2 children)
Well..
a while ago I read a horrible published work because the reviewers did not check (or understand) the math (I talked to them).
It should not have passed a good review, but it did, and I like to believe a preprint would have been better.
(Score: 0) by Anonymous Coward on Monday October 02 2017, @11:22AM (1 child)
In fact, that sounds like your average scientific paper. Barely any novel content, barely supported by evidence and 12 authors, barely any of whom contributed.
(Score: 0) by Anonymous Coward on Monday October 02 2017, @01:15PM
stop poking the bare.
(Score: 4, Insightful) by Anonymous Coward on Monday October 02 2017, @08:03AM
I see you made an effort to think about this, but I disagree.
Peer review wasn't really established until the sixties and seventies.
It was certainly quite rare before the second world war, see here this bit about Einstein in 1936: http://physicstoday.scitation.org/doi/full/10.1063/1.2117822. [scitation.org]
A few hundred years ago, scientists literally communicated by letters to each other.
I assume "Annals" (or annuals) appeared as a way to make these letters easier to reference.
Peer review today still has a fuzzy definition.
Ten years ago I was told that as a reviewer I just have to make sure the paper is not crack-pot science and other people are able to understand enough to refute or confirm the results, or reproduce the experiment in the case of experimental sciences.
Today I'm being told that in principle I have to make sure they looked at previous literature properly, paper is well-written, I am able to understand the analytic proofs, basically demanding a lot more of my time.
The point of preprint servers is exactly a proof of authorship.
They are called "preprints" because they are representations of the work before going to print, which can be taken to mean "before peer review".
I believe the initial point of arXiv was to let scientists talk to each other without waiting for the excruciatingly slow journals.
I think average time from submission to publication may have been something like 6 months ore more when arXiv first came along.
If you are worried about the public misinterpreting preprints, I don't think you should.
It is our job to educate them.
Anyone who is willing to ask a scientist will understand in 5 minutes why preprints cannot be seen as settled science.
If journalists ignore this, the solution is to make them have a better code of conduct, not for us to censor ourselves.
Scientists are not supposed to hide the inner working of science from the public, they are supposed to be as open as possible, so that the public can trust them.
If there are fights, let them see the fights.
If they don't like the fights, let them complain, and we will explain why the fights happen.
We're definitely more civilized than politicians and actors, and they're perfectly fine with those idiots fighting with each other.
(Score: 0) by Anonymous Coward on Monday October 02 2017, @02:47PM
We don't want some high priests convene behind closed doors to "make science". If some fools think openness means science is untrustworthy, that is their problem. It's exactly the opposite. Some smart people might get grasp of how it all works. Life is not black and white but complex. Also scientist have prejudices and egos, like any other group of people. Watching science unfold might not be beautiful if there are bitter quarrels and especially if consensus sides with the wrong explanation. But if we can say for sure somebody did side with the wrong thing, then science has progressed so we know it. Science is all we have besides blind faith in collections of contradictory arcane writings and unquestioning obedience of questionable contemporary authorities.
(Score: 3, Insightful) by meustrus on Monday October 02 2017, @05:36PM
I see a lot of people answering a different question than the one posed. Comments advocate either for all first drafts to be made publicly available or for research to continue appearing in journals as the source of truth.
But that doesn't answer the question, which was "Should Scientists Be Posting Their Work Online Before Peer Review?" (emphasis mine).
My answer to that question is yes. The process described in the summary sounds like crowd-sourced peer review happening out in the open. How is this bad, as long as we still end up with a collection of peer-reviewed papers in the journals?
The only concern would be if laypeople came across the rough drafts and tried to make news out of them. Anybody who is willing to and capable of sifting through the firehose of papers going through a place like bioRxiv is by definition not a layperson, so I don't see that argument holding water.
If there isn't at least one reference or primary source, it's not +1 Informative. Maybe the underused +1 Interesting?
(Score: 2) by sbgen on Tuesday October 03 2017, @01:15AM
Some things come to mind that I did not see explicitly mentioned in the comments. I am a scientist so I am interested in the publication matter - that is my declaration of conflict of interest.
1 - "Science" journal argues against prepublication servers (bioRxiv in the case mentioned here). Of course it does, its very business model depends on scientific journals being closed and science getting accessed only via sanctioned journals. Do you know how much it costs to access an article published in Science for the taxpayer? I can only access their articles from the University. I keep reading the adage "Follow the money" in all articles but did not see it here
2 - When you submit an article to a preprint server you get a DOI number (Digital Object Number). This is a document that tells who submitted it and when. That establishes the attribution credentials and guards against some one else stealing your idea
3 - Not all the articles on preprint servers are good or peer reviewed or even noticed by the majority. However those that are good certainly float up. This situation is no different than that existing in "science" or other commercial scientific journals.
4 - If scientists are squabbling in open about a work, as a practitioner and taxpayer I count my blessings. This is good, an open source approach. I hope those on this site understand it well.
Warning: Not a computer expert, but got to use it. Yes, my kind does exist.