SoylentNews is people
SoylentNews
Richard Horton writes that a recent symposium on the reproducibility and reliability of biomedical research discussed one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with science (PDF), one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. According to Horton, editor-in-chief of The Lancet, a United Kingdom-based medical journal, the apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world or retrofit hypotheses to fit their data.
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative. Tony Weidberg says that the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication following several high-profile errors,. By filtering results through independent working groups, physicists are encouraged to criticize. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. "The good news is that science is beginning to take some of its worst failings very seriously," says Horton. "The bad news is that nobody is ready to take the first step to clean up the system."
[Editor's Comment: Original Submission]
(Score: 3, Insightful) by looorg on Tuesday May 26 2015, @08:31PM
Indeed. As long as my entire career, future and paycheck is tied to some kind of Publish or Perish-metric then that is what I and others are going to do. The only viable method is the shotgun-method; you get your name in on as many things as possible. Some will be boring duds that go straight to the archive and some will be okay. Some might even turn out to be really good. You could spend 10-20-30 years on your "THE ONE"-project but the personal risk vs reward for that one is really low, and possibly not even allowed anymore. I guess if my position was tenured and I was in my sixties and just didn't give a fuck anymore then sure I could do that. But until that time I'm doing what is going to possibly take me there. Is that sad? Yes. Do I wish I could do other things? Yes. Have I cancelled or reworked projects (sexed 'em up)? Sure. Why? Because when was the last time someone read a report where your entire model, data and project failed? Not going to happen. If you get grant money you succeed. If you can't succeed you massage the fuck out of that data until it looks like you did, or at least vaguely so.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @08:51PM
Well I sure as hell don't massage my data.
But I agree that the incentives are all wrong. Nothing worse than feeling like there is pressure for a certain result, or that the story should be simple.
These things are changing though.
(Score: 2) by looorg on Tuesday May 26 2015, @09:00PM
Well I sure as hell don't massage my data.
Right. I wasn't as clear as I should have been. I'll try and rectify that now. I don't change the actual data (cause that would be dishonest and fraud). You change the parameters and other things such as the perspective and/or put limits on your question(s) and query. You don't make up data.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @10:16PM
Sorry. Didn't mean to impugn of your integrity. I definitely feel pressure for things to work out certain ways sometimes. I've been pretty luck so far in that my field is relatively understudied, so even null results are somewhat novel, or have examined population differences with actually large effect sizes consistent with suspected brain damage.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @10:01PM
Only publishing positive results has the same effect on the field as massaging the data. The journals are part of the problem.