Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Monday August 04 2014, @07:45PM   Printer-friendly
from the stored-in-the-round-filing-cabinet dept.

The Conversation looked at one of the problems with science publication; scientists are under pressure to appear "interesting", because journals want to publish interesting headlines and attention grabbing articles. Thankfully I've not yet seen a science article titled "Seven reasons the dinosaurs became extinct. Number 4 will AMAZE you" or "These psychology students took part in a study on generosity. What happens next will SHOCK you".

This is particularly true of the most prestigious journals like Science and Nature. What this means in practice is that journals don't like to publish negative or mixed results studies where you predicted you would find something but actually didn't, or studies where you found a mix of conflicting results.

If your results are unambiguously negative, there is not much you can do. Foreseeing long months of re-submissions to increasingly obscure journals, you consign your study to the file-drawer for a rainy day that will likely never come.

But if your results are less clear-cut? What if some of them suggest your theory was right, but some don't? Again, you could struggle for months or years, scraping the bottom of the journal barrel to find someone to publish the whole lot.

Or you could "simplify". After all, most of your results are in line with your predictions, so your theory is probably right. Why not leave those "aberrant" results out of the paper? There is probably a good reason why they turned out like that. Some anomaly. Nothing to do with your theory really.

Nowhere in this process do you feel like you are being deceptive. You just know what type of papers are easiest to publish, so you chip off the "boring" complications to achieve a clearer, more interesting picture. Sadly, the complications are probably closer to messy reality. The picture you publish, while clearer, is much more likely to be wrong.

Science is supposed to have a mechanism for correcting these sorts of errors. It is called replication, and it is one of the cornerstones of the scientific method. Someone else replicates what you did to see if they get the same results. Unfortunately, replication is another thing the science journals consider "boring" so no one is doing it any more. You can publish your tweaked and nudged and simplified results, safe in the knowledge that no-one will ever try exactly the same thing again and find something different.

We are making huge, life-altering decisions on the basis of bad information. All because we have created a system which treats scientists like journalists; which tells them to give us what is interesting instead of what is true.

The author suggests that the solution is one that we already have; Open Access Journals. All we have to do is continue changing the scientific publishing model so it no longer has anything to do with "interest" and is more open to publishing everything, as long as the methodology is sound. Open Access journals like PLOS ONE already do this. They publish everything they receive that is methodologically sound, whether it is straightforward or messy, headline-grabbing or mind-numbingly boring.

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2, Informative) by drasha on Monday August 04 2014, @08:17PM

    by drasha (1201) on Monday August 04 2014, @08:17PM (#77335)

    Yeah, publishing in open-access journals is fine and dandy as long as you are not forced to publish in highest impacting journals as often as possible (translation - never). And here is the core of the problem - ever since you start in the academia, you are pushed to this game and you grind your teeth and participate, because you want your PhD, grants, projects, etc.

    Although I admire the open-access journal movement, the system has to change first before they become a real alternative. Meanwhile the open-access idea is exploited by big publishers that charge ridiculous sums for making some papers free to view.

  • (Score: 2) by umafuckitt on Monday August 04 2014, @08:25PM

    by umafuckitt (20) on Monday August 04 2014, @08:25PM (#77340)

    Really important results that many people care about often are replicated directly. The vast majority of studies likely aren't replicated directly, instead what tends to happen is that other people build on them. In many cases they have to replicate some or all of them original work in order to extend it. So if the original stuff is wrong, it will come out in the wash. I've seen this happen in my field, where some lines research (usually those that looked shaky in the first place) just seem to vanish over time.

    • (Score: 3, Interesting) by c0lo on Monday August 04 2014, @09:15PM

      by c0lo (156) Subscriber Badge on Monday August 04 2014, @09:15PM (#77365) Journal

      Really important results that many people care about often are replicated directly.

      Not always. E.g. for social psychology it started only recently [psmag.com] and resulted in endless [discovermagazine.com] bruhahas [wordpress.com].

      --
      https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
      • (Score: 2, Troll) by umafuckitt on Tuesday August 05 2014, @07:19AM

        by umafuckitt (20) on Tuesday August 05 2014, @07:19AM (#77504)

        But social psychology doesn't produce *important* results :) Seriously, nothing in that field really matters for anything. It's just Daily Mail science. Stuff that matters gets replicated: things like measurement of the speed of light, mass of the proton, the science behind things like MRI, and gene cloning, etc.

  • (Score: 3, Insightful) by VLM on Monday August 04 2014, @08:28PM

    by VLM (445) Subscriber Badge on Monday August 04 2014, @08:28PM (#77341)

    Another problem with positive bias is some of the ambiguous to negative results are scientifically important to progress.

    It took like 50 years to finally measure the redshift of light in earth's gravitational field because its such a PITA. If they had to operate under modern constraints, it might never, ever get done because we can only talk about good news and without standing on the shoulders of giants who none the less failed, they might have never achieved it experimentally.

    This is also stereotypically a topic that engineers use to make fun of the scientists, because so much of engineering is figuring out and optimizing failure modes, glorying in what the scientists refuse to talk about.

  • (Score: 4, Insightful) by gringer on Monday August 04 2014, @09:01PM

    by gringer (962) on Monday August 04 2014, @09:01PM (#77359)

    In the research institutes I have worked at, almost everyone says that they are working on their "Nature paper". They say it in a sarcastic way, but I figure that with so many people saying it there must be a grain of truth in there somewhere.

    I don't think picking open access journals will improve on the plethora of bad science. People are interested in positive results, and results that fit their own patterns and ideas of the world. Negative results will mostly get ignored by readers, or stacked on top of the continually growing "back burner".

    Further, funding bodies are not particularly keen to fund people who aren't producing interesting research, so it's difficult to get money for negative research. It's also harder to get money for any research at all when your publication history is largely negative in Nature (or some other more willing journal).

    --
    Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
    • (Score: 2) by opinionated_science on Monday August 04 2014, @09:17PM

      by opinionated_science (4031) on Monday August 04 2014, @09:17PM (#77366)

      yes, and this particularly true when there are mistakes in articles that get propagated for decades until someone else contradicts them.

      Or worse, work from 40 years ago gets ignored because it was not in original digital form, and hasn't been cited recently.

      Science has grown enormously in the last 100 years, but funding has not kept up.

      Remember, the big 3 - SS+Welfare (1.3 Tril),Military (820 Bil) and national debt interest (223 bil). All science agencies COMBINED approx 68 billion. That's EVERYTHING. NIH,NSF,DOD,DOE,NASA,USDA etc...

      Makes you wonder how anything gets done...

      • (Score: 2) by kaszz on Tuesday August 05 2014, @12:22AM

        by kaszz (4211) on Tuesday August 05 2014, @12:22AM (#77410) Journal

        Let's see:
        NIH - Health
        NSF - General curiosity?
        DOD - Killing "enemies"
        DOE - Get energy
        NASA - Space
        USDA - Farming

        Anyway considering what you just found. It gets done, but too slow. Global warming seems to progress faster than say fusion research, liquid fission, thorium reactor, sufficiently efficient batteries for cars-laptops, cures for high impact diseases for things like malaria spread when the heat goes up or TB, etc..

        I think we need to get back to the blue sky science model. Too many really good discoveries are found "by mistake".
         

      • (Score: 0) by Anonymous Coward on Tuesday August 05 2014, @12:44AM

        by Anonymous Coward on Tuesday August 05 2014, @12:44AM (#77418)

        the big 3 - SS

        Social Security is an insurance fund.
        Not only is it self-funding (and solvent 100.00 percent until 2033[1]), other federal programs borrow from the fund and are assessed interest.

        People who rag on Social Security demonstrate how little they know about money.

        [1] ...and that assumes that nothing will be done before it becomes 99.99 percent solvent.

        -- gewg_

    • (Score: 1, Interesting) by Anonymous Coward on Monday August 04 2014, @10:18PM

      by Anonymous Coward on Monday August 04 2014, @10:18PM (#77385)

      Scientists are very interested in negative results, but they are often not worth publishing due to a lack of perceived importance by tenure and funding committees. If a researcher works on Oxysterol-binding protein-related protein 2, then they would probably like to know if they should bother buying the Calreticulin antibody if someone has already shown that they don't interact.

  • (Score: 2) by etherscythe on Monday August 04 2014, @11:15PM

    by etherscythe (937) on Monday August 04 2014, @11:15PM (#77395) Journal

    I think this indicates a call for a new way to search through results digitally. Something like a Dewey Decimal System for scientific research. Instead of having to be on a curated front page or whatever, if you can get a hard-link number (assigned by peer review, I guess) for your paper (and/or a series of special tags) then you can publish to your field of study just by adding those filing numbers on your papers in an open-access system, and there are endless opportunities for referencing other papers. Now, not only can a much larger number of papers be accepted and possibly cross-referenced, but the requirements change for "impactfulness" (both good and bad). If magazines can go algorithmic based on trending fields of study, as long as you're doing something important you're likely to be cited by a paper making a big splash eventually. It doesn't matter that thousands of people didn't read your paper initially, as long as readers or reviewers of the paper that did go big have a relevant link to your work later, automatically.

    In other words, get the managers out of the process, and the political crap will go away too. This will refocus "traditional" publishing because there won't need to be a gatekeeper that everyone has to compete to get past; instead they will get to focus on the most interesting research to dress up and editorialize in a way the researchers themselves or a computer could not do.

    I'm not close enough to the process to really suggest any more details, but as someone who has tried to research on various subjects I would LOVE a more centralized interface to get at the greater volume of research that is not interlinked in anything like an accessible way. Like a Science Wiki To Rule Them All, with click-simple bibliography and a crawler constantly adding content from vetted open sources (and of course only degreed or otherwise accredited academics have edit privileges). We've managed to accept unified naming of plants and animals [wikipedia.org] well enough to be useful, I bet we could do the same more universally.

    --
    "Fake News: anything reported outside of my own personally chosen echo chamber"
    • (Score: 2) by kaszz on Tuesday August 05 2014, @12:25AM

      by kaszz (4211) on Tuesday August 05 2014, @12:25AM (#77411) Journal

      There might be a lot of interesting stuff found from non-accredited researchers.

  • (Score: 1, Informative) by Anonymous Coward on Tuesday August 05 2014, @05:59AM

    by Anonymous Coward on Tuesday August 05 2014, @05:59AM (#77492)
  • (Score: 0) by Anonymous Coward on Wednesday August 06 2014, @09:30AM

    by Anonymous Coward on Wednesday August 06 2014, @09:30AM (#77950)

    I haven't seen anyone else say it, so I will. In the science community, we keep saying how the religious nuts don't know what their talking about when they say they don't trust science. Well, they have good reason not to trust science. This article is one of several examples.

    With that said, those nuts are still just as nutty, but we scientists need to get our house in order. I can't call this real science anymore. It's "making money".