SoylentNews is people
SoylentNews
The scientific paper—the actual form of it—was one of the enabling inventions of modernity. Before it was developed in the 1600s, results were communicated privately in letters, ephemerally in lectures, or all at once in books. There was no public forum for incremental advances. By making room for reports of single experiments or minor technical advances, journals made the chaos of science accretive. Scientists from that point forward became like the social insects: They made their progress steadily, as a buzzing mass.
The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little "computation" contributed to the results was done by hand and could be verified in the same way.
The more sophisticated science becomes, the harder it is to communicate results. Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it's contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you've actually discovered, clearly enough that someone else can discover it for themselves.
Source: The Scientific Paper Is Obsolete
(Score: 5, Insightful) by Justin Case on Monday April 09 2018, @01:57PM (11 children)
It's bad enough that the code is usually of very amateur quality, and sometimes not available for replication.
But the actual science? Why is that paywalled? In most cases the taxpayers already paid for the (often wasted) research. Then the researcher has to pay some journal to print their results. Then others have to pay to read it.
It is well past time to let the journals go the way of newspapers: obsolete for decades now. Publish all of your tool chain and data on the web. Make it free for everyone.
"Oh but peer review..." yup, still need that. But, well, some peers are more equal than others. Both before and after initial release, many peers should get to rate the work. In an arrangement similar to web of trust, reviewers could be rated to arrive at a net credibility factor.
This needs to happen folks! Science is currently held hostage with most of the work locked away for the inside few.
(Score: 4, Insightful) by PiMuNu on Monday April 09 2018, @02:05PM (5 children)
> Why is that paywalled?
Almost no science done in UK, funded by research councils, is paywalled. I cannot comment on other nations.
> Publish all of your tool chain and data on the web
Fine, but bear in mind that it does take years to understand the tool chain for any reasonably sophisticated data analysis. So what you suggest doesn't fix anything, while adding some bureaucracy on the researchers.
(Score: 3, Interesting) by Justin Case on Monday April 09 2018, @02:17PM
I guess I was thinking something along the lines of a makefile, with dependencies. At least you could recreate the results even if you don't immediately understand them.
Then, potentially, gather some new observations or measurements and run it again. Different outcome? Why?
(Score: 3, Informative) by TheRaven on Monday April 09 2018, @04:11PM
sudo mod me up
(Score: 2) by Wootery on Tuesday April 10 2018, @09:06AM (2 children)
It's a good thing to make it easier to find software bugs that impact published results, no? It also makes it easier to reproduce results, by enabling other researchers to re-implement as much or as little as they like. You really think that's without any value?
(Score: 2) by PiMuNu on Tuesday April 10 2018, @10:00AM (1 child)
> You really think that's without any value?
I guess the work I do is specific to the bespoke instrumentation I use. There is little use, beyond the general algorithms (which are described in a paper somewhere), to anyone to take that code elsewhere.
(Score: 2) by Wootery on Tuesday April 10 2018, @11:20AM
There's value in providing real working code, both in terms of quality-assurance for the publication (more eyeballs), and in terms of providing direct value to others. Your point is that in your case, the direct value is diminished as the code is tied to one platform, but the first advantage remains.
To fail to publish the source-code is to fail to properly document the experiment.
Implementing pseudocode correctly is not as easy as we might like to think - annoyingly I'm now unable to find the article, but I recall reading that most programmers introduce bugs even when given a relatively simple pseudocode->implementation exercise.
(Score: 0) by Anonymous Coward on Monday April 09 2018, @02:23PM (2 children)
Scientists have a publishing key and get reputation as other scientists cryptographically sign off on their key or directly on their work as applicable. If someone turns out to be a fraud, you revoke signatures which in turn will reduce their credibility as well as the chain of trust for others who they signed.
Collectively this will act like Slashdot/SoylentNews only for scientists reputations. Collusion and Cabals will still be possible, but individual readers will have a far easier time filtering for that based on their own parameters than they will trying to select 'trustworthy' journals in the modern age, most of whom will publish anything for the right amount of money.
(Score: 3, Informative) by JoeMerchant on Monday April 09 2018, @02:26PM
Scientists are barely more likely to understand and use hands-on asymmetric cryptography than the rest of the world.
It has been practical since the 1980s to sign and encrypt everything from e-mail, to twitter feeds, it remains sub-fringe in adoption.
🌻🌻 [google.com]
(Score: 3, Informative) by PiMuNu on Monday April 09 2018, @05:26PM
This is fixing a non-problem, i.e. trustworthiness of a paper. It is usually obvious to spot when a paper is balls, by reading the paper and the analysis done. E.g. do the authors quote a systematic error and state how that error was found? No - then it is junk science.
(Score: 2) by JoeMerchant on Monday April 09 2018, @02:24PM
The process is centuries old, it's amazing that there aren't more people with their hand in the till.
Look at real-estate transfer as a reference. Be amazed that there's not a submission fee, a fee for peer review, a fee for the software enabling both of those, taxes, legal reviews, etc.
I agree with the sentiment: it's time for an overhaul, a "free science" movement if you like - one which requires replication before people take it seriously.
Oh, and we need a muzzle on the press as they run around yapping about the "latest findings" instead of the "most recently confirmed findings."
🌻🌻 [google.com]
(Score: 3, Funny) by DannyB on Monday April 09 2018, @04:03PM
Its a PAY-per, not a paper.
The lower I set my standards the more accomplishments I have.