Stories
Slash Boxes
Comments

SoylentNews is people

posted by cmn32480 on Saturday November 14 2015, @06:04PM   Printer-friendly
from the they-will-try-craps-next dept.

The field of psychology has recently been embarrassed by failed attempts to repeat the results of classic textbook experiments, and a mounting realization that many papers are the result of commonly accepted statistical shenanigans rather than careful attempts to test hypotheses.

Now Ed Yong writes at The Atlantic that Anna Dreber at the Stockholm School of Economics has created a stock market for scientific publications, where psychologists bet on published studies based on how reproducible they deemed the findings. Based on Robin Hanson's classic paper "Could Gambling Save Science," that proposed a market-based alternative to peer review called "idea futures," the market would allow scientists to formally "stake their reputation", and offer clear incentives to be careful and honest while contributing to a visible, self-consistent consensus on controversial (or routine) scientific questions.

Here's how it works. Each of 92 participants received $100 for buying or selling stocks on 41 studies that were in the process of being replicated. At the start of the trading window, each stock cost $0.50. If the study replicated successfully, they would get $1. If it didn't, they'd get nothing. As time went by, the market prices for the studies rose and fell depending on how much the traders bought or sold. The participants tried to maximize their profits by betting on studies they thought would pan out, and they could see the collective decisions of their peers in real time. The final price of the stocks, at the end of two-week experiment, reflected the probability that each study would be successfully replicated, as determined by the collective actions of the traders. In the end, the markets correctly predicted the outcomes of 71 percent of the replications—a statistically significant, if not mind-blowing score.

"It blew us all away," says Dreber. "There is some wisdom of crowds; people have some intuition about which results are true and which are not," adds Dreber. "Which makes me wonder: What's going on with peer review? If people know which results are really not likely to be real, why are they allowing them to be published?"


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by jmorris on Sunday November 15 2015, @05:51AM

    by jmorris (4844) on Sunday November 15 2015, @05:51AM (#263571)

    Time doesn't permit taking all of that on, but will shoot a few holes in the key bits of bogosity.

    So long as the world was wide, and there was room for everyone...

    Doesn't matter. The Industrial Revolution was started in England at a time when a population increase had left them far more people than their social order knew what to do with. The rise of the factories was the solution that worked and changed the world. With the right mindset there is opportunity in just about any situation.

    The pinch will be very ugly if our room shrinks thanks to Climate Change before we... [Malthusian blah blah deleted]

    Look at the article we are commenting on again. Climate science has made no valid predictions. All of the models ran far enough back to be judged against actual events, i.e. twenty or so years ago, are now outside their error bars. The precise scientific term for that is WRONG. Now we can't even make new predictions because the very datasets are 'adjusted' and in many cases unavailable to the public for outside inspection. The models themselves are closed software and unavailable for inspection. This is not how Science works.

    But the big tell is the reaction of Climate Science to the word 'Denier.' Were they scientists they would have denounced the political hacks trying to outlaw dissent. They didn't, telling those with eyes to see everything needed.

    At this point the continued belief in AGW is harder to sustain than Christians believing in Transubstantiation or the Mullahs in Iran waiting for the dude to climb out outta that well. Bluntly, it is nuts.

    They seem to really want war, keep maligning the Muslims as terrorists eager to war upon us. Exterminating the entire Islamic faith and all its followers, without destroying the West, if even remotely possible, would open up a lot of room, yes, but at what cost?

    Yea, when people are crucifying people, hacking off heads and posting the proudly posting the video to social media we certainly should not believe they mean us harm. Makes perfect sense in fact to give them atom bombs!

    NOT.

    It probably isn't required to exterminate the entire population of Muslims. What is required is two things. One is to kill their dream of a restored Caliphate exerting a world dominating influence. That means destroying ISIS in such a way they can plainly see that ISIS never had a chance once we seriously decided to put an end to them. It means over the top violence followed up by ruthlessly hunting down the survivors like hasn't been seen since the Nazihunters after WWII. Second is to convince them in such a way that there remains zero doubt of our resolve that they have precisely two options. Stop their policy of quietly encouraging their nutters to wage jihad or we will invoke Ann Coulter's solution. That they can, by renouncing their dreams of conquest, live with us in peace as equals or they can be ended as a people, as a religion and as a threat forever. That Mecca can be reduced to radioactive rubble, every mosque on Islamic soil razed, Islam made an outlaw political idea in exactly the way Nazism was. In short, we make them an offer they don't refuse.

    Why do we want to drill for more oil?

    Because we need the stuff and the odds are that we are going to continue needing the stuff long after thee and me are worm chow. Someday we might run much of our world on fusion power and that will truly be a day worth living to see dawn. But nobody even has a proposal to replace jet fuel for example.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 2) by bzipitidoo on Tuesday November 17 2015, @03:45AM

    by bzipitidoo (4388) on Tuesday November 17 2015, @03:45AM (#264199) Journal

    Hmph, you are as blind as the worst of the Climate Change deniers. Demanding that climate models be tested by shoveling more and longer periods of data into them until they break down is asking the wrong thing of them, as well as asking too much. No model can perfectly replicate reality. The whole point of a model is to simplify reality the minimum necessary, to work with the always limited and incomplete data available, so that the tools we have can be applied, and results of practical value can be generated in a reasonable amount of time. Asking for more is akin to saying photography is useless and photos are WRONG because they never have enough information to enable a complete reproduction of a scene, right down to the quantum level, allowing the recreation of any dead people who happen to be in it, and so photos and videos should not be admissible evidence in court. You make an argument of that sort, and make it too brief so you can get to the fun part quicker, bashing away at AGW as if it was now beyond debate that scientists have it all wrong and are a bunch of frauds and/or dummies.

    You are being blithe when you say "belief in AGW", and say that such is no different than belief in Transubstantiation and other faith based beliefs. Science is not about belief. Science takes NOTHING on faith. It is all about facts, evidence, and logical deductions, inferences, and conclusions. We operate on the assumption that observable reality is reality, pending evidence to the contrary. It may be that no such evidence will ever be found. We will probably want to improve our models of reality as we learn more-- General Relativity is very good, but it isn't the ToE. I don't "believe" in AGW. I accept that there is sufficient evidence and good enough modeling to conclude that AGW is real, and a problem.

    • (Score: 2) by jmorris on Wednesday November 18 2015, @03:32AM

      by jmorris (4844) on Wednesday November 18 2015, @03:32AM (#264680)

      I guess I failed in communicating. I'm saying the Climate Change Industry is not science. They do not behave as scientists should, they do not follow the scientific method and therefore what comes out of Climatology is not science. It is the core of a new age green religion if one is being generous and knowing fraud if one isn't so inclined.

      You confirm my observation. I took the prediction of the warmers in the way they were published, as predictions with error bars and a confidence interval; i.e. as if they were doing science. The predicted results did not appear, therefore the models are useless. Perhaps after a careful analysis of why they failed they can try again.... but again it will take some time for the results to be tested and in the interval we should hold their past failure against them, especially since they are making extraordinary claims and demanding an extraordinary, civilization reorganizing response to their unproven theory. I'm the one behaving according to the teachings of the Scientific Method, Rational thought and the general Enlightenment ideas.

      Contrast to your pathetic assertions. I'm being unreasonable, I'm "Demanding that climate models be tested by shoveling more and longer periods of data into them until they break down is asking the wrong thing of them." No, I'm comparing their predictions to reality in exactly the way the authors of the models claimed they should be evaluated in the time frames they claimed confidence predicting results for. When reality differed they changed the goalposts and you now come here and have the brass balls to tell me I'm the one making unreasonable demands. If the models lacked actual predictive power they should have said so. Too bad the models were the only argument the warmers had. Look in a dictionary, what is the word for a firmly held opinion not supported by physical evidence? Your sad devotion to that ancient religion doesn't inspire confidence in your reasoning ability.

      I don't "believe" in AGW. I accept that there is sufficient evidence and good enough modeling to conclude that AGW is real, and a problem.

      Really. Show me a model that isn't outside the error bars. That is how science works. Inside the error bars is good, outside is bad. What other evidence is there? Average global temps have been much higher than they are currently, they have also been a lot lower. Before man discovered fire, before the Industrial Revolution. Further attempts to even study the issue would now be pointless. The historical record has been destroyed (See ClimateGate) nobody working in the area is trustworthy at this point since they all behave more as politicians and priests compared to scientists. None outside the Climatology (and face it, the definition of Climatology is now the study of AGW in a perfect circular reasoning fallacy) industry without a decade of time to devote to a careful study of the literature and acquiring several advanced degrees can possibly sort out the lies from any science lurking in the morass.

      Maybe AGW is correct and we will all suffer greatly as a result of people like me saying "A Pox on your house!." Maybe. That is going to be on your team though for lying. We used to teach tales like the Boy Who Cried Wolf for a reason you know.

      • (Score: 2) by bzipitidoo on Wednesday November 18 2015, @06:44AM

        by bzipitidoo (4388) on Wednesday November 18 2015, @06:44AM (#264733) Journal

        > Further attempts to even study the issue would now be pointless. The historical record has been destroyed

        Ridiculous! The record has not been destroyed. Have you thought about what it would take to destroy all records of past weather? There are many scattered all over the world, with copies. Then there is all the evidence from the environment, things like glacial cores, tree rings, lake bottom sediments, and so on. If records had indeed been all lost or destroyed, we can recreate the data from those sources.