Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Friday September 16 2016, @09:46AM   Printer-friendly
from the go-figure! dept.

Claims that the "the science isn't settled" with regard to climate change are symptomatic of a large body of ignorance about how science works.

So what is the scientific method, and why do so many people, sometimes including those trained in science, get it so wrong?

The first thing to understand is that there is no one method in science, no one way of doing things. This is intimately connected with how we reason in general.

[...] Those who demand the science be "settled" before we take action are seeking deductive certainty where we are working inductively. And there are other sources of confusion.

One is that simple statements about cause and effect are rare since nature is complex. For example, a theory might predict that X will cause Y, but that Y will be mitigated by the presence of Z and not occur at all if Q is above a critical level. To reduce this to the simple statement "X causes Y" is naive.

Another is that even though some broad ideas may be settled, the details remain a source of lively debate. For example, that evolution has occurred is certainly settled by any rational account. But some details of how natural selection operates are still being fleshed out.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 5, Interesting) by Justin Case on Friday September 16 2016, @11:09AM

    by Justin Case (4239) on Friday September 16 2016, @11:09AM (#402714) Journal

    I read a great article some 15 years ago pointing out that scientists are people too; they have needs like continued employment, and desires like perhaps fame.

    So the scientist who is most scrupulously careful in all experiments and analysis will still be emotional when deciding what to study.

    Later I worked with some of the Mars Rover people, and I saw that it was true. They were driven by goals such as "find water" and "find life". So they launched rovers with tools specifically chosen for detecting water etc. They sent those rovers to places that "looked interesting".

    They should have selected a random sample of places to explore. But nobody wanted to be the one to spend a career going "yep, sand, just like we thought". So they're never going to really discover the truth. At best they'll find what they wanted to find.

    I call it "backwards logic": start with the conclusion you want, build up evidence in support, and ignore anything that doesn't help you win. Kinda like how my dad defends his cherished belief in young-earth creationism.

    Same flaw exists for a lot of scientific problems: cure cancer, save the world, etc. Starting with the conclusion is not science.

    Well that's my conclusion anyhow!

    Starting Score:    1  point
    Moderation   +3  
       Insightful=1, Interesting=2, Total=3
    Extra 'Interesting' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   5  
  • (Score: 3, Informative) by shrewdsheep on Friday September 16 2016, @12:44PM

    by shrewdsheep (5215) on Friday September 16 2016, @12:44PM (#402738)

    Well, I work in academia and would have some trouble stating what the scientific method would be exactly (IMHO there no generally agreed upon definition).

    That being said, I disagree with your criticism of "backward logic". What counts in my every day life circles around studies. A study starts with a research question and then goes on to design a study to answer this question in the most efficient way possible (in theory). Study design includes what to measure, how so, on which subjects and which statistical analysis to use to arrive at the answer. The research question is always (yes, always) guided by a-priori knowledge. Why is this important? A common criticism nowadays is lack of replication. One big reason is that a-priori chances for a scientific discovery are low thereby creating a lot of noise (i.e. false discoveries due to multiplicity). False discoveries cannot be replicated (with high probability). What you propose is to lower a-priori chances even further by looking even at research questions that would not normally come up (you are bit derogatory in implying emotional reasons). Apart from the cost, scientific progress would be slowed down. If the research question is well defined and the study well designed your answer will be yes or no (well not-yes) at a chosen level of statistical significance. It is true that (most) people are driven by fame and influence but the choice of research questions is competitive amongst scientists. You will be judge by your peers as to how well you chose your research questions. If the group of peers has deeper understanding of the subject matter, good research questions will bubble up by allocation of grant money and scientific promotion. Yes, that's the ideal situation and practice deviates but I believe that there exists at least some scientific integrity at work in academia. So, if you become famous beyond your academic peers by choosing the "sexy" topic that makes the news, more power to you (as long as you have followed the scientific method :-)

  • (Score: 4, Insightful) by Anonymous Coward on Friday September 16 2016, @12:56PM

    by Anonymous Coward on Friday September 16 2016, @12:56PM (#402742)

    Later I worked with some of the Mars Rover people, and I saw that it was true. They were driven by goals such as "find water" and "find life". So they launched rovers with tools specifically chosen for detecting water etc. They sent those rovers to places that "looked interesting".

    Of course. Imagine they sent the Mars Rover to a place where they just expected sand, and found just sand. What would happen? They probably will have a hard time arguing for funding ("Oh, you want to test whether there's really sand there? And you say, it's very likely? So what is the point of wasting money on this?"). If they find what they expect, they probably won't get it published ("oh, you found sand on Mars? Sorry, that's not interesting. Come back when you find something new.") In short, they'll risk their career.

    I've once saw in TV something about the discovery of pale blue galaxies in areas of the sky where previously there had been thought to be nothing. They interviewed the astronomers, who told that they didn't tell the truth about what they planned to do, because that way they never would have been given the observation time. Instead they made up another planned observation, which got them the observation time. It wasn't said what they would have done if they indeed had found nothing, but I guess they would have claimed that they made a mistake, causing the telescope point to the wrong place in the sky.

    So they're never going to really discover the truth. At best they'll find what they wanted to find.

    They are going to discover the truth about their questions, namely whether on those places they selected there is water or not. It's not as if they'd find water just because they want to find water. If they find water, they find it because it is there. And if they don't find water, it's also significant.

    Such a selection doesn't make their work the slightest bit less worth; it just means that other interesting stuff might remain undiscovered.

    Note that already the selection of your subject is a selection what you are going to look for. Would you say that the discoveries at CERN are less valuable due to the fact that the scientists there actively decided to be particle physicists instead of e.g. organic chemists, and therefore ended up searching for fundamental particles instead of new organic molecules?

    I call it "backwards logic": start with the conclusion you want, build up evidence in support, and ignore anything that doesn't help you win.

    That's not what you described before. Just because they included instruments to detect water, they won't automatically find water. The detectors will only show water if there actually is water. Now if they ignored actual sensor readings that show no water where they expected some, that would be problematic. But there's no indication that they do.

    Kinda like how my dad defends his cherished belief in young-earth creationism.

    No. Did your father device any experiments to test young-earth creationism that could fail to demonstrate young-earth creationism? Because those Mars Rover people definitely could fail to find water. And if they didn't find water that would be significant because they looked at the places where water is most likely to be found, and because the Mars Rover had instruments looking for water. "I sent a rover looking for sand to Sahara, and didn't find water, therefore there's no water on Earth" wxould not be convincing. "I sent water detectors to the middle of where we thought the Atlantic Ocean should be, and found no water there" on the other hand would be a very significant finding.

    Same flaw exists for a lot of scientific problems: cure cancer, save the world, etc.

    Neither cure cancer, nor save the world is a scientific problem, although both certainly involve solving certain scientific problems. Seems you do not even know what science is.

    Starting with the conclusion is not science.

    And it is not what those Mars Rover people did. They didn't say "there's water on Mars, so let's find it." They said "Let's look if there is water on Mars, because if there is, that would be big." Note the "if". They didn't start with the conclusion, they started with the question. The question was: "Is there water on Mars?" They didn't know whether the conclusion would be "there is water on Mars" or "there is no water on Mars".

    • (Score: 1) by kanweg on Friday September 16 2016, @03:22PM

      by kanweg (4737) on Friday September 16 2016, @03:22PM (#402828)

      Very insigthful

      Bert

  • (Score: 4, Insightful) by takyon on Friday September 16 2016, @02:35PM

    by takyon (881) <takyonNO@SPAMsoylentnews.org> on Friday September 16 2016, @02:35PM (#402802) Journal

    I'll tell you what. Increase NASA's budget to allow a new rover to land on the planet every month, or to put a permanent manned base on the planet, and then you can study all the boring sand piles you want. Until then, priorities will be based on limited resources and lofty goals. Or maybe you believe we should look for life after we have finished contaminating or terraforming Mars?

    --
    [SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
  • (Score: 0) by Anonymous Coward on Friday September 16 2016, @04:40PM

    by Anonymous Coward on Friday September 16 2016, @04:40PM (#402860)

    I call it "backwards logic": start with the conclusion you want, build up evidence in support, and ignore anything that doesn't help you win. Kinda like how my dad defends his cherished belief in young-earth creationism.

    While I think Karl Popper [wikipedia.org] is well worth studying, but I suspect the article that will best encapsulates the differences between your dad and those scientists is this: http://www.csicop.org/si/show/field_guide_to_critical_thinking [csicop.org]

    In particular, the Mars Rover team had a falsifiable hypothesis, they comprehensively studied where they though water would be (from the evidence they had), then they tested it. The hypothesis "Water exists in location X" was a very testable statement. Sure, they hoped to demonstrate water did exist, but it was falsifiable. Does your dad have a falsifiable hypothesis, comprehensively studied the existing knowledge and then taken the time to demonstrate it with sufficiency? Has he tried to develop a method for dating the earth?

    Starting with a "conclusion" (a hypothesis, like "Water exists in X location") is certainly science in that you are able to disprove it. It appears to me the Mars Rover people did just that, tried to disprove their theory, by intentionally going to the most likely place to have water and see if there was any.

    In my opinion, falsifiability is the most important aspect of science today, but some of the other pieces James Lett mentions are valuable too.

    - JCD

  • (Score: 3, Interesting) by bzipitidoo on Friday September 16 2016, @05:12PM

    by bzipitidoo (4388) on Friday September 16 2016, @05:12PM (#402870) Journal

    Starting with a conclusion is okay, if often not the best approach. Doing so makes it more likely to be wrong and that you are wasting your time, especially if the hypothesized conclusion is not based on reason. But it's not inherently wrong to begin with a conclusion, then try to find out if it's right or wrong.

    A big problem is some people are unwilling to discard a conclusion even when it has been shown to be very wrong. Instead, they cherry pick data, or just flat make up evidence. Data that contradicts the desired conclusion is dismissed, smeared as being unreliable, or even as made up, same as they'd make up favorable data. It could be that some suspect everyone of doing it, see conspiracies everywhere, because they would do it themselves, and can't understand why anyone wouldn't do it. They don't see why they shouldn't do it, and get stuck in a mental swamp in which they have no idea which information is correct and which is wrong, and no way to find out. They don't appreciate that science is our best tool for getting out of that swamp, but that it doesn't work too well if it's held in contempt and deliberately misused.

    • (Score: 3, Insightful) by tfried on Friday September 16 2016, @09:10PM

      by tfried (5534) on Friday September 16 2016, @09:10PM (#402944)

      A big problem is some people are unwilling to discard a conclusion even when it has been shown to be very wrong. Instead, they cherry pick data, or just flat make up evidence.

      But see, that cherry picking shows that your theory is still alive and kicking!

      To be quite clear, I do think, GP is bringing this because he disagrees with AGW, and disagreeing with AGW becomes increasingly hard without explaining how a vast majority of scientists can be wrong. He himself is, very much, starting from his conclusion, and defending it uphill, and beyond reason.

      However, that does not mean that his line of argument is all wrong. Having worked in science for a bit, I actually think it is quite valid, and something to be wary of. Personally, I have been in the situation of starting from a conclusion. Essentially, our research question / conclusion was of the kind of theory A is better than theory B. And boy, theory B is boring! So we invested a significant amount of resources into our study. Even setting up the methods for my study was a huge effort (at the scale I was working at, at least). And then after about two years (the study needed a delay period, too) we had a huge amount of data, and finally, we could set out to answer our key research question, and ... it kind of sort of was not a clear cut case. So, being the stats guy in the project, I spent a long time with that data, and tell you what, using the right methods, the right subgroups, or the right kind of exclusions, I found enough corners where our beloved theory A was doing great. But some pedantic drive kept me going on until I had convinced myself, that boring theory B simply did a much better job at capturing the overall results. That was a long and painful process, and it was enabled by being young, having a lot of time on hands, and low pressure from the seniors. Not sure I would have reached the same conclusion with a bit more nudging from my fellow research team members.

      So far, so good, science won (I hope), but the real staggering realizations started to set in when we were publishing our findings, and going through peer review. Peer review was a bitch. It was giving us quite some headaches, some over ridiculous misunderstandings, many over hair-splitting on words, some over very real substantial problems. But none were about the sensitive spots of my analysis. Today, I am quite convinced, if we had simply run with theory A, we would never have been caught in peer review, and the world would not have known any better.

      That was a pretty confined area of research, not too many people working on it world-wide (and that does reflect the lack of importance). I'm sure, more eyes do improve the situation. But do not, ever, underestimate the means, and the motivation of small groups of scientists to nudge results into the direction that they honestly (but mistakenly) believe to be right.