Richard Horton writes that a recent symposium on the reproducibility and reliability of biomedical research discussed one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with science (PDF), one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. According to Horton, editor-in-chief of The Lancet, a United Kingdom-based medical journal, the apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world or retrofit hypotheses to fit their data.
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative. Tony Weidberg says that the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication following several high-profile errors,. By filtering results through independent working groups, physicists are encouraged to criticize. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. "The good news is that science is beginning to take some of its worst failings very seriously," says Horton. "The bad news is that nobody is ready to take the first step to clean up the system."
[Editor's Comment: Original Submission]
(Score: 4, Insightful) by takyon on Tuesday May 26 2015, @06:36PM
Most of the summary talks about medical science, which is complicated and focused on safety. There are incentives to mislead on safety and efficacy, and possible steps [soylentnews.org] to remedy the problems.
With physics, the skepticism may be too high. Almost nobody (in physics) believed the faster-than-light neutrino result, and it was checked out and debunked in due course. Almost nobody believed that emdrive can work (for good reason, too much secrecy), but the NASA employees apparently finding positive results [soylentnews.org] are using their lunch money to test it. Insert your cold fusion speculation here.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 1, Insightful) by Anonymous Coward on Tuesday May 26 2015, @07:18PM
Yep, and some "science" aren't even science - usually ones with adjectives.
And such usage of the term "science" to lump all the varied disciplines is a BIG part of the problem.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @07:43PM
awesome points.
i wish it WERE all a good old big conspiracy.
i have a good perfect way to color it: the peoples in power are aware of the problems (and then some) and can't seem to reconcile the "prettiness" or "handsomeness" of the mating of the powerhouses (think king of spain offspring marries london girl island to end the war something) thus leading to our situation.
one would think that a test tube baby with a surrogate mother would solve all the problems (who gets to control which technology creation industry) but this would not guarantee a stable tachyon emitter-stream from the future -aka- love.
thus on paper (or testtube) the mating of the two powerhouses would solve the problem temporarily (like now) but leave the future state ... un-loved.
wat?
nobody with future power should grow up unloved!
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @08:07PM
i don't understand wat u mean with tachyon? there's no detector for it!
replying to my own post then i would suggest the example of fukushima!
growing up (consuming matter and influencing the environment) and living in a stream emitted by hollywood which conquered japan and gave them "wrong" tachyon emitters () one could argue that the atoms being released now influence other atoms but when in the future the influenced atoms need to confirm their path they are meet by a shameful nothingness.
that is to say that the observer (made up of non fukushima matter) regards other observers as historyless because they cannot trace their source because their dead have turn into a void.
?
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @08:52PM
Did somebody forget to take his meds today?
(Score: 2) by PartTimeZombie on Wednesday May 27 2015, @12:52AM
Why can't I mod this +1 Incomprehensible?
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @03:18PM
This line reminded me of Gilgamesh:
"But Mother! If you will not suckle me, how can I thrive!"
(Score: 0) by Placenta on Wednesday May 27 2015, @12:41AM
Is computer science considered a science?
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @11:46PM
The social 'sciences' are perhaps the worst when it comes to pseudoscience and bad science. Subjectivity, bias, lack of scientific rigor, and no consideration of alternative possibilities than the ones the researchers raised all seem to be common issues.
(Score: 4, Insightful) by kaszz on Tuesday May 26 2015, @06:39PM
Could it perhaps be related to that research people are pushed by department heads to publish or be crossed over in the paycheck list? Or that a publication track record may be an advantage when seeking funds?
Perhaps those "blue sky" project style management had something going? and that curiosity driven research has been lost since and become a industry printing publications?
Just some thoughts..
(Score: 4, Insightful) by VLM on Tuesday May 26 2015, @06:56PM
Which fits in as a result of the "graduate ten times as many PHDs are there are PHD level job openings". If you don't want the results of brutal cutthroat competition, then don't make a competitive situation. Academia really is messed up and I'm glad I didn't go down that path.
(Score: 3, Insightful) by Kell on Wednesday May 27 2015, @01:49AM
We here in academia are pushed to graduate more students. It's not that we want to take on inferior or subpar students and train them for a job they'll never get (and aren't capable of, anyway), but rather it's the case that more students bring more income into the university and god help you if you try to reject or flunk a full fee paying student. I've been straight up told that I need more PhD students by the time I get to tenure review.
Scientists ask questions. Engineers solve problems.
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @08:38AM
Maybe a better model would be to pay universities only if you failed. That would have a twofold effect: Universities would have to balance their failure rates (too little, and they don't generate income; too much, and they won't get any students), thus making passing the exam a meaningful measure against. On the other hand, it would give an extra incentive to the students to be good (because being good, they could save money), and to those who wouldn't have a chance pass anyway, to stay away (and thus improve the situation for those who do have a chance).
(Score: 4, Insightful) by Thexalon on Tuesday May 26 2015, @08:14PM
More to the point, "publish or perish" is all about attempting to quantify something that really isn't quantifiable, so that it can be turned into performance metrics that can then be used as an excuse to cut people and keep the salaries of scientists from getting too expensive from the point of view of college administrators (the administrators' salaries, of course, need to be increased dramatically).
As soon as you create a performance metric, smart people will find a way to fake it, guaranteed.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @12:35PM
As soon as you create a performance metric, smart people will find a way to fake it, guaranteed.
Indeed and sometimes it really bites you https://en.wikipedia.org/wiki/Cobra_effect [wikipedia.org]
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @02:08PM
More to the point, "publish or perish" is all about attempting to quantify something that really isn't quantifiable, so that it can be turned into performance metrics that can then be used as an excuse to cut people and keep the salaries of scientists from getting too expensive from the point of view of college administrators (the administrators' salaries, of course, need to be increased dramatically).
As soon as you create a performance metric, smart people will find a way to fake it, guaranteed.
Without a metric, how would you determine which of the thousands of scientists should our limited resources be funneled to? If you agree to use a metric, which metric would be better than the admittedly terrible publish-or-perish model?
Put it this way. A random stranger comes up to you and says, "Give me money to study exothermaldynamics. No, I won't promise you any results or anything else. You can trust me I'm using this money well, I'm a scientist."
Maybe if you know this person personally you would trust them. Maybe if you were proficient enough in exothermaldynamics to be confident to determine a legitimate question from a pure money grab you would give them some money. But neither of those scale at all. So what is the better model we should be using?
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @03:25PM
Well, in the 1600-1700s science was done by people with wealthy and powerful patrons. The scientists would come over for dinner and entertain the guests. If the science was not interesting enough , the scientists would also get into feuds with each other which would still entertain and contribute to a kind of "science race" amongst the patrons.
I don't necessarily think that worked better, but it is one option.
(Score: 2) by aristarchus on Wednesday May 27 2015, @06:49PM
The scientists would come over for dinner and entertain the guests.
Yes, and it's all harmless fun, until somebody bursts their bladder! https://en.wikipedia.org/wiki/Tycho_Brahe [wikipedia.org]
(Score: 3, Insightful) by Thexalon on Wednesday May 27 2015, @03:31PM
Nobody is suggesting that. There's a fairly good way of vetting somebody who's trying to get grant money:
1. First, you check the random strangers' educational qualifications. For example, an exothermaldynamicist would be expected to either have a doctorate or be working on their dissertation. You would want them to have done well in their coursework, which you can get from their college transcripts.
2. Second, you check their previous work, if any. If they're somebody new working on their dissertation or something like that, then you'll understand them not having much of a record but you'll probably be a bit stingier with the grants.
3. Third, you ask the acknowledged experts of exothermaldynamics to see what they think of the proposal and the person who's proposing it. Answers like "total crackpot!" or "hmm, there might be something to that, it would be worth a try" should give you some good guidance.
4. Fourth, you get opinions on the random stranger from everybody who knows or has worked with them, particularly academic advisors and professors and such.
The fact that there are far more qualified scientists out there than there is funding for them is truly shameful, because it means that we're intentionally holding back the rate of scientific discovery due to a fear of losing small green pieces of paper.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.
(Score: 3, Insightful) by dcollins on Tuesday May 26 2015, @08:27PM
It's not just the department heads. As non-academic business people have taken over universities in the last few decades, they're the ones that keep arbitrarily tightening the screws in terms of increased number of publications in a shortened time frame for permanent employment. That stuff has basically been taken out of the hands of the academics at this point.
Immediate Example 1: This morning I received this month's "AFT On Campus" magazine with a cover story on "The pernicious effects of corporate influence" which is basically the same issue.
Immediate Example 2: Five minutes ago I got an email from faculty setting up a "workload concerns" committee to pushback on administration where I teach on the issue (for which I would not have high hopes).
http://www.aft.org/periodical/aft-campus [aft.org]
(Score: 1) by Placenta on Tuesday May 26 2015, @10:19PM
How have "non-academic business people" taken over colleges when college administration tends to be made of people with 30+ years in academia? These are people who entered college at 18, and never left. They have never worked in industry or business. This is even usually the case for those who studied and taught business!
(Score: 2, Interesting) by gamechanger on Tuesday May 26 2015, @11:43PM
The appointment of people out of industry is increasingly happening in senior executive roles in Australian universities though not in large numbers yet.
When academics are appointed to these roles, they tend to be people who have shifted to the "administrator" track from the researcher track much
earlier in their career than they did 30 or more years ago. Vice Chancellors used to be people who had had distinguished research careers and who took on the role
towards the end of their career. Now they are younger ambitious go-getters dazzled by the seven figure salary packages.
The pressure from department heads to publish results from pressure in the entire system to be high (and yet higher) in the university and discipline rankings
from Shanghai Jiao Tong, or Times, or QS or ... These in turn lead to more paying customers - international students. "Branding" is the mot du jour in
universities in Australia. This is what is driving science in Australia and I suspect elsewhere, but because of the enormous pressure for international students (and income),
I suspect the effect here is greater! And then you wonder why we are seeing bad science published?
(Score: 2) by Kell on Wednesday May 27 2015, @01:53AM
Hello fellow Aussie academic! I agree with all the points you've made - I've seen it too, myself. I seriously think about leaving academia every 6 months or so. If my field wasn't so small domestically in Australia, I'd probably have left long ago... As it is, I'm 9 months away from submitting for continuing. I can't even think of any other jobs that have a 5 year probationary period.
Scientists ask questions. Engineers solve problems.
(Score: 2) by dcollins on Wednesday May 27 2015, @02:42AM
Example: CUNY Board of Trustees -- 15 people who actually wield power over ~20 colleges in CUNY, the largest urban university in the nation. The first guy, Chairman Schmidt, you could call a lifelong academic -- albeit chairman of a global private school network (formerly Yale). From what I can see the rest are all non-academic industry people.
http://www.cuny.edu/about/trustees/board.html [cuny.edu]
Case 1: In the last few years the trustees took control of the CUNY curriculum and demanded a reduction in credit-hours for all degrees (so as to show increased graduation rates). After much shenanigans, a no-confidence vote was taken by faculty, which came in 92% "no confidence". The Board of Trustees basically said "we don't care" and forced its implementation anyway.
Case 2: Last year Chairman Schmidt, who is on Lynn Cheney's American Council of Trustees and Alumni (ACTA), headed a report recommending that even more functions be taken away from faculty and given to trustees selected from the business community. From the report: "faculty cannot be the last and determining voice regarding academic value, academic quality, and academic strategy... it is lay trustees – with considerable life and community experience – who can bring the big picture to bear in determining what graduates will need...".
http://www.psc-cuny.org/clarion/december-2014/benno-schmidt-backs-report-calling-trustees-reduce-faculty-authority [psc-cuny.org]
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @04:15AM
It's not just that, but the people who attend college and university are increasingly doing so to get goods jobs or make money. They don't care much about actual education or having an academic understanding of the universe. So, the business people start trying to get more people into colleges who have these motivations so they can take their money, and universities end up becoming half-assed trade schools to suit these fools.
(Score: 3, Insightful) by looorg on Tuesday May 26 2015, @08:31PM
Indeed. As long as my entire career, future and paycheck is tied to some kind of Publish or Perish-metric then that is what I and others are going to do. The only viable method is the shotgun-method; you get your name in on as many things as possible. Some will be boring duds that go straight to the archive and some will be okay. Some might even turn out to be really good. You could spend 10-20-30 years on your "THE ONE"-project but the personal risk vs reward for that one is really low, and possibly not even allowed anymore. I guess if my position was tenured and I was in my sixties and just didn't give a fuck anymore then sure I could do that. But until that time I'm doing what is going to possibly take me there. Is that sad? Yes. Do I wish I could do other things? Yes. Have I cancelled or reworked projects (sexed 'em up)? Sure. Why? Because when was the last time someone read a report where your entire model, data and project failed? Not going to happen. If you get grant money you succeed. If you can't succeed you massage the fuck out of that data until it looks like you did, or at least vaguely so.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @08:51PM
Well I sure as hell don't massage my data.
But I agree that the incentives are all wrong. Nothing worse than feeling like there is pressure for a certain result, or that the story should be simple.
These things are changing though.
(Score: 2) by looorg on Tuesday May 26 2015, @09:00PM
Well I sure as hell don't massage my data.
Right. I wasn't as clear as I should have been. I'll try and rectify that now. I don't change the actual data (cause that would be dishonest and fraud). You change the parameters and other things such as the perspective and/or put limits on your question(s) and query. You don't make up data.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @10:16PM
Sorry. Didn't mean to impugn of your integrity. I definitely feel pressure for things to work out certain ways sometimes. I've been pretty luck so far in that my field is relatively understudied, so even null results are somewhat novel, or have examined population differences with actually large effect sizes consistent with suspected brain damage.
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @10:01PM
Only publishing positive results has the same effect on the field as massaging the data. The journals are part of the problem.
(Score: 2) by cyrano on Tuesday May 26 2015, @07:01PM
So scientist are human after all.
It's no surprise given the effort big pharma and others have put in towards corruption of everything, except profit, of course.
What about politicians, lawyers, or, more closely related, psychologists?
The quieter you become, the more you are able to hear. - Kali [kali.org]
(Score: 5, Informative) by gringer on Tuesday May 26 2015, @07:33PM
Have a look at this long article to see a few examples of how difficult it is to fix this problem:
http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ [slatestarcodex.com]
My favourite is section IV:
The idea was to plan an experiment together, with both of them agreeing on every single tiny detail. They would then go to a laboratory and set it up, again both keeping close eyes on one another. Finally, they would conduct the experiment in a series of different batches. Half the batches (randomly assigned) would be conducted by Dr. Schlitz, the other half by Dr. Wiseman. Because the two authors had very carefully standardized the setting, apparatus and procedure beforehand, “conducted by” pretty much just meant greeting the participants, giving the experimental instructions, and doing the staring.
The results? Schlitz’s trials found strong evidence of psychic powers, Wiseman’s trials found no evidence whatsoever.
Take a second to reflect on how this makes no sense. Two experimenters in the same laboratory, using the same apparatus, having no contact with the subjects except to introduce themselves and flip a few switches – and whether one or the other was there that day completely altered the result. For a good time, watch the gymnastics they have to do to in the paper to make this sound sufficiently sensical to even get published. This is the only journal article I’ve ever read where, in the part of the Discussion section where you’re supposed to propose possible reasons for your findings, both authors suggest maybe their co-author hacked into the computer and altered the results.
The problem seemed to be that the scientist's preconceptions were altering the outcome of their study. Trying to "fix" that would require that you carry out your experiments in the absence of a hypothesis to be tested, which goes against one of the core steps in the scientific method.
The conclusion of the writer of the linked article is that the majority of science wouldn't be able to stand up to the rigour that is required to discount something that is obvious quackery (parapsychology), and we are in fact chasing our own tails in carrying out meta-analyses to try to make good science out of an aggregation of bad science:
The highest level of the Pyramid of Scientific Evidence is meta-analysis. But a lot of meta-analyses are crap. This meta-analysis got p < 1.2 * 10^-10 for a conclusion I'm pretty sure is false, and it isn’t even one of the crap ones. Crap meta-analyses look more like this, or even worse.
How do I know it’s crap? Well, I use my personal judgment. How do I know my personal judgment is right? Well, a smart well-credentialed person like James Coyne agrees with me. How do I know James Coyne is smart? I can think of lots of cases where he’s been right before. How do I know those count? Well, John Ioannides has published a lot of studies analyzing the problems with science, and confirmed that cases like the ones Coyne talks about are pretty common. Why can I believe Ioannides’ studies? Well, there have been good meta-analyses of them. But how do I know if those meta-analyses are crap or not? Well…
Picture of dragon swallowing its own tail, Personal Opinion -> Expert Opinion -> Case Reports -> Cohort Studies -> Randomised Controlled Trials -> Meta Analysis -> Personal Opinion -> ...
Once you get away from the low-hanging fruit (and sometimes even then), science is a minefield of guesswork.
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 2) by FatPhil on Wednesday May 27 2015, @11:21AM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @03:31PM
The name for that is collecting data... how is that not valid science? I'd say if all the hypothesis you can come up with is "variable A is correlated somehow with variable B" why bother? You aren't going to be able to rule out all the possible explanations. The collecting data part is much more valuable then "testing" some vague hypothesis that won't prove anything.
(Score: 2) by gringer on Thursday May 28 2015, @03:56AM
The name for that is collecting data... how is that not valid science?
It's far too easy to collect data to fit your own preconceived ideas. Yes, I agree that data collection is part of the scientific method, but it's not something that can be done in a perfect fashion.
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 0) by Anonymous Coward on Thursday May 28 2015, @04:23AM
That's fine though, since other people will have different ideas to compare with the data. As long as the data is reported in enough detail I don't see where the problem arises.
You need to stamp collect before it is possible to devise a real theory, I think you agree and I can see little room for disagreement on that point. My second claim is that testing a theory that predicts something vague like A is correlated with B doesn't really add anything to the discussion. If you don't detect a correlation you can just conclude 'need more data', if you do see one there will be any number of alternates to consider. So it also seems straightforward that testing such theories is a fools errand.
(Score: 2) by darkfeline on Wednesday May 27 2015, @10:31PM
Or maybe psychic powers only work when in the presence of people who possess "amplifier" powers?
I'm not a proponent of parapsychology, but there's still a hell of a lot we don't know, and the scientific method works.
Join the SDF Public Access UNIX System today!
(Score: 2) by Geezer on Tuesday May 26 2015, @08:01PM
From the Enlightenment up to just before World War 2, after which much science was weaponized and monetized by the military-industrial complex, science for its own sake was the noble quest. Has militarism and profiteering taken us back to the wizard-for-hire quackery of medieval alchemy?
(Score: 0) by Anonymous Coward on Tuesday May 26 2015, @08:12PM
The idea that you will get funded to repeat someone else's work is hilarious!! Here's the deal - you spend 1-2 years writing a grant and having it battered by reviewers until you make it (a) sexy and original and (b) boring and practical. One in 20 may get funded, then you've got about a year before you need to restart the process on a new topic fulfilling criterias (a) and (b). There is no money for (a) or (b) alone. No funding = no job next year in a career you spent 10 years getting into. Funding bodies don't care about reproducibility. Funnily enough, drug companies do care and will pay for some studies - but only in regard to very (scientifically) boring aspects of accuracy and precision of particular tests; work that will be unpublishable - see (a).
(Score: 4, Interesting) by marcello_dl on Tuesday May 26 2015, @09:53PM
Let me adopt one stance that many "science people" hate:
Scientific practice needs nothing fixed. It's either followed or not. When it's not, it's not science.
Experiments behind closed doors? Occultism
Massaging results? Fraud
Half truths corroborated with data? Propaganda
And so on.
To be clear, many "science people" hate this stance when I apply it to religion.
(Score: 4, Interesting) by gringer on Tuesday May 26 2015, @09:59PM
Even when you do all this, you run into problems without good solutions. And if you choose to say "well, they've clearly done X wrong, so it's not science" even for those studies, then you run the risk of discounting almost everything as not science [wikipedia.org].
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @04:45AM
X is not Y. This is not a No True Scotsman; it's just a fact. Stop confusing No True Scotsman with people saying that something is not the same as this other thing.
(Score: 2) by marcello_dl on Wednesday May 27 2015, @02:32PM
I'd call this "the no-no true scotsman" fallacy.
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @08:31PM
I call that the "You just committed the no-no true scotsman fallacy" fallacy.
(Score: 1, Funny) by Anonymous Coward on Tuesday May 26 2015, @10:11PM
Experiments behind closed doors? Occultism
Experiments performed with open doors? Ebola walks out the door.
(Score: 1) by khallow on Wednesday May 27 2015, @02:09AM
For example of the latter problem of distinguishing science from not, consider your assertion "experiments behind closed doors == occultism". How do you determine that the doors are closed? For example, suppose you ask for data to my experiments and I helpfully dump a lot of data. A few weeks later after you've determined that the data is mostly irrelevant, I apologize "Oops, my grad student sent over the wrong stuff. When I get back to the college and things settle down in a few weeks, I'll get it worked out." Lather. Rinse. Repeat. After a few months or years of this, how are you going to show that I'm deliberately and very passive aggressively not providing you with the data (the "closed doors" situation)? "I don't know what marcello_dl's problem is. I provided everything he asked for."
(Score: 2) by marcello_dl on Wednesday May 27 2015, @06:18PM
Starting calling things with their name is a good start IMHO.
(Score: 1) by khallow on Wednesday May 27 2015, @11:34PM
Starting calling things with their name is a good start IMHO.
Why should the people who are benefiting from the current situation go along with that? Science didn't get redefined in the past few decades. What's been happening didn't start because someone forgot to add a "and this time we meant it" clause to the definition of science. I think it happened because there has been a huge disengagement in education and research between the activity and the outcome. For example, there's little consideration of what happens to the oversupply of people who get advanced degrees, particularly PhDs, in scientific fields, particularly those which are hard to match to endeavors outside of academia. I think that alone is a huge driver of the problems we're seeing.
Another important example of this disconnect is in the public funding of research, particularly large research projects. Politicians just aren't interested in research for the most part. But they are interested in getting reelected. So having their name associated with the creation of a big project is a boon. And they'll be far away doing other things, should the project collapse or just not do anything interesting.
I think the combination of the two problems above - vast overproduction of potential researchers combined with sucking up much of the best and brightest of them into research projects which no one in control really cares about - are big drivers of current bad scientific practices. I think we're lucky to get as much as we do. And unless we address the key problems here, I think we'll continue to have an unproductive system that wastes a vast portion of humanity's resources and people.
(Score: 2) by marcello_dl on Thursday May 28 2015, @05:13PM
> What's been happening didn't start because someone forgot to add a "and this time we meant it" clause to the definition of science
I agree, if I might go OT, science and all the rest are obviously supported or boycotted by the powerful classes according to the usefulness for them. This is a fact.
My theory is: Science has been useful to transform society, removing as much control as possible from the old style family (which counted big nuclei able to attend to farm or small enterprises). Now, family is 3 persons who are literally not able to survive without a system, and no way to relearn old habits from grandpas. No more need for science as we knew it. No more need for democratic representation either, because aristocracy has been wiped out by removing all systems competing with financial power. The old society days are over since the year 1990.
(Score: 1, Interesting) by Anonymous Coward on Tuesday May 26 2015, @10:21PM
Reproducibility is not a sensitive issue. There are plenty of papers that are published on the topic.
One of the problems is that nobody thinks that they are part of the problem. Everyone thinks they are an above average driver and all accidents are the other person's fault.
Not many scientists actually try to publish bad science. They are victims of cognitive biases.
(Score: 2) by FatPhil on Wednesday May 27 2015, @10:05AM
This is a good read:
"""
Bayesian Critique of Statistics in Health: The Great
Health Hoax
by Robert Matthews
The plain fact is that 70 years ago Ronald Fisher gave scientists
a mathematical machine for turning baloney into breakthroughs,
and flukes into funding. It is time to pull the plug.
"""
That was over a decade ago, and is written for those who already understand the problem. This one's far more explanatory:
An investigation of the false discovery rate and the misinterpretation of p-values -- David Colquhoun
DOI: 10.1098/rsos.140216 Published 19 November 2014
http://rsos.royalsocietypublishing.org/content/1/3/140216
It should be required reading for everyone going into academia, no matter which discipline. It's worth reading to the end, it never stops not pulling punches (even down to naming wrongdoers).
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @02:40PM
Here are some more discussions of how p-values are (mis)used:
http://www.nature.com/news/scientific-method-statistical-errors-1.14700 [nature.com]
http://www.nature.com/news/statistics-p-values-are-just-the-tip-of-the-iceberg-1.17412 [nature.com]
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @01:19AM
s/t
(Score: 2) by engblom on Wednesday May 27 2015, @04:47AM
I had high confidence in science. I am from a science focused country were people trust science more than anything. Schools are required to teach what mainstream science says.
I have to say I was greatly disappointed when I entered university. Much of what we call knowledge today is just guesses made by people without any further proof. I have seen cases where evidence has shown something else than what is considered knowledge, but because the old model was made long time ago people are reluctant to accept the new evidence and selectively picks what fits the old model. Sometimes we have even pure frauds as this one [theguardian.com]. Often peer reviews are missing, or done by someone running the same agenda.
There is good science happening at this moment too, but to blindly trust everything called "science" is dangerous. I guess the biggest reason behind this problem is how the research grants are given. It often encourages to do frauds as then you keep getting grants. My own country do not have much of this problem as the researchers get their salary regardless of what the results are, but as much of academic research happens outside of this country, it is a problem.
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @06:57PM
Much of what we call knowledge today is just guesses made by people without any further proof.
So it is kind of like accusations of moderation abuse on Soylent News, then?
(Score: 0) by Anonymous Coward on Wednesday May 27 2015, @08:27PM
I saw a study a few years ago where the researcher asked all these college kids how long rapists should be put in prison. Then, the researcher showed some of them some porn videos. Finally, they were asked again how long rapists should be put in prison. For the people who watched porn videos, the prison sentences were reduced by a few years. The conclusion put forth by the researcher was that porn makes people callous towards women. They didn't seriously consider any alternative possibilities and the study didn't truly represent the population at large. They just reached a totally arbitrary conclusion about some subjective notion of callousness and called it a day; no direct, objective evidence of anything.
The social 'sciences' seem to be the worst about this.