SoylentNews

upstart writes:

Reporting bias makes homeopathy trials look like homeopathy works:

One of the more productive ways that the methods of science can be used is to look at the scientific process itself. A "meta-science" study (like a recent one published on brain scans) can help tell us when research approaches aren't producing reliable data and can potentially show what we might need to change to get those approaches to work.

Now, someone has applied a bit of meta-science to an area of research where we shouldn't expect to see improvements: homeopathy. A group of Austrian researchers looked into why a reasonable fraction of the clinical trials on homeopathy produce positive results. The biggest factor, the researchers found, is that the trials that show homeopathy is ineffective are less likely to get published.

There are plenty of ways to test potential treatments, but over the years, problems have been identified in almost all of them. That's left the double-blind, randomized clinical trial as the most trusted method of getting rid of some of the biases that make other approaches less reliable. But even in double-blind trials, problems can creep in. There's always a bias toward publishing positive results—ones where the treatments have an effect.

As a result, we can't always be sure whether we are seeing positive results because a treatment works or because negative results simply aren't getting published. This has been a notable issue with some of the fad "cures" for COVID-19.

To deal with that issue, the field has settled on preregistering clinical trials. In these cases, the design of the trial, the outcomes being measured, and other details are placed in a public database before the trial even starts. Many research journals agreed that preregistration would be a requirement for later publication, meaning that anyone who hoped to publish results in the future would have a compelling reason to preregister. But unregistered trials can usually still get published in lower-profile journals.

This can help us identify when only positive results are being published. And that's one of the analyses that was done by the Austrian researchers.

Journal Reference:
Gerald Gartlehner, Robert Emprechtinger, Marlene Hackl, et al. Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis [open], BMJ Evidence-Based Medicine (DOI: 10.1136/bmjebm-2021-111846)

Original Submission