Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Monday April 10 2017, @09:17AM   Printer-friendly
from the draw-curve-then-select-points-to-plot dept.

Scott Gottlieb, President Trump's nominee to run the FDA, is a proponent of adaptive clinical trials, which would allow adjustments of trials as they are ongoing:

In 2006, Scott Gottlieb, then a deputy commissioner at the U.S. Food and Drug Administration (FDA), stood before an audience of clinicians and researchers to sing the praises of a new approach to drug trials. Instead of locking in a study's design from the start, researchers could build in options that would allow them to adjust along the way, based on the data they had collected. They could make the trial larger or smaller, for instance, add or remove arms, or change how incoming patients get assigned to them. Gottlieb predicted such adaptive trial designs, the topic of the conference he attended that distant summer in Washington, D.C., would "tell us more about safety and benefits of drugs, in potentially shorter time frames."

This week, as President Donald Trump's nominee to head FDA, Gottlieb sat before Republican lawmakers hungry for promises of "shorter time frames" for drug and device approvals, and again expressed his zeal—repeatedly—for adaptive trial designs. If confirmed to be FDA's head, as expected, Gottlieb suggested he'd promote wider use of the approach.

But for all their promise, many adaptive trial features still aren't commonplace. And Gottlieb will face a number of obstacles to encouraging their wider use, experts tell ScienceInsider.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by bradley13 on Monday April 10 2017, @09:48AM (14 children)

    by bradley13 (3053) on Monday April 10 2017, @09:48AM (#491566) Homepage Journal

    Why is this of relevance? A quick skim of what an adaptive trial is all about, and...sounds like a variation that would be appropriate in some circumstances. Adapting a trial in progress requires extra care not to mess up the trial results. An example might be discovering the dosage limits of a medicine: successively increasing/decreasing dosage until some effect appears/disappears. Really, a rather minor point.

    If this is the most important issue to be discussed, I suppose the guy will be approved with no problems?

    --
    Everyone is somebody else's weirdo.
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 3, Insightful) by GungnirSniper on Monday April 10 2017, @11:27AM

    by GungnirSniper (1671) on Monday April 10 2017, @11:27AM (#491581) Journal

    Republicans will speak to their base of the business benefits, and Democrats will speak to their base of the safety concerns. He will be approved, but both sides need to use his carcass for promotion first.

    Pharma is one area were we probably have too much regulation now in response to the free-wheeling problems that created weight loss drugs with long-term side effects, or "thalidomide babies" with missing limbs or "flippers". So the balance of cost and risk is eternally in play.

  • (Score: 5, Insightful) by FatPhil on Monday April 10 2017, @12:12PM (5 children)

    by FatPhil (863) <pc-soylentNO@SPAMasdf.fi> on Monday April 10 2017, @12:12PM (#491597) Homepage
    It also permits p-hacking.
    --
    Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
    • (Score: 1) by Zipf on Monday April 10 2017, @05:29PM (1 child)

      by Zipf (2400) on Monday April 10 2017, @05:29PM (#491760)

      This is true -- buy only if those who conduct the trail are unaware of p-hacking. (they are not.)

      I found the following to be rather enlightening:

      http://www.intelligencesquaredus.org/debates/fdas-caution-hazardous-our-health [intelligencesquaredus.org]

      Gottlieb and Huber basically convince those who argue against adaptive trials that adaptive trials are what we need.

      • (Score: 2) by FatPhil on Tuesday April 11 2017, @08:30AM

        by FatPhil (863) <pc-soylentNO@SPAMasdf.fi> on Tuesday April 11 2017, @08:30AM (#492179) Homepage
        > This is true -- buy only if those who conduct the trail are unaware of p-hacking

        or are *perfectly aware* of p-hacking

        > (they are not [unaware].)

        you're close to working out the problem now.
        --
        Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
    • (Score: 2) by AthanasiusKircher on Monday April 10 2017, @05:53PM (1 child)

      by AthanasiusKircher (5291) on Monday April 10 2017, @05:53PM (#491776) Journal

      Does this really qualify as "p-hacking"? P-hacking generally implies taking existing data and trying out oodles of possible statistical correlations to find one (or more) that appear to have a "significant" result. (Or other equivalent data analysis methods than obscure the experimental design and data collection in such a way as to overestimate significance.)

      But this isn't a post-hoc statistical analysis procedure. This is actively manipulating the study and data collection while in progress. I think this actually falls on a different level of study manipulation than p-hacking (which is generally referring to the analysis phase).

      There may be times when an adaptive procedure is justified, but each adaptation generally decreases statistical power and makes it less likely the study the results will mean anything. A rigorous statistical analysis should actually determine that most adaptive studies have LESS significant results, which would be the opposite of p-hacking. (Note that I'm relying on the idea that researchers would honestly report their method and results.)

      The main way to combat p-hacking is to specify analysis methods in advance, rather than allowing ad hoc data manipulation after-the-fact. The effects of potential adaptation may also be able to be quantified in advance (depending on exactly what sort of adaptation is allowed), along with their subsequent effect on the statistical power. But a better design might generally be to use an adaptive trial to identify potential good treatment procedures and then design a more rigorous second study to verify with a locked-down a priori procedure and data analysis method.

      But I'm assuming your point was that a bad use of adaptive design is to fudge the experimental method as you go but analyze the data as if nothing weird happened (which might inflate significance). Yeah, that's the way I'd be afraid of drug companies "cheating" too, though I don't know if I'd call that p-hacking in the normal sense. That would, to me, qualify as more active direct study manipulation, more easily rising to the level of deliberate professional misconduct. If this is done correctly, the parameters of adaptive studies would need to be built into the proposed study design in advance, as well as their potential impact on the way the results are handled.

      • (Score: 2) by FatPhil on Tuesday April 11 2017, @08:39AM

        by FatPhil (863) <pc-soylentNO@SPAMasdf.fi> on Tuesday April 11 2017, @08:39AM (#492184) Homepage
        There are several ways to p-hack. One is post-hoc trawling for relevance in a huge pool of data. Another is simply by running multiple experiments, and chosing (stopping at) the one that has the desired outcome. That latter form is probably the more familiar form of p-hacking: https://www.xkcd.com/882/ Changing the experiment is creating more experiments. (And introducing more flexibility to delete outliers.)
        --
        Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
    • (Score: 0) by Anonymous Coward on Monday April 10 2017, @07:47PM

      by Anonymous Coward on Monday April 10 2017, @07:47PM (#491887)

      My Pee doesn't have an interface, not even a USB port... I tested it no wifi hotspot either, so good luck!

  • (Score: 4, Insightful) by MrGuy on Monday April 10 2017, @12:54PM (2 children)

    by MrGuy (1007) on Monday April 10 2017, @12:54PM (#491606)

    Adapting a trial in progress requires extra care not to mess up the trial results.

    Another way of putting this point is that when you adapt a trial in progress, it's very easy to bias the results. You have to be both intellectually honest and scientifically rigorous to avoid this.

    Allowing adaptive trials will increase efficiency in scientific testing and reduce time to market if drug makers work hard to avoid bias. They will reduce safety and lead to getting dangerous drugs approved and into the market if they don't.

    The FDA's role in the process is to ensure drugs are safe. The concern is that, given the current administration's focus on reducing regulatory oversight, the FDA will lack either the authority or the will to ensure such trials are conducted responsibly, and not used by companies to cheat.

    • (Score: 3, Insightful) by Immerman on Monday April 10 2017, @01:29PM (1 child)

      by Immerman (3985) on Monday April 10 2017, @01:29PM (#491611)

      Indeed. It's an strategy that is much better suited to "exploratory" research, which is what most medical testing is. However, it does make intentionally biasing the studies easier, and there is already plenty of evidence of repeated dishonest testing by the big pharmaceutical companies.

      I could see it working well if we also complemented it by adding the requirement for independent verification of the results before approval. Verification can readily test only the narrow final claims. Just loosening the scientific rigor of the standards though seems like a bad idea.

      On the other hand - pharmaceutical companies already cheat routinely, so it could be argued that there isn't actually anything lost.

      Perhaps an encouragement to honesty with lower overhead - such as large mandatory fines for any company found to have falsified data or otherwise unfairly biased their research, intentionally or otherwise. Encourage independent review within the company. Make the fines payable directly to the FDA coffers in order to discourage corruption at least at the institutional level.

      • (Score: 2) by MrGuy on Monday April 10 2017, @02:27PM

        by MrGuy (1007) on Monday April 10 2017, @02:27PM (#491638)

        On the other hand - pharmaceutical companies already cheat routinely, so it could be argued that there isn't actually anything lost.

        I think there's a huge difference when, facing a class action lawsuit, a company can claim "We conducted rigorous scientific studies using industry standard methods overseen and approved by the FDA!" As opposed to "You caught us - we fudged the numbers to get the result we wanted and lied about it in our reports."

        It might not reduce liability to zero, but being able to claim with a straight face that you did a scientifically accurate study and the FDA oversaw and approved it is a pretty good defense.

  • (Score: 2) by DannyB on Monday April 10 2017, @03:49PM

    by DannyB (5839) Subscriber Badge on Monday April 10 2017, @03:49PM (#491681) Journal

    Nothing wrong with an adaptive trial if the people doing it are "adapting" it for honest and ethical reasons.

    Therein lies the highly likely problem.

    New Clinical Study Shows Promise. People can be happy and enjoy the rest of their lives with high doses of morphine and arsenic! Who knew!

    Trump directs FDA to expedite approval with executive order.

    --
    People today are educated enough to repeat what they are taught but not to question what they are taught.
  • (Score: 2) by davester666 on Monday April 10 2017, @07:59PM (2 children)

    by davester666 (155) on Monday April 10 2017, @07:59PM (#491894)

    The new "adaptive" means they can just remove the people who die and/or get ill from the study, so that the medication can be approved based on the initial trial. They don't have to keep doing it over and over until they get the results needed for approval (and burying the "bad" trials with attorney-client privilege)...

    • (Score: 2) by FatPhil on Tuesday April 11 2017, @08:42AM

      by FatPhil (863) <pc-soylentNO@SPAMasdf.fi> on Tuesday April 11 2017, @08:42AM (#492186) Homepage
      > The new "adaptive" means they can just remove the people who die

      "Outliers" in statistical terminology.
      --
      Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
    • (Score: 1) by purple_cobra on Wednesday April 12 2017, @08:30PM

      by purple_cobra (1435) on Wednesday April 12 2017, @08:30PM (#493050)

      See Ben Goldacre's various media interviews/appearances and his book 'Bad Pharma' (he also wrote 'Bad Science') for more on this kind of thing; he was pretty vocal about Tamiflu, amongst other useless drugs.