SoylentNews

upstart [soylentnews.org] writes:

████ # This file was generated bot-o-matically! Edit at your own risk. ████

FDA rushed out agency-wide AI tool [arstechnica.com]:

Text settings

Under the Trump administration, the Food and Drug Administration is eagerly embracing artificial intelligence tools that staff members are reportedly calling rushed, buggy, overhyped, and inaccurate.

On Monday, the FDA publicly announced the agency-wide rollout of a large language model (LLM) called Elsa [fda.gov], which is intended to help FDA employees—"from scientific reviewers to investigators." The FDA said the generative AI is already being used to "accelerate clinical protocol reviews, shorten the time needed for scientific evaluations, and identify high-priority inspection targets."

"It can summarize adverse events to support safety profile assessments, perform faster label comparisons, and generate code to help develop databases for nonclinical applications," the announcement promised.

In a statement, FDA Chief AI Officer Jeremy Walsh trumpeted the rollout, saying: "Today marks the dawn of the AI era at the FDA[. W]ith the release of Elsa, AI is no longer a distant promise but a dynamic force enhancing and optimizing the performance and potential of every employee."

Meanwhile, FDA Commissioner Marty Makary highlighted the speed with which the tool was rolled out. "I set an aggressive timeline to scale AI agency-wide by June 30," Makary said. "Today’s rollout of Elsa is ahead of schedule and under budget, thanks to the collaboration of our in-house experts across the centers."

Rushed and inaccurate

However, according to a report from NBC News [nbcnews.com], Elsa could have used some more time in development. FDA staff tested Elsa on Monday with questions about FDA-approved products or other public information, only to find that it provided summaries that were either completely or partially wrong.

FDA staffers who spoke with Stat news, meanwhile, called the tool "rushed" [statnews.com] and said its capabilities were overinflated by officials, including Makary and those at the Department of Government Efficiency (DOGE), which was headed by controversial billionaire Elon Musk. In its current form, it should only be used for administrative tasks, not scientific ones, the staffers said.

"Makary and DOGE think AI can replace staff and cut review times, but it decidedly cannot," one employee said. The staffer also said that the FDA has failed to set up guardrails for the tool's use. "I’m not sure in their rush to get it out that anyone is thinking through policy and use," the FDA employee said.

According to Stat, Elsa is based on Anthropic's Claude LLM and is being developed by consulting firm Deloitte. Since 2020, Deloitte has been paid $13.8 million to develop the original database of FDA documents that Elsa's training data is derived from. In April, the firm was awarded a $14.7 million contract to scale the tech across the agency. The FDA said that Elsa was built within a high-security GovCloud environment and offers a "secure platform for FDA employees to access internal documents while ensuring all information remains within the agency."

Previously, each center within the FDA was working on its own AI pilot. However, after cost-cutting in May, the AI pilot originally developed by the FDA's Center for Drug Evaluation and Research, called CDER-GPT, was selected to be scaled up to an FDA-wide version and rebranded as Elsa.

FDA staffers in the Center for Devices and Radiological Health told NBC News that their AI pilot, CDRH-GPT, is buggy, isn't connected to the Internet or the FDA's internal system, and has problems uploading documents and allowing users to submit questions.

Original Submission