New guidelines come on the heels of recommendations covering post-market modifications to AI products, including those incorporated into systems used by clinical laboratories
Artificial intelligence (AI) is booming in healthcare, and as the technology finds its way into more medical devices and clinical laboratory diagnostic test technologies the US Food and Drug Administration (FDA) has stepped up its efforts to provide regulatory guidance for developers of these products. This guidance will have an impact on the development of new lab test technology that uses AI going forward.
In December, the FDA issued finalized recommendations for submitting information about planned modifications to AI-enabled healthcare products. Then, in January, the federal agency issued draft guidance that covers product management and marketing submission more broadly. It is seeking public comments on the latter document through April 7.
“The FDA has authorized more than 1,000 AI-enabled devices through established premarket pathways,” said Troy Tazbaz, director of the Digital Health Center of Excellence at the FDA’s Center for Devices and Radiological Health, in a press release announcing the draft guidance.
This guidance “would be the first to provide total product life cycle recommendations for AI-enabled devices, tying together all design, development, maintenance and documentation recommendations, if and when finalized,” Healthcare IT News reported.
“Today’s draft guidance brings together relevant information for developers, shares learnings from authorized AI-enabled devices, and provides a first point-of-reference for specific recommendations that apply to these devices, from the earliest stages of development through the device’s entire life cycle,” said Troy Tazbaz (above), director of the Digital Health Center of Excellence at the FDA Center for Devices and Radiological Health, in a press release. The new guidance will likely affect the development of new clinical laboratory diagnostic technologies that use AI. (Photo copyright: LinkedIn.)
Engaging with FDA
One key takeaway from the guidance is that manufacturers “should engage with the FDA early to ensure that the testing to support the marketing submission for an AI-enabled device reflects the agency’s total product lifecycle, risk-based approach,” states an analysis from consulting firm Orrick, Herrington and Sutcliffe LLP.
Another key point is transparency, Orrick noted. For example, manufacturers should be prepared to offer details about the inputs and outputs of their AI models and demonstrate “how AI helps achieve a device’s intended use.”
Manufacturers should also take steps to avoid bias in data collection for these models. For example, they should gather evidence to determine “whether a device benefits all relevant demographic groups similarly to help ensure that such devices are safe and effective for their intended use,” Orrick said.
New Framework for AI in Drug Development
On the same day that FDA announced the device guidelines, the agency also proposed a framework for regulating use of AI models in developing drugs and biologics.
“AI can be used in various ways to produce data or information regarding the safety, effectiveness, or quality of a drug or biological product,” the federal agency stated in a press release. “For example, AI approaches can be used to predict patient outcomes, improve understanding of predictors of disease progression and process, and analyze large datasets.”
The press release noted that this is the first time the agency has proposed guidance on use of AI in drug development.
These include “bias and reliability problems due to variability in the quality, size, and representativeness of training datasets; the black-box nature of AI models in their development and decision-making; the difficulty of ascertaining the accuracy of a model’s output; and the dangers of data drift and a model’s performance changing over time or across environments. Any of these factors, in FDA’s thinking, could negatively impact the reliability and relevancy of the data sponsors provide FDA.”
The FDA also plans to participate in direct testing of AI-enabled healthcare tools. In October, the FDA and the Department of Veterans Affairs (VA) announced that they will launch “a joint health AI lab to evaluate promising emerging technologies,” according to Nextgov/FCW.
Elnahal said the facility will allow federal agencies and private entities “to test applications of AI in a virtual lab environment.” The goal is to ensure that the tools are safe and effective while adhering to “trustworthy AI principles,” he said.
“It’s essentially a place where you get rapid but effective evaluation—from FDA’s standpoint and from VA’s standpoint—on a potential new application of generative AI to, number one, make sure it works,” he told Nextgov/FCW.
He added that the lab will be set up with safeguards to ensure that the technologies can be tested safely.
“As long as they go through the right security protocols, we’d essentially be inviting parties to test their technology with a fenced off set of VA data that doesn’t have any risk of contagion into our actual live systems, but it’s still informative and simulated,” he told Nextgov/FCW.
There has been an explosion in the use of AI, machine learning, deep learning, and natural language processing in clinical laboratory diagnostic technologies. This is equally true of anatomic pathology, where AI-powered image analysis solutions are coming to market. That two federal agencies are motivated to establish guidelines on working relationships for evaluating the development and use of AI in healthcare settings tells you where the industry is headed.
While many of the major gains promised by electronic health records (EHRs) and big data remain elusive, Geisinger Health’s Unified Data Architecture demonstrates how big data might help healthcare providers and clinical laboratories optimize care, improve outcomes, and control costs as the technology evolves
Use of big data in healthcare gets plenty of hoopla these days. Many experts predict great things as clinical laboratory test data is pooled with other patient information and demographic data. But there are many technical problems to be overcome before the full potential of healthcare big data can be translated into ways that improve the health of individuals.
Big data in healthcare is essential to the success of both precision medicine and population health management. However, without the ability to consolidate other data sources and provide intuitive ways for healthcare providers to access, analyze, and utilize the data coming from the various sources, such as clinical laboratory and anatomic pathology test results, much of the data can be underutilized or overlooked.
Consolidating Data to Create Cohesive Snapshots of Patient Health
The HBR report attributes Geisinger’s ability to utilize big healthcare data to its Unified Data Architecture (UDA). According to a Healthcare Informatics article, Geisinger’s UDA was based on Hadoop and other open source software. According to the doctors who wrote the HBR report, “… pulling meaningful data aggregated from many sources back out of EHRs has historically been vexingly complex. The potential insight from these data are limited in practice by the shortcomings of traditional data repositories.”
Geisinger’s UDA addresses two key issues the Healthcare Informatics authors see as obstacles to the expanded, easier use of big healthcare data:
Lack of ways to deal with unstructured patient notes that do not adhere to traditional database organizational structures; and
Data silos created when multiple departments collect data but use separate storage systems.
Using natural language processing (NLP), the UDA system can pull critical information from long-form written reports or analyses.
Big data graphic above from Nuance, developer of intelligent systems for healthcare and other industries, illustrates the challenges involved in acquiring, sifting, managing, and utilizing big data in healthcare. (Graphic copyright: Nuance.)
Geisinger’s system connects nurses on the floor, medical technologists in the clinical laboratory, and surgeons in operating rooms to the same pools of data. However, it also pulls in data from external sources, such as pathology groups, other reference or medical laboratories, and even patient-worn mobile medical devices. The HBR report states, “The integration of data from Health Information Exchanges, clinical departmental systems (such as radiology and cardiology), patient satisfaction surveys, and health and wellness apps provides us with a detailed, longitudinal view of the patient.”
Big Data Helps Healthcare Professionals Spot Future Worries
Geisinger doctors found that AAAs typically are discovered during care for another condition. Often, the conditions for which the patient seeks care are more serious than the small AAA and it isn’t mentioned. While AAAs might be noted in patient records, healthcare providers typically do not look for the data. Thus, left untreated, a AAA can develop into a serious condition that could have been prevented.
NLP enables Geisinger doctors to analyze UDA data for warning signs of AAA and create follow-up and treatment plans that might otherwise remain overlooked. According to the HBR report, this program has led to 12 lifesaving operations to date that might otherwise have been missed.
Real-Time, Comprehensive Updates Offer Big Gains in Combating Sepsis
Big healthcare data shows potential for treating many life-threatening conditions, such as sepsis. Prompt treatment is essential to positive outcomes in sepsis cases. Physicians at Geisinger use the company’s UDA data to both pinpoint when sepsis indicators appeared, as well as to consolidate data from across a patient’s care continuum to optimize treatment.
Instead of sorting through disparate streams of data from various operational areas and reports, data is combined into a consolidated dashboard featuring real-time physiologic metrics, such as:
The HBR report notes, “By tracking, aggregating, and synthesizing all sepsis-patient data, we expect we will be able to both reduce the incidence of hospital-acquired sepsis and improve its management.”
Using Big Data to Track Surgical Supply Chains and Waste
With the unique cost and outcome aspects of each surgical case, and the differences in payouts from payers, creating big data for tracking the efficiency and waste of surgeries is difficult without a big picture view of the factors. Using their UDA, Geisinger can track the exact supplies used in an operation along with the outcome, recovery, cost, and follow-up data related to the procedure.
“This gives surgeons and administrators an important new view of how they perform comparatively from both a cost and outcome perspective,” noted the HBR report’s authors.
Big data is still a developing technology. Nevertheless, programs such as at Geisinger Health offer useful lessons into how data streaming from clinical laboratories, pathology assays, operating rooms, intensive care units, and even personal health-tracking devices might be combined to provide a unified patient record. That would make it possible for caregivers to use analytical tools to tailor each patient’s care and treatment to his or her specific conditions and physiology.
Watson is capable of assessing health data, including medical laboratory test results
When IBM’s Watson “supercomputer” squared off against human contestants on the Jeopardy game show last February, there certainly were some pathologists and clinical laboratory managers watching this “man versus machine” battle of knowledge. But those pathologists and medical lab managers did not realize that IBM intends for Watson to play a major role in helping physicians diagnose and treat disease.
IBM is designing Watson to use analytical algorithms to support how physicians assess information as they evaluate patients. In this role, it is likely that Watson will be fed laboratory test data and evidence-based medicine algorithms as part of the data it draws upon to help physicians more accurately diagnose disease and come up with appropriate treatment plans. (more…)
