New guidelines come on the heels of recommendations covering post-market modifications to AI products, including those incorporated into systems used by clinical laboratories
Artificial intelligence (AI) is booming in healthcare, and as the technology finds its way into more medical devices and clinical laboratory diagnostic test technologies the US Food and Drug Administration (FDA) has stepped up its efforts to provide regulatory guidance for developers of these products. This guidance will have an impact on the development of new lab test technology that uses AI going forward.
In December, the FDA issued finalized recommendations for submitting information about planned modifications to AI-enabled healthcare products. Then, in January, the federal agency issued draft guidance that covers product management and marketing submission more broadly. It is seeking public comments on the latter document through April 7.
“The FDA has authorized more than 1,000 AI-enabled devices through established premarket pathways,” said Troy Tazbaz, director of the Digital Health Center of Excellence at the FDA’s Center for Devices and Radiological Health, in a press release announcing the draft guidance.
This guidance “would be the first to provide total product life cycle recommendations for AI-enabled devices, tying together all design, development, maintenance and documentation recommendations, if and when finalized,” Healthcare IT News reported.
“Today’s draft guidance brings together relevant information for developers, shares learnings from authorized AI-enabled devices, and provides a first point-of-reference for specific recommendations that apply to these devices, from the earliest stages of development through the device’s entire life cycle,” said Troy Tazbaz (above), director of the Digital Health Center of Excellence at the FDA Center for Devices and Radiological Health, in a press release. The new guidance will likely affect the development of new clinical laboratory diagnostic technologies that use AI. (Photo copyright: LinkedIn.)
Engaging with FDA
One key takeaway from the guidance is that manufacturers “should engage with the FDA early to ensure that the testing to support the marketing submission for an AI-enabled device reflects the agency’s total product lifecycle, risk-based approach,” states an analysis from consulting firm Orrick, Herrington and Sutcliffe LLP.
Another key point is transparency, Orrick noted. For example, manufacturers should be prepared to offer details about the inputs and outputs of their AI models and demonstrate “how AI helps achieve a device’s intended use.”
Manufacturers should also take steps to avoid bias in data collection for these models. For example, they should gather evidence to determine “whether a device benefits all relevant demographic groups similarly to help ensure that such devices are safe and effective for their intended use,” Orrick said.
New Framework for AI in Drug Development
On the same day that FDA announced the device guidelines, the agency also proposed a framework for regulating use of AI models in developing drugs and biologics.
“AI can be used in various ways to produce data or information regarding the safety, effectiveness, or quality of a drug or biological product,” the federal agency stated in a press release. “For example, AI approaches can be used to predict patient outcomes, improve understanding of predictors of disease progression and process, and analyze large datasets.”
The press release noted that this is the first time the agency has proposed guidance on use of AI in drug development.
These include “bias and reliability problems due to variability in the quality, size, and representativeness of training datasets; the black-box nature of AI models in their development and decision-making; the difficulty of ascertaining the accuracy of a model’s output; and the dangers of data drift and a model’s performance changing over time or across environments. Any of these factors, in FDA’s thinking, could negatively impact the reliability and relevancy of the data sponsors provide FDA.”
The FDA also plans to participate in direct testing of AI-enabled healthcare tools. In October, the FDA and the Department of Veterans Affairs (VA) announced that they will launch “a joint health AI lab to evaluate promising emerging technologies,” according to Nextgov/FCW.
Elnahal said the facility will allow federal agencies and private entities “to test applications of AI in a virtual lab environment.” The goal is to ensure that the tools are safe and effective while adhering to “trustworthy AI principles,” he said.
“It’s essentially a place where you get rapid but effective evaluation—from FDA’s standpoint and from VA’s standpoint—on a potential new application of generative AI to, number one, make sure it works,” he told Nextgov/FCW.
He added that the lab will be set up with safeguards to ensure that the technologies can be tested safely.
“As long as they go through the right security protocols, we’d essentially be inviting parties to test their technology with a fenced off set of VA data that doesn’t have any risk of contagion into our actual live systems, but it’s still informative and simulated,” he told Nextgov/FCW.
There has been an explosion in the use of AI, machine learning, deep learning, and natural language processing in clinical laboratory diagnostic technologies. This is equally true of anatomic pathology, where AI-powered image analysis solutions are coming to market. That two federal agencies are motivated to establish guidelines on working relationships for evaluating the development and use of AI in healthcare settings tells you where the industry is headed.
Infection control teams and clinical laboratory managers may want to look at this new product designed to improve the diagnosis and treatment of sepsis
Accurate and fast diagnosis of sepsis for patients arriving in emergency departments is the goal of a new product that was just cleared by the federal Food and Drug Administration (FDA). It is also the newest example of how artificial intelligence (AI) continues to find its way into pathology and clinical laboratory medicine.
Sepsis is one of the deadliest killers in US hospitals. That is why there is interest in the recent action by the FDA to grant marketing authorization for an AI-powered sepsis detection software through the agency’s De Novo Classification Request. The DNCR “provides a marketing pathway to classify novel medical devices for which general controls alone, or general and special controls, provide reasonable assurance of safety and effectiveness for the intended use, but for which there is no legally marketed predicate device,” the FDA’s website states.
Unlike a single analyte assay that is run in a clinical laboratory, Prenosis’ AI/ML software uses 22 diagnostic and predictive parameters, along with ML algorithms, to analyze data and produce a clinically actionable answer on sepsis.
It is important for clinical laboratory managers and pathologists to recognize that this diagnostic approach to sepsis brings together a number of data points commonly found in a patient’s electronic health record (EHR), some of which the lab generated and others the lab did not generate.
“Sepsis is a serious and sometimes deadly complication. Technologies developed to help prevent this condition have the potential to provide a significant benefit to patients,” said Jeff Shuren, MD, JD, Director of the FDA’s Center for Devices and Radiological Health, in a statement. “The FDA’s authorization of the Prenosis Sepsis ImmunoScore software establishes specific premarket and post-market requirements for this device type.” Clinical laboratory EHRs contain some of the data points Prenosis’ diagnostic software uses. (Photo copyright: US Food and Drug Administration.)
How it Works
To assist doctors diagnose sepsis, the ImmunoScore software is first integrated into the patient’s hospital EHR. From there, it leverages 22 parameters including:
White blood cell count to produce a score that informs caregivers of the patient’s risk for sepsis within 24 hours, MedTech Dive reported.
Instead of requiring a doctor or nurse to look at each parameter separately, the SaMD tool uses AI “to evaluate all those markers at once”, CNBC noted. It then produces a risk score and four discrete risk stratification categories (low, medium, high, and very high) which correlate to “a patient’s risk of deterioration” represented by:
By sharing these details—a number from one to 100 for each of the 22 diagnostic and predictive parameters—Sepsis ImmunoScore helps doctors determine which will likely contribute most to the patient’s risk for developing sepsis, MedTech Dive reported.
“A lot of clinicians don’t trust AI products for multiple reasons. We are trying very hard to counter that skepticism by making a tool that was validated by the FDA first, and then the second piece is we’re not trying to replace the clinician,” Bobby Reddy Jr., PhD, Prenosis co-founder and CEO, told MedTech Dive.
Big Biobank and Blood Sample Data
Prenosis, which says its goal is the “enabling [of] precision medicine in acute care” developed Sepsis ImmunoScore using the company’s own biobank and a dataset of more than 100,000 blood samples from more than 25,000 patients.
AI algorithms drew on this biological/clinical dataset—the largest in the world for acute care patients suspected of having serious infections, according to Prenosis—to “elucidate patterns in rapid immune response.”
“It does not work without data, and the data started at Carle,” said critical care specialist Karen White, MD, PhD, Carle Foundation Hospital, St. Louis, MO, in the news release. “The project involved a large number of physicians, research staff, and internal medicine residents at Carle who helped recruit patients, collect data, and samples,” she said.
Opportunity for Clinical Laboratories
Sepsis is a life-threatening condition based on an “extreme response to an infection” that affects nearly 1.7 million adults in the US each year and is responsible for 350,000 deaths, according to US Centers for Disease Control and Prevention (CDC) data.
A non-invasive diagnostic tool like Sepsis ImmunoScore will be a boon to emergency physicians and the patients they treat. Now that the FDA has authorized the SaMD diagnostic tool to go to market, it may not be long before physicians can use the information it produces to save lives.
Clinical laboratory managers inspired by the development of Sepsis ImmunoScore may want to look for similar ways they can take certain lab test results and combine them with other data in an EHR to create intelligence that physicians can use to better treat their patients. The way forward in laboratory medicine will be combining lab test results with other relevant sets of data to create clinically actionable intelligence for physicians, patients, and payers.
FDA says the move will make it easier to gain authorization for other clinical laboratory tests to utilize at-home collection kits
In another sign of how diagnostic testing is responding to changing consumer preferences, the US Food and Drug Administration (FDA) granted marketing authorization to LetsGetChecked for the company’s Simple 2 test for chlamydia and gonorrhea, which includes at-home collection of samples sent to the test developer’s clinical laboratories in the US and in Ireland.
This marks the first time the FDA has cleared a diagnostic test for either condition in which samples are collected at home. It’s also the first test with at-home sample collection to be authorized for any sexually transmitted infection (STI) other than HIV, the FDA said in a new release.
Samples are collected through a vaginal swab or urine sample. “Results are delivered online in approximately 2-5 days with follow-up virtual consultations and treatment available if needed,” the company press release states.
Previously authorized tests for the conditions required sample collection at the point of care. The company also offers telehealth and online pharmacy services.
“This authorization marks an important public health milestone, giving patients more information about their health from the privacy of their own home,” said Jeff Shuren, MD, JD (above), Director of the FDA’s Center for Devices and Radiological Health. “We are eager to continue supporting greater consumer access to diagnostic tests, which helps further our goal of bringing more healthcare into the home.” With this emphasis on at-home testing from the FDA, clinical laboratories in the US and Ireland will likely be processing more at-home collected samples. (Photo copyright: FDA.)
Simple 2 Process and Costs
Prior to collecting the sample, the user goes online to complete a questionnaire and activate the kit, the FDA news release notes.
LetsGetChecked, headquartered in New York City and Dublin, Ireland, says its US labs are CLIA– and CAP-certified. The company currently offers more than 30 at-home tests covering STIs, men’s health, women’s health, and COVID-19, at prices ranging from $89 to $249 per test.
The Simple 2 test costs $99, and is not covered by insurance, Verywell Health reported. Consumers can get discounts by subscribing to quarterly, semiannual, or annual tests.
New Regulatory Pathway
The FDA said it reviewed the test under its De Novo regulatory pathway, which is intended for “low- to moderate-risk devices of a new type,” according to the news release.
“Along with this De Novo authorization, the FDA is establishing special controls that define the requirements related to labeling and performance testing,” the agency stated. “When met, the special controls, in combination with general controls, provide a reasonable assurance of safety and effectiveness for tests of this type.”
This creates a new regulatory classification, the agency said, that will make it easier for similar devices to obtain marketing authorization.
Citing data from the federal Centers for Disease Control and Prevention (CDC), the FDA news release states that chlamydia and gonorrhea are the most common bacterial STIs in the US. The CDC estimates that there were 1.6 million cases of chlamydia and more than 700,000 cases of gonorrhea in 2021.
“Typically, both infections can be easily treated, but if left untreated, both infections can cause serious health complications for patients, including infertility,” the news release states. “Expanding the availability of STI testing can help patients get quicker results and access to the most appropriate treatment, ultimately helping to curb the rising rates of STIs.”
Experts Praise the FDA’s Authorization of the Lab Test
STI experts contacted by STAT said they welcomed the FDA’s move.
“There are many people who would like to be tested for STIs who may not know where to go or who have barriers to accessing medical care,” said Jodie Dionne, MD, Associate Professor of Medicine in the University of Alabama at Birmingham (UAB) Division of Infectious Diseases. “If we are going to do a better job of reaching more sexually active people for STIs … we need to be creative about how to get them tested and treated in a way that is highly effective and works for them.”
Family physician Alan Katz, MD, a professor at the University of Hawaii John A Burns School of Medicine, told STAT that the Hologic assay is also used by clinicians who treat people in remote locations to diagnose STIs and is regarded as being highly accurate.
“This option is exceptionally useful for individuals who live in rural areas or are geographically distanced from a clinic where STI testing can be done and there is no telehealth option available,” he told STAT.
With this latest move, the FDA is recognizing that it is time to give consumers more control over their healthcare. This is a signal to clinical laboratories that they should be developing their own strategies and offerings that serve consumers who want to order their own tests. Of course, many states still require a physician’s signature on lab test orders, but that is likely to change over time.
InspectIR COVID-19 Breathalyzer identifies a chemical signature associated with SARS-CoV-2 in about three minutes with 91.2% sensitivity and 99.3% specificity
One company is hoping that it can make breathalyzers a viable, easier way to screen for SARS-CoV-2. It will soon have the opportunity to learn if consumers will accept this form of screening for COVID-19, as its device recently obtained an Emergency Use Authorization from the FDA.
On April 14, 2022, InspectIR Systems, LLC, of Frisco, Texas, was granted the US Food and Drug Administration’s first-ever emergency use authorization (EUA202006) for a portable breath test device designed to screen for SARS-CoV-2 infection. Clinical laboratories that perform COVID-19 testing will want to compare the high-level sensitivity of this breath test compared to rapid antigen tests currently used for COVID-19 screening.
The device is about the size of a carry-on suitcase. It provides test results in less than three minutes and is currently authorized for use with subjects who are 18 or older.
The FDA’s EUA limits use of the device to “a qualified, trained operator under the supervision of a healthcare provider licensed or authorized by state law to prescribe tests,” the federal agency said. The test “can be performed in environments where the patient specimen is both collected and analyzed, such as doctor’s offices, hospitals, and mobile testing sites.”
The InspectIR COVID-19 Breathalyzer device “is yet another example of the rapid innovation occurring with diagnostic tests for COVID-19,” said Jeffrey Shuren, MD, JD (above), director of the FDA’s Center for Devices and Radiological Health (CDRH), in the news release. A portable device that can identify SARS-CoV-2 infections in a few minutes with 91% specificity may be of great interest to clinical laboratory companies operating COVID-19 popup testing sites around the nation. (Photo copyright: US Food and Drug Administration.)
In granting the authorization, the FDA cited results of a study with 2,409 participants in which the test had sensitivity (correct positive results) of 91.2% and specificity (correct negative results) of 99.3%. “The test performed with similar sensitivity in a follow-up clinical study focused on the Omicron variant,” the agency stated.
“The FDA continues to support the development of novel COVID-19 tests with the goal of advancing technologies that can help address the current pandemic and better position the US for the next public health emergency,” said Jeffrey Shuren, MD, JD, director of the FDA’s Center for Devices and Radiological Health (CDRH), in the news release.
In its coverage of the EUA, CNET noted that the InspectIR breath test is more sensitive than rapid antigen tests but not as sensitive as PCR tests. The FDA advised that people who receive a positive test result with the InspectIR COVID-19 Breathalyzer should follow up with a PCR molecular test.
How the InspectIR COVID-19 Breathalyzer Works
InspectIR LLC was founded in 2017 by Tim Wing and John Redmond, Forbes reported. Their original goal was to develop a breathalyzer for detection of cannabis or opioid use. However, with the onset of the COVID-19 pandemic, the entrepreneurs decided to adapt the technology into a SARS-CoV-2 diagnostic test.
As described in the FDA’s EUA documents, a subject breathes into the device using a sterilized one-time-use straw. A pre-concentrator collects and concentrates the five targeted VOCs, all from the ketone and aldehyde families of organic compounds. These go to a Residual Gas Analyzer, and an algorithm determines whether the sample contains the chemical signature associated with a SARS-CoV-2 infection.
Redmond told Forbes that the specific mix of VOCs is proprietary. The article notes that Wing, Redmond, and Verbeck have patented the pre-concentrator technology.
The devices are manufactured at a Pfeiffer Vacuum Inc. facility in Indiana. The InspectIR founders told Forbes they expect to produce 100 units per week in a start-up phase with plans to ramp up as sales increase. They also plan to look at applications for other respiratory diseases.
InspectIR has not announced exact pricing, but Time reports that the company will lease the equipment to clients, and that pricing per test will be comparable to rapid antigen tests.
InspectIR’s first breathalyzer device is receiving much positive coverage from the media. Should it prove to effective at spotting COVID-19 at popup testing sites, it may supplant traditional clinical laboratory rapid antigen tests as the screening test of choice.
Since the pandemic began, federal investigators are specifically looking for patterns of fraud in Medicare claims data for COVID-19 clinical laboratory testing
Last month, the federal Department of Health and Human Services (HHS) Office of Inspector General (OIG) announced it had been investigating trends in Medicare claims data that could indicate patterns of fraud in the billing for COVID-19 clinical laboratory tests, Modern Healthcare reported.
Stretching back to at least March, fraudulent actors offering fake SARS-CoV-2 tests have preyed on vulnerable Americans in a wide variety of ways during the public health emergency, according to published reports. Some scam operators have gone into nursing homes and long-term care facilities to collect cash from unsuspecting elders in exchange for swab collections and phony testing, the New York Times reported.
Since the declaration of the public health emergency in the US, the federal Centers for Medicare and Medicaid Services (CMS) no longer requires a lab test requisition signed by a treating physician or other provider for COVID-19 testing. “The strong demand for and limited supply of SARS-CoV-2 tests, along with the move by CMS to relax rules for certain test orders during the pandemic, makes the situation a potentially ripe one for fraud,” Modern Healthcare stated.
Plus, a lack of clarity about the medical necessity of COVID-19 tests could raise the liability risk for law-abiding clinical laboratories. All of these factors make COVID-19 testing fraud a potential bombshell for clinical laboratories conducting coronavirus testing that may get caught up in federal investigations.
Feds Step Up Enforcement
Shortly after the pandemic arrived in the US, the FBI, the Better Business Bureau (BBB), the FDA, the federal Department of Health and Human Services (HHS), and other federal and local authorities have frequently warned doctors, hospitals, and healthcare consumers about the potential for fraud by unscrupulous companies purporting to offer legitimate clinical laboratory testing for COVID-19. A June 26 FBI press release stated, “Scammers are marketing fraudulent and/or unapproved COVID-19 antibody tests, potentially providing false results.”
Some of the fraudsters behind these scams have operated online and through social media and email. While others have conducted these scams in person or over the phone, noted the press release.
And yet, despite the warnings, the scams and news articles about them have continued to spread throughout the COVID-19 pandemic.
Various Forms of Fraud and Their Consequences
In many of these scams, fraudsters seek to collect consumers’ personal information, including names, dates of birth, and Social Security numbers, as well as other forms of personal health information, such as Medicare or private health insurance data, the FBI reported. Scammers can use that information in medical insurance fraud schemes or to commit identity theft, the agency added.
Additionally, any fake or inaccurate COVID-19 tests or assays that the FDA has not allowed for use could provide doctors with false results, potentially creating a dangerous situation for patients.
The New York Times (NYT) recently reported that the FBI had issued a warning “about scammers who advertise fraudulent COVID-19 antibody tests as a way to obtain personal information that can be used for identity theft or medical insurance fraud.”
Three days after the FBI issued its warning about the COVID-19 antibody testing scam, the BBB added an alert to its website: “BBB Scam Alert: Want a COVID-19 test? There’s a scam for that.” BBB also provided advice to consumers about how to avoid testing scams.
On June 17, the FDA reported that it issued warning letters to three companies for marketing adulterated and misbranded COVID-19 antibody tests, stated an FDA news release. The agency sent warning letters to:
In the FDA’s announcement, Jeff Shuren, MD, JD (above), Director of the FDA’s Center for Devices and Radiological Health, said “When tests are marketed inappropriately, with inaccurate or misleading claims—such as the ability to perform the test completely at home, or that the test is authorized, cleared, or approved when it is not—they put the health of Americans at risk. Such conduct will not be tolerated by the FDA, and we will continue to monitor tests marketed in the US, taking appropriate action as warranted.” (Photo copyright: The Food and Drug Administration.)
Scams Reported Just in April
On April 17, the New York Times reported that a special agent with the HHS OIG noted that impostors seeking Medicare or Medicaid information posed as doctors or laboratory technicians to offer fake tests in nursing homes and assisted living facilities.
Earlier in April, The Texas Tribune reported that the owner of a freestanding emergency room in Laredo, Texas, spent $500,000 to buy 20,000 rapid COVID-19 tests for patients suspected of having COVID-19. Health officials in Laredo planned to establish a drive-through testing site and then administer tests that came from a manufacturer in China to detect active infections. After trying to validate the tests, city health officials found they were unreliable and unusable.
An April 9 report from the news department of the AARP (American Association of Retired Persons) stated that federal officials have found fake coronavirus testing sites in many states, including Alabama, Arizona, Florida, Georgia, Kentucky, New York, and Washington state.
The FBI, according to AARP, investigated several fake test sites in Louisville, Ky., after a city official reported that people in personal protective equipment (PPE) were collecting biological specimens from residents. Those seeking tests were told to pay $240 in cash or give their Medicare, Medicaid, or Social Security cards to verify their identity.
Fake drive-up testing sites were reported at gas stations and other locations in Louisville over a four-day period, the AARP reported.
On April 2, WRGB TV in Albany, N.Y., reported that scammers pretending to be from the New York State Department of Health (NYSDOH) were taking money and insurance information from people in exchange for fake coronavirus tests. One woman told police she got a fake test at a drive-up site in a Little League parking lot.
North Greenbush police said the scammers identified themselves as being with NYSDOH and collected money and insurance information from multiple people. Police and state officials said the DOH had no connection to the collection site in the parking lot.
Lessons for Lab Directors
For clinical laboratory directors and all clinical lab scientists, the lesson from these stories is to be wary of strangers offering COVID-19 testing, while also making certain to post information for customers about the legitimacy of your lab’s COVID-19 rapid molecular and serological tests. Doing so might involve providing proof that the FDA has allowed your tests to be used for the coronavirus.
Also, medical laboratories should ensure that all employees collecting specimens in public places display proper identification.