FDA cites ‘risk of false results’ and concerns about labeling and ‘performance claims’ in its official warning letter to Innova, a company with connections to Chinese firms
By many standards, the US government’s response to the COVID-19 pandemic has been phenomenal. However, the many emergency use authorizations (EUAs) awarded by the US federal Food and Drug Administration (FDA) to bring as many COVID-19 tests to market as quickly as possible means some of those tests in use today at clinical laboratories nationwide have not undergone the normal pre-market review and clearance typically required by the FDA.
But in its recall announcement, the FDA described Innova’s recall of its SARS-CoV-2 Antigen Rapid Test as a “Class 1 recall” and gave the stern warning, “Use of these devices may cause serious injuries or death.”
And in its public Safety Communication, the federal agency wrote, “The FDA has significant concerns that the performance of the test has not been adequately established, presenting a risk to health. In addition, labeling distributed with certain configurations of the test includes performance claims that did not accurately reflect the performance estimates observed during the clinical studies of the tests. Finally, the test has not been authorized, cleared, or approved by the FDA for commercial distribution or use in the United States, as required by law.”
In May 2021, Innova Medical Group of Pasadena, Calif., announced it would start producing millions of its COVID-19 SARS-CoV-2 Antigen Rapid test kits (above) per day in the United Kingdom by opening a production facility in Rhymney, South Wales. (Photo copyright: Innova Medical Group, Inc.)
FDA Warns Public to Stop Using Innova’s Rapid Antigen COVID-19 Test
Widescale COVID-19 testing has been viewed as key to containing community spread of the SARS-CoV-2 coronavirus, and fast, inexpensive rapid COVID-19 testing is a necessity in that fight.
However, as clinical laboratory scientists know, rapid tests are not as specific as molecular polymerase chain reaction (PCR) tests, which means there is a higher chance of false negatives and false positives with a COVID-19 rapid test than with a molecular test. When diagnosing COVID-19, a PCR test is considered the gold-standard, though results can take multiple days to produce.
Nevertheless, according to the Innova Europe website, the Innova rapid antigen test has a sensitivity on symptomatic individuals of 97% and a specificity of 99% and is the most widely used test in the world. More than 500 million units are in circulation.
Regardless, in its June 10th warning, the FDA called for the public to stop using the Innova Medical Group SARS-CoV-2 Antigen Rapid test for diagnostic use.
“The FDA has significant concerns that the performance of the test has not been adequately established, presenting a risk to health,” the FDA stated. “In addition, labeling distributed with certain configurations of the test includes performance claims that did not accurately reflect the performance estimates observed during the clinical studies of the tests. Finally, the test has not been authorized, cleared, or approved by the FDA for commercial distribution or use in the United States, as required by law.”
In its warning, the FDA recommended anyone in possession of Innova tests “destroy the tests by placing them in the trash” or return the tests to Innova.
The Innova SARS-CoV-2 Antigen Rapid test is also distributed under the names:
Innova COVID-19 Self-Test Kit (3T Configuration),
Innova Medical Group SARS-CoV-2-Antigen Rapid Test (7T Configuration), and
Innova Medical Group SARS-CoV-2-Antigen Rapid Test (25T Configuration).
Innova Medical Group was formed in March 2020 by Charles Huang, PhD, founder and chairman of private-equity firm Pasaca Capital. The Pasaca website states Innova worked with its primary contract manufacturer, China-based Xiamen Biotime Biotechnology Co., for several months to design “a highly accurate rapid antigen test for COVID-19.”
“The simple test takes less than five minutes to administer and generates results in as little as 20 minutes without the need for a machine,” the website states. “Equally important, Innova and its partner have been able to manufacture the product at scale, presently in excess of ten million kits per day.”
However, The Los Angeles Times claims that in September 2020 Innova “secured a vast supply of rapid coronavirus tests from an obscure Chinese manufacturer before established pharmaceutical companies could do so.” The LA Times adds that Innova distributed more than 70,000 tests in the United States even though the FDA had not acted on Innova’s application to sell its tests domestically.
This may have contributed to the FDA’s dire warning to discontinue use and discard the Innova tests.
UK’s MHRA Disagrees with FDA Warning
But in the UK, it is a different story. According to The Guardian, Innova’s lateral flow tests are the cornerstone of “Operation Moonshot”, the government’s mass testing plan aimed at reducing community transmission by identifying asymptomatic COVID-19 positive people using an inexpensive, quick-response test distributed for home use and to workplaces, schools, and test centers.
In “Rapid COVID Tests Used in Mass UK Programme Get Scathing US Report,” The Guardian reports that “criticism of the Innova test has been fierce” in the UK following the FDA’s “scathing review” of its rapid antigen test. However, after investigating the concerns raised by the FDA, the UK’s Medicines and Healthcare Products Regulatory Agency (MHRA) reiterated that the Innova lateral flow tests were safe to use.
“We have now concluded our review of the risk assessment and are satisfied that no further action is necessary or advisable at this time,” said Graeme Tunbridge, MHRA Director of Devices, in a UK government response statement which announced that the MHRA was extending the Exceptional Use Authorization (EUA) for the tests use in its national asymptomatic testing program through Aug. 28, 2021. “This has allowed us to extend the EUA to allow ongoing supply of these [lateral flow devices] over the coming months. People can be assured of the MHRA’s work to continuously monitor the tests in use; as is our standard process.”
Innova Defends Its Test, FDA Repeats Its Warning
An Innova spokesperson told The Guardian: “The Innova rapid antigen test has been widely used, studied, tested, scrutinized and analyzed with data supporting the efficacy of the test from the largest mass testing program out of the UK. Innova is confident about the quality of its product.”
However, the FDA maintains Innova’s COVID-19 lateral flow test included labeling that provided “false and misleading” estimates of the test’s clinical performance. In its warning letter to Innova, the FDA also pointed out that the clinical study data Innova submitted as part of its EUA request was “identical to data previously provided by other manufacturers in their EUA requests. The data reliability and accuracy issues noted herein raise significant concerns that the performance of the SARS-CoV-2 Antigen Rapid Qualitative Test has not been adequately established, and that the products distributed by Innova without FDA approval, clearance, or authorization could present a serious risk to the public health.”
Pathologists and clinical laboratory professionals in this country will want to watch carefully to see if efforts to increase regulatory scrutiny of diagnostic tests in the UK spills across the Atlantic.
In the absence of a “gold standard,” researchers are finding a high frequency of false negatives among SARS-CoV-2 RT-PCR tests
Serology tests designed to detect antibodies to the SARS-CoV-2 coronavirus that causes the COVID-19 illness have been dogged by well-publicized questions about accuracy. However, researchers also are raising concerns about the accuracy of molecular diagnostics which claim to detect the actual presence of the coronavirus itself.
“Diagnostic tests, typically involving a nasopharyngeal swab, can be inaccurate in two ways,” said Steven Woloshin, MD, MS, in a news release announcing a new report that “examines challenges and implications of false-negative COVID-19 tests.” Woloshin is an internist, a professor at Dartmouth Institute, and co-director of the Geisel School of Medicine at Dartmouth.
“A false-positive result mistakenly labels a person infected, with consequences including unnecessary quarantine and contact tracing,” he stated in the news release. “False-negative results are far more consequential, because infected persons who might be asymptomatic may not be isolated and can infect others.”
Woloshin led a team of Dartmouth researchers who analyzed two studies from Wuhan, China, and a literature review by researchers in Europe and South America that indicated diagnostic tests for COVID-19 are frequently generating false negatives. The team published their results in the June 5 New England Journal of Medicine (NEJM).
For example, one research team in Wuhan collected samples from 213 hospitalized COVID-19 patients and found that an approved RT-PCR test produced false negatives in 11% of sputum samples, 27% of nasal samples, and 40% of throat samples. Their research was published on the medRxiv preprint server and has not been peer-reviewed.
The literature review Woloshin’s team studied was also published on medRxiv, titled, “False-Negative Results of Initial Rt-PCR Assays for COVID-19: A Systematic Review.” It indicated that the rate of false negatives could be as high as 29%. The authors of the review looked at five studies that had enrolled a total of 957 patients. “The collected evidence has several limitations, including risk of bias issues, high heterogeneity, and concerns about its applicability,” they wrote. “Nonetheless, our findings reinforce the need for repeated testing in patients with suspicion of SARS-Cov-2 infection.”
Another literature review, published in the Annals of Internal Medicine, titled, “Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction–Based SARS-CoV-2 Tests by Time Since Exposure,” estimated the probability of false negatives in RT-PCR tests at varying intervals from the time of exposure and symptom onset. For example, the authors found that the median false-negative rate was 38% if a test was performed on the day of symptom onset, versus 20% three days after onset. Their analysis was based on seven studies, five of which were peer-reviewed, with a total of 1330 test samples.
Doctors also are seeing anecdotal evidence of false negatives. For example, clinicians at UC San Diego Health medical center treated a patient with obvious symptoms of COVID-19, but two tests performed on throat samples were negative. However, a third test, using a sample from a bronchial wash, identified the virus, reported Medscape.
Sensitivity and Specificity of COVID-19 Clinical Laboratory Tests
The key measures of test accuracy are sensitivity, which refers to the ability to detect the presence of the virus, and specificity, the ability to determine that the targeted pathogen is not present. “So, a sensitive test is less likely to provide a false-negative result and a specific test is less likely to provide a false-positive result,” wrote Kirsten Meek, PhD, medical writer and editor, in an article for ARUP Laboratories.
“Analytic” sensitivity and specificity “represent the accuracy of a test under ideal conditions in which specimens have been collected from patients with either high viral loads or a complete absence of exposure,” she wrote. However, “sensitivity and specificity under real-world conditions, in which patients are more variable and specimen collection may not be ideal, can often be lower than reported numbers.”
In a statement defending its ID Now molecular point-of-care test, which came under scrutiny during a study of COVID-19 molecular tests by NYU Langone Health, Northwell Health, and Cleveland Clinic, according to MedTech Dive, Abbott Laboratories blamed improper sample collection and handling for highly-publicized false negatives produced by its rapid test. An FDA issued alert about the test on May 14 noted that Abbott had agreed to conduct post-market studies to identify the cause of the false negatives and suggest remedial actions.
Issues with Emergency Use Authorizations
In their NEJM analysis, Woloshin et al point to issues with the FDA’s process for issuing Emergency Use Authorizations (EUAs). For example, they noted variations in how manufacturers are conducting clinical evaluations to determine test performance. “The FDA prefers the use of ‘natural clinical specimens’ but has permitted the use of ‘contrived specimens’ produced by adding viral RNA or inactivated virus to leftover clinical material,” they wrote.
When evaluating clinical performance, manufacturers ordinarily conduct an index test of patients and compare the results with reference-standard test, according to the Dartmouth researchers. For people showing symptoms, the reference standard should be a clinical diagnosis performed by an independent adjudication panel. However, they wrote, “it is unclear whether the sensitivity of any FDA-authorized commercial test has been assessed in this way.” Additionally, a reference standard for determining sensitivity in asymptomatic people “is an unsolved problem that needs urgent attention to increase confidence in test results for contact-tracing or screening purposes.”
“To truly determine false negatives, you need a gold standard test, which is essentially as close to perfect as we can get,” Stephen Rawlings, MD, PhD, (above), a resident physician of internal medicine and infectious diseases fellow at UC San Diego’s Center for AIDS Research (CFAR), who has been working on SARS-CoV-2 test validation since March. “But there just isn’t one yet for coronavirus,” he told Medscape. (Photo copyright: University of California, San Diego.)
Continued adherence to current measures, such as physical distancing and surface disinfection.
Development of highly sensitive and specific tests or combinations of tests to minimize the risk of false-negative results and ongoing transmission based on a false sense of security.
Improved RT-PCR tests and serological assays.
Development and communication of clear risk-stratified protocols for management of negative COVID-19 test results.
“These protocols must evolve as diagnostic test, transmission, and outcome statistics become more available,” they wrote.
Meanwhile, clinical laboratories remain somewhat on their own at selecting which COVID-19 molecular and serology tests they want to purchase and run in their labs. Complicating such decisions is the fact that many of the nation’s most reputable in vitro diagnostics manufacturers cannot produce enough of their COVID-19 tests to meet demand.
Consequently, when looking to purchase tests for SARS-CoV-2, smaller medical laboratory organizations find themselves evaluating COVID-19 kits developed by little-known or even brand-new companies.
