In the absence of a “gold standard,” researchers are finding a high frequency of false negatives among SARS-CoV-2 RT-PCR tests
Serology tests designed to detect antibodies to the SARS-CoV-2 coronavirus that causes the COVID-19 illness have been dogged by well-publicized questions about accuracy. However, researchers also are raising concerns about the accuracy of molecular diagnostics which claim to detect the actual presence of the coronavirus itself.
“Diagnostic tests, typically involving a nasopharyngeal swab, can be inaccurate in two ways,” said Steven Woloshin, MD, MS, in a news release announcing a new report that “examines challenges and implications of false-negative COVID-19 tests.” Woloshin is an internist, a professor at Dartmouth Institute, and co-director of the Geisel School of Medicine at Dartmouth.
“A false-positive result mistakenly labels a person infected, with consequences including unnecessary quarantine and contact tracing,” he stated in the news release. “False-negative results are far more consequential, because infected persons who might be asymptomatic may not be isolated and can infect others.”
Woloshin led a team of Dartmouth researchers who analyzed two studies from Wuhan, China, and a literature review by researchers in Europe and South America that indicated diagnostic tests for COVID-19 are frequently generating false negatives. The team published their results in the June 5 New England Journal of Medicine (NEJM).
For example, one research team in Wuhan collected samples from 213 hospitalized COVID-19 patients and found that an approved RT-PCR test produced false negatives in 11% of sputum samples, 27% of nasal samples, and 40% of throat samples. Their research was published on the medRxiv preprint server and has not been peer-reviewed.
The literature review Woloshin’s team studied was also published on medRxiv, titled, “False-Negative Results of Initial Rt-PCR Assays for COVID-19: A Systematic Review.” It indicated that the rate of false negatives could be as high as 29%. The authors of the review looked at five studies that had enrolled a total of 957 patients. “The collected evidence has several limitations, including risk of bias issues, high heterogeneity, and concerns about its applicability,” they wrote. “Nonetheless, our findings reinforce the need for repeated testing in patients with suspicion of SARS-Cov-2 infection.”
Another literature review, published in the Annals of Internal Medicine, titled, “Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction–Based SARS-CoV-2 Tests by Time Since Exposure,” estimated the probability of false negatives in RT-PCR tests at varying intervals from the time of exposure and symptom onset. For example, the authors found that the median false-negative rate was 38% if a test was performed on the day of symptom onset, versus 20% three days after onset. Their analysis was based on seven studies, five of which were peer-reviewed, with a total of 1330 test samples.
Doctors also are seeing anecdotal evidence of false negatives. For example, clinicians at UC San Diego Health medical center treated a patient with obvious symptoms of COVID-19, but two tests performed on throat samples were negative. However, a third test, using a sample from a bronchial wash, identified the virus, reported Medscape.
Sensitivity and Specificity of COVID-19 Clinical Laboratory Tests
The key measures of test accuracy are sensitivity, which refers to the ability to detect the presence of the virus, and specificity, the ability to determine that the targeted pathogen is not present. “So, a sensitive test is less likely to provide a false-negative result and a specific test is less likely to provide a false-positive result,” wrote Kirsten Meek, PhD, medical writer and editor, in an article for ARUP Laboratories.
“Analytic” sensitivity and specificity “represent the accuracy of a test under ideal conditions in which specimens have been collected from patients with either high viral loads or a complete absence of exposure,” she wrote. However, “sensitivity and specificity under real-world conditions, in which patients are more variable and specimen collection may not be ideal, can often be lower than reported numbers.”
In a statement defending its ID Now molecular point-of-care test, which came under scrutiny during a study of COVID-19 molecular tests by NYU Langone Health, Northwell Health, and Cleveland Clinic, according to MedTech Dive, Abbott Laboratories blamed improper sample collection and handling for highly-publicized false negatives produced by its rapid test. An FDA issued alert about the test on May 14 noted that Abbott had agreed to conduct post-market studies to identify the cause of the false negatives and suggest remedial actions.
Issues with Emergency Use Authorizations
In their NEJM analysis, Woloshin et al point to issues with the FDA’s process for issuing Emergency Use Authorizations (EUAs). For example, they noted variations in how manufacturers are conducting clinical evaluations to determine test performance. “The FDA prefers the use of ‘natural clinical specimens’ but has permitted the use of ‘contrived specimens’ produced by adding viral RNA or inactivated virus to leftover clinical material,” they wrote.
When evaluating clinical performance, manufacturers ordinarily conduct an index test of patients and compare the results with reference-standard test, according to the Dartmouth researchers. For people showing symptoms, the reference standard should be a clinical diagnosis performed by an independent adjudication panel. However, they wrote, “it is unclear whether the sensitivity of any FDA-authorized commercial test has been assessed in this way.” Additionally, a reference standard for determining sensitivity in asymptomatic people “is an unsolved problem that needs urgent attention to increase confidence in test results for contact-tracing or screening purposes.”
“To truly determine false negatives, you need a gold standard test, which is essentially as close to perfect as we can get,” Stephen Rawlings, MD, PhD, (above), a resident physician of internal medicine and infectious diseases fellow at UC San Diego’s Center for AIDS Research (CFAR), who has been working on SARS-CoV-2 test validation since March. “But there just isn’t one yet for coronavirus,” he told Medscape. (Photo copyright: University of California, San Diego.)
Continued adherence to current measures, such as physical distancing and surface disinfection.
Development of highly sensitive and specific tests or combinations of tests to minimize the risk of false-negative results and ongoing transmission based on a false sense of security.
Improved RT-PCR tests and serological assays.
Development and communication of clear risk-stratified protocols for management of negative COVID-19 test results.
“These protocols must evolve as diagnostic test, transmission, and outcome statistics become more available,” they wrote.
Meanwhile, clinical laboratories remain somewhat on their own at selecting which COVID-19 molecular and serology tests they want to purchase and run in their labs. Complicating such decisions is the fact that many of the nation’s most reputable in vitro diagnostics manufacturers cannot produce enough of their COVID-19 tests to meet demand.
Consequently, when looking to purchase tests for SARS-CoV-2, smaller medical laboratory organizations find themselves evaluating COVID-19 kits developed by little-known or even brand-new companies.
Just eleven common tests and procedures blamed for 93% of low-value services and 89% of wasted spending
Overuse of medical laboratory tests and diagnostic procedures has been a long-standing issue among pathologists and other healthcare providers. Now a Washington State healthcare watchdog organization has put a $282-million price tag on the cost of what it calls “wasteful” spending for “low-value” clinical laboratory tests and other procedures in a single year.
A study by the nonprofit Washington Health Alliance (WHA) examined insurance claims between July 2015 and June 2016 from 1.3 million patients who received one of 47 procedures or tests that had been previously labeled by the US Preventive Services Task Force and Choosing Wisely campaign as overused.
Many pathologists and clinical laboratory managers are familiar with the “Choosing Wisely” initiative. This is a collaboration involving most medical specialty associations. These associations are highlighting a list of medical lab tests and other diagnostic procedures that are ordered inappropriately and with greatest frequency. Thus, the WHA study—involving 1.3 million patients—provides more evidence and credibility in support of the “Choosing Wisely” campaign.
In its study, the WHA determined that, over this 12-month period:
More than 45% of the healthcare services examined were determined to be of low value because they have been shown to provide little benefit in certain clinical scenarios;
622,341 (47.9%) patients underwent a low-value test or procedure they didn’t need;
36% of spending went to low-value services, resulting in an estimated $282 million in wasteful spending; and,
Eleven common tests, procedures, and treatments—such as: preoperative tests, laboratory studies prior to surgery, and too frequent cancer screenings—account for 93% of low-value services and 89% of the estimated wasted spending.
The WHA found that problematic procedures and tests aren’t necessarily “big-ticket” services but collectively result in unnecessary costs.
“Many of the services are individually low cost, and, therefore, a doctor or patient may not consider it problematic,” Susie Dade, author of the WHA report and Deputy Director at the Washington Health Alliance, told The News Tribune.
“In the alliance’s analysis, we found that about 80% of the low-value services examined for this report are low cost (meaning less than about $500). However, we all pay the price with increased premiums and healthcare costs,” she noted. Precision medicine approaches to personalized healthcare reduces these costs and improves outcomes.
Population-based screening for 25(OH)-D deficiency;
Imaging for uncomplicated low back pain in the first six weeks;
Preoperative EKG, chest x-ray, and pulmonary function testing prior to low-risk surgery;
Cardiac stress testing; and,
Imaging for uncomplicated headache.
Nancy Giunto (above left), Executive Director of the Washington Health Alliance, called her state’s results “stunning” and noted the WHA report provides “a clear opportunity to educate patients and engage healthcare stakeholders on areas of improvement.” Such an individualized approach to healthcare is at the heart of precision medicine. She’s shown above with previous WHA Executive Directors Mary McWilliams and Margaret Stanley. (Photo copyright: Washington Health Alliance.)
US Healthcare Culture Partly to Blame
“The list of 11 is a starting point for us,” Dade told Modern Healthcare. “I think they will become a rallying point for coming up with specific education and specific interventions.”
Dade suggests the healthcare culture in the US plays a role in the epidemic of unnecessary testing.
“In some cases patients ask for or agree to things because they don’t realize the potential for harm, be it physical, emotional, or financial, that can happen with unnecessary tests, procedures or medications,” Dade told The News Tribune. “In other cases, tests are ordered by providers, with little input or even awareness of the patient.
“In a ‘more is better’ culture in healthcare, there is a belief that it’s better to have additional tests because it’s better to be safe than sorry. However, this doesn’t account for the potential for different kinds of harm.”
“The medical system is still dominated by a payment system that pays providers for doing tests and procedures. Incentives matter. As long as people are paid more to do more they will tend to do too much,” Welch said.
Value-based Care the Solution to Overuse
The Washington Healthcare Alliance believes one key to eliminating overuse can be found in the transition from fee-for-service healthcare to value-based reimbursement models. The WHA report states: “We need to keep our collective ‘foot on the gas’ to transition from paying for volume to paying for value in healthcare.” In addition, the alliance suggests value-based provider contacts should include measures of overuse, and not just measures of access and underuse.
The report also suggests consumers take a more active role in their healthcare decision making by asking these five questions before having a medical test or procedure:
Do I really need this test or procedure?
What are the risks and side effects?
Are there simpler, safer options?
What happens if I don’t do anything?
How much does it cost, and will my insurance pay for it?
Clinical laboratories and anatomic pathology groups can help put the brakes on unnecessary laboratory testing and procedures by guiding physicians toward evidence-based medical care and a renewed focus on “do no harm.”
“What we strive for is substantially reducing the risk of preventable harm,” the report states. “Reducing unnecessary overuse of healthcare services is one important way to do this. The result of the ‘more is always better’ culture present in today’s healthcare delivery seems to be ‘first, do something.’ It is time to get back to ‘first, do no harm.”
