News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel

News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel
Sign In

Multiple Studies Raise Questions About Reliability of Clinical Laboratory COVID-19 Diagnostic Tests

In the absence of a “gold standard,” researchers are finding a high frequency of false negatives among SARS-CoV-2 RT-PCR tests

Serology tests designed to detect antibodies to the SARS-CoV-2 coronavirus that causes the COVID-19 illness have been dogged by well-publicized questions about accuracy. However, researchers also are raising concerns about the accuracy of molecular diagnostics which claim to detect the actual presence of the coronavirus itself.

“Diagnostic tests, typically involving a nasopharyngeal swab, can be inaccurate in two ways,” said Steven Woloshin, MD, MS, in a news release announcing a new report that “examines challenges and implications of false-negative COVID-19 tests.” Woloshin is an internist, a professor at Dartmouth Institute, and co-director of the Geisel School of Medicine at Dartmouth.

“A false-positive result mistakenly labels a person infected, with consequences including unnecessary quarantine and contact tracing,” he stated in the news release. “False-negative results are far more consequential, because infected persons who might be asymptomatic may not be isolated and can infect others.”

Woloshin led a team of Dartmouth researchers who analyzed two studies from Wuhan, China, and a literature review by researchers in Europe and South America that indicated diagnostic tests for COVID-19 are frequently generating false negatives. The team published their results in the June 5 New England Journal of Medicine (NEJM).

For example, one research team in Wuhan collected samples from 213 hospitalized COVID-19 patients and found that an approved RT-PCR test produced false negatives in 11% of sputum samples, 27% of nasal samples, and 40% of throat samples. Their research was published on the medRxiv preprint server and has not been peer-reviewed.

The literature review Woloshin’s team studied was also published on medRxiv, titled, “False-Negative Results of Initial Rt-PCR Assays for COVID-19: A Systematic Review.” It indicated that the rate of false negatives could be as high as 29%. The authors of the review looked at five studies that had enrolled a total of 957 patients. “The collected evidence has several limitations, including risk of bias issues, high heterogeneity, and concerns about its applicability,” they wrote. “Nonetheless, our findings reinforce the need for repeated testing in patients with suspicion of SARS-Cov-2 infection.”

Another literature review, published in the Annals of Internal Medicine, titled, “Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction–Based SARS-CoV-2 Tests by Time Since Exposure,” estimated the probability of false negatives in RT-PCR tests at varying intervals from the time of exposure and symptom onset. For example, the authors found that the median false-negative rate was 38% if a test was performed on the day of symptom onset, versus 20% three days after onset. Their analysis was based on seven studies, five of which were peer-reviewed, with a total of 1330 test samples.

Doctors also are seeing anecdotal evidence of false negatives. For example, clinicians at UC San Diego Health medical center treated a patient with obvious symptoms of COVID-19, but two tests performed on throat samples were negative. However, a third test, using a sample from a bronchial wash, identified the virus, reported Medscape.

The lesson for clinicians is that they can’t rely solely on test results but must also consider their own observations of the patient, Joshua Metlay, MD, PhD, of Massachusetts General Hospital told Medscape.

Sensitivity and Specificity of COVID-19 Clinical Laboratory Tests

The key measures of test accuracy are sensitivity, which refers to the ability to detect the presence of the virus, and specificity, the ability to determine that the targeted pathogen is not present. “So, a sensitive test is less likely to provide a false-negative result and a specific test is less likely to provide a false-positive result,” wrote Kirsten Meek, PhD, medical writer and editor, in an article for ARUP Laboratories.

“Analytic” sensitivity and specificity “represent the accuracy of a test under ideal conditions in which specimens have been collected from patients with either high viral loads or a complete absence of exposure,” she wrote. However, “sensitivity and specificity under real-world conditions, in which patients are more variable and specimen collection may not be ideal, can often be lower than reported numbers.”

In a statement defending its ID Now molecular point-of-care test, which came under scrutiny during a study of COVID-19 molecular tests by NYU Langone Health, Northwell Health, and Cleveland Clinic, according to MedTech Dive, Abbott Laboratories blamed improper sample collection and handling for highly-publicized false negatives produced by its rapid test. An FDA issued alert about the test on May 14 noted that Abbott had agreed to conduct post-market studies to identify the cause of the false negatives and suggest remedial actions.

Issues with Emergency Use Authorizations

In their NEJM analysis, Woloshin et al point to issues with the FDA’s process for issuing Emergency Use Authorizations (EUAs). For example, they noted variations in how manufacturers are conducting clinical evaluations to determine test performance. “The FDA prefers the use of ‘natural clinical specimens’ but has permitted the use of ‘contrived specimens’ produced by adding viral RNA or inactivated virus to leftover clinical material,” they wrote.

When evaluating clinical performance, manufacturers ordinarily conduct an index test of patients and compare the results with reference-standard test, according to the Dartmouth researchers. For people showing symptoms, the reference standard should be a clinical diagnosis performed by an independent adjudication panel. However, they wrote, “it is unclear whether the sensitivity of any FDA-authorized commercial test has been assessed in this way.” Additionally, a reference standard for determining sensitivity in asymptomatic people “is an unsolved problem that needs urgent attention to increase confidence in test results for contact-tracing or screening purposes.”

Stephen Rawlings, MD, PhD
“To truly determine false negatives, you need a gold standard test, which is essentially as close to perfect as we can get,” Stephen Rawlings, MD, PhD, (above), a resident physician of internal medicine and infectious diseases fellow at UC San Diego’s Center for AIDS Research (CFAR), who has been working on SARS-CoV-2 test validation since March. “But there just isn’t one yet for coronavirus,” he told Medscape. (Photo copyright: University of California, San Diego.)

In a perspective for Mayo Clinic Proceedings, Colin P. West, MD, PhD; Victor M. Montori, MD, MSc; and Priya Sampathkumar, MD, offered four recommendations for addressing concerns about testing accuracy:

  • Continued adherence to current measures, such as physical distancing and surface disinfection.
  • Development of highly sensitive and specific tests or combinations of tests to minimize the risk of false-negative results and ongoing transmission based on a false sense of security.
  • Improved RT-PCR tests and serological assays.
  • Development and communication of clear risk-stratified protocols for management of negative COVID-19 test results.

“These protocols must evolve as diagnostic test, transmission, and outcome statistics become more available,” they wrote.

Meanwhile, clinical laboratories remain somewhat on their own at selecting which COVID-19 molecular and serology tests they want to purchase and run in their labs. Complicating such decisions is the fact that many of the nation’s most reputable in vitro diagnostics manufacturers cannot produce enough of their COVID-19 tests to meet demand.

Consequently, when looking to purchase tests for SARS-CoV-2, smaller medical laboratory organizations find themselves evaluating COVID-19 kits developed by little-known or even brand-new companies.

—Stephen Beale

Related Information:

New Report Examines Challenges and Implications of False-Negative COVID-19 Tests

Questions about COVID-19 Test Accuracy Raised Across the Testing Spectrum

COVID-19 Test Results: Don’t Discount Clinical Intuition

FDA Provides New Tool to Aid Development and Evaluation of Diagnostic Tests That Detect SARS-CoV-2 Infection

EUA Authorized Serology Test Performance

Emergency Use Authorization (EUA) Information and List of All Current EUAs 

Coronavirus (COVID-19) Update: FDA Provides Promised Transparency for Antibody Tests

Understanding Medical Tests: Sensitivity, Specificity, and Positive Predictive Value

Webinar Part 1: Quality Issues Your Clinical Laboratory Should Know Before You Buy or Select COVID-19 Serology Tests

Webinar Part 2: Achieving High Confidence Levels in the Quality and Accuracy of Your Clinical Lab’s Chosen COVID-19 Serology Tests, featuring James Westgard, PhD

‘Low-Value’ Medical Lab Tests and Other Overused Medical Procedures Led to $282-Million in ‘Wasted’ Healthcare Spending in Washington State in One Year, Washington Health Alliance Reports

Just eleven common tests and procedures blamed for 93% of low-value services and 89% of wasted spending

Overuse of medical laboratory tests and diagnostic procedures has been a long-standing issue among pathologists and other healthcare providers. Now a Washington State healthcare watchdog organization has put a $282-million price tag on the cost of what it calls “wasteful” spending for “low-value” clinical laboratory tests and other procedures in a single year.

A study by the nonprofit Washington Health Alliance (WHA) examined insurance claims between July 2015 and June 2016 from 1.3 million patients who received one of 47 procedures or tests that had been previously labeled by the US Preventive Services Task Force and Choosing Wisely campaign as overused.

Many pathologists and clinical laboratory managers are familiar with the “Choosing Wisely” initiative. This is a collaboration involving most medical specialty associations. These associations are highlighting a list of medical lab tests and other diagnostic procedures that are ordered inappropriately and with greatest frequency. Thus, the WHA study—involving 1.3 million patients—provides more evidence and credibility in support of the “Choosing Wisely” campaign.

In its study, the WHA determined that, over this 12-month period:

  • More than 45% of the healthcare services examined were determined to be of low value because they have been shown to provide little benefit in certain clinical scenarios;
  • 622,341 (47.9%) patients underwent a low-value test or procedure they didn’t need;
  • 36% of spending went to low-value services, resulting in an estimated $282 million in wasteful spending; and,
  • Eleven common tests, procedures, and treatments—such as: preoperative tests, laboratory studies prior to surgery, and too frequent cancer screenings—account for 93% of low-value services and 89% of the estimated wasted spending.

The WHA found that problematic procedures and tests aren’t necessarily “big-ticket” services but collectively result in unnecessary costs.

“Many of the services are individually low cost, and, therefore, a doctor or patient may not consider it problematic,” Susie Dade, author of the WHA report and Deputy Director at the Washington Health Alliance, told The News Tribune.

“In the alliance’s analysis, we found that about 80% of the low-value services examined for this report are low cost (meaning less than about $500). However, we all pay the price with increased premiums and healthcare costs,” she noted. Precision medicine approaches to personalized healthcare reduces these costs and improves outcomes.

Opportunity for Education/Improvement

The report, “First, Do No Harm: Calculating Health Care Waste in Washington State,” used the new MedInsight Health Waste Calculator from actuarial consulting firm Milliman, to produce the analysis.

The report found the following 11 tests and procedures (listed in descending order based on volume) were the most overused low-value services:

  1. Too frequent cervical cancer screening in women;
  2. Preoperative baseline laboratory studies before low-risk surgery;
  3. Unnecessary imaging for eye disease;
  4. Annual EKGs or cardiac screening in low-risk, asymptomatic individuals;
  5. Prescribing antibiotics for acute upper respiratory and ear infections;
  6. PSA (prostate specific antigen) screening;
  7. Population-based screening for 25(OH)-D deficiency;
  8. Imaging for uncomplicated low back pain in the first six weeks;
  9. Preoperative EKG, chest x-ray, and pulmonary function testing prior to low-risk surgery;
  10. Cardiac stress testing; and,
  11. Imaging for uncomplicated headache.


Nancy Giunto (above left), Executive Director of the Washington Health Alliance, called her state’s results “stunning” and noted the WHA report provides “a clear opportunity to educate patients and engage healthcare stakeholders on areas of improvement.” Such an individualized approach to healthcare is at the heart of precision medicine. She’s shown above with previous WHA Executive Directors Mary McWilliams and Margaret Stanley. (Photo copyright: Washington Health Alliance.)

US Healthcare Culture Partly to Blame

“The list of 11 is a starting point for us,” Dade told Modern Healthcare. “I think they will become a rallying point for coming up with specific education and specific interventions.”

Dade suggests the healthcare culture in the US plays a role in the epidemic of unnecessary testing.

“In some cases patients ask for or agree to things because they don’t realize the potential for harm, be it physical, emotional, or financial, that can happen with unnecessary tests, procedures or medications,” Dade told The News Tribune. “In other cases, tests are ordered by providers, with little input or even awareness of the patient.

“In a ‘more is better’ culture in healthcare, there is a belief that it’s better to have additional tests because it’s better to be safe than sorry. However, this doesn’t account for the potential for different kinds of harm.”

  1. Gilbert Welch, MD, MPH, a professor at The Dartmouth Institute, told National Public Radio (NPR) part of the blame should be placed on a healthcare system that often incentivizes healthcare providers to do more than what is medically necessary.

“The medical system is still dominated by a payment system that pays providers for doing tests and procedures. Incentives matter. As long as people are paid more to do more they will tend to do too much,” Welch said.

Value-based Care the Solution to Overuse

The Washington Healthcare Alliance believes one key to eliminating overuse can be found in the transition from fee-for-service healthcare to value-based reimbursement models. The WHA report states: “We need to keep our collective ‘foot on the gas’ to transition from paying for volume to paying for value in healthcare.” In addition, the alliance suggests value-based provider contacts should include measures of overuse, and not just measures of access and underuse.

The report also suggests consumers take a more active role in their healthcare decision making by asking these five questions before having a medical test or procedure:

  1. Do I really need this test or procedure?
  2. What are the risks and side effects?
  3. Are there simpler, safer options?
  4. What happens if I don’t do anything?
  5. How much does it cost, and will my insurance pay for it?

Clinical laboratories and anatomic pathology groups can help put the brakes on unnecessary laboratory testing and procedures by guiding physicians toward evidence-based medical care and a renewed focus on “do no harm.”

“What we strive for is substantially reducing the risk of preventable harm,” the report states. “Reducing unnecessary overuse of healthcare services is one important way to do this. The result of the ‘more is always better’ culture present in today’s healthcare delivery seems to be ‘first, do something.’ It is time to get back to ‘first, do no harm.”

—Andrea Downing Peck

Related Information:

First, Do No Harm: Calculating Health Care Waste in Washington State

New Study Finds Hundreds of Thousands of Washington Patients Receive Unnecessary Tests, Procedures and Treatments

Washington Residents’ Tab for Unneeded Care in a Year: $280 million-plus

Unnecessary Medical Care: More Common than You Might Imagine

You’re Getting Nickel and Dimed for Low-Value Medical Tests in Washington, Report Says