In the absence of a “gold standard,” researchers are finding a high frequency of false negatives among SARS-CoV-2 RT-PCR tests
Serology tests designed to detect antibodies to the SARS-CoV-2 coronavirus that causes the COVID-19 illness have been dogged by well-publicized questions about accuracy. However, researchers also are raising concerns about the accuracy of molecular diagnostics which claim to detect the actual presence of the coronavirus itself.
“Diagnostic tests, typically involving a nasopharyngeal swab, can be inaccurate in two ways,” said Steven Woloshin, MD, MS, in a news release announcing a new report that “examines challenges and implications of false-negative COVID-19 tests.” Woloshin is an internist, a professor at Dartmouth Institute, and co-director of the Geisel School of Medicine at Dartmouth.
“A false-positive result mistakenly labels a person infected, with consequences including unnecessary quarantine and contact tracing,” he stated in the news release. “False-negative results are far more consequential, because infected persons who might be asymptomatic may not be isolated and can infect others.”
Woloshin led a team of Dartmouth researchers who analyzed two studies from Wuhan, China, and a literature review by researchers in Europe and South America that indicated diagnostic tests for COVID-19 are frequently generating false negatives. The team published their results in the June 5 New England Journal of Medicine (NEJM).
For example, one research team in Wuhan collected samples from 213 hospitalized COVID-19 patients and found that an approved RT-PCR test produced false negatives in 11% of sputum samples, 27% of nasal samples, and 40% of throat samples. Their research was published on the medRxiv preprint server and has not been peer-reviewed.
The literature review Woloshin’s team studied was also published on medRxiv, titled, “False-Negative Results of Initial Rt-PCR Assays for COVID-19: A Systematic Review.” It indicated that the rate of false negatives could be as high as 29%. The authors of the review looked at five studies that had enrolled a total of 957 patients. “The collected evidence has several limitations, including risk of bias issues, high heterogeneity, and concerns about its applicability,” they wrote. “Nonetheless, our findings reinforce the need for repeated testing in patients with suspicion of SARS-Cov-2 infection.”
Another literature review, published in the Annals of Internal Medicine, titled, “Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction–Based SARS-CoV-2 Tests by Time Since Exposure,” estimated the probability of false negatives in RT-PCR tests at varying intervals from the time of exposure and symptom onset. For example, the authors found that the median false-negative rate was 38% if a test was performed on the day of symptom onset, versus 20% three days after onset. Their analysis was based on seven studies, five of which were peer-reviewed, with a total of 1330 test samples.
Doctors also are seeing anecdotal evidence of false negatives. For example, clinicians at UC San Diego Health medical center treated a patient with obvious symptoms of COVID-19, but two tests performed on throat samples were negative. However, a third test, using a sample from a bronchial wash, identified the virus, reported Medscape.
Sensitivity and Specificity of COVID-19 Clinical Laboratory Tests
The key measures of test accuracy are sensitivity, which refers to the ability to detect the presence of the virus, and specificity, the ability to determine that the targeted pathogen is not present. “So, a sensitive test is less likely to provide a false-negative result and a specific test is less likely to provide a false-positive result,” wrote Kirsten Meek, PhD, medical writer and editor, in an article for ARUP Laboratories.
“Analytic” sensitivity and specificity “represent the accuracy of a test under ideal conditions in which specimens have been collected from patients with either high viral loads or a complete absence of exposure,” she wrote. However, “sensitivity and specificity under real-world conditions, in which patients are more variable and specimen collection may not be ideal, can often be lower than reported numbers.”
In a statement defending its ID Now molecular point-of-care test, which came under scrutiny during a study of COVID-19 molecular tests by NYU Langone Health, Northwell Health, and Cleveland Clinic, according to MedTech Dive, Abbott Laboratories blamed improper sample collection and handling for highly-publicized false negatives produced by its rapid test. An FDA issued alert about the test on May 14 noted that Abbott had agreed to conduct post-market studies to identify the cause of the false negatives and suggest remedial actions.
Issues with Emergency Use Authorizations
In their NEJM analysis, Woloshin et al point to issues with the FDA’s process for issuing Emergency Use Authorizations (EUAs). For example, they noted variations in how manufacturers are conducting clinical evaluations to determine test performance. “The FDA prefers the use of ‘natural clinical specimens’ but has permitted the use of ‘contrived specimens’ produced by adding viral RNA or inactivated virus to leftover clinical material,” they wrote.
When evaluating clinical performance, manufacturers ordinarily conduct an index test of patients and compare the results with reference-standard test, according to the Dartmouth researchers. For people showing symptoms, the reference standard should be a clinical diagnosis performed by an independent adjudication panel. However, they wrote, “it is unclear whether the sensitivity of any FDA-authorized commercial test has been assessed in this way.” Additionally, a reference standard for determining sensitivity in asymptomatic people “is an unsolved problem that needs urgent attention to increase confidence in test results for contact-tracing or screening purposes.”
“To truly determine false negatives, you need a gold standard test, which is essentially as close to perfect as we can get,” Stephen Rawlings, MD, PhD, (above), a resident physician of internal medicine and infectious diseases fellow at UC San Diego’s Center for AIDS Research (CFAR), who has been working on SARS-CoV-2 test validation since March. “But there just isn’t one yet for coronavirus,” he told Medscape. (Photo copyright: University of California, San Diego.)
Continued adherence to current measures, such as physical distancing and surface disinfection.
Development of highly sensitive and specific tests or combinations of tests to minimize the risk of false-negative results and ongoing transmission based on a false sense of security.
Improved RT-PCR tests and serological assays.
Development and communication of clear risk-stratified protocols for management of negative COVID-19 test results.
“These protocols must evolve as diagnostic test, transmission, and outcome statistics become more available,” they wrote.
Meanwhile, clinical laboratories remain somewhat on their own at selecting which COVID-19 molecular and serology tests they want to purchase and run in their labs. Complicating such decisions is the fact that many of the nation’s most reputable in vitro diagnostics manufacturers cannot produce enough of their COVID-19 tests to meet demand.
Consequently, when looking to purchase tests for SARS-CoV-2, smaller medical laboratory organizations find themselves evaluating COVID-19 kits developed by little-known or even brand-new companies.
Pathologists might be able to help overburdened doctors by adding medical laboratory support services that assist providers in selecting the right tests and identifying the best therapeutic options for patients
In a new Stanford University School of Medicine study published in the July 9, 2018, issue of Mayo Clinic Proceedings, researchers indicate that physician burnout may be as big a cause of medical errors as unsafe healthcare environments. This highlights an opportunity for clinical laboratory professionals and pathologists to help physicians improve both diagnostic accuracy and the selection of the most appropriate therapies.
The study found that exhausted providers were twice as likely to report making a medical error. However, it’s a complex problem with no easy solutions.
“Just trying to fix the setting of healthcare environments in order to prevent errors is not sufficient,” Stanford University’s Daniel Tawfik, MD, MS, the study’s lead author, told Reuters Health. “We also need to address the actual underlying human factors that contribute to errors—specifically looking at physician burnout.”
Nevertheless, while there is no one-size-fits-all solution to physician burnout, clinical laboratory managers and pathologists potentially could help overburdened providers reduce burnout and fatigue by adopting new lab testing support services designed to assist physicians in selecting the right tests and identifying the best therapeutic options for their patients.
Medical Errors Third-Leading Cause of Death in America
Stanford researchers wanted to learn how physician burnout contributes to medical errors which, according to Johns Hopkins, is the third-leading cause of death in the US. They surveyed 6,695 physicians from across America. Of the respondents:
More than 54% reported symptoms of burnout;
33% experienced excessive fatigue;
Nearly 7% had thoughts of suicide; and,
Roughly 4% reported a failing safety grade in their primary work area.
Even in medical units judged to have excellent safety records, the study found rates of medical errors nearly tripled when physicians working in those units had high levels of burnout. The prevalence of errors became similar to a non-burned-out physician working in a unit with a safety grade of “acceptable” or “poor.”
“We found that physicians with burnout had more than twice the odds of self-reported medical error, after adjusting for specialty, work hours, fatigue, and work unit safety rating,” Tawfik noted in a Stanford news release. “We also found that low safety grades in work units were associated with three to four times the odds of medical errors.”
According to the study, overall, 10.5% of physicians surveyed acknowledged in the prior three months making:
An error in judgment;
A wrong diagnosis;
A technical mistake during a procedure;
Prescribing a wrong drug/dosage; and/or,
Ordering medication/intervention for the wrong patient.
While more than half of mistakes (55.4%) did not affect patient outcomes, or only caused a temporary problem (22.6%), more than 5% of errors did lead to major permanent health problems and 4.5% resulted in a patient death, the study found.
Radiologists, neurosurgeons, and emergency medicine specialists had the highest prevalence of error rates, with more than 21% of providers in each of those fields acknowledging recent mistakes.
Physicians reporting errors were more likely to have symptoms of overall burnout (77.6% versus 51.5%), as well as fatigue (46.6% versus 31.2%), than error-free providers. Physicians reporting recent errors also had a higher prevalence of suicidal thoughts (12.7% versus 5.8%), the study found.
Ted Hole, MD, a family practice physician in Ventura, Calif., is not surprised by the correlation between medical mistakes and overall well-being. “If your brain isn’t working right, you’re going to make errors,” Hole told the Ventura County Star. “That’s what burnout does. It makes your brain not work right.”
Stanford Connects Physician Burnout and Poor Workspace Safety Ratings
In their paper, the Stanford researchers argue a “combination of physician-targeted burnout interventions and unit-targeted patient improvement measures” are needed to tackle the problem of medical errors. Physicians who gave their work units an excellent, very good, or acceptable safety grade were less likely to make a medical error than those who described workplace safety as poor or failing.
Of the physicians who reported a poor or failing work unit safety grade, nearly 25% reported a recent error. Errors were incrementally lower for work units with higher safety grades regardless of physician burnout levels.
“This indicates both the burnout level as well as work unit safety characteristics are independently related to the risk of errors,” Tait Shanafelt, MD, Director of the Stanford WellMD Center and Associate Dean of the School of Medicine, noted in a Stanford statement.
“Today, most organizations invest substantial resources and have a system-level approach to improve safety on every work unit,” he said in the Stanford news release. “We need a holistic and systems-based approach to address the epidemic of burnout among healthcare providers if we are truly going to create the high-quality healthcare system we aspire to.”
Tait Shanafelt, MD (above), is Director of the Stanford WellMD Center, Associate Dean of the School of Medicine, and an author of the Stanford study. He maintains the “epidemic of burnout” among healthcare providers should receive as much attention as safety issues. Shanafelt became Stanford Medicine’s first Chief Wellness Officer in 2017. (Photo copyright: Stanford School of Medicine.)
Burnout Among Physicians Increasing
Other studies, including Medscape’s “Lifestyle Report 2017: Race and Ethnicity, Bias and Burnout,” confirm an upward trend in burnout rates among US physicians. In the Medscape study, 51% of physicians surveyed reporting being “burned out,” defined as a loss of enthusiasm for work, feelings of cynicism, and a low sense of personal accomplishment. Since the Medscape Lifestyle Report first queried physicians about burnout in 2013, the number of providers reporting burnout has increased 25%.
Physician burnout has been attributed to a variety of factors, including:
Excessive workloads;
Financial stress;
Extra hours spent on clerical work or EHR-related tasks; and,
Loss of human-to-human interaction between physician and patient.
Robert Lum, MD, an Oxnard, Calif.-based radiation oncologist, blames the shift to corporate-owned medical practices for some of the reported increases in burnout among physicians. Lum told the Ventura County Star he stays upbeat by never losing sight of why he became a physician.
“If you focus on the reason you went into medicine in the first place, which is to help people and marvel at the miracles modern medicine is able to do, then you’ll have less burnout,” he said.
Nevertheless, other solutions also can help. Clinical laboratories play a key role in maximizing physician/patient encounters. By extension, physicians and laboratories are linked in unique ways that enable labs to reduce physician burden and ensure positive healthcare outcomes.
No clear trend in pathology industry as to the demand for services versus the supply of pathologists
Healthcare providers often describe themselves at being part of a family of caregivers, and right now, one member, the physician, is under extreme pressure. So much so, experts are calling on other members of the family, such as pathology groups and clinical laboratories, to seek ways to proactively ease the burdens on America’s doctors.
A survey by the Physicians Foundation reveals that 80% of physicians reported being overextended or at capacity, with no time to see additional patients. And the number of physicians considering an early exit from patient-centered roles is increasing. If allowed to continue, such a trend will negatively impact patients’ access to care. (more…)
