Group’s report also suggests that at-home clinical laboratory tests for COVID-19 that are difficult to use may lead to inaccurate results
At-home clinical laboratory tests for COVID-19 have become quite popular. But how accurate are they? Now, an independent safety organization has investigated COVID-19 rapid antigen tests to find out how easy—or not—they are to use and what that means for the accuracy of the tests’ results.
ECRI (Emergency Care Research Institute) of Plymouth Meeting, Penn., “conducted a usability evaluation to determine if there were any differences in ease of use for the rapid COVID-19 tests,” according to the company’s website.The nonprofit was founded in the 1960s by surgeon and inventor Joel J. Nobel to evaluate medical devices that have been approved by the U.S. Food and Drug Administration (FDA).
“Because of the urgency in providing useful information to consumers as quickly as possible, ECRI selected the seven test kits based on retail availability,” ECRI noted.
ECRI ranked the seven over-the-counter (OTC) at-home rapid antigen tests according to their SUS usability ratings. The System Usability Scale (SUS), invented by John Brooke in 1986, “rates products on a scale of 0 to 100 with 100 being the easiest to use. More than 30 points separated the top and bottom tests analyzed,” according to Managed Healthcare Executive.
Of the seven rapid antigen test kits for COVID-19, ECRI found “noteworthy usability concerns” and “significant differences in ease of use.” None of the tests achieved a SUS rating of “excellent,” ECRI stated in a press release.
Seven Rapid Antigen Tests for SARS-CoV-2 Evaluated
As clinical laboratory scientists and pathologists know, it’s possible for different test methodologies for the same biomarker to produce dissimilar results. Another factor affecting medical laboratory test accuracy is the variability from one manufacturing batch or lot to another. And, as the ECRI report suggests, how a specimen is collected and handled can affect accuracy, reliability, and reproducibility of the test results generated by that specimen.
These are the OTC COVID-19 rapid antigen tests ECRI evaluated and their SUS ratings:
Some tests, the ECRI analysts found, required “fine motor control” or were packed with written instructions ECRI determined were too small for older adults to read.
How ECRI Evaluated the COVID-19 Rapid Antigen Tests
SUS reviewers took each rapid test and completed questionnaires specifying their level of agreement (on a range of one to five) with these statements. (Edited by Dark Daily for space):
Desire to use
Perception of unnecessary complexity
Easy to use
Support of a technical person needed
Too much system inconsistency
Easy to learn for most people
A very cumbersome system to use
Feeling of confidence in use
A need to learn before getting going
ECRI then used an algorithm to derive an aggregate score (from 0 to 100) for each test, the report noted.
“Based on the aggregate SUS scores, none of the COVID-19 test kits would be judged to have ‘excellent’ usability. The On/Go, CareStart, Flowflex test kits we rate as ‘very good’ as the usability score for these kits falls just short of ‘excellent,’” the report said.
Some of the positive responses ECRI received from the SUS participants included:
“One of the simpler tests to use with good, printed instructions,” (On/Go and CareStart).
“Cassette makes handling without touching test strip easy,” (CareStart and Flowflex).
“I think the rapid tests provide some of the best protection we have against the spread of disease, especially as we now have a variant on hand that’s going to be able to cause an awful lot of breakthrough infections,” Kissler told The Atlantic-Journal Constitution.
One way clinical laboratory leaders can help is to reach out in their local markets and provide information on the importance of appropriate sampling and collection for accurate results from rapid COVID-19 antigen testing.
Developers of medical laboratory tests had high hopes that cheap saliva-based tests would compete with at-home OTC tests that use nasal swabs, but skepticism among scientists continues
Reverse-transcription polymerase chain reaction (RT-PCR) technology has become the standard for clinical laboratory diagnostic testing used to detect the presence of the SARS-CoV-2 coronavirus. However, to enable more widespread testing, some public health experts have called for deployment of cheap, rapid, saliva-based antigen tests that could be self-administered by consumers in their homes.
Despite the technology’s lower sensitivity compared with RT-PCR testing, the idea of “fast-and-frequent” universal antigen testing has gained support as a possible game-changer against the outbreak, the New York Times reported.
In a column he wrote for Time in July, Ashish K. Jha, MD, MPH, a practicing General Internist and Dean of the Brown University School of Public Health, described the promise of rapid antigen tests. “Imagine spitting on a special strip of paper every morning and being told two minutes later whether you were positive for COVID-19,” he wrote. “If everyone in the United States did this daily, we would dramatically drop our transmission rates and bring the pandemic under control.”
However, one major issue with antigen testing is sensitivity. “Antigen tests require higher levels of virus than qPCR [quantitative polymerase chain reaction] to return a positive result,” Jha wrote in Time. However, he contends, “the frequency of testing and the speed of results” counter concerns about accuracy.
Even with lower sensitivity, Jha wrote, the quicker test results from antigen tests “would identify viral loads during the most infectious period, meaning those cases we care most about identifying—at the peak period of infectiousness—are less likely to be missed.”
As the FDA explains, RT-PCR molecular tests “detect the virus’ genetic material,” whereas, according to an article published in Nature, titled, “Fast Coronavirus Tests: What They Can and Can’t Do,” antigen tests can “detect specific proteins … on the surface of the virus, and can identify people who are at the peak of infection, when virus levels in the body are likely to be high.”
At-Home Antigen Tests Receive EUAs
The new antigen test developed by Ellume is “the first over-the-counter (OTC) fully at-home diagnostic test for COVID-19,” the FDA said in a press release. The user self-administers a nasal swab and places it in an analyzer connected to a smartphone app. It can deliver results in 20 minutes. The company states that its test has overall sensitivity of 95% and specificity of 97% based on a clinical study of 198 subjects in a simulated home setting.
Jeffrey Shuren, MD, JD, Director of FDA’s Center for Devices and Radiological Health, said in the FDA press release, “This test, like other antigen tests, is less sensitive and less specific than typical molecular tests run in a lab. However, the fact that it can be used completely at home and return results quickly means that it can play an important role in response to the pandemic.”
Ellume expects to deliver about 20 million tests to the US by the end of June 2021. Multiple outlets reported that the test will cost about $30, AP News reported.
Meanwhile, the FDA also authorized at-home use of Abbott’s BinaxNOW rapid antigen test, which was previously authorized for use in point-of-care settings. This test, which requires a prescription, will sell for $25.
In a series of tweets, Harvard’s Mina applauded both moves, but he wrote that they [antigen tests] still fall short of his vision for fast and frequent testing. He described Abbott’s BinaxNOW as “the type of rapid test I have been calling for,” but said he’d like to see tests priced far less and available without a prescription.
Diminishing Prospects for Saliva-based Antigen Tests?
All rapid antigen tests authorized by the FDA so far require nasopharyngeal and/or nasal swab specimens, and it appears that it may be a long time, if ever, before saliva-based antigen tests are available. The New York Times (NYT) reported in October that two companies working on antigen tests—E25Bio and OraSure (NASDAQ:OSUR)—have dropped plans to enable use of saliva.
One advantage of a saliva-based test is that it would be easier to self-administer. “But as they continued to tinker with their tests, researchers at both E25Bio and OraSure found saliva’s performance to be more lackluster than anticipated, and were forced to pivot,” the New York Times reported. Instead, both companies will seek authorization for use of their tests with nasal swabs.
HHS Contract for Antigen Tests Brings High Rates of False Positives
“With the benefit of hindsight, experts said the Trump administration should have released antigen tests primarily to communities with outbreaks instead of expecting them to work just as well in large groups of asymptomatic people,” ProPublica reported. “Understanding they can produce false results; the government could have ensured that clinics had enough for repeat testing to reduce false negatives and access to more precise PCR tests to weed out false positives.”
A few weeks after the reports from Nevada and Vermont, the FDA issued a letter advising clinical laboratories and healthcare providers about the possibility of false positives, along with steps they could take to improve accuracy.
Though some experts remain hopeful about “fast-and-frequent” testing, others are skeptical and say more research is needed to assess the value of this approach. “We are open to thinking outside the box and coming up with new ways to handle this pandemic,” Esther Babady PhD, D(ABMM) of Memorial Sloan Kettering Cancer Center, told the New York Times. However, she added, “the data for that is what’s missing.”
Nevertheless, were at-home rapid saliva-based antigen tests to become a common choice for healthcare consumers, clinical laboratories that perform RT-PCR testing for COVID-19 could see a marked decrease in orders. Thus, regardless of the current state of antigen testing, its development is worth watching.
Abbott sends the SARS-CoV-2 test results directly to patients’ smartphones, which can be displayed to gain entrance into areas requiring proof of COVID-19 testing
There is no greater example that COVID-19 is a major force for change in the clinical laboratory industry than the fact that—though the US federal government pays 50% of the nation’s total annual healthcare spend of $3.5 trillion—it recently spent $760 million to purchase 150 million COVID-19 tests from Abbott Laboratories (NYSE:ABT), an American multinational medical devices and healthcare company headquartered in Abbott Park, Ill., “to expand strategic, evidence-based testing in the United States,” according to the company’s website.
In August, the federal Food and Drug Administration (FDA) granted an emergency use authorization (EUA) to Abbott for its BinaxNOW portable rapid-response COVID-19 antigen (Ag) test. The credit-card sized test costs $5 and can return clinical laboratory test results in minutes, rather than hours, days, or in some cases, weeks, the Wall Street Journal (WSJ) reported.
The test includes a free smartphone app called NAVICA, which enables those tested to receive their test results directly on their mobile devices—bypassing the patient’s primary care physicians.
According to Abbott’s website, the app “allows people who test negative to get an encrypted temporary digital NAVICA Pass, similar to an airline boarding pass. NAVICA-enabled organizations will be able to verify an individual’s negative COVID-19 test results by scanning the individual’s digital NAVICA Pass to facilitate entry into facilities.”
This feature of Abbott’s new COVID-19 test is a good example of how quickly innovation in the medical laboratory testing profession is bringing new features and new capabilities to the marketplace. By marrying the SARS-CoV-2 test with the NAVICA Pass feature, Abbott hopes to deliver increased value—not just to physicians and their patients—but also to employers with employee screening programs and federal government programs designed to screen federal employees, as well as being used for screening travelers at airports and other transportation hubs.
Abbott appears to be banking that in the future such identification will be required to “enter organizations and other places where people gather,” as the company’s website states.
Testing Limited to CLIA-Certified Clinical Laboratories
An HHS news release announcing the government’s planned distribution of the BinaxNOW tests stated that “Testing will be potentially deployed to schools and to assist with serving other special needs populations.”
In the news release, Alex Azar, HHS Secretary, said, “By strategically distributing 150 million of these tests to where they’re needed most, we can track the virus like never before and protect millions of Americans at risk in especially vulnerable situations.”
Demand for COVID-19 testing has created opportunities for in vitro diagnostics (IVD) companies that can develop and bring tests to market quickly.
Recent issues of Dark Daily’s sister print publication—The Dark Report (TDR)—covered IVD companies’ second quarter (Q2) boom in sales of COVID-19 instruments and tests, while also noting a fall-off in routine clinical laboratory testing during the COVID-19 pandemic.
Abbott Laboratories saw molecular diagnostics sales increase 241% in Q2 driven by $283 million in sales of COVID-19 testing, while rapid diagnostic COVID-19 testing rose 11% on $180 million in sales in Q2, TDR reported, based on Abbott data.
“There is huge economic incentive for diagnostic companies to develop technologies that can be used to create rapid tests that are cheap to perform,” said Robert Michel, Publisher and Editor-in-Chief of TDR and Dark Daily. “In this sense, COVID is a major force for change.”
Thus, Abbott is determined to ensure this product launch is successful and that the test works as promised. According to a news release, “In data submitted to the FDA from a clinical study conducted by Abbott with several leading US research universities, the BinaxNOW COVID-19 Ag Card demonstrated sensitivity of 97.1% (positive percent agreement) and specificity of 98.5% (negative percent agreement) in patients suspected of COVID-19 by their healthcare provider within the first seven days of symptom onset.”
“The massive scale of this test and app will allow tens of millions of people to have access to rapid and reliable testing,” said Joseph Petrosino, PhD, professor and chairman, Molecular Virology and Microbiology, Baylor College of Medicine, in the Abbott news release. “With lab-based tests, you get excellent sensitivity but might have to wait days or longer to get the results. With a rapid antigen test, you get a result right away, getting infectious people off the streets and into quarantine so they don’t spread the virus.”
Abbott has invested hundreds of millions of dollars in two manufacturing facilities where the tests will be made, John Hackett Jr, PhD, an immunologist and Abbott’s Divisional Vice President Applied Research and Technology, and lead scientist on the BinaxNOW project, told The Atlantic.
“Our nation’s frontline healthcare workers and clinical laboratory personnel have been under siege since the onset of this pandemic,” said Charles Chiu, MD, PhD, professor of Laboratory Medicine at University of California, San Francisco, in the Abbott news release. “The availability of rapid testing for COVID-19 will help support overburdened laboratories, accelerate turnaround times, and greatly expand access to people who need it.”
However, other experts are not so sure. In the Atlantic article, Michael Mina MD, PhD, Assistant Professor Epidemiology at Harvard’s T.H. Chan School of Public Health, voiced the need to test both asymptomatic and pre-symptomatic people. “This is the type of [COVID-19] test we have been waiting for—but may not be the test.”
Nevertheless, the federal government’s investment is significant. Abbott plans to start shipping tens of millions of tests in September and produce 50 million tests per month starting in October, Forbes reported.
Shifting Clinical Laboratory Paradigms
BinaxNOW will be performed without doctors’ orders, in a variety of locations, and results go directly to patients’ smartphone—without a pathologist’s interpretation and medical laboratory report. This is new ground and the impact on non-CLIA labs, and on healthcare in general, is yet to be seen.
Clinical laboratory managers will want to monitor the rise of rapid-response tests that can be easily accessed, conducted, and reported on without physician input.
Though some experts claim widespread antibody testing is key to effective public health safety, the WHO warns positive serological tests may not indicate immunity from reinfection or transmission of SARS-CoV-2
It may be the largest program of clinical laboratory testing ever conducted in the United States. Health officials are preparing to undertake large-scale serological surveys (serosurveys) to detect and track previously undetected cases of SARS-CoV-2, the novel coronavirus, that causes the COVID-19 illness.
Microbiologists, epidemiologists, and medical laboratory leaders will be interested in these studies, which are aimed at determining how many adults in the US with no confirmed history of SARS-CoV-2 infection actually possess antibodies to the coronavirus.
Serological screening testing may also enable employers to identify employees who can safely return to their job. And researchers may be able to identify communities and populations that have been most affected by the virus.
Serological Study of COVID-19 Taking Place in Five States
In an interview with Science, Michael Busch, MD, PhD, Senior Vice President, Research and Scientific Affairs of Vitalant (formerly Blood Systems), one of the nation’s oldest and largest nonprofit community blood service providers, and Director of the Vitalant Research Institute, discussed several serological studies in which he is involved. The first study, which he said is being funded by the National Institutes of Health (NIH), is taking place in six metropolitan regions in the US: Seattle, New York City, San Francisco, Los Angeles, Boston, and Minneapolis.
The interesting twist in these studies is that they will test blood samples from people donating blood. In March, participating blood centers in each region started saving 1,000 donor samples per month. Six thousand samples will be assessed monthly for a six-month period using an antibody testing algorithm that enables researchers to monitor how people develop SARS-CoV-2 antibodies over time.
Busch told Science this regional study will evolve into three “national, fully representative serosurveys of the US population using blood donors.” This particular national serosurvey will study 50,000 donations in September and December of 2020 and in November 2021.
“We’re going to be estimating overall antibody prevalence to SARS-CoV-2 within each state, but also map it down within the states to regions and metropolitan urban areas, and look at the differences,” Busch told Science, which called the serosurvey “unprecedented.”
“It’s certainly the largest serosurvey I’ve ever been involved with,” Busch said.
Some of the SARS-CoV-2 serological surveys underway include:
The National Institutes of Health serosurvey involving as many as 10,000 adults in the US who have no confirmed history of infection with SARS-CoV-2, which will analyze blood samples for two types of antibodies—anti-SARS-CoV-2 protein IgG and IgM. Researchers also may perform additional tests to evaluate volunteers’ immune responses to the virus.
A World Health Organization (WHO) coordinated follow-up study to its Solidarity Trial named Solidarity 2, which will “pool data from research groups in different countries to compare rates of infection,” which WHO officials say is ‘critical’ to understanding the true extent of the pandemic and to inform policy, Research Professionals News reported.
In Germany, the Robert Koch Institute, the country’s disease control and prevention agency, is tackling Europe’s first large-scale COVID-19 antibody testing. Its three-phase study will include serological testing on blood from donation centers, followed by testing on blood samples from coronavirus regional hotspots and then the country’s broader population.
But Can Serological Testing Prove Immunity to COVID-19?
The Times of Sweden reported the WHO warned in mid-April that there is no proof recovering from COVID-19 provides immunity.
“There are a lot of countries that are suggesting using rapid diagnostic serological tests to be able to capture what they think will be a measure of immunity,” said Maria Van Kerkhove, PhD, the WHO’s Technical Lead for COVID-19, at a news conference in Geneva, Switzerland, the Times of Sweden reported.
“Right now, we have no evidence that the use of a serological test can show that an individual has immunity or is protected from reinfection,” she said, adding, “These antibody tests will be able to measure that level of seroprevalence—that level of antibodies—but that does not mean that somebody with antibodies [is] immune.”
In addition, the reliability and quality of some serological tests produced in China, as well as some being manufactured in the US, have come into question, the Financial Times reported.
Nevertheless, as serological testing for COVID-19 becomes more widespread, clinical laboratories should plan to play an ever-increasing role in the battle to stop a second wave of the epidemic in this country.
AccuWeather interviewed experts, including pathologists who have analyzed the virus, who say SARS-CoV-2 is susceptible to heat, light, and humidity, while others study weather patterns for their predictions
AccuWeather, as it watched the outbreak of SARS-CoV-2, the novel coronavirus that causes COVID-19, wanted to know what effect that warmer spring temperatures might have on curbing the spread of the virus. There is a good reason to ask this question. As microbiologists, infectious disease doctors, and primary care physicians know, the typical start and end to every flu season is well-documented and closely watched.
As SARS-CoV-2 ravages countries around the world, clinical pathologists and microbiologists debate whether it will subside as temperatures rise in Spring and Summer. Recent analyses suggest it may indeed be a seasonal phenomenon. However, some infectious disease specialists have expressed skepticism.
CNN reported that Nicholls was part of a research team which reproduced the virus in January to study its behavior and evaluate diagnostic tests. Nicholls was also involved in an early effort to analyze the coronavirus associated with the 2003 SARS outbreak involving SARS-CoV, another coronavirus that originated in Asia.
“Sunlight will cut the virus’ ability to grow in half, so the half-life will be 2.5 minutes and in the dark it’s about 13 to 20,” Nicholls told AccuWeather. “Sunlight is really good at killing viruses.” And that, “In cold environments, there is longer virus survival than warm ones.” He added, “I think it will burn itself out in about six months.”
Can Weather Predict the Spread of COVID-19?
Other researchers have analyzed regional weather data to see if there’s a correlation with incidence of COVID-19. A team at the Massachusetts Institute of Technology (MIT) found that the number of cases has been relatively low in areas with warm, humid conditions and higher in more northerly regions. They published their findings in SSRN (formerly Social Science Research Network), an open-access journal and repository for early-stage research, titled “Will Coronavirus Pandemic Diminish by Summer?”
FREE Webinar | What Hospital and Health System Labs Need to Know About Operational Support and Logistics During the COVID-19 Outbreak Wednesday, April 1, 2020 @ 1PM EDT — Register or Stream now
The MIT researchers found that as of March 22, 90% of the
transmissions of SARS-CoV-2 occurred within a temperature range of three to 17
degrees Celsius (37.4 to 62.6 degrees Fahrenheit) and an absolute humidity
range of four to nine grams per cubic meter. Fewer than 6% of the transmissions
have been in warmer climates further south, they wrote.
“Based on the current data on the spread of [SARS-CoV-2], we
hypothesize that the lower number of cases in tropical countries might be due
to warm humid conditions, under which the spread of the virus might be slower
as has been observed for other viruses,” they wrote.
In the US, “the outbreak also shows a north-south divide,”
with higher incidence in northern states, they wrote. The outliers are Oregon,
with fewer than 200 cases, and Louisiana, where, as of March 22, approximately
1,000 had been reported.
There’s been a recent spike in reported cases from warmer
regions in Asia, South America, and Africa, but the MIT researchers attribute
this largely to increased testing.
Still, “there may be several caveats to our work,” they
wrote in their published study. For example, South Korea has been engaged in
widespread testing that includes asymptomatic individuals, whereas other
countries, including the US, have limited testing to a narrower range of
people, which could mean that more cases are going undetected. “Further, the
rate of outdoor transmission versus indoor and direct versus indirect
transmission are also not well understood and environmental related impacts are
mostly applicable to outdoor transmissions,” the MIT researchers wrote.
Even in warmer, more humid regions, they advocate “proper
quarantine measures” to limit the spread of the virus.
The New York Times (NYT) reported that other recent studies have shown a correlation between weather conditions and the incidence of COVID-19 outbreaks as well, though none of this research has been peer reviewed.
Why the Correlation? It’s Unclear, MIT Says
Though the MIT researchers found a strong relation between
the number of cases and weather conditions, “the underlying reasoning behind
this relationship is still not clear,” they wrote. “Similarly, we do not know
which environmental factor is more important. It could be that either
temperature or absolute humidity is more important, or both may be equally or
not important at all in the transmission of [SARS-CoV-2].”
Some experts have looked at older coronaviruses for clues. “The coronavirus is surrounded by a lipid layer, in other words, a layer of fat,” said molecular virologist Thomas Pietschmann, PhD, Director of the Department for Experimental Virology at the Helmholtz Center for Infection Research in Hanover, Germany, in a story from German news service Deutsche Welle. This makes it susceptible to temperature increases, he suggested.
However, Pietschmann cautioned that because it’s a new
virus, scientists cannot say if it will behave like older viruses. “Honestly
speaking, we do not know the virus yet,” he concluded.
How should pathologists and clinical laboratories in this country prepare for COVID-19? Lipsitch wrote that Influenza does tend to be seasonal, in part because cold, dry air is highly conducive to flu transmission. However, “for coronaviruses, the relevance of this factor is unknown.” And “new viruses have a temporary but important advantage—few or no individuals in the population are immune to them,” which means they are not as susceptible to the factors that constrain older viruses in warmer, more humid months.
So, we may not yet know enough to adequately prepare for
what’s coming. Nevertheless, monitoring the rapidly changing data on COVID-19
should be part of every lab’s daily agenda.