Group’s report also suggests that at-home clinical laboratory tests for COVID-19 that are difficult to use may lead to inaccurate results
At-home clinical laboratory tests for COVID-19 have become quite popular. But how accurate are they? Now, an independent safety organization has investigated COVID-19 rapid antigen tests to find out how easy—or not—they are to use and what that means for the accuracy of the tests’ results.
ECRI (Emergency Care Research Institute) of Plymouth Meeting, Penn., “conducted a usability evaluation to determine if there were any differences in ease of use for the rapid COVID-19 tests,” according to the company’s website. The nonprofit was founded in the 1960s by surgeon and inventor Joel J. Nobel to evaluate medical devices that have been approved by the U.S. Food and Drug Administration (FDA).
“Because of the urgency in providing useful information to consumers as quickly as possible, ECRI selected the seven test kits based on retail availability,” ECRI noted.
ECRI ranked the seven over-the-counter (OTC) at-home rapid antigen tests according to their SUS usability ratings. The System Usability Scale (SUS), invented by John Brooke in 1986, “rates products on a scale of 0 to 100 with 100 being the easiest to use. More than 30 points separated the top and bottom tests analyzed,” according to Managed Healthcare Executive.
Of the seven rapid antigen test kits for COVID-19, ECRI found “noteworthy usability concerns” and “significant differences in ease of use.” None of the tests achieved a SUS rating of “excellent,” ECRI stated in a press release.
ECRI published its findings in a report, titled, “Usability of COVID-19 Antigen Home Test Kits.”
Seven Rapid Antigen Tests for SARS-CoV-2 Evaluated
As clinical laboratory scientists and pathologists know, it’s possible for different test methodologies for the same biomarker to produce dissimilar results. Another factor affecting medical laboratory test accuracy is the variability from one manufacturing batch or lot to another. And, as the ECRI report suggests, how a specimen is collected and handled can affect accuracy, reliability, and reproducibility of the test results generated by that specimen.
These are the OTC COVID-19 rapid antigen tests ECRI evaluated and their SUS ratings:
- On/Go, Intrivo Diagnostics: Very Good (82.9)
- CareStart, Intrivo Diagnostics: Very Good (80.8)
- Flowflex, ACON Laboratories: Very Good (79.5)
- QuickVue, Quidel: Good (75.6)
- BinaxNOW, Abbott: Good (73.3)
- InteliSwab, OraSure Technologies: Good (73.3)
- BD Veritor, Becton, Dickinson: Okay (51.8)
Some tests, the ECRI analysts found, required “fine motor control” or were packed with written instructions ECRI determined were too small for older adults to read.
How ECRI Evaluated the COVID-19 Rapid Antigen Tests
SUS reviewers took each rapid test and completed questionnaires specifying their level of agreement (on a range of one to five) with these statements. (Edited by Dark Daily for space):
- Desire to use
- Perception of unnecessary complexity
- Easy to use
- Support of a technical person needed
- Functions well-integrated
- Too much system inconsistency
- Easy to learn for most people
- A very cumbersome system to use
- Feeling of confidence in use
- A need to learn before getting going
ECRI then used an algorithm to derive an aggregate score (from 0 to 100) for each test, the report noted.
“Based on the aggregate SUS scores, none of the COVID-19 test kits would be judged to have ‘excellent’ usability. The On/Go, CareStart, Flowflex test kits we rate as ‘very good’ as the usability score for these kits falls just short of ‘excellent,’” the report said.
Some of the positive responses ECRI received from the SUS participants included:
- “One of the simpler tests to use with good, printed instructions,” (On/Go and CareStart).
- “Cassette makes handling without touching test strip easy,” (CareStart and Flowflex).
- “The QR (quick-response) code-linked instructional video is helpful, but probably not needed,” (QuickVue).
- “Once the swab is inserted into the test card, the test seems less likely to be spilled or disturbed than other test kits,” (BinaxNOW).
Is it Time for Rapid COVID-19 Antigen Tests?
Unlike RT-PCR tests that can take hours or days to return results, rapid antigen tests provide a quick result that’s used for screening worldwide. And with the COVID-19 Omicron variant spreading rapidly around the world, speed is much needed, according to Stephen Kissler, PhD, Research Fellow in the department of immunology and infectious diseases at Harvard’s T.H. Chan School of Public Health.
“I think the rapid tests provide some of the best protection we have against the spread of disease, especially as we now have a variant on hand that’s going to be able to cause an awful lot of breakthrough infections,” Kissler told The Atlantic-Journal Constitution.
One way clinical laboratory leaders can help is to reach out in their local markets and provide information on the importance of appropriate sampling and collection for accurate results from rapid COVID-19 antigen testing.
—Donna Marie Pocius
Related Information:
ECRI Report: Usability of COVID-19 Antigen Home Test Kits
ECRI Finds Significant Gaps in Ease of Use for At-Home COVID Tests