At hospitals where results of molecular COVID-19 testing can take up to several days to return, this new method for identifying potentially infected patients could improve triage
Frustrated by shortages of essential COVID-19 tests and supplies—as well by lengthy coronavirus test turn-around times—researchers at Weill Cornell Medicine have created an Artificial Intelligence (AI) algorithm that can use routine clinical laboratory test data to determine if a patient is infected with SARS-CoV-2, the coronavirus that causes the COVID-19 disease.
This is an important development because the turn-around-time (TAT) for common lab tests is generally much shorter than COVID-19 molecular diagnostics—such as real-time reverse transcription polymerase chain reaction (RT-PCR), currently the most popular coronavirus test—and certainly serological (antibody) diagnostics, which require an infection incubation time of as much as 10-14 days before testing.
Some RT-PCR diagnostic tests for COVID-19, which detect viral RNA on nasopharyngeal swab specimens, can take up to several days to return depending on the test and on the lab’s location. But routine medical laboratory tests generally return much sooner, often within minutes or hours, making this a potential game-changer for triaging infected patients.
Machine Learning Brings AI to COVID-19 Diagnostics
Advances in the use of AI in healthcare have led to the development of machine-learning algorithms that are being utilized as diagnostic tools for anatomic pathology, radiology, and for specific complex diseases, such as cancer. The Weill Cornell scientists wanted to see if alternative lab test results could be used by an algorithmic model to identify people infected with the SARS-CoV-2 coronavirus.
“When patients come to the [emergency department] and the doctor orders several panels of routine lab [tests] and also the [SARS-CoV-2] RT-PCR test, generally the routine test results come back in a couple of hours,” Sarina Yang, MD, PhD (above), one of the authors of the study, told Modern Healthcare. “So, we thought it could be useful to use the routine labs to predict whether the RT-PCR results would be positive or negative to improve the triage process.” Yang is an assistant professor in the Department of Pathology and Laboratory Medicine, and Assistant Director of the central laboratory and Director of the toxicology laboratory at Weill Cornell Medicine. (Photo copyright: Weill Cornell Medicine.)
To perform the research, the team incorporated patients’ age, sex, and race, into a machine learning model that was based on results from 27 routine lab tests chosen from a total of 685 different tests ordered for the patients. The study included 3,356 patients who were tested for SARS-CoV-2 at New York-Presbyterian Hospital/Weill Cornell Medical Center between March 11 and April 29 of this year. The patients ranged in ages from 18 to 101 with the mean age being 56.4 years. Of those patients, 1,402 were RT-PCR positive and the remaining 1,954 were RT-PCR negative.
Using a machine-learning technique known as a gradient-boosting decision tree, the algorithm identified SARS-CoV-2 infections with 76% sensitivity and 81% specificity. When looking at only emergency department (ED) patients, the model performed even better with 80% sensitivity and 83% specificity. ED patients comprised just over half (54%) of the patients used for the study.
Weill Cornell Medicine Algorithm Could Lower False Negative Test Results
The algorithm also correctly identified patients who originally tested negative for COVID-19, but who tested positive for the coronavirus upon retesting within two days. According to the researchers, these results indicated their model could potentially decrease the amount of incorrect test results.
“We are thinking that those potentially false negative patients may demonstrate a different routine lab test profile that might be more similar to those that test positive,” Fei Wang, PhD, Assistant Professor of Healthcare Policy and Research at Weill Cornell Medicine and the study’s senior author, told Modern Healthcare. “So, it offers us a chance to capture those patients who are false negatives.”
The researchers validated their model by comparing the results with patients seen at New York Presbyterian Hospital/Lower Manhattan Hospital during the same time period. Among those patients, 496 were RT-PCR positive and 968 were negative and the algorithmic model performed with 74% specificity and 76% sensitivity.
preliminarily identify high-risk SARS-CoV-2 infected patients before RT-PCR results are available,
risk stratify patients in the ED,
select patients who need relatively urgent retesting if initial RT-PCR results are negative,
help isolate infected patients earlier, and
assist in the identification of SARS-CoV-2 infected patients in areas where RT-PCR testing is unavailable due to financial or supply constraints.
Early Results of Study Promising, But More Research is Needed
Wang noted that more research is needed on the algorithm and that he and his colleagues are currently working on ways to improve the model. They are hoping to test it with different conditions and geographies.
“Our model in the paper was built on data from when New York was at its COVID peak,” he told Modern Healthcare. “At that time, we were not doing wide PCR testing, and the patients who were getting tested were pretty sick.”
At the time of the study, the positivity rate for COVID-19 at New York-Presbyterian Hospital was in the 40% to 50% range. That was substantially higher than the current positivity rate, which is in the 2% to 3% range, Modern Healthcare reported.
“This model we built in a population in New York in a certain time period, so we can’t guarantee that it will work well universally,” Wang told Modern Healthcare.
It’s exciting to think that advances in software algorithms may one day make it possible to combine routine clinical laboratory testing and create diagnostics that identify diseases in ways the individual tests were not originally designed to do.
This study is an example that researchers in AI and informatics are working to bring new tools and diagnostic capabilities to clinical laboratories. Also, this is a demonstration of how a patient’s results from multiple other types of lab tests can by analyzed using AI and similar analytical algorithms to diagnose a health condition unrelated to the original reasons for performing those tests.
If this can be demonstrated with other diseases and health conditions, it would open up one more way that pathologists and clinical laboratory scientists can contribute to more accurate diagnoses and improved selection of the most appropriate therapies for individual patients.
The CDC and US Navy study reveals common symptoms and suggests best protective measures to prevent spread in enclosed environments that clinical labs and pathology groups could use to protect their staff members
Results from a study conducted by the US Navy and the federal Centers for Disease Control and Prevention (CDC) of sailors onboard the USS Theodore Roosevelt during the recent COVID-19 outbreak aboard the ship may be useful for pathologists and clinical laboratory managers. The study also provides public health and infectious disease specialists with an opportunity to learn more about how the SARS-CoV-2 coronavirus spreads in enclosed environments.
The aircraft carrier garnered headlines in April due to a widespread outbreak of the coronavirus among its crew. The investigators asked crewmembers to complete a questionnaire and provide samples for a serological antibody test and molecular diagnostics test, reported the Navy’s Bureau of Medicine and Surgery (BUMED). The goal was to learn more about the disease and how it spreads in high-density environments. The COVID-19 tests were conducted April 20-24 while the ship was docked in Guam.
“This study paints a picture of current and prior SARS-CoV-2 infection among young adults living in close quarters,” said the study’s lead author Dan Payne, PhD, an epidemiologist at the CDC, in the BUMED statement. “This data will contribute to understanding COVID-19 in the US military, as well as among young adults in other close communal environments.”
Participation in the study was voluntary. At the time of testing, a total of 1,417 service members were still on the ship or at the base in Guam, the researchers wrote in their study. Among them, 383 crewmembers agreed to complete the survey and provide a blood sample for an enzyme-linked immunosorbent assay (ELISA) antibody test. Out of that group, 267 also provided nasal swab samples for a reverse transcription polymerase chain reaction (RT-PCR) molecular diagnostic test.
A clinical laboratory team from the Naval Hospital Guam and the Navy and Marine Corps Public Health Center are seen above collecting surveys, nasal swabs, and blood samples from sailors assigned to the aircraft carrier USS Theodore Roosevelt (CVN 71) during the investigation into the COVID-19 outbreak in April. The ship’s medical leaders established an emergency command center, initiated a roving and deep cleaning team, and continually educated the crew on social distancing and proper protective procedures and behaviors, to mitigate the spread of the SARS-CoV-2 coronavirus. (Photo copyright: US Navy.)
The questionnaire sought information about sailors’ demographic factors, health history, symptoms, and preventive behaviors, such as mask wearing and physical distancing. Crewmembers who tested positive for reactive antibodies received an additional test to detect presence of neutralizing antibodies that inhibit the virus.
The median age of participants was 30 years. About 75% were male. Only 28 (7.3%) reported comorbidities such as a history of asthma, diabetes, hypertension, or immunosuppression, which are considered risk factors for developing serious cases of the COVID-19 disease.
Key findings of the CDC/Navy’s study:
228 participants (59.7%) tested positive for reactive antibodies. Of those, 135 (59.2%) tested positive for neutralizing antibodies.
235 participants had previously tested positive in a SARS-CoV-2 diagnostic test. Of those, 212 (90.2%) tested positive for reactive antibodies.
A total of 238 participants had a previous or current SARS-CoV-2 infection. Of these, 18.5% reported no symptoms.
Of the 194 sailors who reported symptoms, 115 (59.3%) sought medical care, and two were hospitalized.
The most frequently reported symptoms were headache (66.5%), loss of taste or smell or both (61.3%); myalgia (56.2%); runny nose (55.7%); and fatigue (55.2%).
The most effective preventive measures were avoidance of common areas, increased physical distancing, and use of face coverings.
“What we saw was that most of the infections were actually mild, in addition to those that were asymptomatic,” Payne told reporters after the study was published, reported CNN. “And this is perhaps different from studies of older Americans, or maybe even those who were hospitalized already, and certainly much different from those with underlying health conditions.”
But with the high number of asymptomatic cases, “symptom-based surveillance might not detect all infections,” noted the researchers, who cautioned that “the analysis was conducted on a convenience sample of persons who might have had a higher likelihood of exposure, and all information was based on self-report, raising the possibility of selection and recall biases.”
In January, the crew of the Roosevelt totaled about 4,800 sailors, reported Defense One. However, after docking in Guam, many sailors were moved to hotel rooms for quarantine. As of May 5, at least 1,156 crewmembers had tested positive for infection, Stars and Stripes reported, and one had died.
Sailors aboard the USS Theodore Roosevelt were examined prior to reboarding the ship following off-ship quarantine or isolation. Only those found to be negative for the coronavirus, were allowed to board the ship. (Photo copyright: US Navy.)
Impact of COVID-19 on the USS Theodore Roosevelt’s Crew
As of April 6, 172 crew members had tested positive for COVID-19, including the ship’s captain Brett Crozier. At that time, 61% of the crew had received clinical laboratory testing and 1,999 sailors had been moved off the ship into quarantine, reported Defense One. By the next day, 270 sailors tested positive, a 57% increase from the previous day.
By April 14, 589 crew members were diagnosed positive for COVID-19. With 92% of the crew tested, 3,922 were found to be negative for the infection. Nevertheless, 4,024 sailors—nearly 83% of the crew—were moved into isolation quarters off-ship to prevent spread of the coronavirus.
In their study, the Navy/CDC researchers concluded: “In this convenience sample of young, healthy US service members experiencing close contact aboard an aircraft carrier, those with previous or current SARS-CoV-2 infection experienced mild illness overall, and nearly 20% were asymptomatic. Approximately one third of participants reported fever, myalgia, and chills and had higher odds of SARS-CoV-2 infection than did persons who reported cough and shortness of breath. Participants reporting anosmia (loss of sense of smell) or ageusia (loss of sense of taste) had 10 times the odds of having infection, compared with those who did not.
“In this sample of intensely exposed subjects, assessed at a single point in time, results demonstrated that antibodies developed and that, at the time of specimen collection, many of these were neutralizing antibodies. … This is a promising indicator of immunity, and in several participants, neutralizing antibodies were still detectable >40 days after symptom onset. Ongoing studies assessing the humoral antibody response over time will aid the interpretation of serologic results in an outbreak investigation such as this.
“These results provide new indications of symptomatology of SARS-CoV-2 infections and serologic responses among a cohort of young US adults living in a congregate environment and contribute to a better understanding of COVID-19 epidemiology in the US military. The findings reinforce the importance of nonpharmaceutical interventions such as wearing a face covering, avoiding common areas, and observing social distancing to lower risk for infection in similar congregate living settings.”
Not all the specific lessons learned from this COVID-19 outbreak aboard a US Navy vessel will be applicable to clinical laboratories and anatomic pathology groups. Nevertheless, it is probable that the data gleaned from the CDC/Navy study aboard the USS Theodore Roosevelt will someday mean civilian Americans can count on improved responses to disease outbreaks from the nation’s testing laboratories.
Though experts say an antigen test is not as accurate as PCR tests, its low cost, ease of use, and widespread availability make it a boon for clinical labs performing COVID-19 testing
As former FDA commissioner Scott Gottlieb, MD, explained on Face the Nation, “this kind of technology is a real game changer … it’s a very rapid test that could be used in a doctor’s office. Doctors now have about forty thousand of these Sofia machines already installed in their offices … you do a simple nasal swab and the test itself scans for the antigens that the virus produces.
“The test is about 85% sensitive. So, let’s say a hundred people come into a doctor’s office who have COVID-19, eighty-five of them are going to be able to be tested positive with this test very quickly. It’s a cheap test. It’ll probably be about five dollars a test and you can get a result within five minutes … you’re getting a very fast result and you can start to take action immediately.
“The company itself said that they’re going to be able to produce about two hundred thousand of these tests starting right away. But in several weeks, they’ll be able to produce up to 1.5 million a week. So, this dramatically expands our testing capacity as long as doctors are able to run these tests in their offices.”
In an interview on CBS’ Face the Nation, former FDA Commissioner Scott Gottlieb, MD (above), said, “These antigen-based tests aren’t as reliable, meaning they’re not as sensitive. So, they’re going to miss some patients who have COVID. But in the hands of a doctor who already has a high index of suspicion that the patient may have the disease … they allow you to dramatically expand testing. And they’re very cheap. They’re very easy to perform. And again, most doctors have these machines already in their offices.” (Photo copyright: US Food and Drug Administration.)
Other LDTs That Have Received EUAs
Here’s a look at other laboratory-developed tests from major manufacturers that have received emergency-use authorizations from the FDA:
This test is designed for use with Abbott’s m2000 RealTime system, which is installed in about 200 US medical laboratories, the company says. It can run up to 470 patient samples in 24 hours. As of a May 11 statement, the company said it had shipped more than two million tests in the US.
This test is designed for use with Abbott’s Alinity m system, which the company describes as its “most advanced laboratory molecular instrument,” with the ability to run up to 1,080 tests in 24 hours, according to a press release.
This is a rapid test designed for use with the ID Now system, a compact portable instrument for point-of-care settings such as urgent care clinics. As of May 11, Abbott said it had shipped more than 1.7 million tests in the US, and that it planned to increase manufacturing capacity to two million tests per month.
However, the test has encountered some stumbling blocks. On May 14, the FDA issued an alert stating that the ID Now COVID-19 test could produce inaccurate negative results. This came after researchers at NYU Langone Health, Northwell Health, and Cleveland Clinic reported problems with the test, according to MedTech Dive. Abbott issued a statement suggesting that the problems were due to improper sample collection and handling, however, the FDA said that Abbott had agreed to conduct post-market studies to identify the cause of the false negatives and suggest remedial actions.
This is a qualitative test designed to detect the presence of IgG antibodies following a SARS-CoV-2 infection. The FDA authorized use of the assay on Abbott’s Architect i2000SR system in April, and then followed up with a May 11 EUA for its use on the Alinity i system. In a statement, Abbott said it planned to ship 30 million tests globally starting in May.
In a March statement, the FDA touted this as the first point-of-care COVID-19 test to receive an EUA. The company estimates the detection time as approximately 45 minutes. It is designed for use with Cepheid’s GeneXpert Dx diagnostic software and GeneXpert Infinity systems, which have nearly 5,000 US installations, according to a Cepheid statement.
This test runs on Hologic’s Panther system, which, according to a Hologic press release, can provide results in about three hours and run more than 1,000 tests per day. The company claims that more than 1,000 Panther systems are installed in US labs, and that it expects to produce an average of one million tests per week.
Ortho’s antibody test is designed for use with its VITROS XT 7600, 3600, 5600, and ECi/ECiQ immunodiagnostic systems, which, the company says are installed in more than 1,000 US labs. The Total Reagent Pack is a qualitative test that detects the presence of all antibodies against SARS-CoV-2.
On April 24, Ortho announced it had received another FDA EUA, this one for its Anti-SARS-CoV-2 IgG test, which detects the presence of IgG antibodies. In a statement, the company said it expects to produce “several million” IgG tests per month.
This test is designed for use with Roche’s cobas 6800 and 8800 systems. The 6800 can process up to 384 results in an eight-hour shift, Roche says, compared with 1,056 results for the 8800 model. The company says results are available in about 3.5 hours. In a statement, Roche said it planned to ship 400,000 tests per week.
Roche describes this as a qualitative antibody test that can be used on cobas e series immunoassay analyzers. Testing time is 18 minutes. As of May 19, the test was live at more than 20 US labs, “with plans in the next several weeks to increase to more than 200 commercial and hospital lab sites with the ability to perform millions of tests per week,” the company stated in a press release.
It’s likely the FDA will continue to issue emergency-use authorizations as the agency receives more applications from IVD manufacturers.
