Clinical laboratories working with AI should be aware of ethical challenges being pointed out by industry experts and legal authorities
Experts are voicing concerns that using artificial
intelligence (AI) in healthcare could present ethical challenges that need
to be addressed. They say databases and algorithms may introduce bias into the
diagnostic process, and that AI may not perform as intended, posing a potential
for patient harm.
If true, the issues raised by these experts would have major
implications for how clinical
laboratories and anatomic
pathology groups might use artificial intelligence. For that reason,
medical laboratory executives and pathologists should be aware of possible
drawbacks to the use of AI and machine-learning
algorithms in the diagnostic process.
Is AI Underperforming?
AI’s ability to improve diagnoses, precisely target
therapies, and leverage healthcare data is predicted to be a boon to precision medicine and personalized
healthcare.
For example, Accenture
(NYSE:ACN) says that hospitals will spend $6.6 billion on AI by 2021. This
represents an annual growth rate of 40%, according
to a report from the Dublin, Ireland-based consulting firm, which states,
“when combined, key clinical health AI applications can potentially create $150
billion in annual savings for the United States healthcare economy by 2026.”
But are healthcare providers too quick to adopt AI?
Accenture defines AI as a “constellation of technologies
from machine learning to natural
language processing that allows machines to sense, comprehend, act, and
learn.” However, some experts say AI is not performing as intended, and that it
introduces biases in healthcare worthy of investigation.
Keith Dreyer, DO, PhD, is Chief Data Science Officer at Partners Healthcare and Vice Chairman of Radiology at Massachusetts General Hospital (MGH). At a World Medical Innovation Forum on Artificial Intelligence covered by HealthITAnalytics, he said, “There are currently no measures to indicate that a result is biased or how much it might be biased. We need to explain the dataset these answers came from, how accurate we can expect them to be, where they work, and where they don’t work. When a number comes back, what does it really mean? What’s the difference between a seven and an eight or a two?” (Photo copyright: Healthcare in Europe.)
What Goes in Limits What Comes Out
Could machine learning lead to machine decision-making that
puts patients at risk? Some legal authorities say yes. Especially when computer
algorithms are based on limited data sources and questionable methods, lawyers
warn.
How can AI provide accurate medical insights for people when
the information going into databases is limited in the first place? Ossorio
pointed to lack of diversity in genomic
data. “There are still large groups of people for whom we have almost no
genomic data. This is another way in which the datasets that you might use to
train your algorithms are going to exclude certain groups of people
altogether,” she told HDM.
She also sounded the alarm about making decisions about
women’s health when data driving them are based on studies where women have
been “under-treated compared with men.”
“This leads to poor treatment, and that’s going to be
reflected in essentially all healthcare data that people are using when they
train their algorithms,” Ossorio said during a Machine Learning for Healthcare (MLHC) conference
covered by HDM.
How Bias Happens
Bias can enter healthcare data in three forms: by humans, by
design, and in its usage. That’s according to David Magnus, PhD, Director
of the Stanford Center for
Biomedical Ethics (SCBE) and Senior Author of a paper published in the New England
Journal of Medicine (NEJM) titled, “Implementing Machine
Learning in Health Care—Addressing Ethical Challenges.”
The paper’s authors wrote, “Physician-researchers are
predicting that familiarity with machine-learning tools for analyzing big data
will be a fundamental requirement for the next generation of physicians and
that algorithms might soon rival or replace physicians in fields that involve
close scrutiny of images, such as radiology and anatomical pathology.”
In a news
release, Magnus said, “You can easily imagine that the algorithms being
built into the healthcare system might be reflective of different, conflicting
interests. What if the algorithm is designed around the goal of making money?
What if different treatment decisions about patients are made depending on
insurance status or their ability to pay?”
In addition to the possibility of algorithm bias, the
authors of the NEJM paper have other concerns about AI affecting
healthcare providers:
“Physicians must adequately understand how
algorithms are created, critically assess the source of the data used to create
the statistical models designed to predict outcomes, understand how the models
function and guard against becoming overly dependent on them.
“Data gathered about patient health, diagnostics,
and outcomes become part of the ‘collective knowledge’ of published literature
and information collected by healthcare systems and might be used without
regard for clinical experience and the human aspect of patient care.
“Machine-learning-based clinical guidance may
introduce a third-party ‘actor’ into the physician-patient relationship, challenging
the dynamics of responsibility in the relationship and the expectation of
confidentiality.”
“We need to be cautious about caring for people based on what algorithms are showing us. The one thing people can do that machines can’t do is step aside from our ideas and evaluate them critically,” said Danton Char, MD, Lead Author and Assistant Professor of Anesthesiology, Perioperative, and Pain Medicine at Stanford, in the news release. “I think society has become very breathless in looking for quick answers,” he added. (Photo copyright: Stanford Medicine.)
Acknowledge Healthcare’s Differences
Still, the Stanford researchers acknowledge that AI can
benefit patients. And that healthcare leaders can learn from other industries,
such as car companies, which have test driven AI.
“Artificial intelligence will be pervasive in healthcare in a
few years,” said
Nigam Shah, PhD, co-author of the NEJM paper and Associate Professor of Medicine at Stanford, in the news release. He added that healthcare leaders need to be aware of the “pitfalls” that have happened in other industries and be cognizant of data.
“Be careful about knowing the data from which you learn,” he
warned.
AI’s ultimate role in healthcare diagnostics is not yet fully
known. Nevertheless, it behooves clinical laboratory leaders and anatomic
pathologists who are considering using AI to address issues of quality and
accuracy of the lab data they are generating. And to be aware of potential
biases in the data collection process.
Clinical laboratory leaders interested in positioning their labs to be paid for added-value services will get knowledge, insights, and more at upcoming third annual Clinical Lab 2.0 Workshop in November
It’s a critical time for medical laboratories. Healthcare is transitioning from a fee-for-service payment system to new value-based payment models, creating disruption and instability in the clinical lab test market. In addition, payers are cutting reimbursement for many lab tests.
These are among the market factors leading some pathologists
and clinical lab leaders to seek new or alternative sources of revenue to keep
the lights on and the machines running in their laboratories. Some might say,
it’s a dark time for the lab industry.
“This is not the time to be shy or timid,” he declared. “The
quantitative value of medical laboratory domain is significant and will be lost
if not exploited or leveraged.”
Shotorbani has reason to be positive. In recent years the Project Santa Fe Foundation (PSFF) has emerged to advocate for, and teach, the Clinical Lab 2.0 model. Clinical Lab 2.0 is an approach which focuses on longitudinal clinical laboratory data to augment population health in new payment arrangements.
Earlier this year, PSFF filed for 501(c) status, according to a news release. It is now positioned as a nonprofit organization, guided by a board of directors whose mission is “to create a disruptive value paradigm and alternative payment model that defines placement of diagnostic services in healthcare.”
Progressing Toward Clinical Lab 2.0
At the 24th Annual Executive War College on Lab and Pathology Management held in New Orleans last May, the nation’s first ever Clinical Lab 2.0 “Shark Tank” competition was won by Aspenti Health, a full-service diagnostic laboratory specializing in toxicology screening.
“This project, as well as all of the other cases that were presented, were quite strong and all were aligned with the mission of the Clinical Lab 2.0 movement,” said Shotorbani, in a news release. “This movement transforms the analytic results from a laboratory into actionable intelligence at the patient visit in partnership with front-liners and clinicians—allowing for identification of patient risks—and arming providers with insights to guide therapeutic interventions.
“Further, it reduces the administrative burden on providers by collecting SDH [social determinants of health] predictors in advance and tying them to outcomes of interest,” he continued. “By bringing SDH predictors to the office visit, it enables providers to engage in SDH without relying on their own data collection—a current care gap in many practices. The lab becomes a catalyst helping to manage the population we serve.”
Aspenti Health’s Shark Tank entry, “Integration of the Clinical Laboratory and Social Determinants of Health in the Management of Substance Use,” focused on the social factors tied to the co-use of opioids and benzodiazepines, a combination that puts patients at higher risk of drug-related overdose or death.
The project revealed that the top-two predictors of co-use
were the prescribing provider practice and the patient’s age.
“They did an interesting thing—what clinical laboratories
alone cannot do—the predictive value of lab test data mapped by zip code for
patients admitted in partnership with social determinants of health. This helps
to create delivery models to potentially help prevent opioid overdose,” said
Shotorbani, who sees economic implications for chronic conditions.
“If clinical laboratories have that ability to do that in
acute conditions such as opioid overdose, what is our opportunity to use lab test
data in chronic conditions, such as diabetes? The cost of healthcare is in
chronic conditions, and that is where clinical lab data has an essential role—to
support early detection and early prevention,” he added.
“This is often described as the transition from volume to value because this trend will fundamentally change how all clinical laboratories and anatomic pathology groups are paid,” said Khosrow Shotorbani (above), MBA, MT(ASCP), Executive Director of the Project Santa Fe Foundation (PSFF), during his presentation at the 22nd annual Executive War College in New Orleans. “This shift from volume to value also will create new winners and losers in the clinical lab industry,” he declared. “Not every lab organization will take the timely action required to introduce the value-based laboratory testing services that hospitals, physicians, and payers will need. (Photo copyright: Albuquerque Business First.)
Clinical Laboratory Data is Health Business Data
One clinical laboratory working toward that opportunity is TriCore Reference Laboratories in Albuquerque, N.M. It recently launched Diagnostic Optimization with the goal of improving the health of their communities.
“TriCore turned to this business model,” Shotorbani
explained. “It is actively pursuing the strategy of intervention, prevention,
and cost avoidance. TriCore is in conversation with health plans on how its lab
test data and other data sets can be combined and analyzed to risk-stratify a
population and to identify care gaps and assist in closing gaps.
“Further, TriCore is identifying high-risk patients early
before they are admitted to hospitals and ERs—the whole notion of facilitating
intervention between the healthcare provider and the potential person who may
get sick,” he added. “These are no longer theoretical goals. They are
realizations. Now the challenge is for Project Santa Fe to help other lab
organizations develop similar value-added collaborations in their communities.”
Renee Ennis, TriCore’s Chief Financial Officer, told American Healthcare Leader, “Women go in (to an ER) for some condition, and the lab finds out they are pregnant before anyone else,” she said, adding that TriCore reaches out to insurers who can offer care coordinators for prenatal services.
“There is definitely a movement within the industry in this
direction [of Clinical Lab 2.0],” she added. “But others might not be moving as
quickly as we are. As a leader in this transition, I think a lot of eyes are on
what we are doing and how we are doing it.”
Why Don’t More Lab Leaders Move Their Labs to Clinical
Lab 2.0?
So, what holds labs back from pursing Clinical Lab 2.0?
Shotorbani pointed to a couple of possibilities:
A lab’s traditional focus on volume while not
developing partnerships (such as with pharmacy colleagues) inside the
organization; and
Limited longitudinal data due to a provider’s
sale of lab outreach services or outsourcing the lab.
“The whole notion of Clinical Lab 2.0 is basically connecting the longitudinal data—the Holy Grail of lab medicine. That is the business model. Without the longitudinal view, the ability to become a Clinical Lab 2.0 is extremely limited,” added Shotorbani.
New Clinical Lab 2.0 Workshop Focuses on Critical ‘Pillars’
Project Santa Fe Foundation will host the Third Annual Clinical Lab. 2.0 Workshop in Chicago on November 3-5. New this year are sessions aligned with Clinical Lab 2.0 “pillars” of leadership, standards, and evidence. The conference will feature panels addressing:
C-suite Drivers: moderated by Mark Dixon, President of The Mark Dixon Group;
How medical laboratories can show value through process improvement methods and analytics will be among many key topics presented at the upcoming Lab Quality Confab conference
Quality management is the clinical laboratory’s best strategy for surviving and thriving in this era of shrinking lab budgets, PAMA price cuts, and value-based payment. In fact, the actions laboratories take in the next few months will set the course for their path to clinical success and financial sustainability in 2020 and beyond.
But how do medical laboratory managers and pathologists address these challenges while demonstrating their lab’s value? One way is through process improvement methods and another is through the use of analytics.
Clinical pathologists, hospital lab leaders, and independent lab executives have told Dark Daily that the trends demanding their focus include:
Ensuring needed resources and appropriate tests,
while the lab is scrutinized by insurance companies and internally by hospital
administration;
“Our impact on patient care, in many cases, is very
indirect. So, it is difficult to point to outcomes that occur. We know things
we do matter and change patient care, but objectively showing that is a real
struggle. And we are being asked to do more than we ever had before, and those
are the two big things that keep me up at night these days,” he added.
This is where process improvement methods and analytics are
helping clinical laboratories understand critical issues and find opportunities
for positive change.
“You need to have a strategy that you can adapt to a changing landscape in healthcare. You have to use analytics to guide your progress and measure your success,” Patricia Nortmann, System Director of Laboratory Services at St. Elizabeth Healthcare, Erlanger, Ky., told Dark Daily.
Clinical Laboratories Can Collaborate Instead of Compete
Prior to a joint venture with TriHealth in Cincinnati, St. Elizabeth lab leaders used data to inform their decision-making. Over about 12 years preceding the consolidation of labs they:
Implemented front-end automation outside the core area and in the microbiology lab.
“We are now considered a regional reference lab in the state
of Kentucky for two healthcare organizations—St. Elizabeth and TriHealth,”
Nortmann said.
Thanks to these changes, the lab more than doubled its
workload, growing from 2.1 million to 4.3 million outreach tests in the core
laboratory, she added.
Christopher Doern, PhD (left), Director of Microbiology and Associate Professor of Pathology at Virginia Commonwealth University Health System; Patricia Nortmann (center), System Director of Laboratory Services at St. Elizabeth Healthcare; and Joseph Cugini (right), Manager Client Solutions at Health Network Laboratories, will present practical solutions and case studies in quality improvement and analytics for clinical laboratory professionals at the 13th Annual Lab Quality Confab, October 15-16, 2019, at the Hyatt Regency in Atlanta, Ga. (Photo copyright: The Dark Report.)
Using Analytics to Test the Tests
Clinical laboratories also are using analytics and information technology (IT) to improve test utilization.
At VCH Health, Doern said an analytics solution interfaces
with their LIS, providing insights into test orders and informing decisions
about workflow. “I use this analytics system in different ways to answer
different questions, such as:
How are clinicians using our tests?
When do things come to the lab?
When should we be working on them?
“This is important for microbiology, which is a very delayed
discipline because of the incubation and growth required for the tests we do,”
he said.
Using analytics, the lab solved an issue with Clostridium
difficile (C diff) testing turnaround-time (TAT) after associating it with
specimen transportation.
Inappropriate or duplicate testing also
can be revealed through analytics. A physician may reconsider a test after discovering
another doctor recently ordered the same test. And the technology can guide
doctors in choosing tests in areas where the related diseases are obscure, such
as serology.
Avoiding Duplicate Records While
Improving Payment
Another example of process
improvement is Health Network Laboratories (HNL) in Allentown, Pa. A team there established an enterprise master patient index (EMPI) and implemented digital tools to find and eliminate
duplicate patient information and improve lab financial indicators.
“The system uses trusted sources of data to make sure data is clean and the lab has what it needs to send out a proper bill. That is necessary on the reimbursement side—from private insurance companies especially—to prevent denials,” Joseph Cugini, HNL’s Manager Client Solutions, told Dark Daily.
HNL reduced duplicate records in its database from 23% to
under one percent. “When you are talking about several million records, that is
quite a significant improvement,” he said.
Processes have improved not only on the billing side, but in
HNL’s patient service centers as well, he added. Staff there easily find
patients’ electronic test orders, and the flow of consumers through their
visits is enhanced.
Learn More at Lab Quality Confab Conference
Cugini, Doern, and Nortmann will speak on these topics and more during the 13th Annual Lab Quality Confab (LQC), October 15-16, 2019, at the Hyatt Regency in Atlanta, Ga. They will offer insights, practical knowledge, and case studies involving Lean, Six Sigma, and other process improvement methods during this important 2-day conference, a Dark Dailynews release notes.
Register for LQC, which is produced by Dark Daily’s sister publication The Dark Report, online at https://www.labqualityconfab.com/register, or by calling 512-264-7103.
OIG suggests better use of analytics by CMS could prevent gaming of the system by providers; clinical laboratories can help through test utilization management technology
In 2015, CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP) as part of the Patient Protection and Affordable Care Act (ACA). The HACRP program incentivizes hospitals to lower their HAI rates by adjusting reimbursements according to the inpatient quality reporting (hospital IQR) data provided by the healthcare providers. Hospital IQR data is the basis on which CMS validates a hospital’s HAI rate (among other things CMS is tracking) to determine the hospital’s reimbursement rate for that year.
CMS, in 2016, met its regulatory requirement to validate inpatient quality reporting data;
It reviewed data of 400 randomly selected hospitals as well as 49 hospitals targeted for failing to report half their HAIs, or for low scores in the prior year’s validation process;
However, OIG also reported that CMS did not include hospitals that displayed abnormal data patterns in its targeted sample. Targeting those hospitals, according to the OIG, could identify inaccurate reporting.
CMS staff had identified 96 hospitals with aberrant data patterns, but did not target them for validation—even though the agency can select up to 200 targeted hospitals for review, Becker’s Hospital Review pointed out.
Dollars More Important than Deaths
According to the OIG report, Medicare excluded in its investigation dozens of hospitals with suspected HAI reporting. This is odd since the CMS and the Centers for Disease Control (CDC) apparently are aware that some healthcare providers have manipulated data to improve their quality measure scores and thus increase their reimbursement rates.
“Collecting and analyzing quality data is increasingly central to Medicare programs that link payments to quality and value. Therefore, it is important for CMS to ensure that hospitals are not gaming [manipulating data to improve scores] their reporting of quality data,” the OIG report noted.
“There are greater requirements for what a company says about a washing machine’s performance than there is for a hospital on quality of care. And this needs to change,” stated Peter Pronovost, MD, PhD, in the Kaiser Health News article. “We require auditing of financial data, but we don’t require auditing of healthcare quality data, and that implies that dollars are more important than deaths.” Pronovost is Senior Vice President for Patient Safety and Quality at Johns Hopkins University School of Medicine.
Peter Pronovost, MD, PhD (above) testifying on preventable deaths before the Senate Subcommittee on Primary Health and Aging in 2014. He is Senior Vice President for Patient Safety and Quality at Johns Hopkins University School of Medicine in Baltimore. Pronovost told Kaiser Health News that there are no uniform standards for reviewing data that hospitals report to Medicare. (Photo copyright: US Senate Committee on Health, Education, Labor and Pensions.)
Medicare Missed Hospitals with Suspected HAI Data
CMS should have done an in-depth review of many hospitals that submitted “aberrant data patterns” in 2013 and 2014, the OIG stated in its report. According to a Kaiser HealthNews article, such patterns could include:
A rapid change in results;
Improbably low infection rates; and
Assertions that infections nearly always struck before patients arrived at the hospital.
“There’s a certain amount of blind faith that hospitals are going to tell the truth. It’s a bit much to expect that if they had a bad record they are going to fess up to it,” noted Lisa McGiffert, Director of the Safe Patient Project at Consumers Union, in the Kaiser Health News article.
CMS Needs Better Data Analytics
So, what does the OIG advise CMS to do? The agency called for “better use of analytics to ensure the integrity of hospital-reported quality data.” Specifically, OIG suggested CMS:
Identify hospitals with abnormal percentages of patients who had infections on admission;
Apply risk scores to identify hospitals with high propensity to manipulate reporting;
Use experiences to create and improve models that identify hospitals most likely to game their reporting.
CMS’ Administrator Seema Verma reportedly responded, “We will continue to evaluate the use of better analytics as feasible, based on Medicare’s operational capabilities.”
Medical Laboratory Diagnostic Testing Part of Gaming the System
A 2015 CMS/CDC joint statement noted “three ways that hospitals may be deviating from CDC’s definitions for reportable HAIs,” and two involve diagnostic test ordering. According to the OIG report, they include:
Overculturing: Diagnostic tests may be overutilized by providers in absence of clinical symptoms. Hospitals may use positive results to game their data by claiming infections that appeared days later were present on admission and thus not reportable.
Underculturing: Hospitals underculture when they do not order diagnostic tests in the presence of clinical symptoms. By not ordering the test, the hospital does not learn whether the patient truly has an infection and, therefore, the hospital does not have to report it.
Adjudication: Hospital administrative staff may inappropriately overrule those who report infections. HAIs are, therefore, not shared.
Clinical Laboratories Can Help
One in 25 people each day receives an HAI, CDC estimates. The OIG findings should be a reminder to medical laboratories and pathology groups that quality measures and patient outcomes are often transparent to media, patients, and the public.
One way medical laboratories in hospitals and health systems can help is by investing in utilization management technology and protocols that ensure appropriate lab test utilization. Informing doctors on the availability of appropriate diagnostic tests based on patients’ existing conditions, unique physiologies, or medical histories, could help prevent hospitals from inadvertently or deliberately game the system.
Clearly, transparency in healthcare is increasing. That means there will be more news stories revealing federal agencies’ failures to respond to healthcare data in ways that could have protected patients and the public. Clinical laboratories don’t want to be included in negative reporting.
By consolidating information, automating data collection, and harnessing new cloud computing technologies, doctors hope to silence the endless array of alarms and inject efficiency and personalization into the critical care experience
Some healthcare experts believe it is time that intensive care units undergo a workflow redesign to improve the quality of care they deliver, while reducing or eliminating design elements that contribute to errors. Clinical laboratories have a stake in this redesign effort, as they provide medical laboratory tests for patients in ICUs.
“What I want to do for the ICU is what Steve Jobs did for the iPhone,” said Peter Pronovost, PhD, MD, in an article published in STAT. Pronovost is working to improve both the flow of information and delivery of care in the ICU of Johns Hopkins Hospital in Baltimore, Maryland. (more…)
