Pathologists and clinical laboratory scientists may find one hospital’s use of a machine-learning platform to help improve utilization of lab tests both an opportunity and a threat
Variation in how individual physicians order, interpret, and act upon clinical laboratory test results is regularly shown by studies in peer-reviewed medical journals to be one reason why some patients get great outcomes and other patients get less-than-desirable outcomes. That is why many healthcare providers are initiating efforts to improve how physicians utilize clinical laboratory tests and other diagnostic procedures.
This effort came about after clinical and administrative leadership at Flagler Hospital realized that only about one-third of its physicians regularly followed certain medical decision-making guidelines or clinical order sets. Armed with these insights, staff members decided to find a solution that reduced or removed variability from their healthcare delivery.
Reducing Variability Improves Care, Lowers Cost
Variability in physician care has been linked to increased healthcare costs and lower quality outcomes, as studies published in JAMA and JAMA Internal Medicine indicate. Such results do not bode well for healthcare providers in today’s value-based reimbursement system, which rewards increased performance and lowered costs.
Clinical order sets are designed to be used as part of clinical decision support systems (CDSS) installed by hospitals for physicians to standardize care and support sound clinical decision making and patient safety.
However, when doctors don’t adhere to those pre-defined standards, the results can be disadvantageous, ranging from unnecessary services and tests being performed to preventable complications for patients, which may increase treatment costs.
“Over the past few decades we’ve come to realize clinical variation plays an important part in the overuse of medical care and the waste that occurs in healthcare, making it more expensive than it should be,” Michael Sanders, MD (above) Flagler’s Chief Medical Information Officer, told Modern Healthcare. “Every time we’re adding something that adds cost, we have to make sure that we’re adding value.” (Photo copyright: Modern Healthcare.)
Flagler’s AI project involved uploading clinical,
demographic, billing, and surgical information to the AyasdiAI platform, which then
employed machine learning to analyze the data and identify trends. Flagler’s
physicians are now provided with a fuller picture of their patients’ conditions,
which helps identify patients at highest risk, ensuring timely interventions that
produce positive outcomes and lower costs.
How Symphony AyasdiAI Works
The AyasdiAI application utilizes a category of mathematics called topological data analysis (TDA) to cluster similar patients together and locate parallels between those groups. “We then have the AI tools generate a carepath from this group, showing all events which should occur in the emergency department, at admission, and throughout the hospital stay,” Sanders told Healthcare IT News. “These events include all medications, diagnostic tests, vital signs, IVs, procedures and meals, and the ideal timing for the occurrence of each so as to replicate the results of this group.”
Caregivers then examine the data to determine the optimal
plan of care for each patient. Cost savings are figured into the overall
equation when choosing a treatment plan.
Flagler first used the AI program to examine trends among their pneumonia patients. They determined that nebulizer treatments should be started as soon as possible with pneumonia patients who also have chronic obstructive pulmonary disease (COPD).
“Once we have the data loaded, we use [an] unsupervised
learning AI algorithm to generate treatment groups,” Sanders told Healthcare
IT News. “In the case of our pneumonia patient data, Ayasdi produced nine
treatments groups. Each group was treated similarly, and statistics were given
to us to understand that group and how it differed from the other groups.”
Armed with this information, the hospital achieved an 80% greater physician adherence to order sets for pneumonia patients. This resulted in a savings of $1,350 per patient and reduced the readmission rates for pneumonia patients from 2.9% to 0.4%, reported Modern Healthcare.
The development of a machine-learning platform designed to
reduce variation in care (by helping physicians become more consistent at
following accepted clinical care guidelines) can be considered a warning shot
across the bow of the pathology profession.
This is a system that has the potential to become interposed
between the pathologist in the medical laboratory and the physicians who refer
specimens to the lab. Were that to happen, the deep experience and knowledge
that have long made pathologists the “doctor’s doctor” will be bypassed.
Physicians will stop making that first call to their pathologists, clinical
chemists, and laboratory scientists to discuss a patient’s condition and
consult on which test to order, how to interpret the results, and get guidance
on selecting therapies and monitoring the patient’s progress.
Instead, a “smart software solution” will be inserted into
the clinical workflow of physicians. This solution will automatically guide the
physician to follow the established care protocol. In turn, this will give the
medical laboratory the simple role of accepting a lab test order, performing
the analysis, and reporting the results.
If this were true, then it could be argued that a laboratory
test is a commodity and hospitals, physicians, and payers would argue that they
should buy these commodity lab tests at the cheapest price.
Researchers find a savings of more than one million dollars and prevention of hundreds, if not thousands, of adverse drug events could have been had with machine learning system
Support for artificial intelligence (AI) and machine learning (ML) in healthcare has been mixed among anatomic pathologists and clinical laboratory leaders. Nevertheless, there’s increasing evidence that diagnostic systems based on AI and ML can be as accurate or more accurate at detecting disease than systems without them.
Dark Daily has covered the development of artificial intelligence and machine learning systems and their ability to accurately detect disease in many e-briefings over the years. Now, a recent study conducted at Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) suggests machine learning can be more accurate than existing clinical decision support (CDS) systems at detecting prescription medication errors as well.
The study was partially retrospective in that the
researchers compiled past alerts generated by the CDS systems at BWH and MGH
between 2009-2011 and added them to alerts generated during the active part of
the study, which took place from January 1, 2012 to December 31, 2013, for a
total of five years’ worth of CDS alerts.
They then sent the same patient-encounter data that generated those CDS alerts to a machine learning platform called MedAware, an AI-enabled software system developed in Ra’anana, Israel.
MedAware was created for the “identification and prevention
of prescription errors and adverse drug effects,” notes the study, which goes
on to state, “This system identifies medication issues based on machine
learning using a set of algorithms with different complexity levels, ranging
from statistical analysis to deep learning with neural networks. Different
algorithms are used for different types of medication errors. The data elements
used by the algorithms include demographics, encounters, lab test results,
vital signs, medications, diagnosis, and procedures.”
The researchers then compared the alerts produced by
MedAware to the existing CDS alerts from that 5-year period. The results were
astonishing.
According to the study:
“68.2% of the alerts generated were unique to
the MedAware system and not generated by the institutions’ CDS alerting system.
“Clinical outlier alerts were the type least
likely to be generated by the institutions’ CDS—99.2% of these alerts were
unique to the MedAware system.
“The largest overlap was with dosage alerts,
with only 10.6% unique to the MedAware system.
“68% of the time-dependent alerts were unique to
the MedAware system.”
Perhaps even more important was the results of the cost
analysis, which found:
“The average cost of an adverse event
potentially prevented by an alert was $60.67 (range: $5.95–$115.40).
“The average adverse event cost per type of
alert varied from $14.58 (range: $2.99–$26.18) for dosage outliers to $19.14
(range: $1.86–$36.41) for clinical outliers and $66.47 (range: $6.47–$126.47)
for time-dependent alerts.”
The researchers concluded that, “Potential savings of $60.67 per alert was mainly derived from the prevention of ADEs [adverse drug events]. The prevention of ADEs could result in savings of $60.63 per alert, representing 99.93% of the total potential savings. Potential savings related to averted calls between pharmacists and clinicians could save an average of $0.047 per alert, representing 0.08% of the total potential savings.
“Extrapolating the results of the analysis to the 747,985
BWH and MGH patients who had at least one outpatient encounter during the
two-year study period from 2012 to 2013, the alerts that would have been fired
over five years of their clinical care by the machine learning medication
errors identification system could have resulted in potential savings of
$1,294,457.”
Savings of more than one million dollars plus the prevention
of potential patient harm or deaths caused by thousands of adverse drug events
is a strong argument for machine learning platforms in diagnostics and
prescription drug monitoring.
Researchers Say Current Clinical Decision Support Systems
are Limited
Machine learning is not the same as artificial intelligence. ML is a “discipline of AI” which aims for “enhancing accuracy,” while AI’s objective is “increasing probability of success,” explained Tech Differences.
Healthcare needs the help. Prescription medication errors cause patient harm or deaths that cost more than $20 billion annually, states a Joint Commission news release.
CDS alerting systems are widely used to improve patient
safety and quality of care. However, the BWH-MGH researchers say the current
CDS systems “have a variety of limitations.” According to the study:
“One limitation is that current CDS systems are rule-based and can thus identify only the medication errors that have been previously identified and programmed into their alerting logic.
“Further, most have high alerting rates with many false positives, resulting in alert fatigue.”
Commenting on the value of adding machine learning
medication alerts software to existing CDS hospital systems, the BWH-MGH
researchers wrote, “This kind of approach can complement traditional rule-based
decision support, because it is likely to find additional errors that would not
be identified by usual rule-based approaches.”
However, they concluded, “The true value of such alerts is
highly contingent on whether and how clinicians respond to such alerts and
their potential to prevent actual patient harm.”
Future research based on real-time data is needed before machine
learning systems will be ready for use in clinical settings, HealthITAnalytics
noted.
However, medical laboratory leaders and pathologists will
want to keep an eye on developments in machine learning and artificial
intelligence that help physicians reduce medication errors and adverse drug
events. Implementation of AI-ML systems in healthcare will certainly affect
clinical laboratory workflows.
Misdiagnosis by doctors leads to many of the cases that we hear about in the news (or on the TV show “House.”). We live in an age where doctors are under pressure to see as many patients in as little time as possible. Not surprising, then, that many physicians often diagnose the most obvious medical condition they deem appropriate without full and detailed consideration of what alternative medical conditions may also be present.
Kaiser Permanente and the Veterans Health Administration are bringing the issue of misdiagnosis to the forefront with their adoption of a Web-based “decision support” software program called “Isabel.” Isabel and similar systems help doctors by offering an array of possible diagnoses they might not have considered or prompting them to perform appropriate tests on patients with certain symptoms. In a study at the VA Medical Center in Northport, NY, Isabel suggested the correct diagnosis in 98% of cases in which the system was used. Doctors have recognized that this system is an excellent training tool for residents and an invaluable reminder that the simplest explanation is not always the right one when it comes to medical conditions.
I spoke to a friend of mine who is a general practice doctor at the Scott & White Clinic in Georgetown, TX. The facility was on the verge of adopting a decision support program that involved PDAs programmed to suggest an appropriate diagnosis based on symptoms in each general practice exam room. “At what point,” she asked, “am I even necessary anymore? I’m starting to question why I even needed to go to medical school – Anyone could use this thing and come up with the right diagnosis!” Unfortunately, my friend’s attitude will likely be mirrored by many doctors who are set in their ways and unfamiliar with this technology. It’s true that, in a large portion of medical cases, the right answer is a simple one, but decision support programs assist doctors who use them correctly to consider alternative conditions, which may save a patient’s life.
Dark Daily predicts that use of clinical decision support systems like Isabel will increase in coming years. It is a logical consequence of the patient safety movement as well as the motivations provided by pay-for-performance programs. Another reason why health care facilities are likely to embrace these systems is that they can electronically document that the physician did the right thing for the patient, based on the fact that the clinical decision support system agreed with the physicians’ evaluation of symptoms and likely medical conditions.
What remains to be seen is how such clinical decision support systems impact laboratory test ordering patterns and how clinicians follow up on laboratory test results. Clinical laboratory managers and pathologists in health systems and hospitals already using such systems report that overall test utilization declines in the weeks following implementation. Going forward, they say that physicians begin to increase their consultations with pathologists and technical lab staff. So the early evidence is that clinical decision support systems can encourage physicians to make better use of the clinical laboratory’s expertise.
