News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel

News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel
Sign In

Florida Hospital Utilizes Machine Learning Artificial Intelligence Platform to Reduce Clinical Variation in Its Healthcare, with Implications for Medical Laboratories

Pathologists and clinical laboratory scientists may find one hospital’s use of a machine-learning platform to help improve utilization of lab tests both an opportunity and a threat

Variation in how individual physicians order, interpret, and act upon clinical laboratory test results is regularly shown by studies in peer-reviewed medical journals to be one reason why some patients get great outcomes and other patients get less-than-desirable outcomes. That is why many healthcare providers are initiating efforts to improve how physicians utilize clinical laboratory tests and other diagnostic procedures.

At Flagler Hospital, a 335-bed not-for-profit healthcare facility in St. Augustine, Fla., a new tool is being used to address variability in clinical care. It is a machine learning platform called Symphony AyasdiAI for Clinical Variation Management (AyasdiAI) from Silicon Valley-based SymphonyAI Group. Flagler is using this system to develop its own clinical order set built from clinical data contained within the hospital’s electronic health record (EHR) and financial systems.

This effort came about after clinical and administrative leadership at Flagler Hospital realized that only about one-third of its physicians regularly followed certain medical decision-making guidelines or clinical order sets. Armed with these insights, staff members decided to find a solution that reduced or removed variability from their healthcare delivery.

Reducing Variability Improves Care, Lowers Cost

Variability in physician care has been linked to increased healthcare costs and lower quality outcomes, as studies published in JAMA and JAMA Internal Medicine indicate. Such results do not bode well for healthcare providers in today’s value-based reimbursement system, which rewards increased performance and lowered costs.

“Fundamentally, what these technologies do is help us recognize important patterns in the data,” Douglas Fridsma, PhD, an expert in health informatics, standards, interoperability, and health IT strategy, and CEO of the American Medical Informatics Association (AMIA), told Modern Healthcare.

Clinical order sets are designed to be used as part of clinical decision support systems (CDSS) installed by hospitals for physicians to standardize care and support sound clinical decision making and patient safety.

However, when doctors don’t adhere to those pre-defined standards, the results can be disadvantageous, ranging from unnecessary services and tests being performed to preventable complications for patients, which may increase treatment costs.

“Over the past few decades we’ve come to realize clinical variation plays an important part in the overuse of medical care and the waste that occurs in healthcare, making it more expensive than it should be,” Michael Sanders, MD (above) Flagler’s Chief Medical Information Officer, told Modern Healthcare. “Every time we’re adding something that adds cost, we have to make sure that we’re adding value.” (Photo copyright: Modern Healthcare.)

Flagler’s AI project involved uploading clinical, demographic, billing, and surgical information to the AyasdiAI platform, which then employed machine learning to analyze the data and identify trends. Flagler’s physicians are now provided with a fuller picture of their patients’ conditions, which helps identify patients at highest risk, ensuring timely interventions that produce positive outcomes and lower costs.

How Symphony AyasdiAI Works

The AyasdiAI application utilizes a category of mathematics called topological data analysis (TDA) to cluster similar patients together and locate parallels between those groups. “We then have the AI tools generate a carepath from this group, showing all events which should occur in the emergency department, at admission, and throughout the hospital stay,” Sanders told Healthcare IT News. “These events include all medications, diagnostic tests, vital signs, IVs, procedures and meals, and the ideal timing for the occurrence of each so as to replicate the results of this group.”

Caregivers then examine the data to determine the optimal plan of care for each patient. Cost savings are figured into the overall equation when choosing a treatment plan. 

Flagler first used the AI program to examine trends among their pneumonia patients. They determined that nebulizer treatments should be started as soon as possible with pneumonia patients who also have chronic obstructive pulmonary disease (COPD).

“Once we have the data loaded, we use [an] unsupervised learning AI algorithm to generate treatment groups,” Sanders told Healthcare IT News. “In the case of our pneumonia patient data, Ayasdi produced nine treatments groups. Each group was treated similarly, and statistics were given to us to understand that group and how it differed from the other groups.”

Armed with this information, the hospital achieved an 80% greater physician adherence to order sets for pneumonia patients. This resulted in a savings of $1,350 per patient and reduced the readmission rates for pneumonia patients from 2.9% to 0.4%, reported Modern Healthcare.

The development of a machine-learning platform designed to reduce variation in care (by helping physicians become more consistent at following accepted clinical care guidelines) can be considered a warning shot across the bow of the pathology profession.

This is a system that has the potential to become interposed between the pathologist in the medical laboratory and the physicians who refer specimens to the lab. Were that to happen, the deep experience and knowledge that have long made pathologists the “doctor’s doctor” will be bypassed. Physicians will stop making that first call to their pathologists, clinical chemists, and laboratory scientists to discuss a patient’s condition and consult on which test to order, how to interpret the results, and get guidance on selecting therapies and monitoring the patient’s progress.

Instead, a “smart software solution” will be inserted into the clinical workflow of physicians. This solution will automatically guide the physician to follow the established care protocol. In turn, this will give the medical laboratory the simple role of accepting a lab test order, performing the analysis, and reporting the results.

If this were true, then it could be argued that a laboratory test is a commodity and hospitals, physicians, and payers would argue that they should buy these commodity lab tests at the cheapest price.

—JP Schlingman

Related Information:

Flagler Hospital Combines AI, Physician Committee to Minimize Clinical Variation

Flagler Hospital Uses AI to Create Clinical Pathways That Enhance Care and Slash Costs

Case Study: Flagler Hospital, How One of America’s Oldest Cities Became Home to One of America’s Most Innovative Hospitals

How Using Artificial Intelligence Enabled Flagler Hospital to Reduce Clinical Variation

Florida Hospital to Save $20M Through AI-enabled Clinical Variation

The Journey from Volume to Value-Based Care Starts Here

The Science of Clinical Carepaths

Machine Learning System Catches Two-Thirds More Prescription Medication Errors than Existing Clinical Decision Support Systems at Two Major Hospitals

Researchers find a savings of more than one million dollars and prevention of hundreds, if not thousands, of adverse drug events could have been had with machine learning system

Support for artificial intelligence (AI) and machine learning (ML) in healthcare has been mixed among anatomic pathologists and clinical laboratory leaders. Nevertheless, there’s increasing evidence that diagnostic systems based on AI and ML can be as accurate or more accurate at detecting disease than systems without them.

Dark Daily has covered the development of artificial intelligence and machine learning systems and their ability to accurately detect disease in many e-briefings over the years. Now, a recent study conducted at Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) suggests machine learning can be more accurate than existing clinical decision support (CDS) systems at detecting prescription medication errors as well.

The researchers published their findings in the Joint Commission Journal on Quality and Patient Safety, titled, “Using a Machine Learning System to Identify and Prevent Medication Prescribing Errors: A Clinical and Cost Analysis Evaluation.”

A Retrospective Study

The study was partially retrospective in that the researchers compiled past alerts generated by the CDS systems at BWH and MGH between 2009-2011 and added them to alerts generated during the active part of the study, which took place from January 1, 2012 to December 31, 2013, for a total of five years’ worth of CDS alerts.

They then sent the same patient-encounter data that generated those CDS alerts to a machine learning platform called MedAware, an AI-enabled software system developed in Ra’anana, Israel.

MedAware was created for the “identification and prevention of prescription errors and adverse drug effects,” notes the study, which goes on to state, “This system identifies medication issues based on machine learning using a set of algorithms with different complexity levels, ranging from statistical analysis to deep learning with neural networks. Different algorithms are used for different types of medication errors. The data elements used by the algorithms include demographics, encounters, lab test results, vital signs, medications, diagnosis, and procedures.”

The researchers then compared the alerts produced by MedAware to the existing CDS alerts from that 5-year period. The results were astonishing.

According to the study:

  • “68.2% of the alerts generated were unique to the MedAware system and not generated by the institutions’ CDS alerting system.
  • “Clinical outlier alerts were the type least likely to be generated by the institutions’ CDS—99.2% of these alerts were unique to the MedAware system.
  • “The largest overlap was with dosage alerts, with only 10.6% unique to the MedAware system.
  • “68% of the time-dependent alerts were unique to the MedAware system.”

Perhaps even more important was the results of the cost analysis, which found:

  • “The average cost of an adverse event potentially prevented by an alert was $60.67 (range: $5.95–$115.40).
  • “The average adverse event cost per type of alert varied from $14.58 (range: $2.99–$26.18) for dosage outliers to $19.14 (range: $1.86–$36.41) for clinical outliers and $66.47 (range: $6.47–$126.47) for time-dependent alerts.”

The researchers concluded that, “Potential savings of $60.67 per alert was mainly derived from the prevention of ADEs [adverse drug events]. The prevention of ADEs could result in savings of $60.63 per alert, representing 99.93% of the total potential savings. Potential savings related to averted calls between pharmacists and clinicians could save an average of $0.047 per alert, representing 0.08% of the total potential savings.

“Extrapolating the results of the analysis to the 747,985 BWH and MGH patients who had at least one outpatient encounter during the two-year study period from 2012 to 2013, the alerts that would have been fired over five years of their clinical care by the machine learning medication errors identification system could have resulted in potential savings of $1,294,457.”

Savings of more than one million dollars plus the prevention of potential patient harm or deaths caused by thousands of adverse drug events is a strong argument for machine learning platforms in diagnostics and prescription drug monitoring.

“There’s huge promise for machine learning in healthcare. If clinicians use the technology on the front lines, it could lead to improved clinical decision support and new information at the point of care,” said Raj Ratwani, PhD (above), Vice President of Scientific Affairs at MedStar Health Research Institute (MHRI), Director of MedStar Health’s National Center for Human Factors in Healthcare, and Associate Professor of Emergency Medicine at Georgetown University School of Medicine, told HealthITAnalytics. [Photo copyright: MedStar Institute for Innovation.)

Researchers Say Current Clinical Decision Support Systems are Limited

Machine learning is not the same as artificial intelligence. ML is a “discipline of AI” which aims for “enhancing accuracy,” while AI’s objective is “increasing probability of success,” explained Tech Differences.

Healthcare needs the help. Prescription medication errors cause patient harm or deaths that cost more than $20 billion annually, states a Joint Commission news release.

CDS alerting systems are widely used to improve patient safety and quality of care. However, the BWH-MGH researchers say the current CDS systems “have a variety of limitations.” According to the study:

  • “One limitation is that current CDS systems are rule-based and can thus identify only the medication errors that have been previously identified and programmed into their alerting logic.
  • “Further, most have high alerting rates with many false positives, resulting in alert fatigue.”

Alert fatigue leads to physician burnout, which is a big problem in large healthcare systems using multiple health information technology (HIT) systems that generate large amounts of alerts, such as: electronic health record (EHR) systems, hospital information systems (HIS), laboratory information systems (LIS), and others.

Commenting on the value of adding machine learning medication alerts software to existing CDS hospital systems, the BWH-MGH researchers wrote, “This kind of approach can complement traditional rule-based decision support, because it is likely to find additional errors that would not be identified by usual rule-based approaches.”

However, they concluded, “The true value of such alerts is highly contingent on whether and how clinicians respond to such alerts and their potential to prevent actual patient harm.”

Future research based on real-time data is needed before machine learning systems will be ready for use in clinical settings, HealthITAnalytics noted. 

However, medical laboratory leaders and pathologists will want to keep an eye on developments in machine learning and artificial intelligence that help physicians reduce medication errors and adverse drug events. Implementation of AI-ML systems in healthcare will certainly affect clinical laboratory workflows.

—Donna Marie Pocius

Related Information:

AI and Healthcare: A Giant Opportunity

Using a Machine Learning System to Identify and Prevent Medication Prescribing Errors:  A Clinical and Cost Analysis Evaluation

Machine Learning System Accurately Identifies Medication Errors

Journal Study Evaluates Success of Automated Machine Learning System to Prevent Medication Prescribing Errors

Differences Between Machine Learning and Artificial Intelligence

Machining a New Layer of Drug Safety

Harvard and Beth Israel Deaconess Researchers Use Machine Learning Software Plus Human Intelligence to Improve Accuracy and Speed of Cancer Diagnoses

XPRIZE Founder Diamandis Predicts Tech Giants Amazon, Apple, and Google Will Be Doctors of The Future

Hospitals Worldwide Are Deploying Artificial Intelligence and Predictive Analytics Systems for Early Detection of Sepsis in a Trend That Could Help Clinical Laboratories, Microbiologists

Doctors Get Electronic Help with Their Diagnoses Via Decision Support Software

Misdiagnosis by doctors leads to many of the cases that we hear about in the news (or on the TV show “House.”). We live in an age where doctors are under pressure to see as many patients in as little time as possible. Not surprising, then, that many physicians often diagnose the most obvious medical condition they deem appropriate without full and detailed consideration of what alternative medical conditions may also be present.

Kaiser Permanente and the Veterans Health Administration are bringing the issue of misdiagnosis to the forefront with their adoption of a Web-based “decision support” software program called “Isabel.” Isabel and similar systems help doctors by offering an array of possible diagnoses they might not have considered or prompting them to perform appropriate tests on patients with certain symptoms. In a study at the VA Medical Center in Northport, NY, Isabel suggested the correct diagnosis in 98% of cases in which the system was used. Doctors have recognized that this system is an excellent training tool for residents and an invaluable reminder that the simplest explanation is not always the right one when it comes to medical conditions.

I spoke to a friend of mine who is a general practice doctor at the Scott & White Clinic in Georgetown, TX. The facility was on the verge of adopting a decision support program that involved PDAs programmed to suggest an appropriate diagnosis based on symptoms in each general practice exam room. “At what point,” she asked, “am I even necessary anymore? I’m starting to question why I even needed to go to medical school – Anyone could use this thing and come up with the right diagnosis!” Unfortunately, my friend’s attitude will likely be mirrored by many doctors who are set in their ways and unfamiliar with this technology. It’s true that, in a large portion of medical cases, the right answer is a simple one, but decision support programs assist doctors who use them correctly to consider alternative conditions, which may save a patient’s life.

Dark Daily predicts that use of clinical decision support systems like Isabel will increase in coming years. It is a logical consequence of the patient safety movement as well as the motivations provided by pay-for-performance programs. Another reason why health care facilities are likely to embrace these systems is that they can electronically document that the physician did the right thing for the patient, based on the fact that the clinical decision support system agreed with the physicians’ evaluation of symptoms and likely medical conditions.

What remains to be seen is how such clinical decision support systems impact laboratory test ordering patterns and how clinicians follow up on laboratory test results. Clinical laboratory managers and pathologists in health systems and hospitals already using such systems report that overall test utilization declines in the weeks following implementation. Going forward, they say that physicians begin to increase their consultations with pathologists and technical lab staff. So the early evidence is that clinical decision support systems can encourage physicians to make better use of the clinical laboratory’s expertise.

Related Articles:

Preventing the tragedy of misdiagnosis

Why Doctors So Often Get It Wrong

More on Clinical decision support systems (CDSSs)