News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel

News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel
Sign In

Machine Learning System Catches Two-Thirds More Prescription Medication Errors than Existing Clinical Decision Support Systems at Two Major Hospitals

Researchers find a savings of more than one million dollars and prevention of hundreds, if not thousands, of adverse drug events could have been had with machine learning system

Support for artificial intelligence (AI) and machine learning (ML) in healthcare has been mixed among anatomic pathologists and clinical laboratory leaders. Nevertheless, there’s increasing evidence that diagnostic systems based on AI and ML can be as accurate or more accurate at detecting disease than systems without them.

Dark Daily has covered the development of artificial intelligence and machine learning systems and their ability to accurately detect disease in many e-briefings over the years. Now, a recent study conducted at Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) suggests machine learning can be more accurate than existing clinical decision support (CDS) systems at detecting prescription medication errors as well.

The researchers published their findings in the Joint Commission Journal on Quality and Patient Safety, titled, “Using a Machine Learning System to Identify and Prevent Medication Prescribing Errors: A Clinical and Cost Analysis Evaluation.”

A Retrospective Study

The study was partially retrospective in that the researchers compiled past alerts generated by the CDS systems at BWH and MGH between 2009-2011 and added them to alerts generated during the active part of the study, which took place from January 1, 2012 to December 31, 2013, for a total of five years’ worth of CDS alerts.

They then sent the same patient-encounter data that generated those CDS alerts to a machine learning platform called MedAware, an AI-enabled software system developed in Ra’anana, Israel.

MedAware was created for the “identification and prevention of prescription errors and adverse drug effects,” notes the study, which goes on to state, “This system identifies medication issues based on machine learning using a set of algorithms with different complexity levels, ranging from statistical analysis to deep learning with neural networks. Different algorithms are used for different types of medication errors. The data elements used by the algorithms include demographics, encounters, lab test results, vital signs, medications, diagnosis, and procedures.”

The researchers then compared the alerts produced by MedAware to the existing CDS alerts from that 5-year period. The results were astonishing.

According to the study:

  • “68.2% of the alerts generated were unique to the MedAware system and not generated by the institutions’ CDS alerting system.
  • “Clinical outlier alerts were the type least likely to be generated by the institutions’ CDS—99.2% of these alerts were unique to the MedAware system.
  • “The largest overlap was with dosage alerts, with only 10.6% unique to the MedAware system.
  • “68% of the time-dependent alerts were unique to the MedAware system.”

Perhaps even more important was the results of the cost analysis, which found:

  • “The average cost of an adverse event potentially prevented by an alert was $60.67 (range: $5.95–$115.40).
  • “The average adverse event cost per type of alert varied from $14.58 (range: $2.99–$26.18) for dosage outliers to $19.14 (range: $1.86–$36.41) for clinical outliers and $66.47 (range: $6.47–$126.47) for time-dependent alerts.”

The researchers concluded that, “Potential savings of $60.67 per alert was mainly derived from the prevention of ADEs [adverse drug events]. The prevention of ADEs could result in savings of $60.63 per alert, representing 99.93% of the total potential savings. Potential savings related to averted calls between pharmacists and clinicians could save an average of $0.047 per alert, representing 0.08% of the total potential savings.

“Extrapolating the results of the analysis to the 747,985 BWH and MGH patients who had at least one outpatient encounter during the two-year study period from 2012 to 2013, the alerts that would have been fired over five years of their clinical care by the machine learning medication errors identification system could have resulted in potential savings of $1,294,457.”

Savings of more than one million dollars plus the prevention of potential patient harm or deaths caused by thousands of adverse drug events is a strong argument for machine learning platforms in diagnostics and prescription drug monitoring.

“There’s huge promise for machine learning in healthcare. If clinicians use the technology on the front lines, it could lead to improved clinical decision support and new information at the point of care,” said Raj Ratwani, PhD (above), Vice President of Scientific Affairs at MedStar Health Research Institute (MHRI), Director of MedStar Health’s National Center for Human Factors in Healthcare, and Associate Professor of Emergency Medicine at Georgetown University School of Medicine, told HealthITAnalytics. [Photo copyright: MedStar Institute for Innovation.)

Researchers Say Current Clinical Decision Support Systems are Limited

Machine learning is not the same as artificial intelligence. ML is a “discipline of AI” which aims for “enhancing accuracy,” while AI’s objective is “increasing probability of success,” explained Tech Differences.

Healthcare needs the help. Prescription medication errors cause patient harm or deaths that cost more than $20 billion annually, states a Joint Commission news release.

CDS alerting systems are widely used to improve patient safety and quality of care. However, the BWH-MGH researchers say the current CDS systems “have a variety of limitations.” According to the study:

  • “One limitation is that current CDS systems are rule-based and can thus identify only the medication errors that have been previously identified and programmed into their alerting logic.
  • “Further, most have high alerting rates with many false positives, resulting in alert fatigue.”

Alert fatigue leads to physician burnout, which is a big problem in large healthcare systems using multiple health information technology (HIT) systems that generate large amounts of alerts, such as: electronic health record (EHR) systems, hospital information systems (HIS), laboratory information systems (LIS), and others.

Commenting on the value of adding machine learning medication alerts software to existing CDS hospital systems, the BWH-MGH researchers wrote, “This kind of approach can complement traditional rule-based decision support, because it is likely to find additional errors that would not be identified by usual rule-based approaches.”

However, they concluded, “The true value of such alerts is highly contingent on whether and how clinicians respond to such alerts and their potential to prevent actual patient harm.”

Future research based on real-time data is needed before machine learning systems will be ready for use in clinical settings, HealthITAnalytics noted. 

However, medical laboratory leaders and pathologists will want to keep an eye on developments in machine learning and artificial intelligence that help physicians reduce medication errors and adverse drug events. Implementation of AI-ML systems in healthcare will certainly affect clinical laboratory workflows.

—Donna Marie Pocius

Related Information:

AI and Healthcare: A Giant Opportunity

Using a Machine Learning System to Identify and Prevent Medication Prescribing Errors:  A Clinical and Cost Analysis Evaluation

Machine Learning System Accurately Identifies Medication Errors

Journal Study Evaluates Success of Automated Machine Learning System to Prevent Medication Prescribing Errors

Differences Between Machine Learning and Artificial Intelligence

Machining a New Layer of Drug Safety

Harvard and Beth Israel Deaconess Researchers Use Machine Learning Software Plus Human Intelligence to Improve Accuracy and Speed of Cancer Diagnoses

XPRIZE Founder Diamandis Predicts Tech Giants Amazon, Apple, and Google Will Be Doctors of The Future

Hospitals Worldwide Are Deploying Artificial Intelligence and Predictive Analytics Systems for Early Detection of Sepsis in a Trend That Could Help Clinical Laboratories, Microbiologists

As Accreditation Inspections of Clinical Laboratories Become Tougher, Innovative Labs Use Internal Audits and Continuous Quality Improvements to Help Prevent Deficiencies

Savvy medical laboratory managers conduct internal audits of processes involved in deficiency citations so they can uncover how deficiencies occur and help eliminate recurrences

One trend that places clinical laboratories at risk involves increased regulation of lab processes, along with more thorough accreditation inspections. Compared to past years, both developments mean more ways for lab assessors to find greater numbers of deficiencies.

However, leading laboratory accreditation and quality improvement experts say that many deficiencies could be avoided if lab leaders conducted their own internal audits and continuous quality improvement projects ahead of visits by accrediting authorities.

In an exclusive interview with Dark Daily, Randall Querry, Director of Government Relations at the American Association for Laboratory Accreditation (A2LA) said, “Clinical laboratories can do a better job of preparing for the external assessment by doing an internal audit. That is, watching personnel perform tests and noting if they aren’t following the same sequences that standard operating procedures address before the external assessors arrive.”

A2LA is one of the primary accrediting bodies with authority from the Centers for Medicare and Medicaid Services (CMS) to accredit medical laboratories relative to Clinical Laboratory Improvement Amendments (CLIA). Others include:

“This doesn’t have to be an ‘us against them’ exercise. We are all in this together for continual improvement and to ensure we’re doing a better job at the end of each day—that we have had a win,” said Querry said. 

How Should Clinical Laboratories Conduct Internal Audits?

So, what is the best method for clinical laboratory leaders to conduct their own audits of operations and avoid citations of deficiencies?

Lucia Berte, President of Laboratories Made Better, suggested medical laboratories should “Pick a sequence and follow it through.” In the Dark Daily interview, she suggested labs should focus on:

  • The sequence of receiving samples in the laboratory to make certain they are properly accessioned, processed, and distributed;
  • Steps to setting up and running an analyzer; and
  • The process of ensuring tests’ critical values are reported to ordering clinicians and how reports are made.

An internal audit may suggest areas where the clinical lab is not on target to meet regulatory and accreditation criteria. Or, the lab may discover what Querry calls “gray areas”—places where criteria are currently being met, but a trend suggests there could be problems down the road.

“And in those cases, it’s always good to identify areas of improvement for preventative action. They may not be a top priority—such as a deficiency—but the areas are on the radar screen as something to address to prevent it becoming a worsening problem,” Querry said.

Quality Improvement Processes to Address Deficiencies

Berte notes that citations in one area of the lab may suggest the need for continuous improvement projects across all laboratory departments or sections. For example, an accrediting body may cite chemistry for a deficiency while hematology and other departments do okay. However, that determination can be deceiving. 

“There is always an underlying process. And the better question for the clinical laboratory is ‘can we make an improvement project out of this that can solve this problem not only for the area where it was cited, but perhaps prevent this problem from occurring in other lab [departments] prior to the next external accreditation assessments?’” Berte said.

Lack of Uniformity among a Clinical Laboratory’s Departments

Berte says a common deficiency is “lack of a uniform competency assessment program” for staff throughout the lab. Assessors expect laboratory departments to have the same competency assessment in regard to processes, records, and the way documents are created, she explained.

Lucia Berte (above), President of Laboratories Made Better, advises improving quality of documents as a project across the entire lab. “We still have a lot of silo-mentality in labs—where chemistry is different from hematology which is different from transfusion. Labs should have a uniform approach to the way their documents are written, and this is not necessarily the case,” she told Dark Daily. (Photo copyright: Whitehat Communications.)

Competency-related Citations

Berte also said competency-related citations may happen when documents read by auditors are not in sync with what the officials see in the clinical lab during inspections. “People not doing things in the order in which things have to happen. That’s the disconnect.”

Querry, speaking from the perspective of an assessor, adds, “We see a discrepancy and ask—do they have the appropriate work procedures with them at the workstation? Is it accessible? Where is this discrepancy? We identify it and then it’s up to the lab to address it—in training, and between the written procedure and the process.”

Consistency, he says, is important especially in organizations where staff rotate among lab areas and different shifts.

Quality System Essentials for Clinical Laboratories

The website for the Clinical and Laboratory Standards Institute (CLIA) states that implementing a quality management system in the lab involves use of “quality system essentials (QSEs).” QSEs are key to lab workflow, communication, and training. They include documents and records management, assessments, and continual improvement.

Querry emphasizes that trying to predict what the hot citations may be in 2020 is not as important as focusing on the technical competence of the lab and its resources.

“We are not out to play gotcha. We are going in there, looking at all the systems, and doing a sampling of testing in various departments of the lab. It’s up to the lab to show us it is technically competent to perform those tests. And they have the equipment and records that the equipment has been checked and calibrated and maintained. We have an examination process,” he said.

Experts agree, clinical laboratories that prepare for external assessments with internal audits and continuous improvement programs may reduce deficiencies during inspections.

—Donna Marie Pocius

Related Information:

Top Laboratory Deficiencies Across Accreditation Agencies

Implementing a Quality Management System in the Laboratory

Lab Quality Confab hosted by The Dark Report

25th Annual Executive War College on lab and pathology management April 28 – 29, 2020

With Experienced Baby Boomers Retiring in Ever-Larger Numbers, Clinical Laboratories and Pathology Groups Use New Methods to Improve Productivity, Reduce Costs

All labs face the challenge of coping with shrinking budgets and staffing shortages, which is why coaching, management observation, and continuous improvement initiatives are proving helpful to medical laboratories

It’s the biggest generational shift since baby boomers began working in clinical laboratories and anatomic pathology groups. Across the nation, labs are watching their most experienced and knowledgeable medical technologists and other lab scientists retire. The need to train their replacements while maintaining peak productivity and controlling costs is motivating lab leaders to adopt powerful new management methods.

Innovative lab administrators and pathologists recognize that automation and the ability to leverage the increasing amounts of data produced by today’s innovative diagnostic technologies and assays can only go so far in helping to compensate for declining revenues.

This is why one trend is quietly gaining momentum. There are many medical laboratories, pathology groups, and other diagnostics providers working to help their lab staffs create a culture of continuous, meaningful improvement. The stakes are great. Not only is this essential for financial sustainability, it can be the source of competitive advantage with physicians, patients, and payers in today’s increasingly competitive diagnostic market.

This is why many medical laboratories are turning to continuous improvement systems such as Lean to increase personnel skills, reduce waste, and make the most out of shrinking budgets and margins. Yet, without a solid foundation of staff trained in these methods and a framework of processes to encourage improvement, lean laboratory managers often struggle to see consistent, significant improvements.

Optimizing Staff Performance and Developing Improvement Processes with Coaching and Management Observation

Performance Coaching and Management Observation offer powerful tools for laboratory managers to reinforce improvement efforts. They encourage the success of personnel, leverage existing personnel to meet growing demands while maintaining service levels, and establish an effective foundation for Lean execution.

Benefits of effective coaching and management observation sessions for laboratories include:

  • Enhancement of laboratory manager performance and skills;
  • Improved retention of skilled labor and consistent improvement of personnel skills through individualized assessment and improved communication;
  • Establishing a method of creating and maintaining a culture of continuous improvement, while empowering staff across all levels of lab operations; and,
  • Creating a competitive edge on laboratories struggling to implement Lean processes and other optimizations in response to increased workloads, reduced staff, and tighter budgets.

Stephen Stone (left), Managing Director, Argent Global Services, and, Rita D’Angelo, PhD (right), President and CEO, D’Angelo Advantage, spoke at Lab Quality Confab in 2018 on the benefits of coaching and management observation sessions for clinical laboratories and implementing continuous improvement systems using Lean Production Methods. (Photo copyright: Dark Daily.)

“While many laboratories are familiar with management observation because of competency testing, few laboratories use coaching and management observation as part of their Lean efforts.” Stephen Stone, Managing Director at Argent Global Services told Dark Daily. “Coaching and management observation offers laboratories an effective means to not only increase throughput using existing staffing and encourage sustainable growth, but also increase retention of existing skilled personnel and reduce hiring costs.”

Speaking at Lab Quality Confab in Oct. 2018, Stone highlighted why these later benefits are increasingly important to laboratories. Citing data from LabTestingMatters and the American Society for Clinical Pathology (ASCP), he reported that while the job market for medical technologist and laboratory technician openings should increase by roughly 11,300 openings in 2018, fewer than 5,000 individuals are graduating each year from accredited training programs.

Lab Supervisor Retirements Projected to Exceed Staff Retirements

Of possibly greater concern, he goes on to point out, is that projected retirement rates for supervisors are higher than those of staff. This means laboratories which fail to focus on staff development and retention could face further issues in both leadership and staffing shortages should trends continue.

“Investing in and empowering your staff will improve productivity, improve quality, improve safety, and help laboratories to work toward goals as a cohesive team,” Rita D’Angelo, PhD, President and CEO at D’Angelo Advantage, LLC, told Dark Daily. “As a result, costs and waste drop significantly. Coaching and management observation alongside a culture of continuous improvement can help labs to overcome many of the staffing and budget obstacles faced today.”

Preparing Your Lab for Continuous Improvement

To help labs prepare for these significant trends, D’Angelo and Stone will co-present a 90-minute webinar on Jan. 16th titled, “Performance Coaching and Management Observations to Improve Productivity and Efficiency: Strengthening the Skills of Management to Execute a Lean Lab Transformation.”

The webinar will include essential coaching skills to help laboratory managers pass on the skills to serve as Lean champions to personnel and establish the foundation and structure for a lasting culture of improvement within the laboratory.

C-Level laboratory leadership, laboratory directors, managers and supervisors, and key members of continuous improvement teams also can use the interactive Q/A session following the webinar to gain answers to questions and concerns directly facing their laboratories’ efforts to develop continuous improvement processes or implement Lean methodologies.

(To register for this critical Jan. 16th webinar, click here. Or, copy and paste this URL into your browser:

—Jon Stone

Related Information:

Performance Coaching and Management Observations to Improve Productivity and Efficiency: Strengthening the Skills of Management to Execute a Lean Lab Transformation

Performance Coaching and Management Observations in the Lab: Master this Proven Way to Develop Your Lab’s Managers, Supervisors, and Lab Staff

Secrets of Effective Culture Change in Hospital and Health System Labs: Engaging Staff to Continuously Improve, Sustain Quality, and Regularly Cut Unnecessary Costs

Facing Increasing Pressure from an Evolving Regulatory Environment, Clinical Laboratories Need to be ‘Inspection Ready’ in 2019

Experts say medical laboratories must overcome staffing challenges and develop a culture of continual inspection readiness

Clinical laboratories and anatomic pathology groups will be entering 2019 facing not only financial upheaval from Medicare Part B fee cuts and narrowing payer networks, but also an increasingly rigorous and ever-evolving regulatory environment.

Although the Clinical Laboratory Improvement Amendments (CLIA) requirements have not undergone major changes since debuting in 1992, the medical laboratory industry has become more complex and technology-laden, resulting in old rules being applied to emerging technologies.

“People can get lulled into this sense that we know what standards are and we’re meeting them, but what has happened is the rules haven’t really changed—the industry has,” says Nora L. Hess, MBA, MT(ASCP), Senior Consultant for Accumen, a lab quality improvement company in San Diego, CA. “Technology is now jumping ahead so fast that keeping up with it and understanding how the rules are going to be applied is challenging.”

Hess and Anne T. Daley, MS, Quality Officer at ARUP Laboratories in Salt Lake City, UT, will co-present a 90-minute webinar on Dec. 13th titled, “Make Your Lab Assessment Ready in 2019: Know the Most Common Deficiencies in Accreditation and Certification and What to Expect in the Future.”

The program will review the top deficiencies reported by the College of American Pathologists (CAP), the Joint Commission, A2LA, and COLA.

Anne T. Daley, MS, CMQOE, CSSBB, CLC, MT, DLM (left), and Nora L. Hess, MBA, MT(ASCP), PMP (right), are Senior Consultants with Chi Solutions, an Accumen company. Together, they have extensive first-hand experience guiding clinical laboratories through the rigid and rigorous process of achieving inspection-ready status. (Photo copyrights: Chi Solutions, Inc.)

Why being ‘Inspection Ready’ Can Improve Performance and Increase Revenue

Striving to be “inspection ready” should be the goal of every clinical laboratory and pathology group. However, Daley notes labs typically operate with staffs that are stretched thin by retirements, illnesses, staff shortages, or ancillary demands on administrators’ time, caused by system-wide initiatives that range from electronic health record (EHR) rollouts to integration and consolidation of other labs.

“Most hospital laboratories are staffed at a level where they are working hard just to meet the daily patient care needs,” Daley says. “You add in all these additional projects and something has to give.”

In today’s challenging environment, Hess says laboratories that consistently perform well during the inspection process share several characteristics, including the following:

  • a culture of quality across the laboratory; and,
  • an emphasis on inspection readiness.

“They make inspection readiness a consistent focus across the entire laboratory,” Hess notes. “It becomes part of the calendar. It becomes part of what they talk about all the time. It is a part of their decision-making. It is hardwired into who they are and what they do … successful laboratories are making this part of their day-to-day activities, so it doesn’t get lost in the shuffle and it doesn’t get back-burnered.”

Hess and Daley note the types of deficiencies cited by accreditation agencies tend not to vary much from year to year. Testing personnel competency and proficiency testing are common themes, annually ranking high on top-10 lists of deficiencies found. However, laboratory directors increasingly are being singled out for issues related to qualifications and performance of responsibilities.

(To register for their Dec. 13th webinar or to order a DVD, click here. Or, copy and paste this URL into your browser:

Andrea Downing Peck

Related Information:

Make Your Lab Assessment Ready in 2019: Know the Most Common Deficiencies in Accreditation and Certification

CLIA Inspection–What they Mean and How to Avoid Them

‘We Wanted to Be the Best we Could Possibly Be’: CAP ISO 15189-Accredited Labs on the Difference it Makes

Are CLIA Inspections of Clinical Pathology Laboratories Getting Tougher? Why It Pays to Achieve “Inspection Readiness”

Diagnostic Errors Get the Attention of the Institute of Medicine, Reinforcing Efforts by Nation’s Clinical Pathology Laboratory Scientists to Improve Patient Safety

Along with its assessment of the rate of errors in diagnosis, the IOM has a plan to improve, but will doctors accept the IOM’s advice, or continue business as usual?

Diagnostic errors in the American healthcare system is a problem that is now on the radar screen of policymakers at the Institute of Medicine (IOM). Pathologists and clinical laboratory professionals will welcome this development, because recommendations from the IOM carry weight with Congress.

Thus, should the IOM develop specific actions items intended to reduce medical errors, not only are these suggestions likely to involve more effective use of medical laboratory tests by physicians, but there is a strong probability that Congress might eventually write these recommendations into future healthcare legislation.

The Institute of Medicine is a division of the National Academies of Sciences, Engineering, and Medicine. The IOM recently convened a committee that released a list of recommendations to address the problem of diagnostic errors in medicine. Those recommendations, however, are running up against ingrained mindsets and overconfidence on the part of physicians who are reluctant to include decision-support technology in the diagnostic process. (more…)