Researchers find a savings of more than one million dollars and prevention of hundreds, if not thousands, of adverse drug events could have been had with machine learning system
Support for artificial intelligence (AI) and machine learning (ML) in healthcare has been mixed among anatomic pathologists and clinical laboratory leaders. Nevertheless, there’s increasing evidence that diagnostic systems based on AI and ML can be as accurate or more accurate at detecting disease than systems without them.
Dark Daily has covered the development of artificial intelligence and machine learning systems and their ability to accurately detect disease in many e-briefings over the years. Now, a recent study conducted at Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) suggests machine learning can be more accurate than existing clinical decision support (CDS) systems at detecting prescription medication errors as well.
The study was partially retrospective in that the
researchers compiled past alerts generated by the CDS systems at BWH and MGH
between 2009-2011 and added them to alerts generated during the active part of
the study, which took place from January 1, 2012 to December 31, 2013, for a
total of five years’ worth of CDS alerts.
They then sent the same patient-encounter data that generated those CDS alerts to a machine learning platform called MedAware, an AI-enabled software system developed in Ra’anana, Israel.
MedAware was created for the “identification and prevention
of prescription errors and adverse drug effects,” notes the study, which goes
on to state, “This system identifies medication issues based on machine
learning using a set of algorithms with different complexity levels, ranging
from statistical analysis to deep learning with neural networks. Different
algorithms are used for different types of medication errors. The data elements
used by the algorithms include demographics, encounters, lab test results,
vital signs, medications, diagnosis, and procedures.”
The researchers then compared the alerts produced by
MedAware to the existing CDS alerts from that 5-year period. The results were
astonishing.
According to the study:
“68.2% of the alerts generated were unique to
the MedAware system and not generated by the institutions’ CDS alerting system.
“Clinical outlier alerts were the type least
likely to be generated by the institutions’ CDS—99.2% of these alerts were
unique to the MedAware system.
“The largest overlap was with dosage alerts,
with only 10.6% unique to the MedAware system.
“68% of the time-dependent alerts were unique to
the MedAware system.”
Perhaps even more important was the results of the cost
analysis, which found:
“The average cost of an adverse event
potentially prevented by an alert was $60.67 (range: $5.95–$115.40).
“The average adverse event cost per type of
alert varied from $14.58 (range: $2.99–$26.18) for dosage outliers to $19.14
(range: $1.86–$36.41) for clinical outliers and $66.47 (range: $6.47–$126.47)
for time-dependent alerts.”
The researchers concluded that, “Potential savings of $60.67 per alert was mainly derived from the prevention of ADEs [adverse drug events]. The prevention of ADEs could result in savings of $60.63 per alert, representing 99.93% of the total potential savings. Potential savings related to averted calls between pharmacists and clinicians could save an average of $0.047 per alert, representing 0.08% of the total potential savings.
“Extrapolating the results of the analysis to the 747,985
BWH and MGH patients who had at least one outpatient encounter during the
two-year study period from 2012 to 2013, the alerts that would have been fired
over five years of their clinical care by the machine learning medication
errors identification system could have resulted in potential savings of
$1,294,457.”
Savings of more than one million dollars plus the prevention
of potential patient harm or deaths caused by thousands of adverse drug events
is a strong argument for machine learning platforms in diagnostics and
prescription drug monitoring.
Researchers Say Current Clinical Decision Support Systems
are Limited
Machine learning is not the same as artificial intelligence. ML is a “discipline of AI” which aims for “enhancing accuracy,” while AI’s objective is “increasing probability of success,” explained Tech Differences.
Healthcare needs the help. Prescription medication errors cause patient harm or deaths that cost more than $20 billion annually, states a Joint Commission news release.
CDS alerting systems are widely used to improve patient
safety and quality of care. However, the BWH-MGH researchers say the current
CDS systems “have a variety of limitations.” According to the study:
“One limitation is that current CDS systems are rule-based and can thus identify only the medication errors that have been previously identified and programmed into their alerting logic.
“Further, most have high alerting rates with many false positives, resulting in alert fatigue.”
Commenting on the value of adding machine learning
medication alerts software to existing CDS hospital systems, the BWH-MGH
researchers wrote, “This kind of approach can complement traditional rule-based
decision support, because it is likely to find additional errors that would not
be identified by usual rule-based approaches.”
However, they concluded, “The true value of such alerts is
highly contingent on whether and how clinicians respond to such alerts and
their potential to prevent actual patient harm.”
Future research based on real-time data is needed before machine
learning systems will be ready for use in clinical settings, HealthITAnalytics
noted.
However, medical laboratory leaders and pathologists will
want to keep an eye on developments in machine learning and artificial
intelligence that help physicians reduce medication errors and adverse drug
events. Implementation of AI-ML systems in healthcare will certainly affect
clinical laboratory workflows.
The scientist also employed machine learning “to gauge how easily accessible genes are for transcription” in research that could lead to new clinical laboratory diagnostic tests
Anatomic pathologists and clinical laboratories are of course familiar with the biological science of genomics, which, among other things, has been used to map the human genome. But did you know that a three-dimensional (3D) map of a genome has been created and that it is helping scientists understand how DNA regulates its organization—and why?
The achievement took place at St. Jude Children’s Research Hospital (St. Jude) in Memphis, Tenn. Scientists there created “the first 3D map of a mouse genome” to study “the way cells organize their genomes during development,” a St. Jude news release noted.
Some experts predict that this new approach to understanding how changes happen in a genome could eventually provide new insights that anatomic pathologists and clinical laboratory scientists could find useful when working with physicians to diagnose patients and using the test results to identify the most appropriate therapy for those patients.
In addition to 3D modeling, the researchers applied machine learning to data from multiple sources to see how the organization of the genome changed at different times during development. “The changes are not random, but part of the developmental program of cells,” Dyer said in the news release.
The St. Jude study focused on the rod cells in a mouse retina. That may seem like a narrow scope, but there are more than 8,000 genes involved in retinal development in mice, during which those genes are either turned on or off.
To see what was happening among the cells, the researchers used HI-C analysis, an aspect of ultra-deep chromosome conformation capture, in situ. They found that the loops in the DNA bring together regions of the genome, allowing them to interact in specific ways.
Until this study, how those interactions took place was a
mystery.
“Understanding the way cells organize their genomes during development will help us to understand their ability to respond to stress, injury and disease,”Michael Dyer, PhD (above), Chair of St. Jude’s Developmental Neurobiology Department, co-leader of the Developmental Biology and Solid Tumor Program, and Investigator at Howard Hughes Medical Institute (HHMI), said in the news release. (Photo copyright: St. Jude Children’s Research Hospital.)
The scientists also discovered there were DNA promoters, which encourage gene expression, and also DNA enhancers that increase the likelihood gene expression will occur.
“The research also included the first report of a powerful regulator of gene expression, a super enhancer, that worked in a specific cell at a specific stage of development,” the news release states. “The finding is important because the super enhancers can be hijacked in developmental cancers of the brain and other organs.”
St. Jude goes on to state, “In this study, the scientists determined that when a core regulatory circuit super-enhancer for the VSX2 gene was deleted, an entire class of neurons (bipolar neurons) was eliminated. No other defects were identified. Deletion of the VSX2 gene causes many more defects in retinal development, so the super-enhancer is highly specific to bipolar neurons.”
The St. Jude researchers developed a genetic mouse model of
the defect that scientists are using to study neural circuits in the retina,
the news release states.
Research Technologist Victoria Honnell (left); Developmental Neurobiologist Jackie Norrie, PhD (center); and Postdoctoral Researcher Marybeth Lupo, PhD (right), work in the St. Jude clinical laboratory of Michael Dyer, PhD, using 3D genomic mapping to study gene regulation during development and disease. (Photo copyright: St. Jude Children’s Research Hospital.)
DNA Loops May Matter to Pathology Sooner Rather than
Later
Previous researcher studies primarily used genomic sequencing technology to locate and investigate alterations in genes that lead to disease. In the St. Jude study, the researchers examined how DNA is packaged. If the DNA of a single cell could be stretched out, it would be more than six feet long. To fit into the nucleus of a cell, DNA is looped and bundled into a microscopic package. The St. Jude scientists determined that how these loops are organized regulates how the cell functions and develops.
Scientists around the world will continue studying how the loops in DNA impact gene regulation and how that affects the gene’s response to disease. At St. Jude Children’s Research Hospital, Dyer and his colleagues “used the same approach to create a 3D genomic map of the mouse cerebellum, a brain structure where medulloblastoma can develop. Medulloblastoma is the most common malignant pediatric brain tumor,” noted the St. Jude’s news release.
In addition to providing an understanding of how genes
function, these 3D studies are providing valuable insight into how some
diseases develop and mature. While nascent research such as this may not impact
pathologists and clinical laboratories at the moment, it’s not a stretch to
think that this work may lead to greater understanding of the pathology of
diseases in the near future.
Clinical laboratories working with AI should be aware of ethical challenges being pointed out by industry experts and legal authorities
Experts are voicing concerns that using artificial
intelligence (AI) in healthcare could present ethical challenges that need
to be addressed. They say databases and algorithms may introduce bias into the
diagnostic process, and that AI may not perform as intended, posing a potential
for patient harm.
If true, the issues raised by these experts would have major
implications for how clinical
laboratories and anatomic
pathology groups might use artificial intelligence. For that reason,
medical laboratory executives and pathologists should be aware of possible
drawbacks to the use of AI and machine-learning
algorithms in the diagnostic process.
Is AI Underperforming?
AI’s ability to improve diagnoses, precisely target
therapies, and leverage healthcare data is predicted to be a boon to precision medicine and personalized
healthcare.
For example, Accenture
(NYSE:ACN) says that hospitals will spend $6.6 billion on AI by 2021. This
represents an annual growth rate of 40%, according
to a report from the Dublin, Ireland-based consulting firm, which states,
“when combined, key clinical health AI applications can potentially create $150
billion in annual savings for the United States healthcare economy by 2026.”
But are healthcare providers too quick to adopt AI?
Accenture defines AI as a “constellation of technologies
from machine learning to natural
language processing that allows machines to sense, comprehend, act, and
learn.” However, some experts say AI is not performing as intended, and that it
introduces biases in healthcare worthy of investigation.
Keith Dreyer, DO, PhD, is Chief Data Science Officer at Partners Healthcare and Vice Chairman of Radiology at Massachusetts General Hospital (MGH). At a World Medical Innovation Forum on Artificial Intelligence covered by HealthITAnalytics, he said, “There are currently no measures to indicate that a result is biased or how much it might be biased. We need to explain the dataset these answers came from, how accurate we can expect them to be, where they work, and where they don’t work. When a number comes back, what does it really mean? What’s the difference between a seven and an eight or a two?” (Photo copyright: Healthcare in Europe.)
What Goes in Limits What Comes Out
Could machine learning lead to machine decision-making that
puts patients at risk? Some legal authorities say yes. Especially when computer
algorithms are based on limited data sources and questionable methods, lawyers
warn.
How can AI provide accurate medical insights for people when
the information going into databases is limited in the first place? Ossorio
pointed to lack of diversity in genomic
data. “There are still large groups of people for whom we have almost no
genomic data. This is another way in which the datasets that you might use to
train your algorithms are going to exclude certain groups of people
altogether,” she told HDM.
She also sounded the alarm about making decisions about
women’s health when data driving them are based on studies where women have
been “under-treated compared with men.”
“This leads to poor treatment, and that’s going to be
reflected in essentially all healthcare data that people are using when they
train their algorithms,” Ossorio said during a Machine Learning for Healthcare (MLHC) conference
covered by HDM.
How Bias Happens
Bias can enter healthcare data in three forms: by humans, by
design, and in its usage. That’s according to David Magnus, PhD, Director
of the Stanford Center for
Biomedical Ethics (SCBE) and Senior Author of a paper published in the New England
Journal of Medicine (NEJM) titled, “Implementing Machine
Learning in Health Care—Addressing Ethical Challenges.”
The paper’s authors wrote, “Physician-researchers are
predicting that familiarity with machine-learning tools for analyzing big data
will be a fundamental requirement for the next generation of physicians and
that algorithms might soon rival or replace physicians in fields that involve
close scrutiny of images, such as radiology and anatomical pathology.”
In a news
release, Magnus said, “You can easily imagine that the algorithms being
built into the healthcare system might be reflective of different, conflicting
interests. What if the algorithm is designed around the goal of making money?
What if different treatment decisions about patients are made depending on
insurance status or their ability to pay?”
In addition to the possibility of algorithm bias, the
authors of the NEJM paper have other concerns about AI affecting
healthcare providers:
“Physicians must adequately understand how
algorithms are created, critically assess the source of the data used to create
the statistical models designed to predict outcomes, understand how the models
function and guard against becoming overly dependent on them.
“Data gathered about patient health, diagnostics,
and outcomes become part of the ‘collective knowledge’ of published literature
and information collected by healthcare systems and might be used without
regard for clinical experience and the human aspect of patient care.
“Machine-learning-based clinical guidance may
introduce a third-party ‘actor’ into the physician-patient relationship, challenging
the dynamics of responsibility in the relationship and the expectation of
confidentiality.”
“We need to be cautious about caring for people based on what algorithms are showing us. The one thing people can do that machines can’t do is step aside from our ideas and evaluate them critically,” said Danton Char, MD, Lead Author and Assistant Professor of Anesthesiology, Perioperative, and Pain Medicine at Stanford, in the news release. “I think society has become very breathless in looking for quick answers,” he added. (Photo copyright: Stanford Medicine.)
Acknowledge Healthcare’s Differences
Still, the Stanford researchers acknowledge that AI can
benefit patients. And that healthcare leaders can learn from other industries,
such as car companies, which have test driven AI.
“Artificial intelligence will be pervasive in healthcare in a
few years,” said
Nigam Shah, PhD, co-author of the NEJM paper and Associate Professor of Medicine at Stanford, in the news release. He added that healthcare leaders need to be aware of the “pitfalls” that have happened in other industries and be cognizant of data.
“Be careful about knowing the data from which you learn,” he
warned.
AI’s ultimate role in healthcare diagnostics is not yet fully
known. Nevertheless, it behooves clinical laboratory leaders and anatomic
pathologists who are considering using AI to address issues of quality and
accuracy of the lab data they are generating. And to be aware of potential
biases in the data collection process.
Low prices to encourage consumers to order its WGS service is one way Veritas co-founder and genetics pioneer George Church hopes to sequence 150,000 genomes by 2021
By announcing an annotated whole-genome sequencing (WGS) service to consumers for just $599, Veritas Genetics is establishing a new price benchmark for medical laboratories and gene testing companies. Prior to this announcement in July, Veritas priced its standard myGenome service at $999.
“There is no more comprehensive genetic test than your whole genome,” Rodrigo Martinez, Veritas’ Chief Marketing and Design Officer, told CNBC. “So, this is a clear signal that the whole genome is basically going to replace all other genetic tests. And this [price drop] gets it closer and closer and closer.”
Pathologists and clinical laboratory managers will want to watch to see if Veritas’ low-priced, $599 whole-genome sequencing becomes a pricing standard for the genetic testing industry. Meanwhile, the new price includes not only the sequencing, but also an expert analysis of test results that includes information on more than 200 conditions, Veritas says.
“The focus in our industry is shifting from the cost of sequencing genomes to interpretation capabilities and that’s where our secret sauce is,” said Veritas CEO Mirza Cifric in a news release. “We’ve built and deployed a world class platform to deliver clinically-actionable insights at scale.” The company also says it “achieved this milestone primarily by deploying internally-developed machine learning and AI [artificial intelligence] tools as well as external tools—including Google’s DeepVariant—and by improving its in-house lab operations.”
The myGenome service offers 30x WGS, which Veritas touts in company documentation as the “gold standard” for sequencing, compared to the less-precise 0.4x WGS.
The myGenome service is available only in the United States.
Will Whole-Genome Sequencing Replace Other Genetic Tests?
Veritas was co-founded by George Church, PhD, a pioneer of personal genomics through his involvement with the Harvard Personal Genome Project at Harvard Medical School. In a press release announcing the launch of myGenome in 2016, Veritas described its system as “the world’s first whole genome for less than $1,000, including interpretation and genetic counseling.”
Church predicts that WGS will someday replace other genetic tests, such as the genotyping used by personal genomics and biotechnology company 23andMe.
“Companies like 23andMe that are based on genotyping technology basically opened the market over the last decade,” Martinez explained in an interview with WTF Health. “They’ve done an incredible job of getting awareness in the general population.”
However, he goes on to say, “In genotyping technology, you
are looking at very specific points of the genome, less than half of one
percent, a very small amount.”
Martinez says Veritas is sequencing all 6.4 billion letters
of the genome. And, with the new price point, “we’re closer to realizing that
seismic shift,” he said in the news release.
“This is the inflection point,” Martinez told CNBC.
“This is the point where the curve turns upward. You reach a critical mass when
you are able to provide a product that gives value at a specific price point.
This is the beginning of that. That’s why it’s seismic.”
Rodrigo Martinez (above), Veritas’ Chief Marketing and Design Officer, told CNBC, “The only way we’re going to be able to truly extract the value of the genome for a healthier society is going to be analyzing millions of genomes that have been sequenced. And the only way we can get there is by reducing the price so that more consumers can sequence their genome.” Photo copyright: Twitter.)
Payment Models Not Yet Established by Government, Private
Payers
However, tying WGS into personalized medicine that leads to actionable diagnoses may not be easy. Robin Bennett, PhD (hon.), a board certified senior genetic counselor and Professor of Medicine and Medical Genetics at UW School of Medicine, told CNBC, “[Healthcare] may be moving in that direction, but the payment for testing and for services, it hasn’t moved in the preventive direction. So, unless the healthcare system changes, these tests may not be as useful because … the healthcare system hasn’t caught up to say, ‘Yes, we support payment for this.’”
“Insurers are looking for things where, if you get the
information, there’s something you can do with it and that both the provider
and the patient are willing and able to use that information to do things that
improve their health,” Phillips told CNBC. “Insurers are very interested
in using genetic testing for prevention, but we need to . . . demonstrate that
the information will be used and that it’s a good trade-off between the
benefits and the costs.”
Sequencing for Free If You Share Your Data
Church may have an answer for that as well—get biopharmaceutical companies to foot the bill. Though Veritas’ new price for their myGenome service is significantly lower than before, it’s not free. That’s what Nebula Genomics, a start-up genetics company in Massachusetts co-founded by Church, offers people willing to share the data derived from their sequencing. To help biomedical researchers gather data for their studies, Nebula provides free or partially-paid-for whole-genome sequencing to qualified candidates.
“Nebula will enable individuals to get sequenced at much
lower cost through sequencing subsidies paid by the biopharma industry,” Church
told BioSpace.
“We need to bring the costs of personal genome sequencing close to zero to
achieve mass adoption.”
So, will lower-priced whole-genome sequencing catch on?
Perhaps. It’s certainly popular with everyday people who want to learn their
ancestry or predisposition to certain diseases. How it will ultimately affect
clinical laboratories and pathologists remains to be seen, but one thing is
certain—WGS is here to stay.
Study suggests AI-enabled technology can help clinical laboratories and hospital blood banks save thousands of dollars annually on expensive blood products
Artificial intelligence may prove to be a useful tool in helping hospitals better manage utilization of blood products. That’s one conclusion from a newly-published study done at New York’s Icahn School of Medicine at Mount Sinai. If so, this is a technology improvement that would be welcomed by blood bankers and clinical laboratory managers who must manage the cost and utilization of blood products.
There’s no way around it—blood is expensive. A typical 400- to 600-bed hospital likely budgets upwards of one million dollars annually just for blood products. Almost universally, in hospitals the medical laboratory manages the blood bank. This is where medical technologists trained in blood banking test patients and test blood to ensure whole blood units, or other blood products such as platelets, match and will not trigger a negative reaction when administered to the patient.
When left unmanaged, the cost and utilization of blood bank
products can put the budgets of hospital medical laboratories in the red. Hospitals
also invest a great deal of money training surgeons to accurately assess the
procedure and order the correct amount of blood components prior to surgery.
Therefore, new artificial intelligence (AI) technology that helps pinpoint patients’ blood loss during childbirth will be of interest to blood bankers and hospital laboratory administrators.
Can AI Help Clinical Labs Improve Utilization of Blood Products
in Hospitals?
Physicians at the Icahn School of Medicine at Mount Sinai recently investigated whether “Quantifying blood loss” would improve the use of blood during human childbirth. They published the results of their study in the International Journal of Obstetric Anesthesia.
Their research into 7,618 deliveries (vaginal and cesarean) involved “An observational study comparing blood loss, management, and outcomes between two historical cohorts (August 2016 to January 2017 and August 2017 to January 2018) at an academic tertiary care center. Patients in the intervention group (second period) had blood loss quantified compared with visual estimation for controls,” the research paper notes.
The researchers concluded that “Quantifying blood loss may
result in increased vigilance for vaginal and cesarean delivery. We identified
an association between quantifying blood loss and improved identification of
postpartum hemorrhage, patient management steps, and cost savings.”
The researchers, according to a press release, employed the Triton AI-enabled platform from Gauss Surgical, a silicon valley-based health technology company, to “monitor blood loss in all deliveries (vaginal and cesarean, n=3807) at Mount Sinai Hospital from August 2017 through January 2018 to support the institution’s stage-based hemorrhage protocol.”
The researchers found that use of a monitoring system was
associated with earlier postpartum hemorrhage
intervention and annual cost savings of $172,614 in lab costs and $36,614 in blood
bank costs.
Daniel Katz, MD (above), Director of Obstetric Anesthesia Research and Associate Professor Anesthesiology, Perioperative and Pain Medicine, at Mount Sinai Hospital in New York, said in the news release, “This study demonstrates that efficiently obtaining accurate, real-time blood loss information is critical to the successful implementation of a stage-based hemorrhage protocol.” Katz was the study’s lead author. (Photo copyright: Mount Sinai Hospital.)
Measuring Blood Loss: The Eye versus AI
Gauss has secured Food and Drug Administration (FDA) clearance for Triton and more than 50 US hospitals are using it. Triton provides, in real-time, images of blood-saturated surgical sponges and canisters and uses computer vision and machine learning to pinpoint blood loss, reported MD+DI.
Traditionally, physicians visually estimate blood loss
during procedures. When they are off in their estimates of postpartum
hemorrhage, harmful postpartum health complications and deaths can occur, the
Mount Sinai researchers explained in their paper.
And although other vital signs—heart rate, rhythm, blood
pressure, oxygen level, etc.— are monitored with equipment in the surgical
suite, blood usage is not.
“Blood loss in surgery has been an enigma for decades since the dawn of medicine,” Siddarth Satish, Founder and Chief Executive Officer of Gauss, told MD+DI. “We monitor many other vital signs in surgery, but ultimately there hasn’t been any direct indicator of a patient’s hemoglobin loss.”
Bleeding Better Recognized, Less Blood Transfusions
After the Mount Sinai researchers used the Triton system to
monitor blood loss during 3,807 vaginal and cesarean deliveries from August
2017 to January 2018 at Mount Sinai Hospital, they compared their findings to
3,811 deliveries from August 2016 to January 2017, during which doctors relied
solely on visual estimation of blood loss.
The study found the following, according to the news
release:
Improved hemorrhage recognition in vaginal deliveries of 2.2% and cesarean sections of 12.6% compared to .5% and 6.4%, respectively;
Less blood transfusions needed (vaginal patients): 47% with Triton compared to 71%;
Reduced blood transfusion dose (cesarean section): 1.90 units with Triton compared to 2.52 units;
Cost savings: $209,228 a year (the total of aforementioned lab and blood bank costs).
“What we like about [Gauss] is that it somewhat embodies precision medicine in the sense that you’re using more precise tools of measurement in their first use case,” Garrett Vygantas, MD, MBA, Managing Director for OSF Ventures, the financing arm of OSF Healthcare, who also serves on Gauss Surgical’s board, told MD+DI.
Possible New Resource for Hospital Medical Laboratories
So, will AI quickly become an omnipresent overseer in surgical suites? Hardly. However, AI is in the early stages of finding places in healthcare where it can be useful. “A lot of people are predicting that AI will play a huge role in healthcare … I think it’ll be ever-present. There will be a little bit of AI in everything you’re doing, but I think the actual practice of medicine in its truest form is going to carry forward,” Satish told Fierce Healthcare.
Hospital medical laboratories and blood blanks looking for
new tools to manage blood use may want to look into AI-enabled systems like
Triton. Saving money is not the only benefit. Less transfused blood is better
for patient care as well.
