Might this be a sign that AI platforms like Watson still cannot diagnose the wide range of patients’ conditions as accurately as a board-certified clinical pathologist?
Computer technology evolves so quickly, products often become obsolete before fulfilling their expected potential. Such, apparently, is the case with Watson, the genius artificial intelligence (AI) brainchild of International Business Machines Corp. (IBM) which was going to revolutionize how healthcare providers diagnose disease. In some areas of healthcare, such as analyzing MRIs and X-rays, AI has been a boon. But from a business perspective, Watson has failed to turn a profit for IBM, so it has to go.
In February, The Wall Street Journal (WSJ) reported that IBM is looking to sell its Watson Health unit because it is not profitable, despite bringing in $1 billion annually in revenue. The sale of Watson Health, the article states, would be aligned with IBM’s goal of streamlining the company and focusing its energies on cloud computing and other AI functions. Because one goal of the Watson project was to give physicians a tool to help them diagnose patients more accurately and faster, the problems that prevented Watson from achieving that goal should be of interest to pathologists and clinical laboratory managers, who daily are on the front lines of helping doctors diagnose the most challenging cases.
In a follow-up article, titled, “Potential IBM Watson Health Sale Puts Focus on Data Challenges,” the WSJ wrote, “… some experts found that it can be difficult to apply AI to treating complex medical conditions. Having access to data that represents patient populations broadly has been a challenge, experts told the Journal, and gaps in knowledge about complex diseases may not be fully captured in clinical databases.”
“I believe that we’re many years away from AI products that really positively impact clinical care for many patients,” Bob Kocher, Partner at Venrock, a venture-capital firm that invests in healthcare IT and related services, told the WSJ.
IBM Watson was promoted as a major resource to help improve medical care and support doctors in making more accurate diagnoses. However, in “IBM’s Retreat from Watson Highlights Broader AI Struggles in Health,” the WSJ reported that “IBM spent several billion dollars on acquisitions to build up Watson [Health] … a unit whose marquee product was supposed to help doctors diagnose and cure cancer … A decade later, reality has fallen short of that promise.”
During the years following Watson’s Jeopardy win, Watson Health made some positive advances in the fields of healthcare data analytics, performance measurements, clinical trial recruitment, and healthcare information technology (HIT).
However, Watson Health also experienced some high-profile failures as well. One such failure involved a collaboration with MD Anderson Cancer Center, established in 2013, to help the health systems’ oncologists develop new tools to benefit cancer patients. MD Anderson ended the relationship in 2018 after spending more than $60 million on the project, citing “multiple examples of unsafe and incorrect treatment recommendations,” made by the Watson supercomputer, Healthcare IT News reported.
Watson Health later readjusted the development and sales of its AI drug discovery tools and altered its marketing strategy amid reports of disappointing sales and skepticism surrounding machine learning for medical applications.
Underestimating the Challenge of AI in Healthcare
Since its inception, Watson Health has achieved substantial growth, mainly through a series of acquisitions. Those targeted acquisitions include:
Merge Healthcare, a healthcare imaging software company that was purchased for $1 billion in 2015,
Phytel, a health management software company that was purchased for an undisclosed amount in 2015,
Explorys, a healthcare analytics company that was purchased for an undisclosed amount in 2015, and
Truven Health Analytics, a provider of cloud-based healthcare data, analytics, and insights that was purchased for $2.6 billion in 2016.
“IBM’s Watson Health business came together as a result of several acquisitions,” said Paddy Padmanabhan, founder and CEO of Damo Consulting, a firm that provides digital transformation strategy and advisory services for healthcare organizations. “The decision to sell the business may also have to do with the performance of those units on top of the core Watson platform’s struggles,” he told Healthcare IT News.
It should be noted that these acquisitions involved companies that already had a product in the market which was generating revenue. So, the proposed sale of Watson Health includes not just the original Watson AI product, but the other businesses that IBM put into its Watson Health business division.
Padmanabhan noted that there are many challenges for AI in healthcare and that “historical data is at best a limited guide to the future when diagnosing and treating complex conditions.” He pointed to the failure with MD Anderson (in the use of Watson Health as a resource or tool for diagnosing cancer) was a setback for both IBM and the use of AI in healthcare. However, Padmanabhan is optimistic regarding the future use of AI in healthcare.
“To use an oft-quoted analogy, AI’s performance in healthcare right now is more akin to that of the hedgehog than the fox. The hedgehog can solve for one problem at a time, especially when the problem follows familiar patterns discerned in narrow datasets,” he told Healthcare IT News. “The success stories in healthcare have been in specific areas such as sepsis and readmissions. Watson’s efforts to apply AI in areas such as cancer care may have underestimated the nuances of the challenge.”
Other experts agree that IBM was overly ambitious and overreached with Watson Health and ended up over-promising and under-delivering.
“IBM’s initial approach misfired due to how the solution AI was trained and developed,” Dan Olds, Principal Analyst with Gabriel Consulting Group, told EnterpriseAI. “It didn’t conform well to how doctors work in the real world and didn’t learn from its experiences with real doctors. It was primarily learning from synthetic cases, not real-life cases.”
Was Watson Already Obsolete?
Another issue with Watson was that IBM’s marketing campaign may have exceeded the product’s design capabilities. When Watson was developed, it was built with AI and information technologies (IT) that were already outdated and behind the newest generation of those technologies, noted Tech Republic.
“There were genuine AI innovation triggers at Watson Health in natural language processing and generation, knowledge extraction and management, and similarity analytics,” Jeff Cribbs, Research Vice President at Gartner Research, told Tech Republic. “The hype got ahead of the engineering, as the hype cycle says it almost always will, and some of those struggles became apparent.”
Can Artificial Intelligence Fulfill its Potential in Healthcare?
The fact that IBM is contemplating the sale of Watson Health is another illustration of how difficult it can be to navigate the healthcare industry in the US. It is probable that someday AI could make healthcare diagnostics more accurate and reduce overall costs, however, data challenges still exist and more research and exploration will be needed for AI to fulfill its potential.
“Today’s AI systems are great in beating you at chess or Jeopardy,” Kumar Srinivas, Chief Technology Officer, Health Plans, at NTT DATA Services told Forbes. “But there are major challenges when addressing practical clinical issues that need a bit of explanation as to ‘why.’ Doctors aren’t going to defer to AI-decisions or respond clinically to a list of potential cancer cases if it’s generated from a black box.”
And perhaps that is the biggest challenge of all. For doctors to entrust their patients’ lives to a supercomputer, it better be as good as the hype. But can AI in healthcare ever accomplish that feat?
“AI can work incredibly well when it’s applied to specific use cases,” gastroenterologist Nirav R. Shah, MD, Chief Medical Officer at Sharecare, told Forbes. “With regards to cancer, we’re talking about a constellation of thousands of diseases, even if the focus is on one type of cancer. What we call ‘breast cancer,’ for example, can be caused by many different underlying genetic mutations and shouldn’t really be lumped together under one heading. AI can work well when there is uniformity and large data sets around a simple correlation or association. By having many data points around a single question, neural networks can ‘learn.’ With cancer, we’re breaking several of these principles.”
So, in deciding to divest itself of Watson Health, IBM may simply be as prescient now as it was when it first embraced the concept of AI in healthcare. The tech giant may foresee that AI will likely never replace the human mind of a trained healthcare diagnostician.
If this proves true—at least for several more years—then board-certified clinical pathologists can continue to justifiably refer to themselves as “the doctor’s doctor” because of their skills in diagnosing difficult-to-diagnose patients, and because of their knowledge of which clinical laboratory tests to order and how to interpret those test results.
Wall Street Journal reports IBM losing Watson-for-Oncology partners and clients, but scientists remain confident artificial intelligence will revolutionize diagnosis and treatment of disease
What happens when a healthcare revolution is overhyped? Results fall short of expectations. That’s the diagnosis from the Wall Street Journal (WSJ) and other media outlets five years after IBM marketed its Watson supercomputer as having the potential to “revolutionize” cancer diagnosis and treatment.
“Watson can read all of the healthcare texts in the world in seconds,” John E. Kelly III, PhD, IBM Senior Vice President, Cognitive Solutions and IBM Research, told Wired in 2011. “And that’s our first priority, creating a ‘Dr. Watson,’ if you will.”
However, despite the marketing pitch, the WSJ investigation published in August claims IBM has fallen far short of that goal during the past seven years. The article states, “More than a dozen IBM partners and clients have halted or shrunk Watson’s oncology-related projects. Watson cancer applications have had limited impact on patients, according to dozens of interviews with medical centers, companies and doctors who have used it, as well as documents reviewed by the Wall Street Journal.”
Anatomic pathologists—who use tumor biopsies to diagnose cancer—have regularly wondered if IBM’s Watson would actually help physicians do a better job in the diagnosis, treatment, and monitoring of cancer patients. The findings of the Wall Street Journal show that Watson has yet to make much of a positive impact when used in support of cancer care.
The WSJ claims Watson often “didn’t add much value” or “wasn’t accurate.” This lackluster assessment is blamed on Watson’s inability to keep pace with fast-evolving treatment guidelines, as well as its inability to accurately evaluate reoccurring or rare cancers. Despite the more than $15 billion IBM has spent on Watson, the WSJ reports there is no published research showing Watson improving patient outcomes.
“The discomfort that I have—and that others have had with using it—has been the sense that you never know how much faith you can put in those results,” Wartman said.
Rudimentary Not Revolutionary Intelligence, STAT Notes
IBM’s Watson made headlines in 2011 when it won a head-to-head competition against two champions on the game show “Jeopardy.” Soon after, IBM announced it would make Watson available for medical applications, giving rise to the idea of “Dr. Watson.”
In a 2017 investigation, however, published on STAT, Watson is described as in its “toddler stage,” falling far short of IBM’s depiction of Watson as a “digital prodigy.”
“Perhaps the most stunning overreach is in [IBM’s] claim that Watson-for-Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, ‘even new approaches’ to cancer care,” the STAT article notes. “STAT found that the system doesn’t create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.”
STAT reported it had taken six years for data engineers and doctors to train Watson in just seven types of cancers and keep the system updated with the latest knowledge.
“IBM spun a story about how Watson could improve cancer treatment that was superficially plausible—there are thousands of research papers published every year and no doctor can read them all,” Howard told HealthNewsReview.org. “However, the problem is not that there is too much information, but rather there is too little. Only a handful of published articles are high-quality, randomized trials. In many cases, oncologists have to choose between drugs that have never been directly compared in a randomized trial.”
Howard argues the news media needs to do a better job vetting stories touting healthcare breakthroughs.
“Reporters are often susceptible to PR hype about the potential of new technology—from Watson to ‘wearables’—to improve outcomes,” Howard said. “A lot of stories would turn out differently if they asked a simple question: ‘Where is the evidence?’”
Peter Greulich, a retired IBM manager who has written extensively on IBM’s corporate challenges, told STAT that IBM would need to invest more money and people in the Watson project to make it successful—an unlikely possibility in a time of shrinking revenues at the corporate giant.
“IBM ought to quit trying to cure cancer,” he said. “They turned the marketing engine loose without controlling how to build and construct a product.”
“It’s anybody’s guess who is going to be the first to the market leader in this space,” he said. “Artificial intelligence and big data are coming to doctors’ offices and hospitals. But it won’t necessarily look like the ads on TV.”
How AI and precision medicine plays out for clinical laboratories and anatomic pathologists is uncertain. Clearly, though, healthcare is on a path toward increased involvement of computerized decision-making applications in the diagnostic process. Regardless of early setbacks, that trend is unlikely to slow. Laboratory managers and pathology stakeholders would be wise to keep apprised of these developments.
IBM’s Watson continues to seek a role as a cognitive computing tool of choice for physicians and pathologists in need of evidence-based clinical patient data
Remember IBM’s Watson? It’s been five years since Watson beat human contestants on Jeopardy. Since then, IBM has hoped Watson could be used in healthcare. To that end, some oncologists are exploring the use of Watson in cancer care. This could have implications for anatomic pathologists if oncologists developed a way to use Watson in the diagnosing cancers and identifying appropriate therapies for those cancers.
In 2011, IBM’s Watson supercomputer defeated human contestants for a charity prize during the television show Jeopardy. Just days later, Dark Daily reported on IBM’s goal for Watson to play a major role in helping physicians diagnose and treat disease. Since then, IBM has been exploring ways to commercialize Watson’s cognitive computing platform through partnerships with some of the healthcare industry’s biggest brands. (more…)
Such cognitive robots may also find a role in clinical pathology laboratories
Pathologists and clinical laboratory managers might soon have new cognitive robotic tools to help them diagnose disease. Engineers and emergency medicine specialists at Vanderbilt University have joined together to develop a system of cognitive robots that would reduce the wait times physicians and staff experience in America’s emergency departments (ED).
These cognitive robots would be programmed to perform basic tests and deliver results on patients. By handling these functions, the Vanderbilt development team believes that their cognitive robots would reduce the workload on triage nurses and speed the process of treating patients in the emergency room. (more…)
Watson is capable of assessing health data, including medical laboratory test results
When IBM’s Watson “supercomputer” squared off against human contestants on the Jeopardy game show last February, there certainly were some pathologists and clinical laboratory managers watching this “man versus machine” battle of knowledge. But those pathologists and medical lab managers did not realize that IBM intends for Watson to play a major role in helping physicians diagnose and treat disease.
IBM is designing Watson to use analytical algorithms to support how physicians assess information as they evaluate patients. In this role, it is likely that Watson will be fed laboratory test data and evidence-based medicine algorithms as part of the data it draws upon to help physicians more accurately diagnose disease and come up with appropriate treatment plans. (more…)