News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel

News, Analysis, Trends, Management Innovations for
Clinical Laboratories and Pathology Groups

Hosted by Robert Michel
Sign In

Wall Street Journal Reports IBM May Sell Watson Health Due to Data Challenges and Unprofitability

Might this be a sign that AI platforms like Watson still cannot diagnose the wide range of patients’ conditions as accurately as a board-certified clinical pathologist?

Computer technology evolves so quickly, products often become obsolete before fulfilling their expected potential. Such, apparently, is the case with Watson, the genius artificial intelligence (AI) brainchild of International Business Machines Corp. (IBM) which was going to revolutionize how healthcare providers diagnose disease. In some areas of healthcare, such as analyzing MRIs and X-rays, AI has been a boon. But from a business perspective, Watson has failed to turn a profit for IBM, so it has to go.

In February, The Wall Street Journal (WSJ) reported that IBM is looking to sell its Watson Health unit because it is not profitable, despite bringing in $1 billion annually in revenue. The sale of Watson Health, the article states, would be aligned with IBM’s goal of streamlining the company and focusing its energies on cloud computing and other AI functions. Because one goal of the Watson project was to give physicians a tool to help them diagnose patients more accurately and faster, the problems that prevented Watson from achieving that goal should be of interest to pathologists and clinical laboratory managers, who daily are on the front lines of helping doctors diagnose the most challenging cases.

In a follow-up article, titled, “Potential IBM Watson Health Sale Puts Focus on Data Challenges,” the WSJ wrote, “… some experts found that it can be difficult to apply AI to treating complex medical conditions. Having access to data that represents patient populations broadly has been a challenge, experts told the Journal, and gaps in knowledge about complex diseases may not be fully captured in clinical databases.”

“I believe that we’re many years away from AI products that really positively impact clinical care for many patients,” Bob Kocher, Partner at Venrock, a venture-capital firm that invests in healthcare IT and related services, told the WSJ.

IBM Watson was promoted as a major resource to help improve medical care and support doctors in making more accurate diagnoses. However, in “IBM’s Retreat from Watson Highlights Broader AI Struggles in Health,” the WSJ reported that “IBM spent several billion dollars on acquisitions to build up Watson [Health] … a unit whose marquee product was supposed to help doctors diagnose and cure cancer … A decade later, reality has fallen short of that promise.”

In 2018, Dark Daily covered the beginnings of Watson’s struggles in “IBM’s Watson Not Living Up to Hype, Wall Street Journal and Other Media Report; ‘Dr. Watson’ Has Yet to Show It Can Improve Patient Outcomes or Accurately Diagnose Cancer,” and again in 2019 in “Artificial Intelligence Systems, Like IBM’s Watson, Continue to Underperform When Compared to Oncologists and Anatomic Pathologists.”

 previous Jeopardy champions Ken Jennings and Brad Rutter with Watson on the show in 2011
IBM initially created Watson (above center) to be an AI tool capable of a wide variety of applications, starting with answering questions. In January 2011, Watson made headlines when it defeated previous Jeopardy champions Ken Jennings (left) and Brad Rutter (right) on the popular television game show. After its triumph, IBM announced it would transition Watson for use in medical applications and promoted it as a major resource to help improve medical care and support healthcare professionals in making more precise diagnoses. The company called its new division IBM Watson Health and stated that massive data sets would be the key to accomplish its healthcare mission. That same year, The Dark Report had an IBM executive do a presentation about Watson Health at the Executive War College on Laboratory and Pathology Management. (Photo copyright: CBS News.)

Watson’s Successes and Failures in Healthcare

During the years following Watson’s Jeopardy win, Watson Health made some positive advances in the fields of healthcare data analytics, performance measurements, clinical trial recruitment, and healthcare information technology (HIT). 

However, Watson Health also experienced some high-profile failures as well. One such failure involved a collaboration with MD Anderson Cancer Center, established in 2013, to help the health systems’ oncologists develop new tools to benefit cancer patients. MD Anderson ended the relationship in 2018 after spending more than $60 million on the project, citing “multiple examples of unsafe and incorrect treatment recommendations,” made by the Watson supercomputer, Healthcare IT News reported.

Watson Health later readjusted the development and sales of its AI drug discovery tools and altered its marketing strategy amid reports of disappointing sales and skepticism surrounding machine learning for medical applications.

Underestimating the Challenge of AI in Healthcare

Since its inception, Watson Health has achieved substantial growth, mainly through a series of acquisitions. Those targeted acquisitions include:

  • Merge Healthcare, a healthcare imaging software company that was purchased for $1 billion in 2015,
  • Phytel, a health management software company that was purchased for an undisclosed amount in 2015,
  • Explorys, a healthcare analytics company that was purchased for an undisclosed amount in 2015, and
  • Truven Health Analytics, a provider of cloud-based healthcare data, analytics, and insights that was purchased for $2.6 billion in 2016.

“IBM’s Watson Health business came together as a result of several acquisitions,” said Paddy Padmanabhan, founder and CEO of Damo Consulting, a firm that provides digital transformation strategy and advisory services for healthcare organizations. “The decision to sell the business may also have to do with the performance of those units on top of the core Watson platform’s struggles,” he told Healthcare IT News.

It should be noted that these acquisitions involved companies that already had a product in the market which was generating revenue. So, the proposed sale of Watson Health includes not just the original Watson AI product, but the other businesses that IBM put into its Watson Health business division.

Padmanabhan noted that there are many challenges for AI in healthcare and that “historical data is at best a limited guide to the future when diagnosing and treating complex conditions.” He pointed to the failure with MD Anderson (in the use of Watson Health as a resource or tool for diagnosing cancer) was a setback for both IBM and the use of AI in healthcare. However, Padmanabhan is optimistic regarding the future use of AI in healthcare. 

“To use an oft-quoted analogy, AI’s performance in healthcare right now is more akin to that of the hedgehog than the fox. The hedgehog can solve for one problem at a time, especially when the problem follows familiar patterns discerned in narrow datasets,” he told Healthcare IT News. “The success stories in healthcare have been in specific areas such as sepsis and readmissions. Watson’s efforts to apply AI in areas such as cancer care may have underestimated the nuances of the challenge.”

Other experts agree that IBM was overly ambitious and overreached with Watson Health and ended up over-promising and under-delivering.

“IBM’s initial approach misfired due to how the solution AI was trained and developed,” Dan Olds, Principal Analyst with Gabriel Consulting Group, told EnterpriseAI. “It didn’t conform well to how doctors work in the real world and didn’t learn from its experiences with real doctors. It was primarily learning from synthetic cases, not real-life cases.” 

Was Watson Already Obsolete?

Another issue with Watson was that IBM’s marketing campaign may have exceeded the product’s design capabilities. When Watson was developed, it was built with AI and information technologies (IT) that were already outdated and behind the newest generation of those technologies, noted Tech Republic.

“There were genuine AI innovation triggers at Watson Health in natural language processing and generation, knowledge extraction and management, and similarity analytics,” Jeff Cribbs, Research Vice President at Gartner Research, told Tech Republic. “The hype got ahead of the engineering, as the hype cycle says it almost always will, and some of those struggles became apparent.”

Can Artificial Intelligence Fulfill its Potential in Healthcare?

The fact that IBM is contemplating the sale of Watson Health is another illustration of how difficult it can be to navigate the healthcare industry in the US. It is probable that someday AI could make healthcare diagnostics more accurate and reduce overall costs, however, data challenges still exist and more research and exploration will be needed for AI to fulfill its potential.

“Today’s AI systems are great in beating you at chess or Jeopardy,” Kumar Srinivas, Chief Technology Officer, Health Plans, at NTT DATA Services told Forbes. “But there are major challenges when addressing practical clinical issues that need a bit of explanation as to ‘why.’ Doctors aren’t going to defer to AI-decisions or respond clinically to a list of potential cancer cases if it’s generated from a black box.”

And perhaps that is the biggest challenge of all. For doctors to entrust their patients’ lives to a supercomputer, it better be as good as the hype. But can AI in healthcare ever accomplish that feat?

“AI can work incredibly well when it’s applied to specific use cases,” gastroenterologist Nirav R. Shah, MD, Chief Medical Officer at Sharecare, told Forbes. “With regards to cancer, we’re talking about a constellation of thousands of diseases, even if the focus is on one type of cancer. What we call ‘breast cancer,’ for example, can be caused by many different underlying genetic mutations and shouldn’t really be lumped together under one heading. AI can work well when there is uniformity and large data sets around a simple correlation or association. By having many data points around a single question, neural networks can ‘learn.’ With cancer, we’re breaking several of these principles.”

So, in deciding to divest itself of Watson Health, IBM may simply be as prescient now as it was when it first embraced the concept of AI in healthcare. The tech giant may foresee that AI will likely never replace the human mind of a trained healthcare diagnostician.

If this proves true—at least for several more years—then board-certified clinical pathologists can continue to justifiably refer to themselves as “the doctor’s doctor” because of their skills in diagnosing difficult-to-diagnose patients, and because of their knowledge of which clinical laboratory tests to order and how to interpret those test results.

—JP Schlingman

Related Information:

IBM Explores Sale of IBM Watson Health

IBM’s Retreat from Watson Highlights Broader AI Struggles in Health

Potential IBM Watson Health Sale Puts Focus on Data Challenges

IBM Sale of Watson Health Could Enable Renewed Focus on Cloud Growth

IBM Reportedly Looking to Sell its Unprofitable Watson Health Business

IBM Watson: Why Is Healthcare AI So Tough?

Hoping to Become Heavyweights in Healthcare Big Data, IBM Watson Health Teams Up with Siemens Radiology and In Vitro Diagnostics Businesses

IBM Watson Health to Acquire Truven Health Analytics and Its Millions of Patient Records for $2.6 Billion

Artificial Intelligence Systems, Like IBM’s Watson, Continue to Underperform When Compared to Oncologists and Anatomic Pathologists

IBM’s Watson Not Living Up to Hype, Wall Street Journal and Other Media Report; ‘Dr. Watson’ Has Yet to Show It Can Improve Patient Outcomes or Accurately Diagnose Cancer

IBM’s Watson Not Living Up to Hype, Wall Street Journal and Other Media Report; ‘Dr. Watson’ Has Yet to Show It Can Improve Patient Outcomes or Accurately Diagnose Cancer

Wall Street Journal reports IBM losing Watson-for-Oncology partners and clients, but scientists remain confident artificial intelligence will revolutionize diagnosis and treatment of disease

What happens when a healthcare revolution is overhyped? Results fall short of expectations. That’s the diagnosis from the Wall Street Journal (WSJ) and other media outlets five years after IBM marketed its Watson supercomputer as having the potential to “revolutionize” cancer diagnosis and treatment.

The idea that artificial intelligence (AI) could be used to diagnose cancer and identify appropriate therapies certainly carried with it implications for clinical laboratories and anatomic pathologists, which Dark Daily reported as far back as 2012. It also promised to spark rapid growth in precision medicine. For now, though, that momentum may be stalled.

“Watson can read all of the healthcare texts in the world in seconds,” John E. Kelly III, PhD, IBM Senior Vice President, Cognitive Solutions and IBM Research, told Wired in 2011. “And that’s our first priority, creating a ‘Dr. Watson,’ if you will.”

However, despite the marketing pitch, the WSJ investigation published in August claims IBM has fallen far short of that goal during the past seven years. The article states, “More than a dozen IBM partners and clients have halted or shrunk Watson’s oncology-related projects. Watson cancer applications have had limited impact on patients, according to dozens of interviews with medical centers, companies and doctors who have used it, as well as documents reviewed by the Wall Street Journal.”

Anatomic pathologists—who use tumor biopsies to diagnose cancer—have regularly wondered if IBM’s Watson would actually help physicians do a better job in the diagnosis, treatment, and monitoring of cancer patients. The findings of the Wall Street Journal show that Watson has yet to make much of a positive impact when used in support of cancer care.

The WSJ claims Watson often “didn’t add much value” or “wasn’t accurate.” This lackluster assessment is blamed on Watson’s inability to keep pace with fast-evolving treatment guidelines, as well as its inability to accurately evaluate reoccurring or rare cancers. Despite the more than $15 billion IBM has spent on Watson, the WSJ reports there is no published research showing Watson improving patient outcomes.

Lukas Wartman, MD, Assistant Professor, McDonnell Genome Institute at the Washington University School of Medicine in St. Louis, told the WSJ he rarely uses the Watson system, despite having complimentary access. IBM typically charges $200 to $1,000 per patient, plus consulting fees in some cases, for Watson-for-Oncology, the WSJ reported.

“The discomfort that I have—and that others have had with using it—has been the sense that you never know how much faith you can put in those results,” Wartman said.

Rudimentary Not Revolutionary Intelligence, STAT Notes

IBM’s Watson made headlines in 2011 when it won a head-to-head competition against two champions on the game show “Jeopardy.” Soon after, IBM announced it would make Watson available for medical applications, giving rise to the idea of “Dr. Watson.”

In a 2017 investigation, however, published on STAT, Watson is described as in its “toddler stage,” falling far short of IBM’s depiction of Watson as a “digital prodigy.”

“Perhaps the most stunning overreach is in [IBM’s] claim that Watson-for-Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, ‘even new approaches’ to cancer care,” the STAT article notes. “STAT found that the system doesn’t create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.”

STAT reported it had taken six years for data engineers and doctors to train Watson in just seven types of cancers and keep the system updated with the latest knowledge.

“It’s been a struggle to update, I’ll be honest,” Mark Kris, MD, oncologist at Memorial Sloan Kettering Cancer Center in New York and lead Watson trainer, told STAT. “Changing the system of cognitive computing doesn’t turn on a dime like that. You have to put in the literature, you have to put in the cases.” (Photo copyright: Physician Education Resource.)

Watson Recommended Unsafe and Incorrect Treatments, STAT Reported

In July 2018, STAT reported that internal documents from IBM revealed Watson had recommended “unsafe and incorrect” cancer treatments.

David Howard, PhD, Professor, Health Policy and Management, Rollins School of Public Health at Emory University, blames Watson’s failure in part to the dearth of high-quality published research available for the supercomputer to analyze.

“IBM spun a story about how Watson could improve cancer treatment that was superficially plausible—there are thousands of research papers published every year and no doctor can read them all,” Howard told HealthNewsReview.org. “However, the problem is not that there is too much information, but rather there is too little. Only a handful of published articles are high-quality, randomized trials. In many cases, oncologists have to choose between drugs that have never been directly compared in a randomized trial.”

Howard argues the news media needs to do a better job vetting stories touting healthcare breakthroughs.

“Reporters are often susceptible to PR hype about the potential of new technology—from Watson to ‘wearables’—to improve outcomes,” Howard said. “A lot of stories would turn out differently if they asked a simple question: ‘Where is the evidence?’”

Peter Greulich, a retired IBM manager who has written extensively on IBM’s corporate challenges, told STAT that IBM would need to invest more money and people in the Watson project to make it successful—an unlikely possibility in a time of shrinking revenues at the corporate giant.

“IBM ought to quit trying to cure cancer,” he said. “They turned the marketing engine loose without controlling how to build and construct a product.”

AI Could Still Revolutionize Precision Medicine

Despite the recent negative headlines about Watson, AI continues to offer the promise of one day changing how pathologists and physicians work together to diagnose and treat disease. Isaac Kohane, MD, PhD, Chairman of the Biomedical Informatics Program at Harvard Medical School, told Bloomberg that IBM may have oversold Watson, but he predicts AI one day will “revolutionize medicine.”

“It’s anybody’s guess who is going to be the first to the market leader in this space,” he said. “Artificial intelligence and big data are coming to doctors’ offices and hospitals. But it won’t necessarily look like the ads on TV.”

How AI and precision medicine plays out for clinical laboratories and anatomic pathologists is uncertain. Clearly, though, healthcare is on a path toward increased involvement of computerized decision-making applications in the diagnostic process. Regardless of early setbacks, that trend is unlikely to slow. Laboratory managers and pathology stakeholders would be wise to keep apprised of these developments.

—Andrea Downing Peck

Related Information:

IBM’s Watson Supercomputer Recommended ‘Unsafe and Incorrect’ Cancer Treatments, Internal Documents Show

IBM Pitched its Watson Supercomputer as a Revolution in Cancer Care. It’s Nowhere Close

IBM’s Watson Wins Jeopardy! Next Up: Fixing Health Care

IBM’s Watson Supercomputer Wins Practice Jeopardy Round

Memorial Sloan-Kettering Cancer Center, IBM to Collaborate in Applying Watson Technology to Help Oncologists

IBM Has a Watson Dilemma

MD Anderson Cancer Center’s IBM Watson Project Fails, and So Did the Journalism Related to It

What Went Wrong with IBM’s Watson?

IBM’s Watson Failed Against Cancer but AI Still Has Promise

Will IBM’s ‘Watson on Oncology’ Give Oncologists and Pathologists a Useful Tool for Diagnosing and Treating Various Cancer

Pathologists Take Note: IBM’s Watson to Attack Cancer with Help of WellPoint and Cedars-Sinai

Will IBM’s ‘Watson on Oncology’ Give Oncologists and Pathologists a Useful Tool for Diagnosing and Treating Various Cancers?

IBM’s Watson continues to seek a role as a cognitive computing tool of choice for physicians and pathologists in need of evidence-based clinical patient data

Remember IBM’s Watson? It’s been five years since Watson beat human contestants on Jeopardy. Since then, IBM has hoped Watson could be used in healthcare. To that end, some oncologists are exploring the use of Watson in cancer care. This could have implications for anatomic pathologists if oncologists developed a way to use Watson in the diagnosing cancers and identifying appropriate therapies for those cancers.

In 2011, IBM’s Watson supercomputer defeated human contestants for a charity prize during the television show Jeopardy. Just days later, Dark Daily reported on IBM’s goal for Watson to play a major role in helping physicians diagnose and treat disease. Since then, IBM has been exploring ways to commercialize Watson’s cognitive computing platform through partnerships with some of the healthcare industry’s biggest brands. (more…)

Cognitive Robots in Emergency Departments Could Reduce Wait Times and Help Pathologists with Diagnoses

Such cognitive robots may also find a role in clinical pathology laboratories

Pathologists and clinical laboratory managers might soon have new cognitive robotic tools to help them diagnose disease. Engineers and emergency medicine specialists at Vanderbilt University have joined together to develop a system of cognitive robots that would reduce the wait times physicians and staff experience in America’s emergency departments (ED).

These cognitive robots would be programmed to perform basic tests and deliver results on patients. By handling these functions, the Vanderbilt development team believes that their cognitive robots would reduce the workload on triage nurses and speed the process of treating patients in the emergency room. (more…)

After Taking on Jeopardy Contestants, IBM’s Watson Super Computer Might Be a Resource for Pathologists

Watson is capable of assessing health data, including medical laboratory test results

When IBM’s Watson “supercomputer” squared off against human contestants on the Jeopardy game show last February, there certainly were some pathologists and clinical laboratory managers watching this “man versus machine” battle of knowledge. But those pathologists and medical lab managers did not realize that IBM intends for Watson to play a major role in helping physicians diagnose and treat disease.

IBM is designing Watson to use analytical algorithms to support how physicians assess information as they evaluate patients. In this role, it is likely that Watson will be fed laboratory test data and evidence-based medicine algorithms as part of the data it draws upon to help physicians more accurately diagnose disease and come up with appropriate treatment plans. (more…)

;