Research results call into question the safety and dependability of using artificial intelligence in medical diagnosis, a development that should be watched by clinical laboratory scientists
ChatGPT, an artificial intelligence (AI) chatbot that returns answers to written prompts, has been tested and found wanting by researchers at the University of Florida College of Medicine (UF Health) who looked into how well it could answer typical patient questions on urology. Not good enough according to the researchers who conducted the study.
AI is quickly becoming a powerful new tool in diagnosis and medical research. Some digital pathologists and radiologists use it for data analysis and to speed up diagnostic modality readings. It’s even been said that AI will improve how physicians treat disease. But with all new discoveries there comes controversy, and that’s certainly the case with AI in healthcare.
Many voices in opposition to AI’s use in clinical medicine claim the technology is too new and cannot be trusted with patients’ health. Now, UF Health’s study seems to have confirmed that belief—at least with ChatGPT.
The study revealed that answers ChatGPT provided “fell short of the standard expected of physicians,” according to a UF Health new release, which called ChatGPT’s answers “flawed.”
The questions posed were considered to be common medical questions that patients would ask during a visit to a urologist.
The researchers believes their study is the first of its kind to focus on AI and the urology specialty and which “highlights the risk of asking AI engines for medical information even as they grow in accuracy and conversational ability,” UF Health noted in the news release.
“I am not discouraging people from using chatbots,” said Russell S. Terry, MD (above), an assistant professor in the UF College of Medicine’s department of urology and the study’s senior author, in a UF Health news release. “But don’t treat what you see as the final answer. Chatbots are not a substitute for a doctor.” Pathologists and clinical laboratory managers will want to monitor how developers improve the performance of chatbots and other applications using artificial intelligence. (Photo copyright: University of Florida.)
UF Health ChatGPT Study Details
UF Health’s study featured 13 of the most queried topics from patients to their urologists during office visits. The researchers asked ChatGPT each question three times “since ChatGPT can formulate different answers to identical queries,” they noted in the news release.
The urological conditions the questions covered included:
The researchers then “evaluated the answers based on guidelines produced by the three leading professional groups for urologists in the United States, Canada, and Europe, including the American Urological Association (URA). Five UF Health urologists independently assessed the appropriateness of the chatbot’s answers using standardized methods,” UF Health noted.
Notable was that many of the results were inaccurate. According to UF Health, only 60% of responses were deemed appropriate from the 39 evaluated responses. Outside of those results, the researchers noted in their Urology paper, “[ChatGPT] misinterprets clinical care guidelines, dismisses important contextual information, conceals its sources, and provides inappropriate references.”
When asked, for the most part ChatGPT was not able to accurately provide the sources it referenced for its answers. Apparently, the chatbot was not programmed to provide such sources, the UF Health news release stated.
“It provided sources that were either completely made up or completely irrelevant,” Terry noted in the new release. “Transparency is important so patients can assess what they’re being told.”
Further, “Only 7 (54%) of 13 topics and 21 (54%) of 39 responses met the BD [Brief DISCERN] cut-off score of ≥16 to denote good-quality content,” the researchers wrote in their paper. BD is a validated healthcare information assessment questionnaire that “provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem,” according to the DISCERN website.
ChatGPT often “omitted key details or incorrectly processed their meaning, as it did by not recognizing the importance of pain from scar tissue in Peyronie’s disease. As a result … the AI provided an improper treatment recommendation,” the UF Health study paper noted.
Is Using ChatGPT for Medical Advice Dangerous to Patients?
Terry noted that the chatbot performed better in some areas over others, such as infertility, overactive bladder, and hypogonadism. However, frequently recurring UTIs in women was one topic of questions for which ChatGPT consistently gave incorrect results.
“One of the more dangerous characteristics of chatbots is that they can answer a patient’s inquiry with all the confidence of a veteran physician, even when completely wrong,” UF Health reported.
“In only one of the evaluated responses did the AI note it ‘cannot give medical advice’ … The chatbot recommended consulting with a doctor or medical adviser in only 62% of its responses,” UF Health noted.
For their part, ChatGPT’s developers “tell users the chatbot can provide bad information and warn users after logging in that ChatGPT ‘is not intended to give advice,’” UF Health added.
Future of Chatbots in Healthcare
In UF Health’s Urology paper, the researchers state, “Chatbot models hold great promise, but users should be cautious when interpreting healthcare-related advice from existing AI models. Additional training and modifications are needed before these AI models will be ready for reliable use by patients and providers.”
UF Health conducted its study in February 2023. Thus, the news release points out, results could be different now due to ChatGPT updates. Nevertheless, Terry urges users to get second opinions from their doctors.
“It’s always a good thing when patients take ownership of their healthcare and do research to get information on their own,” he said in the news release. “But just as when you use Google, don’t accept anything at face value without checking with your healthcare provider.”
That’s always good advice. Still, UF Health notes that “While this and other chatbots warn users that the programs are a work in progress, physicians believe some people will undoubtedly still rely on them.” Time will tell whether trusting AI for medical advice turns out well for those patients.
The study reported above is a useful warning to clinical laboratory managers and pathologists that current technologies used in ChatGPT, and similar AI-powered solutions, have not yet achieved the accuracy and reliability of trained medical diagnosticians when answering common questions about different health conditions asked by patients.
Healthcare revenue cycle consultant Jonathan Wiik suggests healthcare providers must prepare their organizations for patients who need help paying increasing medical costs
A recent analysis of this issue by TransUnion Healthcare (NYSE:TRU) states, “patients experienced an 11% increase in average out-of-pocket costs during 2017, rising from $1,630 in Q4 2016 to $1,813 in Q4 2017.” It is a development that should send up red flags to clinical laboratory managers seeking ways to maintain and increase revenues.
“Given the increased payment responsibility, being able to determine a patient’s ability to pay is increasingly important for hospitals,” noted Jonathan Wiik, Principal, Healthcare Strategy at TransUnion Healthcare (TRU). “In order to allow patients to focus on getting the care they need healthcare providers need processes and tools in place to help patients meet their financial obligations and to establish funding mechanisms that will benefit both the patient and provider.” Obviously, this also applies to clinical laboratories.
According to a news release, “The [TRU] analysis also revealed that in 2017, on average, 49% of patient out-of-pocket costs per healthcare visit were below $500; 39% were $501-$1,000; and 12% were more than $1,000.”
For providers, patients’ swelling unpaid balances mean more uncompensated care, the analysis also showed. And that means more unpaid balances for clinical laboratories as well.
“Increasing healthcare costs and patient responsibility is a continuing trend that does not seem to be slowing anytime in the near future,” noted Jonathan Wiik (above) Principal, Healthcare Strategy, at TransUnion Healthcare and author of the new book “Healthcare Revolution: The Patient Is the New Payer,” during a HIMSS 2018 presentation. (Photo copyright: Colorado Managed Care Collaborative.)
Patients Struggle to Pay Amounts Under $500
Each year, more healthcare consumers are forced onto high-deductible health plans (HDHPs) that make them responsible for thousands and even tens of thousands of dollars in upfront costs.
And according to another TRU news release, patients with commercial insurance plans experienced a 67% increase in their financial responsibility over five years. In other words, after insurance plans paid providers, patients still needed to pony up 12.2% of the total bill in 2017, as compared to 8% in 2012.
During the most recent year studied by TransUnion Healthcare, patients’ out-of-pocket costs increased 11%, rising to $1,813 in 2017 from $1,630 in 2016, a news release revealed.
And it doesn’t take a huge bill for patients to feel the pain. TransUnion’s data reveals that 68% of patients with medical bills below $500 did not fully pay what they owed, RevCycle Intelligence reported. This has major implications for clinical laboratories and anatomic pathology groups because many lab charges fall under $500 and TransUnion shows that almost 70% of patients do not pay the full amount of these bills.
According to TRU, medical specialties with the highest out-of-pocket estimated amounts due from patients include:
And, as Dark Daily previously reported, affluent and self-employed people also feel the pinch, as deductibles can be as high as $5,000/year for individuals and more than $10,000/year for a families, whether plans are purchased through the Affordable Care Act (ACA) or employers.
When patients cannot afford to pay their bills, hospitals’ bad debt and charity-care levels rise. Together, bad debt and charity care comprise a provider’s uncompensated care.
“A lot of patients can’t afford these bills, which is why uncompensated care has bounced,” Wiik told Modern Healthcare.
Indeed, uncompensated care was $38.3 billion in 2016, up $2.6 billion since 2015, according to an American Hospital Association (AHA) 2017 fact sheet.
Meanwhile, the Centers for Medicare and Medicaid Services (CMS) reported that Medicare bad debt (the effect of Medicare patients not paying deductibles and co-pays) increased to $3.69 billion in 2016 from $3.14 billion in 2012, a 17% bump, TransUnion Healthcare pointed out.
Consumers Say They Want Prices, Financing Plans
Consumers say healthcare providers are not transparent about costs for procedures, nor do they effectively offer financing options. That’s according to a HealthFirst Financial news release, which states, “More than three-quarters, or 77%, of healthcare consumers say it’s important or very important they know their costs before treatment and 53% want to discuss financing options before care. However, the vast majority of healthcare providers are not satisfying these consumer demands.”
“53% voice concern about the ability to pay a medical bill of less than $1,000;
“35% worried about the ability to pay a bill of less than $500; and,
“16% are concerned about the ability to pay a bill of less than $250.”
These numbers fall well into the amounts clinical laboratories charge for services rendered.
What Can Medical Laboratories Do?
To help their customers pay their bills and improve revenue, Dark Daily suggest labs:
Use software that enables ordering clinicians to process advanced beneficiary notices and prior authorizations for services;
Inform the customer prior to specimen collection about their financial responsibility for the test;
Ask for payment-due at time of the patient encounter;
Share key lab test price data in easily accessible and understandable ways;
Keep credit card information securely on-hand for agreed-to balances patients are responsible for paying; and,
Offer payment options, such as e-billing and financing plans.
As we’ve pointed out many times, because clinical laboratories are dependent on the physicians and hospitals they service, they are particularly vulnerable when patients stop paying their bills.
OPKO Health’s 4Kscore test predicts the rate of high-risk prostate cancer and may become a useful business case study for other labs developing proprietary diagnostic tests
Clinical laboratories and biotech companies with new medical laboratory tests are struggling to win coverage by Medicare and private payers. How big is this problem? There are currently tens of thousands of molecular diagnostic assays and genetic tests offered for clinical use.
Any lab company seeking to obtain an appropriate Current Procedural Terminology (CPT) code, favorable coverage guidelines, and adequate reimbursement from health insurers for its new lab test faces three big challenges, and they are related. First, payers are simply overwhelmed with requests to review new genetic tests. The flood of new test submissions exceeds the capability of payers to respond.
Most Payers May Not Have Right Scientific Expertise to Evaluate Genetic Tests
