Research results call into question the safety and dependability of using artificial intelligence in medical diagnosis, a development that should be watched by clinical laboratory scientists
ChatGPT, an artificial intelligence (AI) chatbot that returns answers to written prompts, has been tested and found wanting by researchers at the University of Florida College of Medicine (UF Health) who looked into how well it could answer typical patient questions on urology. Not good enough according to the researchers who conducted the study.
AI is quickly becoming a powerful new tool in diagnosis and medical research. Some digital pathologists and radiologists use it for data analysis and to speed up diagnostic modality readings. It’s even been said that AI will improve how physicians treat disease. But with all new discoveries there comes controversy, and that’s certainly the case with AI in healthcare.
Many voices in opposition to AI’s use in clinical medicine claim the technology is too new and cannot be trusted with patients’ health. Now, UF Health’s study seems to have confirmed that belief—at least with ChatGPT.
The study revealed that answers ChatGPT provided “fell short of the standard expected of physicians,” according to a UF Health new release, which called ChatGPT’s answers “flawed.”
The questions posed were considered to be common medical questions that patients would ask during a visit to a urologist.
The researchers believes their study is the first of its kind to focus on AI and the urology specialty and which “highlights the risk of asking AI engines for medical information even as they grow in accuracy and conversational ability,” UF Health noted in the news release.
“I am not discouraging people from using chatbots,” said Russell S. Terry, MD (above), an assistant professor in the UF College of Medicine’s department of urology and the study’s senior author, in a UF Health news release. “But don’t treat what you see as the final answer. Chatbots are not a substitute for a doctor.” Pathologists and clinical laboratory managers will want to monitor how developers improve the performance of chatbots and other applications using artificial intelligence. (Photo copyright: University of Florida.)
UF Health ChatGPT Study Details
UF Health’s study featured 13 of the most queried topics from patients to their urologists during office visits. The researchers asked ChatGPT each question three times “since ChatGPT can formulate different answers to identical queries,” they noted in the news release.
The urological conditions the questions covered included:
The researchers then “evaluated the answers based on guidelines produced by the three leading professional groups for urologists in the United States, Canada, and Europe, including the American Urological Association (URA). Five UF Health urologists independently assessed the appropriateness of the chatbot’s answers using standardized methods,” UF Health noted.
Notable was that many of the results were inaccurate. According to UF Health, only 60% of responses were deemed appropriate from the 39 evaluated responses. Outside of those results, the researchers noted in their Urology paper, “[ChatGPT] misinterprets clinical care guidelines, dismisses important contextual information, conceals its sources, and provides inappropriate references.”
When asked, for the most part ChatGPT was not able to accurately provide the sources it referenced for its answers. Apparently, the chatbot was not programmed to provide such sources, the UF Health news release stated.
“It provided sources that were either completely made up or completely irrelevant,” Terry noted in the new release. “Transparency is important so patients can assess what they’re being told.”
Further, “Only 7 (54%) of 13 topics and 21 (54%) of 39 responses met the BD [Brief DISCERN] cut-off score of ≥16 to denote good-quality content,” the researchers wrote in their paper. BD is a validated healthcare information assessment questionnaire that “provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem,” according to the DISCERN website.
ChatGPT often “omitted key details or incorrectly processed their meaning, as it did by not recognizing the importance of pain from scar tissue in Peyronie’s disease. As a result … the AI provided an improper treatment recommendation,” the UF Health study paper noted.
Is Using ChatGPT for Medical Advice Dangerous to Patients?
Terry noted that the chatbot performed better in some areas over others, such as infertility, overactive bladder, and hypogonadism. However, frequently recurring UTIs in women was one topic of questions for which ChatGPT consistently gave incorrect results.
“One of the more dangerous characteristics of chatbots is that they can answer a patient’s inquiry with all the confidence of a veteran physician, even when completely wrong,” UF Health reported.
“In only one of the evaluated responses did the AI note it ‘cannot give medical advice’ … The chatbot recommended consulting with a doctor or medical adviser in only 62% of its responses,” UF Health noted.
For their part, ChatGPT’s developers “tell users the chatbot can provide bad information and warn users after logging in that ChatGPT ‘is not intended to give advice,’” UF Health added.
Future of Chatbots in Healthcare
In UF Health’s Urology paper, the researchers state, “Chatbot models hold great promise, but users should be cautious when interpreting healthcare-related advice from existing AI models. Additional training and modifications are needed before these AI models will be ready for reliable use by patients and providers.”
UF Health conducted its study in February 2023. Thus, the news release points out, results could be different now due to ChatGPT updates. Nevertheless, Terry urges users to get second opinions from their doctors.
“It’s always a good thing when patients take ownership of their healthcare and do research to get information on their own,” he said in the news release. “But just as when you use Google, don’t accept anything at face value without checking with your healthcare provider.”
That’s always good advice. Still, UF Health notes that “While this and other chatbots warn users that the programs are a work in progress, physicians believe some people will undoubtedly still rely on them.” Time will tell whether trusting AI for medical advice turns out well for those patients.
The study reported above is a useful warning to clinical laboratory managers and pathologists that current technologies used in ChatGPT, and similar AI-powered solutions, have not yet achieved the accuracy and reliability of trained medical diagnosticians when answering common questions about different health conditions asked by patients.
Device is latest example that wearable healthcare devices are moving past simple biomarker monitoring and into the area of assisting in rehab
Companies unrelated to traditional clinical laboratory medicine continue to develop wearable devices that enable individuals to monitor their health while also alerting physicians and caregivers in real time when certain biomarkers are out of range.
One recent example is US biotechnology company STAT Health Informatics in Boston, which has developed a wearable device that monitors blood flow to the ear and face “to better understand symptoms such as dizziness, brain fog, headaches, fainting, and fatigue that occur upon standing,” according to a press release. The tiny device is worn in the ear and connects wirelessly to a smartphone app.
Johns Hopkins University clinically tested the STAT device, and according to Medical Device Network, “It can predict a person fainting minutes before it happens and can be worn with more than 90% of devices that go in or around the ear. It can also be left in while sleeping and showering, meaning less likelihood of removing the device and forgetting to replace it.”
Another notable aspect of this invention is that it’s an example of how the ongoing miniaturization of various technologies makes it possible to invent smaller devices but with greater capabilities. In the case of the STAT device, it combines tiny sensors, Bluetooth, and an equally tiny battery to produce a device that fits in the ear and can function for up to three days before needing a recharge.
It’s easy to imagine these technologies being used for other types of diagnostic testing devices that could be managed by clinical laboratories.
“It’s well understood that the ear is a biometric gold mine because of its close proximity to the brain and major arteries. This allows for new biometrics … to be possible,” said Daniel Lee (above), co-founder and CEO of STAT Health, in a press release. “In addition, the ear is largely isolated from data corruption caused by arm motion—a problem that plagues current wearables and prevents them from monitoring heart metrics during many daily tasks. The ear is really the ideal window into the brain and heart.” Clinical laboratory managers may want to watch how this technology is further developed to incorporate other biomarkers for diseases and health conditions. (Photo copyright: STAT Health.)
How STAT Works
Every time the wearer stands, the STAT device tracks the change in response of blood pressure, heart rate, and blood flow to the head. “The device distills all this information into an ‘Up Score’ to track time spent upright. Its ‘Flow Score’ helps users pace their recovery by watching for blood flow abnormalities,” MassDevice reported.
According to the company’s website, STAT is intended for use in individuals who have been diagnosed with conditions known to suffer from drops in blood flow to the head, such as:
As an individual continues to use the device, STAT “learns about each user’s unique body to provide personalized coaching for healthy lifestyle choices,” MassDevice reported.
Another key factor is the technology built into the device. An optical sensor was chosen over ultrasound because STAT Health felt it was both easy to use and provided precise measurements accessing the shallow ear artery, MassDevice reported.
“Despite its small scale, the device incorporates advanced optical sensors, an accelerometer, a pressure sensor, temperature sensors, artificial intelligence (AI)-edge computing, three-day battery life (or more), and a micro solar panel,” Medical Device Network noted.
STAT’s image above demonstrates how truly minute the company’s wearable device is, even though it monitors blood flow to the face and ear looking for signs that the wearer is about to suffer bouts of dizziness or lightheadedness due to a drop in blood flow. (Photo copyright: STAT Health Informatics Inc.)
STAT’s Impact on Users’ Health
STAT’s developers intend the device to help individuals stay on track with their health. “The target population can navigate their condition better. If they’re not standing when they can, they will become deconditioned. This product encourages standing and being upright where possible, as part of rehab,” Lee told Medical Device Network.
Lee has been developing wearable in-ear devices for many years.
“Nobody has realized the ear’s true potential due to the miniaturization and complex systems design needed to make a practical and user-friendly ear wearable,” he told MassDevice. “After multiple engineering breakthroughs, we’ve succeeded in unlocking the ear to combine the convenience and long-term nature of wearables with the high fidelity nature of obtrusive clinical monitors. No other device comes close along the axis of wearability and cardiac signal quality, which is why we believe STAT is truly the world’s most advanced wearable.”
For clinical laboratories, though STAT is not a diagnostic test, it is the latest example of how companies are developing wearable monitoring devices intended to allow individuals to monitor their health. It moves beyond the simple monitoring of Apple Watch and Fitbit. This device can aid individuals during rehab.
Wearable healthcare devices will continue to be introduced that are smaller, allow more precise measurements of target biomarkers, and alert wearers in real time when those markers are out of range. Keeping in tune with the newest developments will help clinical laboratories and pathologists find new ways to support healthcare providers who recommend these devices for monitoring their patients conditions.
Research could lead to improvements in gene therapy and antiviral resistance medications while also possibly leading to a new class of clinical laboratory tests
Scientists at the University of Maryland, Baltimore County (UMBC) have discovered what may be the scariest virus of all—the Vampire Virus. It’s a term that may inspire “Walking Dead” level horror in the wake of the COVID-19 pandemic, and though virologists and microbiologists might be tempted to dismiss them as imaginary, they are all too real. Even more apropos to the Dracula saga, the UM scientists found them in a soil sample. Yikes!
Happily, this ghoulish discovery could have positive implications for gene editing, gene therapy, and the development of new antiviral medications, according to The Conversation. In turn, these positive implications may eventually trigger the need to create new diagnostic tests that clinical laboratories can offer to physicians.
The image above, taken from a University of Maryland news release, shows the satellite virus “latched onto its helper virus.” Discovery of vampire-like viruses that attach at the “neck” of other viruses may lead to important discoveries in the development of gene editing and antiviral therapies. Might clinical laboratories one day collect samples for pharmaceutical developers engaged in combating antiviral drug resistance? (Photo copyright: University of Maryland.)
Spotting a Vampire Virus
According to IFLScience, these tiny vampire viruses were first discovered by undergraduates who believed they were looking at sample contamination when analyzing sequences of bacteriophages from environmental soil samples. But upon repeating the experiment they realized it was no mistake.
In the UMBC news release, bioinformatician Ivan Erill, PhD, Professor of Biological Sciences at the University of Maryland, noted that “some viruses, called satellites, depend not only on their host organism to complete their life cycle, but also on another virus, known as a helper.
“The satellite virus needs the helper either to build its capsid, a protective shell that encloses the virus’ genetic material, or to help it replicate its DNA,” he added. “These viral relationships require the satellite and the helper to be in proximity to each other at least temporarily, but there were no known cases of a satellite actually attaching itself to a helper—until now.”
Although scientists have witnessed viruses working together before, this is the first known instance of a virus directly latching onto another virus’ capsid—rather like a vampire going for the neck.
“When I saw it, I was like, I can’t believe this,” said Tagide deCarvalho, PhD, Assistant Director of Natural and Mathematical Sciences at the University of Maryland and first author of the study, in a UM news release, “No one has ever seen a bacteriophage—or any other virus—attach to another virus.”
“Not everyone has a TEM at their disposal. [With the TEM] I’m able to follow up on some of these observations and validate them with imaging. There’s elements of discovery we can only make using the TEM,” said deCarvalho in the UMBC news release.
Using Vampire Viruses to Develop Better Gene Therapies
Spookily, the comparisons to Dracula and his parasitic brethren do not stop with their freeloading tendencies. The researchers found that some viruses without a satellite attached still showed signs of having been leeched onto before. Those viruses had the equivalent of “bite marks” showing evidence of encountering vampiric viruses in the past.
“It’s possible that a lot of the bacteriophages that people thought were contaminated were actually these satellite-helper systems,” said deCarvalho in the ISME paper.
But what does UMBC’s breakthrough mean for the greater scientific and medical community? Do we need to arm host viruses with silver crosses and necklaces of garlic? Jokes aside, this discovery could lead to further development in research of how to genetically alter viruses and deliver therapeutic elements into cells.
According to Healthline, some gene therapy or “gene editing” already involves the use of viruses. Scientists switch out the programming on a virus and trick it into healing, instead of harming the cells it infiltrates. Therefore, UMBC’s discovery could lead to new breakthroughs battling deadly viruses by using their own parasitic tricks to infiltrate other viruses.
Although groundbreaking and extremely interesting, the research is still in early stages. Any developments from this discovery aren’t likely to impact clinical laboratories any time soon. But after the past few years of battling the COVID-19 variants, this exciting discovery could help find new ways to prevent the next pandemic.
Spectroscopic technique was 91% accurate in identifying the notoriously difficult-to-diagnose disease suggesting a clinical diagnostic test for CFS may be possible
Most clinical pathologists know that, despite their best efforts, scientists have failed to come up with a reliable clinical laboratory blood test for diagnosing myalgic encephalomyelitis (ME), the condition commonly known as chronic fatigue syndrome (CFS)—at least not one that’s ready for clinical use.
But now an international team of researchers at the University of Oxford has developed an experimental non-invasive test for CFS using a simple blood draw, artificial intelligence (AI), and a spectroscopic technique known as Raman spectroscopy.
The approach uses a laser to identify unique cellular “fingerprints” associated with the disease, according to an Oxford news release.
“When Raman was added to a panel of potentially diagnostic outputs, we improved the ability of the model to identify the ME/CFS patients and controls,” Karl Morten, PhD, Director of Graduate Studies and Principal Investigator at Oxford University, told Advanced Science News. Morton led the research team along with Wei Huang, PhD, Professor of Biological Engineering at Oxford.
The researchers claim the test is 91% accurate in differentiating between healthy people, disease controls, and ME/CFS patients, and 84% accurate in differentiating between mild, moderate, and severe cases, the new release states.
“This could be a game changer as we are unsure what causes [ME/CFS] and diagnosis occurs perhaps 10 to 20 years after the condition has started to develop,” said Karl Morten, PhD, Director of Graduate Studies and Principal Investigator at Oxford University. “An early diagnosis might allow us to identify what is going wrong with the potential to fix it before the more long-term degenerative changes are observed.” Though this research may not lead to a simple clinical laboratory blood test for CFS, any non-invasive diagnostic test would enable doctors to help many people. (Photo copyright: Oxford University.)
Need for an ME/CFS Test
The federal Centers for Disease Control and Prevention (CDC) describes ME/CFS as “a serious, long-term illness that affects many body systems,” with symptoms that include severe fatigue and sleep difficulties. Citing an Institute of Medicine (IoM) report, the agency estimates that 836,000 to 2.5 million Americans suffer from the condition but notes that most cases have not been diagnosed.
“One of the difficulties is the complexity of the disease,” said Jonas Bergquist, MD, PhD, Director of the ME/CFS Research Center of Uppsala University in Sweden, told Advanced Science News. “Because it’s a multi-organ disorder, you get symptoms from many different regions of the body with different onsets, though it’s common with post viral syndrome to have different overlapping [symptoms] that disguise the diagnosis.” Bergquist was not involved with the Oxford study.
One key to the Oxford researchers’ technique is the use of multiple artificial intelligence models to analyze the spectral profiles. “These signatures are complex and by eye there are not necessarily clear features that separate ME/CFS patients from other groups,” Morten told Advanced Science News.
“The AI looks at this data and attempts to find features which can separate the groups,” he continued. “Different AI methods find different features in the data. Individually, each method is not that successful at assigning an unknown sample to the correct group. However, when we combine the different methods, we produce a model which can assign the subjects to the different groups very accurately.”
Without a reliable test, “diagnosis of the condition is difficult, with most patients relying on self-report, questionnaires, and subjective measures to receive a diagnosis,” the Oxford press release noted.
But developing such a test has been challenging, Advanced Science News noted.
How Oxford’s Raman Technique Works
Raman spectroscopy uses a laser to determine the “vibrational modes of molecules,” according to the Oxford press release.
“When a laser beam is directed at a cell, some of the scattered photons undergo frequency shifts due to energy exchanges with the cell’s molecular components,” the press release stated. “Raman micro-spectroscopy detects these shifted photons, providing a non-invasive method for single cell analysis. The resulting single cell Raman spectra serve as a unique fingerprint, revealing the intrinsic and biochemical properties and indicating the physiological and metabolic state of the cell.”
The researchers employed the technique on blood samples from 98 subjects, including 61 ME/CFS patients, 16 healthy controls, and 21 controls with multiple sclerosis (MS), Advanced Science reported.
The Oxford scientists focused their attention on peripheral blood mononuclear cells (PBMCs), as previous studies found that these cells showed “reduced energetic function” in ME/CFS patients. “With this evidence, the team proposed that single-cell analysis of PBMCs might reveal differences in the structure and morphology in ME/CFS patients compared to healthy controls and other disease groups such as multiple sclerosis,” the press release states.
Clinical Laboratory Blood Processing and the Oxford Raman Technique
Oxford’s Raman spectroscopic technique “only requires a small blood sample which could be developed as a point-of-care test perhaps from one drop of blood,” the researchers wrote. However, Advanced Science News pointed out that required laser microscopy equipment costs more than $250,000.
In their Advanced Science paper, the researchers note that the test could be made more widely available by transferring blood samples collected by local clinical laboratories to diagnostic centers that have the needed hardware.
“Alternatively, a compact system containing portable Raman instruments could be developed, which would be much cheaper than a standard Raman microscope, and [which] incorporated with microfluidic systems to stream cells through a Raman laser for detection, eliminating the need for lengthy blood sample processing,” the researchers wrote.
They noted that the technique could be adapted to test for other chronic conditions as well, such as MS, fibromyalgia, Lyme disease, and long COVID.
“Our paper is very much a starting point for future research,” Morten told Advanced Science News. “Larger cohorts need to be studied, and if Raman proves useful, we need to think carefully about how a test might be developed.”
Bergquist agreed, stating it’s “not necessarily something you would see in a doctor’s office. It requires a lot of advanced data analysis to use—I still see it as a research methodology. But in the long run, it could be developed into a tool that could be used in a more simplistic way.”
Though a useable diagnostic test may be far off, clinical laboratories should consider how they can aid in ME/CFS research.
Scientists believe the biodegradable device could someday help detect multiple saliva biomarkers. If true, it might provide a new type of test for clinical laboratories
When it comes to tongue depressors, it turns out you can teach an old dog new tricks. Researchers from National and Kapodistrian University of Athens Greece (NKUA) have taken this simple wooden medical tool and developed a high-tech biosensing device that may someday be useful at the point-of-care in hospitals and as a new type of test for clinical laboratories.
Using diode laser engraving, the researchers developed an “eco-friendly disposable sensor that can measure glucose levels and other biomarkers in saliva,” according to LabMedica.
This proof-of-principle biosensing device demonstrates the feasibility of “simultaneous determination of glucose and nitrite in artificial saliva,” according to the NKUA scientists who hope it will help doctors diagnose a variety of conditions.
In their published paper, the scientists at the University of Athens wrote that their wooden electrochemical biosensing tongue depressor (above) “is an easy-to-fabricate disposable point-of-care chip with a wide scope of applicability to other bioassays,” and that “it paves the way for the low-cost and straightforward production of wooden electrochemical platforms.” Might this and other similar biosensing devices eventually find their way to clinical laboratories for use in identifying and tracking certain biomarkers for disease? (Photo copyright: University of Athens.)
How to Make a High-Tech Tongue Depressor
Though wood is affordable and accessible, it doesn’t conduct electricity very well. The researchers’ first attempt to solve this problem was to use the wood as “a passive substrate” to which they coated “metals and carbon-based inks,” LabMedica reported. After that they tried using high-powered lasers to “char specific regions on the wood, turning those spots into conductive graphite.” But that process was complicated, expensive, and a fire hazard.
The researchers eventually turned to “low-power diode lasers” which have been used successfully “to make polyimide-based sensors but have not previously been applied to wooden electronics and electrochemical sensors,” LabMedica noted.
In their Analytical Chemistry paper, the researchers wrote, “A low-cost laser engraver, equipped with a low-power (0.5 W) diode laser, programmably irradiates the surface of the WTD [wooden tongue depressor], forming two mini electrochemical cells (e-cells). The two e-cells consist of four graphite electrodes: two working electrodes, a common counter, and a common reference electrode. The two e-cells are spatially separated via programmable pen-plotting, using a commercial hydrophobic marker pen.”
In other words, the researchers “used a portable, low-cost laser engraver to create a pattern of conductive graphite electrodes on a wooden tongue depressor, without the need for special conditions. Those electrodes formed two electrochemical cells separated by lines drawn with a water-repellent permanent marker,” states a press release from the American Chemical Society.
“The biosensor was then used to quickly and simultaneously measure nitrite and glucose concentrations in artificial saliva. Nitrite can indicate oral diseases like periodontitis, while glucose can serve as a diagnostic for diabetes. The researchers suggest that these low-cost devices could be adapted to detect other saliva biomarkers and could be easily and rapidly produced on-site at medical facilities,” LabMedica reported.
Benefits of Using Wood
One of the major benefits of using wood for their biosensing device is how environmentally friendly it is. “Since wood is a renewable, biodegradable naturally occurring material, the development of conductive patterns on wood substrates is a new and innovative chapter in sustainable electronics and sensors,” the researchers wrote in Analytical Chemistry.
Additionally, the tongue depressor features “An easy-to-fabricate disposable point-of-care chip with a wide scope of applicability to other bioassays, while it paves the way for the low-cost and straightforward production of wooden electrochemical platforms,” the researchers added.
This adds to a growing trend of developing bioassay products that keep the health of our planet in mind.
“This new BC test is non-toxic, naturally biodegradable and both inexpensive and scalable to mass production, currently costing less than $4.00 per test to produce. Its cellulose fibers do not require the chemicals used to manufacture paper, and the test is almost entirely biodegradable,” a UPenn blog post noted.
New Future Tool Use in Clinical Diagnostics
Who could have predicted that the lowly wooden tongue depressor would go high tech with technology that uses lasers to convert it to an electrochemical multiplex biosensing device for oral fluid analysis? This is yet another example of technologies cleverly applied to classic devices that enable them to deliver useful diagnostic information about patients.
And while a biosensing tongue depressor is certainly a diagnostic tool that may be useful for nurses and physicians in clinic and hospital settings, with further technology advancements, it could someday be used to collect specimens that measure more than glucose and nitrites.
Pathologists and clinical laboratory managers will want to stay alert to the concerns voiced by tech experts about the need to exercise caution when using generative AI to assist medical diagnoses
GPTs are an integral part of the framework of a generative artificial intelligence that creates text, images, and other media using generative models. These neural network models can learn the patterns and structure of inputted information and then develop new data that contains similar characteristics.
Through their proposal, the AMA has developed principles and recommendations surrounding the benefits and potentially harmful consequences of relying on AI-generated medical advice and content to advance diagnoses.
“We’re trying to look around the corner for our patients to understand the promise and limitations of AI,” said Alexander Ding, MD (above), AMA Trustee and Associate Vice President for Physician Strategy and Medical Affairs at Humana, in a press release. “There is a lot of uncertainty about the direction and regulatory framework for this use of AI that has found its way into the day-to-day practice of medicine.” Clinical laboratory professionals following advances in AI may want to remain informed on the use of generative AI solutions in healthcare. (Photo copyright: American Medical Association.)
Preventing Spread of Mis/Disinformation
GPTs are “a family of neural network models that uses the transformer architecture and is a key advancement in artificial intelligence (AI) powering generative AI applications such as ChatGPT,” according to Amazon Web Services.
In addition to creating human-like text and content, GPTs have the ability to answer questions in a conversational manner. They can analyze language queries and then predict high-quality responses based on their understanding of the language. GPTs can perform this task after being trained with billions of parameters on massive language datasets and then generate long responses, not just the next word in a sequence.
“AI holds the promise of transforming medicine,” said diagnostic and interventional radiologist Alexander Ding, MD, AMA Trustee and Associate Vice President for Physician Strategy and Medical Affairs at Humana, in an AMA press release.
“We don’t want to be chasing technology. Rather, as scientists, we want to use our expertise to structure guidelines, and guardrails to prevent unintended consequences, such as baking in bias and widening disparities, dissemination of incorrect medical advice, or spread of misinformation or disinformation,” he added.
The AMA plans to work with the federal government and other appropriate organizations to advise policymakers on the optimal ways to use AI in healthcare to protect patients from misleading AI-generated data that may or may not be validated, accurate, or relevant.
Advantages and Risks of AI in Medicine
The AMA’s proposal was prompted by AMA-affiliated organizations that stressed concerns about the lack of regulatory oversight for GPTs. They are encouraging healthcare professionals to educate patients about the advantages and risks of AI in medicine.
“AI took a huge leap with large language model tool and generative models, so all of the work that has been done up to this point in terms of regulatory and governance frameworks will have to be treated or at least reviewed with this new lens,” Sha Edathumparampil, Corporate Vice President, Digital and Data, Baptist Health South Florida, told Healthcare Brew.
According to the AMA press release, “the current limitations create potential risks for physicians and patients and should be used with appropriate caution at this time. AI-generated fabrications, errors, or inaccuracies can harm patients, and physicians need to be acutely aware of these risks and added liability before they rely on unregulated machine-learning algorithms and tools.”
According to the AMA press release, the organization will propose state and federal regulations for AI tools at next year’s annual meeting in Chicago.
In a July AMA podcast, AMA’s President, Jesse Ehrenfeld, MD, stressed that more must be done through regulation and development to bolster trust in these new technologies.
“There’s a lot of discomfort around the use of these tools among Americans with the idea of AI being used in their own healthcare,” Ehrenfeld said. “There was a 2023 Pew Research Center poll [that said] 60% of Americans would feel uncomfortable if their own healthcare provider relied on AI to do things like diagnose disease or recommend a treatment.”
WHO Issues Cautions about Use of AI in Healthcare
In May, the World Health Organization (WHO) issued a statement advocating for caution when implementing AI-generated large language GPT models into healthcare.
A current example of such a GPT is ChatGPT, a large language-based model (LLM) that enables users to refine and lead conversations towards a desired length, format, style, level of detail and language. Organizations across industries are now utilizing GPT models for Question and Answer bots for customers, text summarization, and content generation and search features.
“Precipitous adoption of untested systems could lead to errors by healthcare workers, cause harm to patients, erode trust in AI, and thereby undermine (or delay) the potential long-term benefits and uses of such technologies around the world,” commented WHO in the statement.
WHO’s concerns regarding the need for prudence and oversight in the use of AI technologies include:
Data used to train AI may be biased, which could pose risks to health, equity, and inclusiveness.
LLMs generate responses that can appear authoritative and plausible, but which may be completely incorrect or contain serious errors.
LLMs may be trained on data for which consent may not have been given.
LLMs may not be able to protect sensitive data that is provided to an application to generate a response.
LLMs can be misused to generate and disseminate highly convincing disinformation in the form of text, audio, or video that may be difficult for people to differentiate from reliable health content.
Tech Experts Recommended Caution
Generative AI will continue to evolve. Therefore, clinical laboratory professionals may want to keep a keen eye on advances in AI technology and GPTs in healthcare diagnosis.
“While generative AI holds tremendous potential to transform various industries, it also presents significant challenges and risks that should not be ignored,” wrote Edathumparampil in an article he penned for CXOTECH Magazine. “With the right strategy and approach, generative AI can be a powerful tool for innovation and differentiation, helping businesses to stay ahead of the competition and better serve their customers.”
GPT’s may eventually be a boon to healthcare providers, including clinical laboratories, and pathology groups. But for the moment, caution is recommended.
