Boston Children’s Hospital Hires a Prompt Engineer to Help the Healthcare Organization Deploy and Use Artificial Intelligence Applications
This may be a new ‘sign of the times’ as hospitals, clinical laboratories, and other healthcare providers working with AI find they also need to hire their own prompt engineers
Boston Children’s Hospital last year hired a “prompt engineer” to propel the hospital forward in using Artificial intelligence (AI) as part of its business model. But what is AI prompting? It’s a relatively new term and may not be familiar to clinical laboratory and pathology leaders.
AI “prompting,” according to Florida State University, “refers to the process of interacting with an AI system by providing specific instructions or queries to achieve a desired outcome.”
According to workable.com, prompt engineers specialize “in developing, refining, and optimizing AI-generated text prompts to ensure they are accurate, engaging, and relevant for various applications. They also collaborate with different teams to improve the prompt generation process and overall AI system performance.”
Healthcare institutions are getting more serious about using AI to improve daily workflows and clinical care, including in the clinical laboratory and pathology departments. But adopting the new technology can be disruptive. To ensure the implementation goes smoothly, hospitals are now seeking prompt engineers to guide the organization’s strategy for using AI.
When Boston Children’s Hospital leaders set out to find such a person, they looked for an individual who had “a clinical background [and] who knows how to use these tools. Someone who had experience coding for large language models and natural language processing, but who could also understand clinical language,” according to MedPage Today.
“We got many, many applications, some really impressive people, but we were looking for a specific set of skills and background,” John Brownstein, PhD, Chief Innovation Officer at Boston Children’s Hospital and Professor of Biomedical Informatics at Harvard Medical School, told MedPage Today.
“It was not easy to find [someone]—a bit of a unicorn-type candidate,” noted Brownstein, who is also a medical contributor to ABC News.
After a four-month search, the hospital hired Dinesh Rai, MD, emergency room physician and AI engineer, for the position. According to Brownstein, Rai had “actually practiced medicine, lived in a clinical environment,” and had “successfully launched many [AI] applications on top of large language models,” MedPage Today reported.
“Some of the nuances I bring to the table in terms of being a physician and having worked clinically and understanding really deeply the clinical workflows and how we can implement the [AI] technology—where its limits are, where it can excel, and the quickest way to get things [done],” Dinesh Rai, MD (above), told MedPage Today. “I’m happy to be able to help with all of that.” Hospital clinical laboratory and pathology managers may soon by engaging with prompt engineers to ensure the smooth use of AI in their departments. (Photo copyright: LinkedIn.)
Prompt Engineers are like F1 Drivers
“It’s kind of like driving a car, where basically anyone can drive an automatic car, and anyone can go onto ChatGPT, write some text, and get a pretty solid response,” said Rai, describing the act of AI prompting to MedPage today.
Then, there are “people who know how to drive manual, and there are people who will know different prompting techniques, like chain-of-thought or zero-shot prompting,” he added. “Then you have those F1 drivers who are very intimate with the mechanics of their car, and how to use it most optimally.”
The American Hospital Association (AHA) believes that AI “holds great promise in helping healthcare providers gain insights and improve health outcomes.” In an article titled, “How AI Is Improving Diagnostics, Decision-Making and Care,” the AHA noted that, “Although many questions remain regarding its safety, regulation, and impact, the use of AI in clinical care is no longer in its infancy and is expected to experience exponential growth in the coming years.
“AI is improving data processing, identifying patterns, and generating insights that otherwise might elude discovery from a physician’s manual effort. The next five years will be critical for hospitals and health systems to build the infrastructure needed to support AI technology, according to the recently released Futurescan 2023,” the AHA wrote.
The graphic above is taken from the American Hospital Association’s article about Futurescan’s 2023 survey results on AI in healthcare. “Healthcare executives from across the nation were asked how likely it is that by 2028 a federal regulatory body will determine that Al for clinical care delivery augmentation (e.g., assisted diagnosis and prescription, personalized medication and care) is safe for use by our hospital or health systems,” AHA stated. This would include the use of AI in clinical laboratories and pathology group practices. (Graphic copyright: American Hospital Association.)
The AHA listed the top three opportunities for AI in clinical care as:
- Clinical Decision Tools: “AI algorithms analyze a vast amount of patient data to assist medical professionals in making more informed decisions about care.”
- Diagnostic and Imaging: The use of AI “allows healthcare professionals to structure, index, and leverage diagnostic and imaging data for more accurate diagnoses.”
- Patient Safety: The use of AI improves decision making and optimizes health outcomes by evaluating patient data. “Systems that incorporate AI can improve error detection, stratify patients, and manage drug delivery.”
The hiring of a prompt engineer by Boston Children’s Hospital is another example of how AI is gaining traction in clinical healthcare. According to the Futurescan 2023 survey, nearly half of hospital CEOs and strategy leaders believe that health systems will have the infrastructure in place by 2028 to successfully utilize AI in clinical decision making.
“I’m lucky to [be] in an organization that has recognized the importance of AI as part of the future practice of medicine,” Rai told MedPage Today.
Pathologists and managers of clinical laboratories and genetic testing companies will want to track further advancements in artificial intelligence. At some point, the capabilities of future generations of AI solutions may encourage labs to hire their own prompt engineers.
—JP Schlingman
Related Information:
Why One Hospital Hired an AI Prompt Engineer
This Children’s Hospital is Integrating AI with Healthcare
How Five Healthcare Organizations Are Investing in AI for Patient Care
What is a Large Language Model (LLM)?
How AI is Improving Diagnostics, Decision-making and Care
Prompt Engineer Job Description
Artificial Intelligence in the Operating Room: Dutch Scientists Develop AI Application That Informs Surgical Decision Making during Cancer Surgery
Speedy DNA sequencing and on-the-spot digital imaging may change the future of anatomic pathology procedures during surgery
Researchers at the Center for Molecular Medicine (CMM) at UMC Utrecht, a leading international university medical center in the Netherlands, have paired artificial intelligence (AI) and machine learning with DNA sequencing to develop a diagnostic tool cancer surgeons can use during surgeries to determine in minutes—while the patient is still on the operating table—whether they have fully removed all the cancerous tissue.
The method, “involves a computer scanning segments of a tumor’s DNA and alighting on certain chemical modifications that can yield a detailed diagnosis of the type and even subtype of the brain tumor,” according to The New York Times, which added, “That diagnosis, generated during the early stages of an hours-long surgery, can help surgeons decide how aggressively to operate, … In the future, the method may also help steer doctors toward treatments tailored for a specific subtype of tumor.”
This technology has the potential to reduce the need for frozen sections, should additional development and studies confirm that it accurately and reliably shows surgeons that all cancerous cells were fully removed. Many anatomic pathologists would welcome such a development because of the time pressure and stress associated with this procedure. Pathologists know that the patient is still in surgery and the surgeons are waiting for the results of the frozen section. Most pathologists would consider fewer frozen sections—with better patient outcomes—to be an improvement in patient care.
The UMC Utrecht scientist published their findings in the journal Nature titled, “Ultra-Fast Deep-Learned CNS Tumor Classification during Surgery.”
“It’s imperative that the tumor subtype is known at the time of surgery,” Jeroen de Ridder, PhD (above), associate professor in the Center for Molecular Medicine at UMC Utrecht and one of the study leaders, told The New York Times. “What we have now uniquely enabled is to allow this very fine-grained, robust, detailed diagnosis to be performed already during the surgery. It can figure out itself what it’s looking at and make a robust classification,” he added. How this discovery affects the role of anatomic pathologists and pathology laboratories during cancer surgeries remains to be seen. (Photo copyright: UMC Utrecht.)
Rapid DNA Sequencing Impacts Brain Tumor Surgeries
The UMC Utrecht scientists employed Oxford Nanopore’s “real-time DNA sequencing technology to address the challenges posed by central nervous system (CNS) tumors, one of the most lethal type of tumor, especially among children,” according to an Oxford Nanopore news release.
The researchers called their new machine learning AI application the “Sturgeon.”
According to The New York Times, “The new method uses a faster genetic sequencing technique and applies it only to a small slice of the cellular genome, allowing it to return results before a surgeon has started operating on the edges of a tumor.”
Jeroen de Ridder, PhD, an associate professor in the Center for Molecular Medicine at UMC Utrecht, told The New York Times that Sturgeon is “powerful enough to deliver a diagnosis with sparse genetic data, akin to someone recognizing an image based on only 1% of its pixels, and from an unknown portion of the image.” Ridder is also a principal investigator at the Oncode Institute, an independent research center in the Netherlands.
The researchers tested Sturgeon during 25 live brain surgeries and compared the results to an anatomic pathologist’s standard method of microscope tissue examination. “The new approach delivered 18 correct diagnoses and failed to reach the needed confidence threshold in the other seven cases. It turned around its diagnoses in less than 90 minutes, the study reported—short enough for it to inform decisions during an operation,” The New York Times reported.
But there were issues. Where the minute samples contain healthy brain tissue, identifying an adequate number of tumor markers could become problematic. Under those conditions, surgeons can ask an anatomic pathologist to “flag the [tissue samples] with the most tumor for sequencing, said PhD candidate Marc Pagès-Gallego, a bioinformatician at UMC Utrecht and a co-author of the study,” The New York Times noted.
“Implementation itself is less straightforward than often suggested,” Sebastian Brandner, MD, a professor of neuropathology at University College London, told The Times. “Sequencing and classifying tumor cells often still required significant expertise in bioinformatics as well as workers who are able to run, troubleshoot, and repair the technology,” he added.
“Brain tumors are also the most well-suited to being classified by the chemical modifications that the new method analyzes; not all cancers can be diagnosed that way,” The Times pointed out.
Thus, the research continues. The new method is being applied to other surgical samples as well. The study authors said other facilities are utilizing the method on their own surgical tissue samples, “suggesting that it can work in other people’s hands.” But more work is needed, The Times reported.
UMC Utrecht Researchers Receive Hanarth Grant
To expand their research into the Sturgeon’s capabilities, the UMC Utrecht research team recently received funds from the Hanarth Fonds, which was founded in 2018 to “promote and enhance the use of artificial intelligence and machine learning to improve the diagnosis, treatment, and outcome of patients with cancer,” according to the organization’s website.
The researchers will investigate ways the Sturgeon AI algorithm can be used to identify tumors of the central nervous system during surgery, a UMC Utrecht news release states. These type of tumors, according to the researchers, are difficult to examine without surgery.
“This poses a challenge for neurosurgeons. They have to operate on a tumor without knowing what type of tumor it is. As a result, there is a chance that the patient will need another operation,” said de Ridder in the news release.
The Sturgeon application solves this problem. It identifies the “exact type of tumor during surgery. This allows the appropriate surgical strategy to be applied immediately,” the news release notes.
The Hanarth funds will enable Jeroen and his team to develop a variant of the Sturgeon that uses “cerebrospinal fluid instead of (part of) the tumor. This will allow the type of tumor to be determined already before surgery. The main challenge is that cerebrospinal fluid contains a mixture of tumor and normal DNA. AI models will be trained to take this into account.”
The UMC Utrecht scientists’ breakthrough is another example of how organizations and research groups are working to shorten time to answer, compared to standard anatomic pathology methods. They are combining developing technologies in ways that achieve these goals.
—Kristin Althea O’Connor
Related Information:
Ultra-fast Deep-Learned CNS Tumor Classification during Surgery
New AI Tool Diagnoses Brain Tumors on the Operating Table
Pediatric Brain Tumor Types Revealed Mid-Surgery with Nanopore Sequencing and AI
AI Speeds Up Identification Brain Tumor Type
Four New Cancer Research Projects at UMC Utrecht Receive Hanarth Grants
Rapid Nanopore Sequencing, Machine Learning Enable Tumor Classification during Surgery
Scientists in Italy Develop Hierarchical Artificial Intelligence System to Analyze Bacterial Species in Culture Plates
New artificial intelligence model agrees with interpretations of human medical technologists and microbiologists with extraordinary accuracy
Microbiology laboratories will be interested in news from Brescia University in Italy, where researchers reportedly have developed a deep learning model that can visually identify and analyze bacterial species in culture plates with a high level of agreement with interpretations made by medical technologists.
They initially trained and tested the system to digitally identify pathogens associated with urinary tract infections (UTIs). UTIs are the source for a large volume of clinical laboratory microbiological testing.
The system, known as DeepColony, uses hierarchical artificial intelligence technology. The researchers say hierarchical AI is better suited to complex decision-making than other approaches, such as generative AI.
The researchers published their findings in the journal Nature titled, “Hierarchical AI Enables Global Interpretation of Culture Plates in the Era of Digital Microbiology.”
In their Nature paper, the researchers explained that microbiologists use conventional methods to visually examine culture plates that contain bacterial colonies. The scientists hypothesize which species of bacteria are present, after which they test their hypothesis “by regrowing samples from each colony separately and then employing mass spectroscopy techniques,” to confirm their hypotheses.
However, DeepColony—which was designed for use with clinical laboratory automation systems—looks at high-resolution digital scans of cultured plates and attempts to identify the bacterial strains and analyze them in much the same way a microbiologist would. For example, it can identify species based on their appearance and determine which colonies are suitable for analysis, the researchers explained.
“Working on a large stream of clinical data, and a complete set of 32 pathogens, the proposed system is capable of effectively assisting plate interpretation with a surprising degree of accuracy in the widespread and demanding framework of urinary tract infections,” the study authors wrote. “Moreover, thanks to the rich species-related generated information, DeepColony can be used for developing trustworthy clinical decision support services in laboratory automation ecosystems from local to global scale.”
“Compared to the most common solutions based on single convolutional neural networks (CNN), multi-network architectures are attractive in our case because of their ability to fit into contexts where decision-making processes are stratified into a complex structure,” wrote the study’s lead author Alberto Signoroni, PhD (above), Associate Professor of Computer Science, University of Brescia, and his researcher team in their Nature paper. “The system must be designed to generate useful and easily interpretable information and to support expert decisions according to safety-by-design and human-in-the-loop policies, aiming at achieving cost-effectiveness and skill-empowerment respectively.” Microbiologists and clinical laboratory managers will want to follow the further development of this technology. (Photo copyright: University of Brescia.)
How Hierarchical AI Works
Writing in LinkedIn, patent attorney and self-described technology expert David Cain, JD, of Hauptman Ham, LLP, explained that hierarchical AI systems “are structured in layers, each with its own distinct role yet interconnected in a way that forms a cohesive whole. These systems are significant because they mirror the complexity of human decision-making processes, incorporating multiple levels of analysis and action. This multi-tiered approach allows for nuanced problem-solving and decision-making, akin to a seasoned explorer deftly navigating through a multifaceted terrain.”
DeepColony, the researchers wrote, consists of multiple convolutional neural networks (CNNs) that exchange information and cooperate with one another. The system is structured into five levels—labeled 0 through 4—each handling a different part of the analysis:
- At level 0, the system determines the number of bacterial colonies and their locations on the plate.
- At level 1, the system identifies “good colonies,” meaning those suitable for further identification and analysis.
- At level 2, the system assigns each good colony to a bacterial species “based on visual appearance and growth characteristics,” the researchers wrote, referring to the determination as being “pathogen aware, similarity agnostic.”
The CNN used at this stage was trained by using images of 26,213 isolated colonies comprising 32 bacterial species, the researchers wrote in their paper. Most came from clinical laboratories, but some were obtained from the American Type Culture Collection (ATCC), a repository of biological materials and information resources available to researchers.
- At level 3, the system attempts to improve accuracy by looking at the larger context of the plate. The goal here is to “determine if observed colonies are similar (pure culture) or different (mixed cultures),” the researchers wrote, describing this step as “similarity aware, pathogen agnostic.” This enables the system to recognize variants of the same strain, the researchers noted, and has the effect of reducing the number of strains identified by the system.
At this level, the system uses two “Siamese CNNs,” which were trained with a dataset of 200,000 image pairs.
Then, at level 4, the system “assesses the clinical significance of the entire plate,” the researchers added. Each plate is labeled as:
- “Positive” (significant bacterial growth),
- “No significant growth” (negative), or
- “Contaminated,” meaning it has three or more “different colony morphologies without a particular pathogen that is prevalent over the others,” the researchers wrote.
If a plate is labeled as “positive,” it can be “further evaluated for possible downstream steps,” using MALDI-TOF mass spectrometry or tests to determine susceptibility to antimicrobial measures, the researchers stated.
“This decision-making process takes into account not only the identification results but also adheres to the specific laboratory guidelines to ensure a proper supportive interpretation in the context of use,” the researchers wrote.
Nearly 100% Agreement with Medical Technologists
To gauge DeepColony’s accuracy, the researchers tested it on a dataset of more than 5,000 urine cultures from a US laboratory. They then compared its analyses with those of human medical technologists who had analyzed the same samples.
Agreement was 99.2% for no-growth cultures, 95.6% for positive cultures, and 77.1% for contaminated or mixed growth cultures, the researchers wrote.
The lower agreement for contaminated cultures was due to “a deliberately precautionary behavior, which is related to ‘safety by design’ criteria,” the researchers noted.
Lead study author Alberto Signoroni, PhD, Associate Professor of Computer Science, University of Brescia, wrote in Nature that many of the plates identified by medical technologists as “contaminated” were labeled as “positive” by DeepColony. “We maximized true negatives while allowing for some false positives, so that DeepColony [can] focus on the most relevant or critical cases,” he said.
Will DeepColony replace medical technologists in clinical laboratories any time soon? Not likely. But the Brescia University study indicates the direction AI in healthcare is headed, with high accuracy and increasing speed. The day may not be far off when pathologists and microbiologists regularly employ AI algorithms to diagnose disease.
—Stephen Beale
Related Information:
Hierarchical AI Enables Global Interpretation of Culture Plates in the Era of Digital Microbiology
An AI System Helps Microbiologists Identify Bacteria
University of Oxford Researchers Use Spectroscopy and Artificial Intelligence to Create a Blood Test for Chronic Fatigue Syndrome
Spectroscopic technique was 91% accurate in identifying the notoriously difficult-to-diagnose disease suggesting a clinical diagnostic test for CFS may be possible
Most clinical pathologists know that, despite their best efforts, scientists have failed to come up with a reliable clinical laboratory blood test for diagnosing myalgic encephalomyelitis (ME), the condition commonly known as chronic fatigue syndrome (CFS)—at least not one that’s ready for clinical use.
But now an international team of researchers at the University of Oxford has developed an experimental non-invasive test for CFS using a simple blood draw, artificial intelligence (AI), and a spectroscopic technique known as Raman spectroscopy.
The approach uses a laser to identify unique cellular “fingerprints” associated with the disease, according to an Oxford news release.
“When Raman was added to a panel of potentially diagnostic outputs, we improved the ability of the model to identify the ME/CFS patients and controls,” Karl Morten, PhD, Director of Graduate Studies and Principal Investigator at Oxford University, told Advanced Science News. Morton led the research team along with Wei Huang, PhD, Professor of Biological Engineering at Oxford.
The researchers claim the test is 91% accurate in differentiating between healthy people, disease controls, and ME/CFS patients, and 84% accurate in differentiating between mild, moderate, and severe cases, the new release states.
The researchers published their paper in the journal Advanced Science titled, “Developing a Blood Cell-Based Diagnostic Test for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome Using Peripheral Blood Mononuclear Cells.”
“This could be a game changer as we are unsure what causes [ME/CFS] and diagnosis occurs perhaps 10 to 20 years after the condition has started to develop,” said Karl Morten, PhD, Director of Graduate Studies and Principal Investigator at Oxford University. “An early diagnosis might allow us to identify what is going wrong with the potential to fix it before the more long-term degenerative changes are observed.” Though this research may not lead to a simple clinical laboratory blood test for CFS, any non-invasive diagnostic test would enable doctors to help many people. (Photo copyright: Oxford University.)
Need for an ME/CFS Test
The federal Centers for Disease Control and Prevention (CDC) describes ME/CFS as “a serious, long-term illness that affects many body systems,” with symptoms that include severe fatigue and sleep difficulties. Citing an Institute of Medicine (IoM) report, the agency estimates that 836,000 to 2.5 million Americans suffer from the condition but notes that most cases have not been diagnosed.
“One of the difficulties is the complexity of the disease,” said Jonas Bergquist, MD, PhD, Director of the ME/CFS Research Center of Uppsala University in Sweden, told Advanced Science News. “Because it’s a multi-organ disorder, you get symptoms from many different regions of the body with different onsets, though it’s common with post viral syndrome to have different overlapping [symptoms] that disguise the diagnosis.” Bergquist was not involved with the Oxford study.
One key to the Oxford researchers’ technique is the use of multiple artificial intelligence models to analyze the spectral profiles. “These signatures are complex and by eye there are not necessarily clear features that separate ME/CFS patients from other groups,” Morten told Advanced Science News.
“The AI looks at this data and attempts to find features which can separate the groups,” he continued. “Different AI methods find different features in the data. Individually, each method is not that successful at assigning an unknown sample to the correct group. However, when we combine the different methods, we produce a model which can assign the subjects to the different groups very accurately.”
Without a reliable test, “diagnosis of the condition is difficult, with most patients relying on self-report, questionnaires, and subjective measures to receive a diagnosis,” the Oxford press release noted.
But developing such a test has been challenging, Advanced Science News noted.
How Oxford’s Raman Technique Works
Raman spectroscopy uses a laser to determine the “vibrational modes of molecules,” according to the Oxford press release.
“When a laser beam is directed at a cell, some of the scattered photons undergo frequency shifts due to energy exchanges with the cell’s molecular components,” the press release stated. “Raman micro-spectroscopy detects these shifted photons, providing a non-invasive method for single cell analysis. The resulting single cell Raman spectra serve as a unique fingerprint, revealing the intrinsic and biochemical properties and indicating the physiological and metabolic state of the cell.”
The researchers employed the technique on blood samples from 98 subjects, including 61 ME/CFS patients, 16 healthy controls, and 21 controls with multiple sclerosis (MS), Advanced Science reported.
The Oxford scientists focused their attention on peripheral blood mononuclear cells (PBMCs), as previous studies found that these cells showed “reduced energetic function” in ME/CFS patients. “With this evidence, the team proposed that single-cell analysis of PBMCs might reveal differences in the structure and morphology in ME/CFS patients compared to healthy controls and other disease groups such as multiple sclerosis,” the press release states.
Clinical Laboratory Blood Processing and the Oxford Raman Technique
Oxford’s Raman spectroscopic technique “only requires a small blood sample which could be developed as a point-of-care test perhaps from one drop of blood,” the researchers wrote. However, Advanced Science News pointed out that required laser microscopy equipment costs more than $250,000.
In their Advanced Science paper, the researchers note that the test could be made more widely available by transferring blood samples collected by local clinical laboratories to diagnostic centers that have the needed hardware.
“Alternatively, a compact system containing portable Raman instruments could be developed, which would be much cheaper than a standard Raman microscope, and [which] incorporated with microfluidic systems to stream cells through a Raman laser for detection, eliminating the need for lengthy blood sample processing,” the researchers wrote.
They noted that the technique could be adapted to test for other chronic conditions as well, such as MS, fibromyalgia, Lyme disease, and long COVID.
“Our paper is very much a starting point for future research,” Morten told Advanced Science News. “Larger cohorts need to be studied, and if Raman proves useful, we need to think carefully about how a test might be developed.”
Bergquist agreed, stating it’s “not necessarily something you would see in a doctor’s office. It requires a lot of advanced data analysis to use—I still see it as a research methodology. But in the long run, it could be developed into a tool that could be used in a more simplistic way.”
Though a useable diagnostic test may be far off, clinical laboratories should consider how they can aid in ME/CFS research.
—Stephen Beale
Related Information:
First Ever Diagnostic Test for Chronic Fatigue Syndrome Sparks Hope
Blood Test for Chronic Fatigue Syndrome Found to Be 91% Accurate
Scientists Develop Blood Test for Chronic Fatigue Syndrome
Biomarkers for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS): A Systematic Review
