Researchers intend their new AI image retrieval tool to help pathologists locate similar case images to reference for diagnostics, research, and education
Researchers at Stanford University turned to an unusual source—the X social media platform (formerly known as Twitter)—to train an artificial intelligence (AI) system that can look at clinical laboratory pathology images and then retrieve similar images from a database. This is an indication that pathologists are increasingly collecting and storing images of representative cases in their social media accounts. They then consult those libraries when working on new cases that have unusual or unfamiliar features.
The Stanford Medicine scientists trained their AI system—known as Pathology Language and Image Pretraining (PLIP)—on the OpenPath pathology dataset, which contains more than 200,000 images paired with natural language descriptions. The researchers collected most of the data by retrieving tweets in which pathologists posted images accompanied by comments.
“It might be surprising to some folks that there is actually a lot of high-quality medical knowledge that is shared on Twitter,” said researcher James Zou, PhD, Assistant Professor of Biomedical Data Science and senior author of the study, in a Stanford Medicine SCOPE blog post, which added that “the social media platform has become a popular forum for pathologists to share interesting images—so much so that the community has widely adopted a set of 32 hashtags to identify subspecialties.”
“It’s a very active community, which is why we were able to curate hundreds of thousands of these high-quality pathology discussions from Twitter,” Zou said.
“The main application is to help human pathologists look for similar cases to reference,” James Zou, PhD (above), Assistant Professor of Biomedical Data Science, senior author of the study, and his colleagues wrote in Nature Medicine. “Our approach demonstrates that publicly shared medical information is a tremendous resource that can be harnessed to develop medical artificial intelligence for enhancing diagnosis, knowledge sharing, and education.” Leveraging pathologists’ use of social media to store case images for future reference has worked out well for the Stanford Medicine study. (Photo copyright: Stanford University.)
Retrieving Pathology Images from Tweets
“The lack of annotated publicly-available medical images is a major barrier for innovations,” the researchers wrote in Nature Medicine. “At the same time, many de-identified images and much knowledge are shared by clinicians on public forums such as medical Twitter.”
In this case, the goal “is to train a model that can understand both the visual image and the text description,” Zou said in the SCOPE blog post.
“Pathology is perhaps even more suited to Twitter than many other medical fields because for most pathologists, the bulk of our daily work revolves around the interpretation of images for the diagnosis of human disease,” wrote Jerad M. Gardner, MD, a dermatopathologist and section head of bone/soft tissue pathology at Geisinger Medical Center in Danville, Pa., in a blog post about the Pathology Hashtag Ontology project. “Twitter allows us to easily share images of amazing cases with one another, and we can also discuss new controversies, share links to the most cutting edge literature, and interact with and promote the cause of our pathology professional organizations.”
The researchers used the 32 subspecialty hashtags to retrieve English-language tweets posted from 2006 to 2022. Images in the tweets were “typically high-resolution views of cells or tissues stained with dye,” according to the SCOPE blog post.
The researchers collected a total of 232,067 tweets and 243,375 image-text pairs across the 32 subspecialties, they reported. They augmented this with 88,250 replies that received the highest number of likes and had at least one keyword from the ICD-11 codebook. The SCOPE blog post noted that the rankings by “likes” enabled the researchers to screen for high-quality replies.
They then refined the dataset by removing duplicates, retweets, non-pathology images, and tweets marked by Twitter as being “sensitive.” They also removed tweets containing question marks, as this was an indicator that the practitioner was asking a question about an image rather than providing a description, the researchers wrote in Nature Medicine.
They cleaned the text by removing hashtags, Twitter handles, HTML tags, emojis, and links to websites, the researchers noted.
The final OpenPath dataset included:
116,504 image-text pairs from Twitter posts,
59,869 from replies, and
32,041 image-text pairs scraped from the internet or obtained from the LAION dataset.
The latter is an open-source database from Germany that can be used to train text-to-image AI software such as Stable Diffusion.
Training the PLIP AI Platform
Once they had the dataset, the next step was to train the PLIP AI model. This required a technique known as contrastive learning, the researchers wrote, in which the AI learns to associate features from the images with portions of the text.
As explained in Baeldung, an online technology publication, contrastive learning is based on the idea that “it is easier for someone with no prior knowledge, like a kid, to learn new things by contrasting between similar and dissimilar things instead of learning to recognize them one by one.”
“The power of such a model is that we don’t tell it specifically what features to look for. It’s learning the relevant features by itself,” Zou said in the SCOPE blog post.
The resulting AI PLIP tool will enable “a clinician to input a new image or text description to search for similar annotated images in the database—a sort of Google Image search customized for pathologists,” SCOPE explained.
“Maybe a pathologist is looking at something that’s a bit unusual or ambiguous,” Zou told SCOPE. “They could use PLIP to retrieve similar images, then reference those cases to help them make their diagnoses.”
The Stanford University researchers continue to collect pathology images from X. “The more data you have, the more it will improve,” Zou said.
Pathologists will want to keep an eye on the Stanford Medicine research team’s progress. The PLIP AI tool may be a boon to diagnostics and improve patient outcomes and care.
The focus of the ongoing GenoVA study is to “determine the clinical effectiveness of polygenic risk score testing among patients at high genetic risk for at least one of six diseases measured by time-to-diagnosis of prevalent or incident disease over 24 months,” according to the National Institutes of Health.
The scientists used data obtained from 36,423 patients enrolled in the Mass General Brigham Biobank. The six diseases they researched were:
The polygenic scores were then tested among 227 healthy adult patients to determine their risk for the six diseases. The researchers found that:
11% of the patients had a high-risk score for atrial fibrillation,
7% for coronary artery disease,
8% for diabetes, and
6% for colorectal cancer.
Among the subjects used for the study:
15% of the men in the study had a high-risk score for prostate cancer, and
13% of the women in the study had a high score for breast cancer.
The researchers concluded that the implementation of PRS may help improve disease prevention and management and give doctor’s a way to assess a patient’s risk for these conditions. They published their findings in the journal Nature Medicine, titled, “Development of a Clinical Polygenic Risk Score Assay and Reporting Workflow.”
“We have shown that [medical] laboratory assay development and PRS reporting to patients and physicians are feasible … As the performance of PRS continues to improve—particularly for individuals of underrepresented ancestry groups—the implementation processes we describe can serve as generalizable models for laboratories and health systems looking to realize the potential of PRS for improved patient health,” the researchers wrote.
Using PRS in Clinical Decision Support
Polygenetic risk scores examine multiple genetic markers for risk of certain diseases. A calculation based on hundreds or thousands of these genetic markers could help doctors and patients make personalized treatment decisions, a core tenet of precision medicine.
“As a primary care physician myself, I knew that busy physicians were not going to have time to take an entire course on polygenic risk scores. Instead, we wanted to design a lab report and informational resources that succinctly told the doctor and patient what they need to know to make a decision about using a polygenic risk score result in their healthcare,” epidemiologist Jason Vassy, MD, told The Harvard Gazette. Vassy is Associate Professor, Harvard Medical School at VA Boston Healthcare System and one of the authors of the research.
“This is another great example of precision medicine,” Jason Vassy, MD (above), Adjunct Assistant Professor, General Internal Medicine at Boston University School of Medicine, told WebMD. “There’s always been a tantalizing idea that someone’s genetic makeup might help tailor preventative medicine and treatment.” Personalized clinical laboratory testing is increasingly becoming based on an individual’s genetics. (Photo copyright: Harvard Medical School.)
Increasing Diversity of Patients in Genomic Research
The team did encounter some challenges during their analysis. Because most existing genomic research was performed on persons of European descent, the risk scores are less accurate among non-European populations. The researchers for this study addressed this limitation by applying additional statistical methods to qualify accurate PRS calculations across multiple racial groups.
“Researchers must continue working to increase the diversity of patients participating in genomics research,” said Matthew Lebo, PhD, Chief Laboratory Director, Laboratory Molecular Medicine, at Mass General Brigham and one of the authors of the study. “In the meantime, we were heartened to see that we could generate and implement valid genetic scores for patients of diverse backgrounds,” he told The Harvard Gazette.
The team hopes the scores may be utilized in the future to help doctors and patients make better decisions regarding preventative care and screenings.
“It’s easy to say that everyone needs a colonoscopy at age 45,” Vassy told WebMD. “But what if you’re such a low risk that you could put it off for longer? We may get to the point where we understand risk so much that someone may not need one at all.”
Future of PRS in Clinical Decision Making
The scientists plan to enroll more than 1,000 patients in a new program and track them for two years to assess how medical professionals use PRS in clinical care. It is feasible that patients who are at high risk for certain diseases may opt for more frequent screenings or take preventative medicines to mitigate their risk.
“Getting to that point will take time,” Vassy added. “But I can see this type of information playing a role in shared decision making between doctor and patient in the near future.”
The team also established resources and educational materials to assist both doctors and patients in using the scores.
“It’s still very early days for precision prevention,” Vassy noted, “but we have shown it is feasible to overcome some of the first barriers to bringing polygenic risk scores into the clinic.”
More research and studies are needed to prove the effectiveness of using PRS tests in clinical care and determine its role in customized treatment plans based on personal genetics. Nevertheless, pathologists and medical scientists will want to follow the GenoVA study.
“It is probably most helpful to think of polygenic risk scores as a risk factor for disease, not a diagnostic test or an indication that an individual will certainly develop the disease,” Vassy said. “Most diseases have complex, multifactorial etiologies, and a high polygenic risk score is just one piece of the puzzle.”
Pathologists and clinical laboratory managers may want to stay informed as researchers in the GenoVA study tease new useful diagnostic insights from their ongoing study of the whole human genome. Meanwhile, the GenoVA team is moving forward with the 1,000-patient study with the expectation that this new knowledge may enable earlier and more accurate diagnoses of the health conditions that were the focus of the GenoVA study.
Study conducted on International Space Station found crew’s red blood cells were destroyed 54% faster in space than while on Earth
Hemolysis in blood specimens is something that clinical laboratories deal with every day. Now researchers in Canada have determined that, while astronauts are in space, hemolysis is a causative factor in the condition known as “space anemia.”
Hematologists whose clinical laboratories process a steady volume of complete blood count (CBC) tests to diagnosis anemia will want to take note of this research study, which was conducted at the University of Ottawa and on the International Space Station. Dubbed the “MARROW” study, it may have uncovered not only why astronauts suffer from anemia even a year after returning to Earth, but also how those insights can be applied to treatments for anemia and other blood diseases for Earthbound patients as well.
Anemia is caused by a marked decrease in the number of red blood cells and can lead to weakness, persistent fatigue, and slower brain function, which on Earth is concerning, but in space can be life threatening.
“Space anemia has consistently been reported when astronauts returned to Earth since the first space missions, but we didn’t know why,” said the study’s lead author Guy Trudel, MD, in a University of Ottawa news release.
Trudel is Director of the Bone and Joint Research Laboratory at the Ottawa Hospital Rehabilitation Centre in Canada. He is also a Rehabilitation Physician and Researcher at the Ottawa Hospital and Professor of Medicine at the University of Ottawa, and the principal investigator of the MARROW study, which is investigating the effects of microgravity on bone marrow, according to NASA.
“Our study shows that upon arriving in space, more red blood cells are destroyed, and this continues for the entire duration of the astronaut’s mission,” he added.
Although these scientific findings may not immediately lead to new methodologies for testing human blood for use in clinical laboratories, the insights gleaned from the study could inform future studies designed to learn how to get the body to produce more red blood cells in ways that benefit patients diagnosed with anemia or other blood disorders.
Guy Trudel, MD (above), Rehabilitation Physician and Researcher at the Ottawa Hospital and Professor at the University of Ottawa, is lead author of a Canadian study explaining why astronauts develop “space anemia.” In space, astronauts’ red blood cells are destroyed 54% faster than while on Earth, a finding that could have implications for space tourists and astronauts on longer missions to the moon and Mars, the researchers noted. (Photo copyright: University of Ottawa.)
Effects of Anemia Continue One Year after Returning to Earth
The MARROW research project, which was funded by the Canadian Space Agency (CSA), required the participation of 14 astronauts on the International Space Station.
The researchers began collecting data in October 2015 and completed their final tests in June 2020. They found that astronauts’ bodies destroyed 54% more red blood cells in space than would be normal on Earth, according to the study published in Nature Medicine.
“Thankfully, having fewer red blood cells in space isn’t a problem when your body is weightless,” Trudel said in the news release. “But when landing on Earth, and potentially on other planets or moons, anemia affecting your energy, endurance, and strength can threaten mission objectives. The effects of anemia are felt once you land and must deal with gravity again.”
The MARROW experiment detected the following changes:
During a six-month mission, astronauts’ bodies were destroying 54% more red blood cells than typical preflight rates.
Five of the 13 astronauts who had their blood drawn shortly after landing back on Earth were anemic. Red blood cell levels gradually improved three to four months post-flight.
The rate of red blood cell destruction remained 30% higher one year after landing than before missions to the International Space Station.
“Increased hemolysis as a primary effect of exposure to space constitutes a paradigm shift in our understanding of space anemia … Persistent hemolysis during space missions suggests that the longer the exposure, the worse the anemia,” the study’s authors wrote.
Measurements were made by testing the astronauts’ blood for iron levels and using breath tests to measure exhaled carbon monoxide. One molecule of carbon monoxide is produced every time one molecule of heme, the deep-red pigment in blood cells, is destroyed.
According to the researchers, the discovery that space travel increases red blood cell destruction:
highlights the need to screen astronauts and space tourists for existing blood or health conditions that are affected by anemia;
impacts longer missions to the moon and Mars, which would likely worsen an astronaut’s anemia;
suggests astronauts require an adapted diet; and
shows it is unclear how long the body can maintain this higher rate of destruction and production of red blood cells.
Space Study Could Lead to Better Healthcare on Earth
A 2007 NASA study published in Microgravity Science and Technology blamed space anemia on water loss during space flight decreasing the amount of hemoglobin in red blood cells. The study labeled space anemia a “15-day ailment” because those researchers believed issues resolved within 15 days of crew members returning to Earth.
The MARROW study, however, found much longer-lasting implications for astronauts in space, which could lead to new insights for patients on Earth. The Canadian Space Agency believes the study’s findings could lead to better understanding and monitoring of the effects of physical inactivity on seniors, bedridden patients, and those with reduced mobility or undergoing rehabilitation.
“The findings have implications for understanding the physiological consequences of space flight and anemia in patients on the ground,” Sulekha Anand, PhD, a professor in the Department of Biological Sciences at San Jose State University, told Reuters.
This latest study shows how discoveries in space continue to lead to advancements in scientists’ understanding of how the human body functions. That knowledge may one day provide the foundation for developing new or improved clinical laboratory tests for astronauts as well as everyday earthlings.
One key finding of interest to clinical laboratory scientists is that this research study indicates that the human microbiome may more closely correlate with blood markers of metabolic disease than the genome of individuals
In the search for more sensitive diagnostic biomarkers (meaning the ability to detect disease with smaller samples and smaller quantities of the target biomarker), an international team of researchers has teased out a finding that a panel of multiple biomarkers in the human microbiome is more closely correlated with metabolic disease than genetic markers.
The team also discovered that the foods an individual ate had a more powerful impact on their microbiomes than their genes. The study participants included several sets of identical twins. The researchers found that identical twins shared only about 34% of the same gut microbes. People who were unrelated shared 30% of the same gut microbes.
This is a fascinating insight for pathologists and microbiologists involved in the study of the human microbiome for use in development of precision medicine clinical laboratory testing and drug therapies.
Microbiome Markers for Obesity, Heart Disease, and More
The study began in 2018, when an international team of researchers analyzed the gut microbiomes, diets, and blood biomarkers for cardiometabolic health obtained from 1,100 mostly healthy adults in the United Kingdom (UK) and the United States (US). They collected blood samples from the participants before and after meals to examine blood sugar levels, hormones, cholesterol, and inflammation levels. Sleep and activity levels also were monitored. Participants had to wear a continuous glucose monitor for two weeks during the research period.
The scientists discovered that the composition of a healthy gut microbiome is strongly linked to certain foods, food groups, nutrients, and diet composition. They identified markers for obesity, impaired glucose tolerance, and cardiovascular disease in the gut bacteria.
“When you eat, you’re not just nourishing your body, you’re feeding the trillions of microbes that live inside your gut,” genetic epidemiologist Tim Spector, MD, FmedSCi, told Labroots. Spector is a professor of genetic epidemiology at King’s College London and one of the authors of the study.
The scientists found that a diet rich in nutrient-dense, whole foods was more beneficial to a healthy gut microbiome, which can be an indicator of good health. Individuals who ate minimally processed foods, such as vegetables, nuts, eggs, and seafood were more likely to have healthy gut bacteria than individuals who consumed large amounts of highly processed foods, like juices and other sweetened beverages, processed meats, and refined grains and foods that were high in added sugars and salt.
“It goes back to the age-old message of eating as many whole and unprocessed foods as possible,” Sarah Berry, PhD, a nutrition scientist at King’s College London and a co-author of the study told The New York Times. “What this research shows for the first time is the link between the quality of the food we’re eating, the quality of our microbiomes, and ultimately our health outcomes,” she added.
The researchers concluded that heavily processed foods tend to contain very minimal amounts of fiber, a macronutrient that helps promote good bacteria in the gut microbiome and leads to better metabolic and cardiovascular health.
They found that people who had healthy blood sugar levels following a meal had higher levels of good bacteria called Prevotella copri, a genus of gram-negative bacteria, and Blastocystis, a genus of single-celled heterokont parasites, present in their guts. These bacteria are associated with lower levels of visceral fat, which accumulates around internal organs and increases risk of heart disease.
These “good” microbes also are affiliated with lower levels of inflammation, better blood sugar control, and lower spikes in blood fat and cholesterol levels after meals.
“We were surprised to see such large, clear groups of what we informally call ‘good’ and ‘bad’ microbes emerging from our analysis,” Nicola Segata, PhD (above), told News Medical. Segata is a professor and principal investigator at the Computational Metagenomics Lab at the University of Trento in Italy, and co-author of the study. “It is also exciting to see that microbiologists know so little about many of these microbes that they are not even named yet. This is now a big area of focus for us, as we believe they may open new insights in the future into how we could use the gut microbiome as a modifiable target to improve human metabolism and health,” he added. Pathologists and clinical laboratory scientists who read Dark Daily are already familiar with the plethora of ways the human microbiome is being studied for use in diagnostic testing and drug therapy. (Photo copyright: University of Trento.)
The study also found that different people have wildly varying metabolic responses to the same foods, partially due to the types of bacteria residing in their gut microbiome. The consumption of some foods is better for overall health than other foods, but there is no definitive, one-size-fits-all diet that works for everyone.
“What we found in our study was that the same diet in two different individuals does not lead to the same microbiome, and it does not lead to the same metabolic response. There is a lot of variation,” Andrew Chan, MD, Professor of Medicine at Harvard Medical School, told The New York Times. Chan is also Chief of the Clinical and Translational Epidemiology Unit at Massachusetts General Hospital and co-author of the study.
Small Changes in Diet, Big Impact to Health
The team is now planning a clinical trial to test whether changes in diet can alter levels of good and bad microbes in the gut. If proven to be true, such information could help clinicians design personalized nutritional plans that would enable individuals to improve their gut microbiome and their overall health.
“As a nutritional scientist, finding novel microbes that are linked to specific foods, as well as metabolic health, is exciting,” Berry told News Medical. “Given the highly personalized composition of each individual’s microbiome, our research suggests that we may be able to modify our gut microbiome to optimize our health by choosing the best foods for our unique biology.
“We think there are lots of small changes that people can make that can have a big impact on their health that might be mediated through the microbiome,” Berry told The New York Times.
More research and clinical trials are needed before diagnostic tests that use microbiome biomarkers to detect metabolic diseases can be developed. But these early research findings are a sign to pathologists and clinical laboratory managers that microbiome-based assays may come to play a more significant role in the early detection of several metabolic diseases.
New bioinformatic tool finds gut microbiota may be ‘potential reservoir of bloodstream pathogens’ suggesting patients’ own bodies can be source of infections
Clinical laboratories in hospitals and health networks throughout the nation are collaborating in the priority effort to reduce deaths from sepsis and related blood infections. Now comes news that researchers at Stanford have identified an unexpected source of bloodstream infections. This finding may help medical laboratories contribute to faster and more accurate diagnoses of blood infections, particularly for hospital inpatients.
Lax infection-control practices often are blamed for hospital-acquired infections (HAIs). And HAIs certainly have been responsible for many tragic avoidable deaths. However, new research from Stanford University School of Medicine shows that hospital staff, other patients, or unclean instruments may not be solely responsible for all infections that present during hospital stays. According to Stanford researchers, a patient’s own digestive tract can be the surprising culprit for many bloodstream infections. This finding confirms a common belief that the patient’s microbiome probably is involved in many blood infections.
The researchers published their findings in Nature Medicine.
Bacteria Causing Blood Infections Found in Patients’ Stool Samples After Bone Marrow Transplants
Using a new bioinformatic computational tool called StrainSifter, the Stanford University team rapidly and accurately identified a surprising infection source in a group of hospitalized patients—microbes already living in the patients’ large intestines—a Stanford University news release explained.
The researchers analyzed blood and stool samples from 30 patients who developed bloodstream infections after receiving bone marrow transplants between October 2015 and June 2017 at Stanford Hospital. The researchers sought to determine whether the bacteria isolated from the patients’ blood also was found in stool specimens that had been collected prior to the transplants. The process required sequencing not only the patients’ DNA, but also analyzing the genomes of all the individual microbial strains resident in each patient’s stool.
“Just finding E. coli in a patient’s blood and again in the patient’s stool doesn’t mean they’re the same strain,” Ami Bhatt, MD, PhD, Assistant Professor of Hematology and Genetics at Stanford, explained in the news release. Bhatt served as senior author of the study. (Photo copyright: Stanford University.)
Analysis found that more than one-third of the patients’ stool samples (11) contained detectable levels of the same bacterial strain that had caused those patients’ bloodstream infections.
“Because the gut normally harbors more than 1,000 different bacterial strains, it’s looked upon as a likely culprit of bloodstream infections, especially when the identified pathogen is one known to thrive inside the gut,” Ami Bhatt, MD, PhD, Assistant Professor of Hematology and Genetics at Stanford, said in the news release. “But while this culpability has been assumed—and it’s an entirely reasonable assumption—it’s never been proven. Our study demonstrates that it’s true.”
Clinical and DNA data confirmed the gastrointestinal presence of Escherichia coli and Klebsiella pneumonia, common causes of pneumonia, urinary tract infections, and other potentially serious conditions. In addition, they found other disease-causing pathogens in the gut that they would not have expected to be there.
“We also find cases where typically nonenteric [outside the intestine] pathogens, such as Pseudomonas aeruginosa and Staphylococcus epidermidis, are found in the gut microbiota, thereby challenging the existing informal dogma of these infections originating from environmental or skin sources,” Fiona Tamburini, a senior graduate student, and postdoctoral scholar Tessa Andermann, MD, MPH, Infectious Disease Medical Fellow, wrote in Nature Medicine.
New Tool for Precision Medicine
Bhatt believes being able to trace the source of bloodstream infections will help doctors provide more targeted treatments for HAIs and potentially lead to effective prevention methods. This will create a new opportunity for microbiology laboratories to provide the necessary diagnostic tests designed to guide therapeutic choices of attending physicians.
“Until now, we couldn’t pinpoint those sources with high confidence,” Bhatt said in the news release. “That’s a problem because when a patient has a bloodstream infection, it’s not enough simply to administer broad-spectrum antibiotics. You need to treat the source, or the infection will come back.”
Bhatt says the computational tool has the potential to allow medical practitioners to quickly identify whether a pathogen responsible for a patient’s bloodstream infection came from a break in the skin, leaked through the intestinal wall into the blood, or was passed on through an inserted catheter or other object.
Bhatt’s team focused on the intestines for their study because it’s the home of 1,000 to 2,000 different germs. Dark Daily has reported often on developments involving human gut bacteria (AKA, microbiome) in e-briefings going back to 2013. While these gut bacteria do not typically cause problems, Bhatt said, “It’s only when they show up in the wrong place—due, for example, to leaking through a disrupted intestinal barrier into the bloodstream—that they cause trouble.”
Because nearly 40% of immunocompromised patients who spend up to six weeks in a hospital develop bloodstream infections, the Stanford findings could signal a major breakthrough in preventing HAIs. However, larger studies are needed to validate the researchers’ contention that the gut is a “potential reservoir of bloodstreams pathogens.”
If true, microbiologists and clinical pathologists may in the future have a new method for helping hospitals identify, track, and treat blood-born infections as well as and preventing HAIs.
