Researchers intend their new AI image retrieval tool to help pathologists locate similar case images to reference for diagnostics, research, and education
Researchers at Stanford University turned to an unusual source—the X social media platform (formerly known as Twitter)—to train an artificial intelligence (AI) system that can look at clinical laboratory pathology images and then retrieve similar images from a database. This is an indication that pathologists are increasingly collecting and storing images of representative cases in their social media accounts. They then consult those libraries when working on new cases that have unusual or unfamiliar features.
The Stanford Medicine scientists trained their AI system—known as Pathology Language and Image Pretraining (PLIP)—on the OpenPath pathology dataset, which contains more than 200,000 images paired with natural language descriptions. The researchers collected most of the data by retrieving tweets in which pathologists posted images accompanied by comments.
“It might be surprising to some folks that there is actually a lot of high-quality medical knowledge that is shared on Twitter,” said researcher James Zou, PhD, Assistant Professor of Biomedical Data Science and senior author of the study, in a Stanford Medicine SCOPE blog post, which added that “the social media platform has become a popular forum for pathologists to share interesting images—so much so that the community has widely adopted a set of 32 hashtags to identify subspecialties.”
“It’s a very active community, which is why we were able to curate hundreds of thousands of these high-quality pathology discussions from Twitter,” Zou said.
“The main application is to help human pathologists look for similar cases to reference,” James Zou, PhD (above), Assistant Professor of Biomedical Data Science, senior author of the study, and his colleagues wrote in Nature Medicine. “Our approach demonstrates that publicly shared medical information is a tremendous resource that can be harnessed to develop medical artificial intelligence for enhancing diagnosis, knowledge sharing, and education.” Leveraging pathologists’ use of social media to store case images for future reference has worked out well for the Stanford Medicine study. (Photo copyright: Stanford University.)
Retrieving Pathology Images from Tweets
“The lack of annotated publicly-available medical images is a major barrier for innovations,” the researchers wrote in Nature Medicine. “At the same time, many de-identified images and much knowledge are shared by clinicians on public forums such as medical Twitter.”
In this case, the goal “is to train a model that can understand both the visual image and the text description,” Zou said in the SCOPE blog post.
“Pathology is perhaps even more suited to Twitter than many other medical fields because for most pathologists, the bulk of our daily work revolves around the interpretation of images for the diagnosis of human disease,” wrote Jerad M. Gardner, MD, a dermatopathologist and section head of bone/soft tissue pathology at Geisinger Medical Center in Danville, Pa., in a blog post about the Pathology Hashtag Ontology project. “Twitter allows us to easily share images of amazing cases with one another, and we can also discuss new controversies, share links to the most cutting edge literature, and interact with and promote the cause of our pathology professional organizations.”
The researchers used the 32 subspecialty hashtags to retrieve English-language tweets posted from 2006 to 2022. Images in the tweets were “typically high-resolution views of cells or tissues stained with dye,” according to the SCOPE blog post.
The researchers collected a total of 232,067 tweets and 243,375 image-text pairs across the 32 subspecialties, they reported. They augmented this with 88,250 replies that received the highest number of likes and had at least one keyword from the ICD-11 codebook. The SCOPE blog post noted that the rankings by “likes” enabled the researchers to screen for high-quality replies.
They then refined the dataset by removing duplicates, retweets, non-pathology images, and tweets marked by Twitter as being “sensitive.” They also removed tweets containing question marks, as this was an indicator that the practitioner was asking a question about an image rather than providing a description, the researchers wrote in Nature Medicine.
They cleaned the text by removing hashtags, Twitter handles, HTML tags, emojis, and links to websites, the researchers noted.
The final OpenPath dataset included:
116,504 image-text pairs from Twitter posts,
59,869 from replies, and
32,041 image-text pairs scraped from the internet or obtained from the LAION dataset.
The latter is an open-source database from Germany that can be used to train text-to-image AI software such as Stable Diffusion.
Training the PLIP AI Platform
Once they had the dataset, the next step was to train the PLIP AI model. This required a technique known as contrastive learning, the researchers wrote, in which the AI learns to associate features from the images with portions of the text.
As explained in Baeldung, an online technology publication, contrastive learning is based on the idea that “it is easier for someone with no prior knowledge, like a kid, to learn new things by contrasting between similar and dissimilar things instead of learning to recognize them one by one.”
“The power of such a model is that we don’t tell it specifically what features to look for. It’s learning the relevant features by itself,” Zou said in the SCOPE blog post.
The resulting AI PLIP tool will enable “a clinician to input a new image or text description to search for similar annotated images in the database—a sort of Google Image search customized for pathologists,” SCOPE explained.
“Maybe a pathologist is looking at something that’s a bit unusual or ambiguous,” Zou told SCOPE. “They could use PLIP to retrieve similar images, then reference those cases to help them make their diagnoses.”
The Stanford University researchers continue to collect pathology images from X. “The more data you have, the more it will improve,” Zou said.
Pathologists will want to keep an eye on the Stanford Medicine research team’s progress. The PLIP AI tool may be a boon to diagnostics and improve patient outcomes and care.
As infectious bacteria become even more resistant to antibiotics, chronic disease patients with weakened immune systems are in particular danger
Microbiologists
and clinical
laboratory managers in the United States may find it useful to learn that
exceptionally virulent strains of bacteria are causing increasing numbers of cancer
patient deaths in India. Given the speed with which infectious diseases spread
throughout the world, it’s not surprising that deaths due to similar hospital-acquired
infections (HAIs) are increasing in the US as well.
Recent news reporting indicates that an ever-growing number
of cancer patients in the world’s second most populous nation are struggling to
survive these infections while undergoing chemotherapy and other treatments for
their cancers.
In some ways, this situation is the result of more powerful antibiotics. Today’s modern antibiotics help physicians, pathologists, and clinical laboratories protect patients from infectious disease. However, it’s a tragic fact that those same powerful drugs are making patients with chronic diseases, such as cancer, more susceptible to death from HAIs caused by bacteria that are becoming increasingly resistant to those same antibiotics.
India is a prime example of that devastating dichotomy. Bloomberg
reported that a study conducted by Abdul
Ghafur, MD, an infectious disease physician with Apollo Hospitals in Chennai, India,
et al, concluded that “Almost two-thirds of cancer patients with a
carbapenem-resistant infection are dead within four weeks, vs. a 28-day
mortality rate of 38% in patients whose infections are curable.”
This news should serve as an alert to pathologists, microbiologists,
and clinical laboratory leaders in the US as these same superbugs—which resist
not only antibiotics but other drugs as well—may become more prevalent in this
country.
‘We Don’t Know
What to Do’
The dire challenge facing India’s cancer patients is due to escalating
bloodstream infections associated with carbapenem-resistant
enterobacteriaceae (CRE), a particularly deadly bacteria that has become
resistant to even the most potent carbapenem antibiotics, generally
considered drugs of last resort for dealing with life-threatening infections.
Lately, the problem has only escalated. “We are facing a
difficult scenario—to give chemotherapy and cure the cancer and get a
drug-resistant infection and the patient dying of infections.” Ghafur told Bloomberg.
“We don’t know what to do. The world doesn’t know what to do in this scenario.”
Ghafur added, “However wonderful the developments in the
field of oncology, they are not going to be useful, because we know cancer
patients die of infections.”
Abdul Ghafur, MD (above), an infectious disease physician with Apollo Hospitals in Chennai, India, told The Better India that, “Indians, are obsessed with antibiotics and believe that they can cure almost all infections, including viral infections! Moreover, at least half of the prescriptions by Indian doctors include an antibiotic. Sadly, the public believes that whenever we get cold and cough, we need to swallow antibiotics for three days along with paracetamol [acetaminophen]! This is a myth that urgently needs to disappear!” (Photo copyright: Longitude Prize.)
The problem in India, Bloomberg reports, is
exacerbated by contaminated food and water. “Germs acquired through ingesting
contaminated food and water become part of the normal gut microbiome, but they can
turn deadly if they escape the bowel and infect the urinary tract, blood, and
other tissues.” And chemotherapy patients, who likely have weakened digestive
tracts, suffer most when the deadly germs reach the urinary tract, blood, and surrounding
tissues.
“Ten years ago, carbapenem-resistant superbug infections
were rare. Now, infections such as carbapenem-resistant klebsiella bloodstream
infection, urinary infection, pneumonia, and surgical site infections are a
day-to-day problem in our (Indian) hospitals. Even healthy adults in the
community may carry these bacteria in their gut in Indian metropolitan cities;
up to 5% of people carry these superbugs in their intestines,” Ghafur told The
Better India.
“These patients receive chemotherapy during treatment, which
lead to severe mucositis
of gastrointestinal tract and myelosuppression.
It was hypothesized that the gut colonizer translocate into blood circulation
causing [bloodstream infection],” the AIIMS paper states.
US Cases of C. auris Also Linked to CRE
Deaths in the US involving the fungus Candida auris (C. auris)
have been linked to CRE as well. And, people who were hospitalized outside the
US may be at particular risk.
The CDC reported on
a Maryland resident who was hospitalized in Kenya with a
carbapenemase-producing infection, which was later diagnosed as C. auris. The CDC
describes C. auris as “an emerging drug-resistant yeast of high public concern
… C auris frequently co-occurs with carbapenemase-producing organisms like
CRE.”
The graphic above, developed by the NYT from CDC data, shows that Candida auris is found globally and not restricted to poor or resource-strapped nations. “The fungus seems to have emerged in several locations at once, not from a single source,” the NYT reports. This means clinical laboratories can expect to be processing more tests to identify the deadly fungus. (Graphic copyright: New York Times/CDC.)
Drug-resistant germs are a public health threat that has
grown beyond overuse of antibiotics to an “explosion of resistant fungi,”
reported the New
York Times (NYT).
“It’s an enormous problem. We depend on being able to treat
those patients with antifungals,” Matthew Fisher, PhD,
Professor of Fungal Disease Epidemiology at Imperial College London, told the NYT.
The NYT article states that “Nearly half of patients
who contract C. auris die within 90 days, according to the CDC. Yet the world’s
experts have not nailed down where it came from in the first place.”
Cases of C. auris in the US are showing up in New York, New
Jersey, and Illinois and is arriving on travelers from many countries,
including India, Pakistan, South Africa, Spain, United Kingdom, and
Venezuela.
“It is a creature from the black lagoon,” Tom Chiller, MD,
Chief of the Mycotic
Diseases Branch at the CDC told the NYT. “It bubbled up and now it
is everywhere.”
Since antibiotics are used heavily in agriculture and
farming worldwide, the numbers of antibiotic-resistant infections will likely
increase. Things may get worse, before they get better.
Pathologists, microbiologists, oncologists, and clinical
laboratories involved in caring for patients with antibiotic-resistant
infections will want to fully understand the dangers involved, not just to
patients, but to healthcare workers as well.
Of the five trends described in a report published by Kalorama, only two made the list for both years: Consolidation within the IVD industry and growth in molecular point of care
What a difference one year can make in the most significant trends influencing the in vitro diagnostics (IVD) industry, which also influences clinical laboratories, the largest customers of IVD manufacturers. These insights come from comparing the top five IVD trends for 2016 as identified by Kalorama Information from its top five IVD trends that it says dominated during 2015.
Kalorama is a division of MarketResearch.com, a company that publishes market research in the life sciences. In a report titled, “Five IVD Market Trends to Watch for in 2016,” it published its picks for the top five trends in IVD testing for 2016. The five most prominent trends recognized by the healthcare research marketer are as follows: (more…)
Illumina-bioMérieux Service to Aid Hospital and Public Health Labs
Illumina designated sequencing laboratories with Illumina MiSeq® systems will collaborate with microbiologists working in hospital and public health laboratories to prevent, rapidly track, and contain infectious disease agents in hospitals and communities. (more…)
Separate research projects at University of Washington and in the United Kingdom are producing handheld diagnostic devices to accurately detect Malaria
Two new handheld, point-of-care test (POC) devices for malaria could save millions of lives in third-world countries. At the same time, these POC devices may lead to inexpensive alternatives for diagnosing common diseases in developed nations as well.
Clinical laboratory test developers see a big opportunity in developing assays to detect Malaria. That is because an estimated 200 million cases of malaria are diagnosed annually, resulting in the death of about 100 million people each year.
Recently, two organizations released news about the specific testing devices they have developed to detect malaria. One group is at the University of Washington in Seattle, Washington. The other group is NanoMal, a biotechnology company located in the United Kingdom. (more…)
