Groups representing academic publishers are taking steps to combat paper mills that write the papers and then sell authorship spots
Clinical laboratory professionals rely on peer-reviewed research to keep up with the latest findings in pathology, laboratory medicine, and other medical fields. They should thus be interested in new efforts to combat the presence of “research paper mills,” defined as “profit oriented, unofficial, and potentially illegal organizations that produce and sell fraudulent manuscripts that seem to resemble genuine research,” according to the Committee on Publication Ethics (COPE), a non-profit organization representing stakeholders in academic publishing.
“They may also handle the administration of submitting the article to journals for review and sell authorship to researchers once the article is accepted for publication,” the COPE website states.
In a recent example of how paper mills impact scholarly research, multinational publishing company John Wiley and Sons (Wiley) announced in The Scholarly Kitchen last year that it had retracted more than 1,700 papers published in journals from the company’s Hindawi subsidiary, which specializes in open-access academic publishing.
“In Hindawi’s case, this is a direct result of sophisticated paper mill activity,” wrote Jay Flynn, Wiley’s Executive Vice President and General Manager, Research, in a Scholarly Kitchen guest post. “The extent to which our processes and systems were breached required an end-to-end review of every step in the peer review and publishing process.”
In addition, journal indexer Clarivate removed 19 Hindawi journals from its Web of Science list in March 2023, due to problems with their editorial quality, Retraction Watch reported.
Hindawi later shut down four of the journals, which had been “heavily compromised by paper mills,” according to a blog post from the publisher.
Wiley also announced at that time that it would temporarily pause Hindawi’s special issues publishing program due to compromised articles, according to a press release.
“We urgently need a collaborative, forward-looking and thoughtful approach to journal security to stop bad actors from further abusing the industry’s systems, journals, and the communities we serve,” wrote Jay Flynn (above), Wiley EVP and General Manager, Research and Learning, in an article he penned for The Scholarly Kitchen. “We’re committed to addressing the challenge presented by paper mills and academic fraud head on, and we invite our publishing peers, and the many organizations that work alongside us, to join us in this endeavor.” Clinical laboratory leaders understand the critical need for accurate medical research papers. (Photo copyright: The Scholarly Kitchen.)
Using AI to Detect Paper Mill Submissions
Wiley acquired Hindawi in 2021 in a deal valued at $298 million, according to a press release, but the subsidiary has since become a financial drain for the company.
The journals earn their revenue by charging fees to authors. But in fiscal year 2024, which began last fall, “Wiley expects $35-40 million in lost revenue from Hindawi as it works to turn around journals with issues and retract articles,” Retraction Watch reported, citing an earnings call.
Wiley also revealed that it would stop using the Hindawi brand name and bring the subsidiary’s remaining journals under its own umbrella by the middle of 2024.
The service will incorporate tools to detect signs that submissions originated from paper mills, including similarities with “known papermill hallmarks” and use of “tortured phrases” indicating that passages were translated by AI-based language models, according to a press release.
These tools include:
Papermill Similarity Detection: Checks for known papermill hallmarks and compares content against existing papermills papers.
Problematic Phrase Recognition: Flags unusual alternatives to established terms.
Unusual Publication Behavior Detection: Identifies irregular publishing patterns by paper authors.
Researcher Identity Verification: Helps detect potential bad actors.
Gen-AI Generated Content Detection: Identifies potential misuse of generative AI.
Journal Scope Checker: Analyzes the article’s relevance to the journal.
The company said that the new service will be available through Research Exchange, Wiley’s manuscript submission platform, as early as next year.
Other Efforts to Spot Paper Mill Submissions
Previously, STM announced the launch of the STM Integrity Hub, with a mission “to equip the scholarly communication community with data, intelligence, and technology to protect research integrity,” Program Director Joris van Rossum, PhD, told The Scholarly Kitchen.
In 2023, the group announced that the hub would integrate Papermill Alarm from Clear Skies, a paper mill detection tool launched in 2022 with a focus on cancer research. It uses a “traffic-light rating system for research papers,” according to a press release.
In an announcement about the launch of Wiley’s Papermill Detection service, Retraction Watch suggested that one key to addressing the problem would be to reduce incentives for authors to use paper mills. Those incentives boil down to the pressure placed on many scientists, clinicians, and students to publish manuscripts, according to the research report from STM and COPE.
In one common scenario, the report noted, a paper mill will submit a staff-written paper to multiple journals. If the paper is accepted, the company will list it on a website and offer authorship spaces for sale.
“If a published paper is challenged, the ‘author’ may sometimes back down and ask for the paper to be retracted because of data problems, or they may try to provide additional supporting information including a supporting letter from their institution which is also a fake,” the report noted.
All of this serves as a warning to pathologists and clinical laboratory professionals to carefully evaluate the sources of medical journals publishing studies that feature results on areas of healthcare and lab medicine research that are of interest.
Trifecta of forces at work that will affect the clinical laboratory and pathology industries have been described as a ‘perfect storm’ requiring lab and practice managers to be well informed
Digital pathology, artificial intelligence (AI) in healthcare, and the perfect storm of changing federal regulations, took centerstage at the 29th Executive War College on Diagnostics, Clinical Laboratory, and Pathology Management in New Orleans this week, where more than 1,000 clinical laboratory and pathology leaders convened over three days.
This was the largest number of people ever onsite for what has become the world’s largest event focused exclusively on lab management topics and solutions. Perhaps the highlight of the week was the federal Food and Drug Administration’s (FDA’s) announcement of its final rule on Laboratory Developed Tests (LDTs). Overall, the conference featured more than 120 speakers, many of them national thought leaders on the topic of clinical lab and pathology management. More than 65% of the audience onsite were executive level lab managers.
“The level of interest in the annual Executive War College is testimony to the ongoing need for dynamic, engaging, and highly relevant conference events,” said Robert Michel (above), Editor-in-Chief of Dark Daily and its sister publication The Dark Report, and founder of the Executive War College. “These in-person gatherings present great opportunities for clinical laboratory and pathology managers and leaders to network and speak with people they otherwise might not meet.” (Photo copyright: Dark Intelligence Group.)
Demonstrating Clinical Value
For those who missed the action onsite, the following is a synopsis of the highlights this week.
Lâle White, Executive Chair and CEO of XiFin, spoke about the future of clinical laboratory testing and the factors reshaping the industry. There are multiple dynamics impacting healthcare economics and outcomes—namely rising costs, decreasing reimbursements, and the move to a more consumer-focused healthcare. But it is up to labs, she said, to ensure their services are not simply viewed as a commodity.
“Laboratory diagnostics have the potential to change the economics of healthcare by really gaining efficiencies,” she noted. “And it’s up to labs to demonstrate clinical value by helping physicians manage two key diagnostic decision points—what tests to order, and what to do with the results.”
But even as labs find ways to increase the value offered to clinicians, there are other disruptive factors in play. Consumer-oriented tech companies such as Google, Apple, and Amazon are democratizing access to patient data in unforeseen ways, and Medicare Advantage plans are changing the way claims are processed and paid.
Clinical labs are fundamental components of the public health infrastructure. So, the CDC plans on focusing on delivering high-quality laboratory science, supported by reliable diagnostics and informatics for disease outbreaks and exposures, and engaging with public and private sector partners.
The history of MolDX and Z-Codes were the topics discussed by Gabriel Bien-Willner, MD, PhD, Chief Medical Officer for healthcare claims and transaction processing company Palmetto GBA. Molecular testing is highly complex, and the lack of well-defined billing codes and standardization makes it difficult to know if a given test is reasonable and necessary.
Z-Codes were established to clarify what molecular testing was performed—and why—prompting payers to require both Z-Codes and Current Procedural Terminology (CPT) codes when processing molecular test claims. Medicare’s MolDX program further streamlines the claims process by utilizing expertise in the molecular diagnostics space to help payers develop coverage policies and reimbursement for these tests.
FDA Final Rule on LDT Regulation
Timothy Stenzel, MD, PhD, CEO of Grey Haven Consulting and former director of the FDA’s Office of In Vitro Diagnostics reviewed the latest updates from the FDA’s Final Rule on LDT (laboratory developed test) regulation. Prior to the FDA releasing its final rule, some experts suggested that the new regulations could result in up to 90% of labs discontinuing their LDT programs, impacting innovation, and patient care.
However, the final rule on LDTs is very different from the original proposed rule which created controversy. The final rule actually lowers the regulatory burden to the point that some labs may not have to submit their LDTs at all. The FDA is reviewing dozens of multi-cancer detection assays, some of which have launched clinically as LDTs. The agency is likely to approve those that accurately detect cancers for which there is no formal screening program.
Stenzel explained the FDA’s plan to down-classify most in vitro diagnostic tests, changing them from Class III to Class II, and exempting more than 1,000 assays from FDA review. He also discussed the highlights of the Quality Management System Regulation (QMSR). Launched in January, the QMSR bought FDA requirements in line with ISO 13485, making compliance easier for medical device manufacturers and test developers working internationally.
Looming Perfect Storm of Regulatory Changes
To close out Day 1, Michel took to the stage again with a warning to clinical laboratories about the looming “Perfect Storm” trifecta—the final FDA ruling on LDTs, Z-Code requirements for genetic testing, and updates to CLIA ’92 that could result in patient data being considered a specimen.
Laboratory leaders must think strategically if their labs are to survive the fallout, because the financial stress felt by labs in recent years will only be exacerbated by macroeconomic trends such as:
Staff shortages,
Rising costs,
Decreasing and delayed reimbursements, and
Tightening supply chains.
Lab administrators looking for ways to remain profitable and prosperous should look beyond the transactional Clinical Lab 1.0 fee-for-service model and adopt Clinical Lab 2.0, which embraces HEDIS (Healthcare Effectiveness Data and Information Set) scores and STAR ratings to offer more value to Medicare Advantage and other payers.
Wednesday’s General Session agenda was packed with information about the rise of artificial intelligence, big data, and precision medicine in healthcare. Taking centerstage on the program’s final day was Michael Simpson, President and CEO of Clinisys. Simpson gave a global perspective on healthcare data as the new driver of innovation in diagnostics and patient care.
“The timing of EWC with the release of this policy couldn’t be better,” CEO and founder of Momentum ConsultingValerie Palmieri told Dark Daily in an interview at Monday night’s opening reception. “It’s a great conference to not only catch up with colleagues but really hear and have those difficult discussions about where we are today, where we’re going, and where we need to be.”
Final LDT rule ‘radically’ different than draft
Tim Stenzel, MD, PhD, former director of the FDA’s Office of In Vitro Diagnostics called the finalized rule “radically different” from the proposed rule. In some ways it is less complex: “The bar is lower,” he said, noting that he was voicing his personal views and not those of the federal agency. “I was convinced that there would be lawsuits, but I’m now not sure if that’s advisable.”
Still, laboratory teams will have to parse the more than 500-page document to determine how the final rule relates to their specific circumstances. After that, it won’t be as challenging, Stenzel said.
His advice: First, read the rule. Second, reach out to FDA for help—he’s sure, he said, that the office is geared up to respond to a “ton of questions” about the implications for individual labs and are standing by to answer emails from labs. And, he added in a discussion session, emailing the agency is free.
The final rule will be in force 60 days after it’s published. Stenzel provided a timeline for some of the milestones:
1 Year: Comply with MD(AE) reporting and reporting of corrections and removals.
2 Years: Comply with labeling, registration and listing, and investigational use requirements.
3 Years: QS records and, in some cases, design controls and purchasing controls.
3.5 Years: Comply with high risk (class III) premarket review requirements.
4 Years: Comply with moderate and low-risk premarket review requirements.
Executive Chair and CEO of XiFin, Inc.Lâle White welcomed the audience with a morning keynote entitled “Big Changes in Healthcare” on new regulations and diagnostics players poised to reshape lab testing.
The diagnostics business is in constant flux, she noted, from payer requirements to greater regulatory and compliance burdens on labs. Other factors include the growing senior population and increasingly complex health conditions, rising costs throughout the healthcare ecosystem, falling funding and reimbursement, and staffing shortages.
As for the economic challenges, consumers are increasingly making decisions based on cost, convenience and quality. The population is shifting to Medicare advantage, which is more cost effective. But changes to the star ratings system will mean lower pay for payer organizations. Those companies will, in turn, mitigate their losses by making changes to pre-authorizations and tightening denials, even for clean claims.
Still, White said, more money isn’t the answer.
White urged the audience to use technology, including artificial intelligence and advances in genetic testing, to manage these and other industry changes.
“We need to optimize the tests we order,” she said. “And if we did that, lab diagnostics really has the potential to change the economics of health and improve outcomes.”
The FDA, Stenzel added, is “very interested” in stimulating innovation, building on the laboratory industry’s success in responding swiftly to the COVID pandemic and outbreaks of Monkey Pox, for example.
He shared lessons learned from recent public health emergencies, talked about CDC’s efforts to engage with clinical labs to improve future public health readiness and response and provided an overview of the CDC’s first laboratory-specific center.
“Laboratories are fundamental to public health,” he said. The industry is on the “front lines” when it comes to identifying threats, responding to them, and preparing for future responses.
Robert Michel, Editor-in-Chief of The Dark Report wrapped up the day’s regulatory discussions with a general session on the “regulatory trifecta” that includes the LDT final rule, CLIA regulations, and private payers’ policies for genetic claims.
In a follow-up story, investigative news team in Boston sends a reporter’s cheek swab sample to the same pet DNA testing lab: report states the reporter is part Malamute, Shar Pei, and Labrador Retriever
One pet DNA testing company returned results from human cheek swabs showing two different people were in fact part dog. The resulting local reporting calls into question the accuracy of DNA testing of our beloved furry friends and may impact the trust people have in clinical laboratory genetic testing as well.
Pet DNA analysis is nearly as popular as human DNA analysis. The market is expected to exceed $700 million by the end of the decade, according to Zion Market Research. But are customers getting their money’s worth? One CBS news station in Boston decided to find out.
Last year, the WBZ I-Team, the investigative part of a CBS News station in Boston, looked into the accuracy of pet DNA testing. They reported on a pet owner who questioned the DNA test results she received for her German Shepard. The report indicated that her dog had DNA from more than 10 breeds, besides German Shepard.
During their research, the WBZ investigative reporters learned that pet owners order these tests to reveal what one pet DNA testing company described as understanding “your dog’s unique appearance, behavior, and health.”
“So, the WBZ-TV I-Team came with more tests from different companies to compare. All came back with some German Shepherd, but the percentages ranged from 65% to just 29%. Aside from that, the three companies showed a puzzling hodgepodge of other breeds. One included Great Pyrenees, another came back with Siberian Husky, another listed Korean Jindo, and the list goes on,” WBZ News reported.
The owner of the German Shepard then sent two swab samples from her own cheeks to one of the pet DNA testing companies. The test results indicated that she was 40% Border Collie, 32% Cane Corso, and 28% Bulldog.
The company that performed that DNA testing—DNA My Dog—insisted to the WBZ I-Team that one of the pet owner’s cheek samples contained dog DNA, WBZ News reported.
“The second sample did in fact yield canine DNA. … The results provided would not be possible on a human sample,” Jessica Barnett, Director of Service Operations, DNA My Dog, told WBZ News.
This must have come as a shock to the pet owner, who is probably sure she is not part dog.
“I think that is a red flag for sure,” Lisa Moses, VMD (above), a veterinarian and bioethicist with Harvard Medical School, told WBZ News. “A company should know if they’ve in any basic way analyzed a dog’s DNA, that that is not a dog,” she said. One wonders what might happen if a dog’s DNA was secretly sent to a clinical laboratory performing human genetic testing. What might the results be? (Photo copyright: Harvard Medical School.)
Two Times is the Charm
To continue its investigation into this odd occurrence, the WBZ I-Team decided to repeat the test this year. They sent a cheek saliva sample from one of their own reporters to three different dog DNA testing companies.
According to the I-Team report, one company, Orivet, said the sample “failed to provide the data necessary to perform breed ID analysis. Another company, Wisdom Panel stated the sample “didn’t provide enough DNA to produce a reliable result.”
However, DNA My Dog once again reported that the human sample belonged to a canine. This time the company’s test reported that the DNA sample was 40% Alaskan Malamute, 35% Shar Pei, and 25% Labrador Retriever.
DNA My Dog did not respond to WBZ I-Team’s attempt to contact them for a comment, WBZ News reported.
Wild West of DNA Testing
“I personally do have concerns about the fact that, from a consumer standpoint, you don’t always know what you’re getting when you work with those companies,” said geneticist Elinor Karlsson, PhD, Director of the Vertebrate Genomics Group at the Broad Institute of MIT and Harvard, told WBZ News. “There’s not a lot of rules in this space.”
Karlsson is also founder and Chief Scientist at Darwin’s Ark, a nonprofit organization that combines dog genetics and behavior to advance the understanding of complex canine diseases. People participating in the initiative contribute data about their dogs to an open source database, which is then shared with researchers around the globe. To date, more than 44,000 dogs have been registered with the project.
She hopes that reports like the one from the WBZ I-Team will not dissuade interest in pet genetics, as the science does have significant value when performed correctly.
“We might be able to figure out which dogs are at risk of getting cancer, and screen them more often and be able to diagnose it earlier,” Karlsson said. “We might be able to develop new treatments for that cancer.”
“There isn’t necessarily a gold standard answer for what your dog is,” veterinarian and bioethicist Lisa Moses, VMD, co-director of the Capstone Program for the Master of Science in Bioethics Program at Harvard Medical School, told WBZ News. “A breed is something that we’ve decided, which is based upon essentially the way a dog looks. But that doesn’t necessarily mean that we’re going to know what their genes look like.”
DNA My Dog Awarded ‘Best Budget Dog DNA Test’
In February, US News and World Report published an article rating the best dog DNA tests of 2024. The magazine ranked the DNA My Dog Essential Breed ID Test as the “best budget dog DNA test on the market.” The test sells for $79.99. According to the company’s website, a simple cheek swab yields:
A complete breed breakdown,
Genetic health concerns,
Unique personality traits, and
Bonding tips for dogs and their owners.
“I worry about people making medical decisions … based on one of these tests,” Moses told WBZ News, which added that, “She and some of her colleagues have called on lawmakers to set standards and regulations for pet DNA labs, and to require them to share their databases with each other, for more consistent results.”
The investigation into pet DNA testing by the television news reporters in Boston is a reminder to clinical lab managers and pathologists that DNA testing can be problematic in many ways. Also, when consumers read news stories like this one about inaccurate canine DNA testing, it can cause them to question the accuracy of other types of DNA testing.
Speedy DNA sequencing and on-the-spot digital imaging may change the future of anatomic pathology procedures during surgery
Researchers at the Center for Molecular Medicine (CMM) at UMC Utrecht, a leading international university medical center in the Netherlands, have paired artificial intelligence (AI) and machine learning with DNA sequencing to develop a diagnostic tool cancer surgeons can use during surgeries to determine in minutes—while the patient is still on the operating table—whether they have fully removed all the cancerous tissue.
The method, “involves a computer scanning segments of a tumor’s DNA and alighting on certain chemical modifications that can yield a detailed diagnosis of the type and even subtype of the brain tumor,” according to The New York Times, which added, “That diagnosis, generated during the early stages of an hours-long surgery, can help surgeons decide how aggressively to operate, … In the future, the method may also help steer doctors toward treatments tailored for a specific subtype of tumor.”
This technology has the potential to reduce the need for frozen sections, should additional development and studies confirm that it accurately and reliably shows surgeons that all cancerous cells were fully removed. Many anatomic pathologists would welcome such a development because of the time pressure and stress associated with this procedure. Pathologists know that the patient is still in surgery and the surgeons are waiting for the results of the frozen section. Most pathologists would consider fewer frozen sections—with better patient outcomes—to be an improvement in patient care.
“It’s imperative that the tumor subtype is known at the time of surgery,” Jeroen de Ridder, PhD (above), associate professor in the Center for Molecular Medicine at UMC Utrecht and one of the study leaders, told The New York Times. “What we have now uniquely enabled is to allow this very fine-grained, robust, detailed diagnosis to be performed already during the surgery. It can figure out itself what it’s looking at and make a robust classification,” he added. How this discovery affects the role of anatomic pathologists and pathology laboratories during cancer surgeries remains to be seen. (Photo copyright: UMC Utrecht.)
Rapid DNA Sequencing Impacts Brain Tumor Surgeries
The UMC Utrecht scientists employed Oxford Nanopore’s “real-time DNA sequencing technology to address the challenges posed by central nervous system (CNS) tumors, one of the most lethal type of tumor, especially among children,” according to an Oxford Nanopore news release.
The researchers called their new machine learning AI application the “Sturgeon.”
According to The New York Times, “The new method uses a faster genetic sequencing technique and applies it only to a small slice of the cellular genome, allowing it to return results before a surgeon has started operating on the edges of a tumor.”
Jeroen de Ridder, PhD, an associate professor in the Center for Molecular Medicine at UMC Utrecht, told The New York Times that Sturgeon is “powerful enough to deliver a diagnosis with sparse genetic data, akin to someone recognizing an image based on only 1% of its pixels, and from an unknown portion of the image.” Ridder is also a principal investigator at the Oncode Institute, an independent research center in the Netherlands.
The researchers tested Sturgeon during 25 live brain surgeries and compared the results to an anatomic pathologist’s standard method of microscope tissue examination. “The new approach delivered 18 correct diagnoses and failed to reach the needed confidence threshold in the other seven cases. It turned around its diagnoses in less than 90 minutes, the study reported—short enough for it to inform decisions during an operation,” The New York Times reported.
But there were issues. Where the minute samples contain healthy brain tissue, identifying an adequate number of tumor markers could become problematic. Under those conditions, surgeons can ask an anatomic pathologist to “flag the [tissue samples] with the most tumor for sequencing, said PhD candidate Marc Pagès-Gallego, a bioinformatician at UMC Utrecht and a co-author of the study,” The New York Times noted.
“Implementation itself is less straightforward than often suggested,” Sebastian Brandner, MD, a professor of neuropathology at University College London, told The Times. “Sequencing and classifying tumor cells often still required significant expertise in bioinformatics as well as workers who are able to run, troubleshoot, and repair the technology,” he added.
“Brain tumors are also the most well-suited to being classified by the chemical modifications that the new method analyzes; not all cancers can be diagnosed that way,” The Times pointed out.
Thus, the research continues. The new method is being applied to other surgical samples as well. The study authors said other facilities are utilizing the method on their own surgical tissue samples, “suggesting that it can work in other people’s hands.” But more work is needed, The Times reported.
UMC Utrecht Researchers Receive Hanarth Grant
To expand their research into the Sturgeon’s capabilities, the UMC Utrecht research team recently received funds from the Hanarth Fonds, which was founded in 2018 to “promote and enhance the use of artificial intelligence and machine learning to improve the diagnosis, treatment, and outcome of patients with cancer,” according to the organization’s website.
The researchers will investigate ways the Sturgeon AI algorithm can be used to identify tumors of the central nervous system during surgery, a UMC Utrecht news release states. These type of tumors, according to the researchers, are difficult to examine without surgery.
“This poses a challenge for neurosurgeons. They have to operate on a tumor without knowing what type of tumor it is. As a result, there is a chance that the patient will need another operation,” said de Ridder in the news release.
The Sturgeon application solves this problem. It identifies the “exact type of tumor during surgery. This allows the appropriate surgical strategy to be applied immediately,” the news release notes.
The Hanarth funds will enable Jeroen and his team to develop a variant of the Sturgeon that uses “cerebrospinal fluid instead of (part of) the tumor. This will allow the type of tumor to be determined already before surgery. The main challenge is that cerebrospinal fluid contains a mixture of tumor and normal DNA. AI models will be trained to take this into account.”
The UMC Utrecht scientists’ breakthrough is another example of how organizations and research groups are working to shorten time to answer, compared to standard anatomic pathology methods. They are combining developing technologies in ways that achieve these goals.
