Large Dutch Survey Shines Light on Fraud and Questionable Research Practices in Medical Studies Published in Scientific Journals
About half of nearly 7,000 respondents admitted to sloppy practices, which suggests that pathologists and clinical lab professionals may want to be skeptical about the findings of many papers published in medical journals
It may surprise pathologists and medical laboratory professionals to learn that as many as 10% of surveyed authors of published scientific papers admitted to regularly falsifying or fabricating data! This was one finding in a study conducted by researchers to determine the quality and accuracy of scientific papers that are published in journals.
The National Survey on Research Integrity (NSRI), an organization based in The Netherlands, conducted the research.
In its coverage of the NSRI’s findings, Nature wrote, “Between October and December 2020, study authors contacted nearly 64,000 researchers at 22 universities in the Netherlands, 6,813 of whom completed the survey.”
According to Nature, “An estimated 8% of scientists who participated in an anonymous survey of research practices at Dutch universities confessed to falsifying and/or fabricating data at least once between 2017 and 2020. More than 10% of medical and life-science researchers admitted to committing this type of fraud, the survey found.”
Gowri Gopalakrishna, PhD, an epidemiologist and public health policy scientist with the Amsterdam University Medical Center (AUMC) who helped lead the NSRI study “thinks that the percentage of researchers who confessed to falsifying or fabricating data could be an underestimate,” Nature reported.
Thousands of Researchers Admit to ‘Questionable Research Practices’
Conducted online, the NSRI received responses from nearly 7,000 academics and researchers across a wide range of disciplines. About half admitted to engaging in “questionable research practices” (QRPs), 4.3% admitted to fabrication of data, and 4.2% admitted to falsification of data.
The NSRI presented its survey results in two preprints:
- “Prevalence of Questionable Research Practices, Research Misconduct and Their Potential Explanatory Factors: A Survey Among Academic Researchers in The Netherlands,” and
- “Prevalence of Responsible Research Practices and Their Potential Explanatory Factors: A Survey Among Academic Researchers in The Netherlands.”
The NSRI study authors wrote that QRPs included “subtle trespasses such as not submitting valid negative results for publication, not reporting flaws in study design or execution, selective citation to enhance one’s own findings and so forth.”
An article in Science, titled, “Landmark Research Integrity Survey Finds Questionable Practices Are Surprisingly Common,” notes that the NSRI survey organizers took steps to ensure anonymity of respondents. “So, we have good reason to believe that our outcome is closer to reality than that of previous studies,” Gopalakrishna said.
Publish or Perish
Survey organizers originally sought responses from more than 60,000 researchers, but “many institutions refused to cooperate for fear of negative publicity,” Science reported.
The authors cited “publication pressure,” otherwise known as the “publish or perish” reward system, as the top factor driving questionable research practices. Respondents were “less likely” to engage in questionable research practices, data falsification, or fabrication if they subscribed to scientific norms and perceived a high likelihood of being detected.
According the NSRI findings, within academic ranks, PhD candidates and junior researchers were “most likely” to engage in QRPs, as well as males and people involved in empirical research.
Epidemiologist Gowri Gopalakrishna, PhD (above), a post-doctoral researcher and the project secretary for the NSRI, told Science that advocates for research integrity should pay more attention to “sloppy research practices” as opposed to outright misconduct. “We need to have a positive environment where mistakes can happen, and where there is more focus on responsible conduct, slower science, and taking time for good quality research,” she said. Clinical laboratory professionals would likely agree with Gopalakrishna’s assessment. (Photo copyright: University of Amsterdam Medical Center.)
Tracking Retractions
Retraction Watch, a blog founded in 2010 by medical journalists Ivan Oransky, MD, and Adam Marcus, offers a day-to-day barometer on research integrity. As the name indicates, the blog tracks research studies that have been retracted due to scientific misconduct or other reasons. In 2018, the bloggers launched a searchable database with more than 18,000 papers or conference abstracts that had been retracted.
An analysis by Science, titled, “What a Massive Database of Retracted Papers Reveals about Science Publishing’s ‘Death Penalty’,” looked at about 10,500 retracted journal articles in the database. It found that about half of those retractions involved scientific misconduct, including fabrication, falsification, and plagiarism. Nearly 40% were withdrawn “because of errors, problems with reproducibility, and other issues,” the analysis noted.
The data also indicates that a relatively small number of authors—about 500—accounted for about 25% of the retractions in journals.
In addition to the blog, Oransky and Marcus penned a column for STAT, titled, “The Watchdogs” in which they called attention to scientific misconduct and suggested solutions. Some solutions included:
- Use of scientific integrity czars by journals,
- Greater sharing of raw data,
- More transparency in the peer review process,
- More transparency involving conflicts of interest, and
- Use of statistical analysis to identify data fabrication in advance of publication.
Tips From a Media Watchdog
Gary Schwitzer, founder and Publisher of HealthNewsReview.org, a media watchdog website, offers additional insights. Schwitzer is a longtime medical journalist who also taught health journalism and media ethics at the University of Minnesota.
“Not all studies are the same and no study should necessarily be equated with the truth,” Schwitzer said in a video embedded on the website. People “often lose sight of the fact that journals were meant to be forums for discussions among scientists, not a source of daily news.”
The website includes the following Tips for Analyzing Studies, Medical Evidence, and Health Care Claims:
- Beware of “under-powered” clinical studies with small numbers of participants.
- In larger studies, beware of claims based on subgroup results.
- Be skeptical about health claims based on animal or lab studies.
- Beware of “cause and effect” interpretations in observational studies.
- Beware of health claims based on early-phase clinical trials.
The website also includes a tip sheet for evaluating claims about medical tests.
The NSRI’s research is the latest in a long line of studies into so-called “scientific research,” some of which found “cooked” data and outright fraud. This suggests that pathologists and clinical laboratory professionals should follow the saying caveat emptor (“Let the buyer beware”) when absorbing research published in scientific journals or presented at meetings.
—Stephen Beale
Related Information
Largest Study Ever on Research Integrity Launches, Aimed at All Researchers in the Netherlands
A Huge Database of Scientific Retractions Is Live. That’s Great for Science
The Real Plague Affecting Science? It Isn’t Fraud
What a Massive Database of Retracted Papers Reveals about Science Publishing’s ‘Death Penalty’
Why Our Peer Review System Is a Toothless Watchdog
Science Isn’t Broken. It’s Just a Hell of a Lot Harder Than We Give It Credit For
Q/A with Dr. Ivan Oransky from Retraction Watch
The Science of This Pandemic Is Moving at Dangerous Speeds
Op-ed: Covering Science at Dangerous Speeds
The Watchdogs: We’ll Sniff Out Scientific Misconduct
Why Most Published Research Findings Are False
Tips for Analyzing Studies, Medical Evidence and Health Care Claims
There’s a Way to Spot Data Fakery. All Journals Should Be Using It
Does Science Self-Correct? What We’ve Learned at Retraction Watch
Retractions, Post-Publication Peer Review and Fraud
Ivan Oransky Co-Founder of Retraction Watch Discusses Scientific Research Integrity
