About half of nearly 7,000 respondents admitted to sloppy practices, which suggests that pathologists and clinical lab professionals may want to be skeptical about the findings of many papers published in medical journals
It may surprise pathologists and medical laboratory professionals to learn that as many as 10% of surveyed authors of published scientific papers admitted to regularly falsifying or fabricating data! This was one finding in a study conducted by researchers to determine the quality and accuracy of scientific papers that are published in journals.
In its coverage of the NSRI’s findings, Nature wrote, “Between October and December 2020, study authors contacted nearly 64,000 researchers at 22 universities in the Netherlands, 6,813 of whom completed the survey.”
According to Nature, “An estimated 8% of scientists who participated in an anonymous survey of research practices at Dutch universities confessed to falsifying and/or fabricating data at least once between 2017 and 2020. More than 10% of medical and life-science researchers admitted to committing this type of fraud, the survey found.”
Gowri Gopalakrishna, PhD, an epidemiologist and public health policy scientist with the Amsterdam University Medical Center (AUMC) who helped lead the NSRI study “thinks that the percentage of researchers who confessed to falsifying or fabricating data could be an underestimate,” Nature reported.
Thousands of Researchers Admit to ‘Questionable Research Practices’
Conducted online, the NSRI received responses from nearly 7,000 academics and researchers across a wide range of disciplines. About half admitted to engaging in “questionable research practices” (QRPs), 4.3% admitted to fabrication of data, and 4.2% admitted to falsification of data.
The NSRI presented its survey results in two preprints:
The NSRI study authors wrote that QRPs included “subtle trespasses such as not submitting valid negative results for publication, not reporting flaws in study design or execution, selective citation to enhance one’s own findings and so forth.”
Survey organizers originally sought responses from more than 60,000 researchers, but “many institutions refused to cooperate for fear of negative publicity,” Science reported.
The authors cited “publication pressure,” otherwise known as the “publish or perish” reward system, as the top factor driving questionable research practices. Respondents were “less likely” to engage in questionable research practices, data falsification, or fabrication if they subscribed to scientific norms and perceived a high likelihood of being detected.
According the NSRI findings, within academic ranks, PhD candidates and junior researchers were “most likely” to engage in QRPs, as well as males and people involved in empirical research.
Epidemiologist Gowri Gopalakrishna, PhD (above), a post-doctoral researcher and the project secretary for the NSRI, told Science that advocates for research integrity should pay more attention to “sloppy research practices” as opposed to outright misconduct. “We need to have a positive environment where mistakes can happen, and where there is more focus on responsible conduct, slower science, and taking time for good quality research,” she said. Clinical laboratory professionals would likely agree with Gopalakrishna’s assessment. (Photo copyright: University of Amsterdam Medical Center.)
Tracking Retractions
Retraction Watch, a blog founded in 2010 by medical journalists Ivan Oransky, MD, and Adam Marcus, offers a day-to-day barometer on research integrity. As the name indicates, the blog tracks research studies that have been retracted due to scientific misconduct or other reasons. In 2018, the bloggers launched a searchable database with more than 18,000 papers or conference abstracts that had been retracted.
An analysis by Science, titled, “What a Massive Database of Retracted Papers Reveals about Science Publishing’s ‘Death Penalty’,” looked at about 10,500 retracted journal articles in the database. It found that about half of those retractions involved scientific misconduct, including fabrication, falsification, and plagiarism. Nearly 40% were withdrawn “because of errors, problems with reproducibility, and other issues,” the analysis noted.
The data also indicates that a relatively small number of authors—about 500—accounted for about 25% of the retractions in journals.
In addition to the blog, Oransky and Marcus penned a column for STAT, titled, “The Watchdogs” in which they called attention to scientific misconduct and suggested solutions. Some solutions included:
Use of statistical analysis to identify data fabrication in advance of publication.
Tips From a Media Watchdog
Gary Schwitzer, founder and Publisher of HealthNewsReview.org, a media watchdog website, offers additional insights. Schwitzer is a longtime medical journalist who also taught health journalism and media ethics at the University of Minnesota.
“Not all studies are the same and no study should necessarily be equated with the truth,” Schwitzer said in a video embedded on the website. People “often lose sight of the fact that journals were meant to be forums for discussions among scientists, not a source of daily news.”
The website also includes a tip sheet for evaluating claims about medical tests.
The NSRI’s research is the latest in a long line of studies into so-called “scientific research,” some of which found “cooked” data and outright fraud. This suggests that pathologists and clinical laboratory professionals should follow the saying caveat emptor (“Let the buyer beware”) when absorbing research published in scientific journals or presented at meetings.
Radboud University researchers fear oncology, molecular biology, pharmacology, and other cell-centric medical research efforts are at risk due to verification that at least 30,000 studies published in 33,000 scientific journals included data derived from misidentified or contaminated cell lines
Many research findings that underpin the science behind various diagnostic technologies used regularly by clinical laboratories and anatomic pathology groups may not be valid. This is because a large number of published studies may have used misidentified or contaminated cell lines.
Biomedical scientists have known for a long time that many research papers exist containing reports on the wrong cells due to cell line misidentification. And yet, few studies have measured the true scope of the problem. Until now. Researchers at Radboud University in the Netherlands have determined that this problem may have influenced the findings of thousands of published research studies and upon which many other research studies were conducted.
Because clinical laboratories and anatomic pathology groups use assays and diagnostic tests that are developed as a result of these research studies, identifying how many published papers have inaccurate findings that cannot be duplicated would affect how and when it is appropriate for physicians to order certain medical laboratory tests and rely on the results.
Additionally, cancer research is based on cell line studies as well. Thus, it may prove necessary to restudy existing published findings and revise them as appropriate. In turn, these new findings might change how and when some cancer tests are ordered and the results interpreted.
“We considered a reference to this original article as a good proxy for the usage of a cell line,” the researchers noted in their study published in the journal PLOS ONE. “Since typically the original papers are focused on reporting the establishment of the cell line only.”
They focused on misidentified cell lines that were caused by HeLa cells, also known as “immortalized cells.” HeLa cells have been used in scientific research for decades. They were the first mass-producible cells that could be used in vitro, making them highly desirable for biomedical research.
However, the process of creating immortalized cells involves mutation, during which contamination can be introduced by other cells. Immortalized cells can be identified as one type of cell when in fact they are actually another type of cell.
Research scientists have been aware of this problem for about as long as immortalized cells have been in use. They attempt to take it into account when completing their analyses, though not always successfully.
The Radboud researchers found 32,655 records of primary literature based on contaminated cell lines. They then cross-referenced the ICLAC Register of Misidentified Cell Lines with a range of databases to determine if articles were available for each of the 451 cell lines listed on Table One of the ICLAC Register.
With this information, they further researched published articles in the Web of Science database using cell line identifiers. They noted both primary literature and any citation report entries for each cell line.
The researchers noted in their published study, “As we only searched for cell lines known to be misidentified, this constitutes a conservative estimate of the scale of contamination in the primary literature. Moreover, to avoid false positives, we excluded several cell lines, such as the ones with non-unique identifiers or the cell lines for which verified stock is still in circulation.”
Their estimate for secondary contaminated literature based off primary articles is larger still. “In total, we can conservatively estimate the citations to the primary contaminated primary literature at over 500,000, excluding self-citations,” the authors noted in their PLOS ONE article. “Thereby leaving traces in a substantial share of the biomedical literature.” They concluded, “… the amount of research potentially building on false grounds remains worrisome.”
Impact of Contaminated Cell Lines on Research, Clinical Laboratory Communities
Many of the assays and diagnostic tests performed by clinical laboratories and pathology groups were developed using cell line research. Should further scrutiny into the ability to duplicate and verify study findings fail to produce positive outcomes, it might call into question the validity and appropriate use of these tests.
For the research community, these findings represent yet another call to promote accountability and define standards for verifying authenticity of cell lines to further strengthen research findings.
The Radboud researchers ranked the number of contaminated articles they discovered by research area. Top affected areas include:
Oncology
Molecular Biology
Pharmacology
Cell Biology
Immunology
The distribution of contaminated primary literature over the research areas as defined by Web of Science. Only the 25 most affected research areas are included. (Graphic copyright: PLOS ONE.)
Addressing the Problem of Cell Line Contamination and Misidentification
Adapting the ever-growing body of published medical literature to reflect the known misidentifications, as well as the possibility of invalid results, will be a major undertaking. Ultimately, resolving this problem could require changes to practices and procedures currently used by research facilities and medical laboratories.
While the cost to authenticate cell lines adds to the bottom line of research projects, the money spent on research that becomes invalidated by misidentified cell lines is far greater.
In a 2015 Retraction Watch article, Leonard P. Freeman, PhD, President, Global Biological Standards Institute, notes, “An NIH RePORT search identified 9,000 active projects using cell lines, totaling $3.7 billion. Required use of authentication techniques would affect over $900 million in research dollars annually.”
Additionally, failure to adapt authentication as a part of standard operations brings other consequences. “A 2004 survey reported that just one-third of laboratories authenticate their cell lines,” Freeman noted. “10 years later, a Sigma-Aldrich survey found that only 37% of respondents ‘validate the purity and identity before first use’ of cell lines. Understanding the existing barriers that prevent implementation of universal cell authentication is central to changing this sad state of affairs.”
Mixed Recommendations for Fixing Inaccurate Published Studies
Of course, none of this will change the vast body of archived literature that might contain errors due to misidentification. Recommendations for addressing this aspect of the problem vary. The Radboud study authors suggest posting notes on any previously published articles stating that misidentified cell lines were used.
However, in a STAT article, Ivan Oransky, MD, and Adam Marcus, Managing Editor, Gastroenterology and Endoscopy News, co-founders of Retraction Watch, recommend more severe measures. “When we polled readers of Retraction Watch last December about the issue, 55% said journals should correct papers known to describe contaminated or misidentified cell lines, and more than 40% said retraction was the right choice.”
Thanks to the Radboud study, as cell lines continue to power the innovations of modern biomedical research, concerns will surely increase surrounding cell-line authentication and research findings. For pathology groups and medical laboratories, staying abreast of these developments will work to ensure data validity and reduce reputation and liability concerns.
