Retractions and controversies over coronavirus research show that the process of science is working as it should
- Written by Mark R. O'Brian, Professor and Chair of Biochemistry, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, The State University of New York
Several high-profile papers on COVID-19 research have come under fire from people in the scientific community in recent weeks. Two articles addressing the safety of certain drugs when taken by COVID-19 patients were retracted[1], and researchers are calling for the retraction of a third paper that evaluated behaviors that mitigate coronavirus transmission[2].
Some people are viewing the retractions as an indictment of the scientific process[3]. Certainly, the overturning of these papers is bad news, and there is plenty of blame to go around.
But despite these short-term setbacks, the scrutiny and subsequent correction of the papers actually show that science is working. Reporting of the pandemic is allowing people to see, many for the first time, the messy business of scientific progress.
Scientific community quickly responds to flawed research
In May, two papers were published on the safety of certain drugs for COVID-19 patients. The first, published in the New England Journal of Medicine, claimed that a particular heart medication was in fact safe for COVID-19 patients[4], despite previous concerns. The second, published in The Lancet, claimed that the antimalarial drug hydroxychloroquine increased the risk of death[5] when used to treat COVID-19.
The Lancet paper caused the World Health Organization to briefly halt studies investigating hydroxychloroquine[6] for COVID-19 treatment.
The paper published in The Lancet claimed that hydroxychloroquine increased risk of death in COVID-19 patients, but was retracted when other scientists discovered the data used for the study was unreliable.
The Lancet/Mandeep R Mehra, Sapan S Desai, Frank Ruschitzka, Amit N Patel[7]
Within days, over 200 scientists signed an open letter[8] highly critical of the paper, noting that some of the findings were simply implausible. The database provided by the tiny company Surgisphere – whose website is no longer accessible – was unavailable during peer review of the paper or to scientists and the public afterwards, preventing anyone from evaluating the data. Finally, the letter suggested that it was unlikely this company was able to obtain the hospital records alleged to be in the database when no one else had access to this information.
[The Conversation’s science, health and technology editors pick their favorite stories. Weekly on Wednesdays[9].]
By early June, both the Lancet[10] and New England Journal of Medicine[11] articles were retracted, citing concerns about the integrity of the database the researchers used in the studies. A retraction is the withdrawal of a published paper because the data underlying the major conclusions of the work are found to be seriously flawed. These flaws are sometimes, but not always, due to intentional scientific misconduct.
The urgency to find solutions to the COVID-19 pandemic certainly contributed to the publication of sloppy and possibly fraudulent science[12]. The quality control measures that minimize the publication of bad science failed miserably in these cases.
Imperfect and iterative
The retraction of the hydroxychloroquine paper in particular drew immediate attention not only because it placed science in a bad light, but also because President Trump had touted the drug[13] as an effective treatment for COVID-19 despite the lack of strong evidence.
Responses in the media were harsh. The New York Times declared that “The pandemic claims new victims: prestigious medical journals[14].” The Wall Street Journal accused the Lancet of “politicized science[15],” and the Los Angeles Times claimed that the retracted papers “contaminated global coronavirus research[16].”
These headlines may have merit, but perspective is also needed. Retractions are rare[17] – only about 0.04% of published papers are withdrawn – but scrutiny, update and correction are common. It is how science is supposed to work, and it is happening in all areas of research relating to SARS-CoV-2.
Doctors have learned that the disease targets numerous organs[18], not just the lungs as was initially thought. Scientists are still working on understanding whether COVID-19 patients develop immunity[19] to the disease. And to close the case on hydroxychloroquine, three new large studies[20] published after the Lancet retraction indicate that the malaria drug is indeed ineffective in preventing or treating COVID-19.
Since the beginning of scientific publishing, peer review has helped weed out bad science, but public discourse between researchers has easily played as big a role.
Public Domain[21]
Science is self-correcting
Before a paper is published, it undergoes peer review by experts in the field who recommend to the journal editor whether it should be accepted for publication, rejected or reconsidered after modification. The reputation of the journal is dependent on high-quality peer review, and once a paper is published, it is in the public domain, where it can then be evaluated and judged by other scientists.
The publication of the Lancet and the New England Journal of Medicine papers failed at the level of peer review. But scrutiny by the scientific community – likely spurred on by the public spotlight on coronavirus research – caught the mistakes in record time.
The hydroxychloroquine article published in The Lancet was retracted only 13 days after it was published. By contrast, it took 12 years for the Lancet to retract the fraudulent article that incorrectly claimed vaccinations cause autism[22].
It is not yet known whether these papers involved deliberate scientific misconduct, but mistakes and corrections are common, even for top scientists. For example, Linus Pauling[23], who won the Nobel Prize for discovering the structure of proteins, later published an incorrect structure of DNA[24]. It was subsequently corrected by Watson and Crick[25]. Mistakes and corrections are a hallmark of progress, not foul play.
Importantly, these errors were exposed by other scientists. They were not uncovered by some policing body or watchdog group.
This back-and-forth between academics is foundational to science. There is no reason to believe that scientists are more virtuous than anyone else. Rather, the mundane human traits of curiosity, competitiveness, self-interest and reputation come into play before and after publication are what allow science to regulate itself. A model based on robust evidence emerges while the weaker one is abandoned.
Living with uncertainty
From high school classes and textbooks, science seems like a body of well-known facts and principles that are straightforward and incontrovertible. These sources view science in hindsight and often make discoveries seem inevitable, even dull.
In reality, scientists learn as they go. Uncertainty is inherent to the path of discovery, and success is not guaranteed. Only 14% of drugs and therapies[26] that go through human clinical trials ultimately win FDA approval, with less than a 4% success rate for cancer drugs.
The process of science generally takes place below the radar of public awareness, and so this uncertainty is not generally in view. However, Americans are paying close attention[27] to the COVID-19 pandemic, and many are, for the first time, seeing the sausage as it is being made.
Although the recent retractions may be unappetizing, medical science has been very successful over the long run. Smallpox has been eradicated, infections are treated with antibiotics rather than amputation and pain management during surgery has advanced well beyond biting on a stick.
The system is by no means perfect, but it is pretty darned good.
References
- ^ retracted (www.nytimes.com)
- ^ mitigate coronavirus transmission (metrics.stanford.edu)
- ^ indictment of the scientific process (www.wsj.com)
- ^ was in fact safe for COVID-19 patients (doi.org)
- ^ hydroxychloroquine increased the risk of death (doi.org)
- ^ halt studies investigating hydroxychloroquine (www.who.int)
- ^ The Lancet/Mandeep R Mehra, Sapan S Desai, Frank Ruschitzka, Amit N Patel (www.thelancet.com)
- ^ open letter (doi.org)
- ^ Weekly on Wednesdays (theconversation.com)
- ^ the Lancet (doi.org)
- ^ New England Journal of Medicine (doi.org)
- ^ sloppy and possibly fraudulent science (theconversation.com)
- ^ President Trump had touted the drug (www.forbes.com)
- ^ The pandemic claims new victims: prestigious medical journals (www.nytimes.com)
- ^ politicized science (www.wsj.com)
- ^ contaminated global coronavirus research (www.latimes.com)
- ^ Retractions are rare (doi.org)
- ^ targets numerous organs (doi.org)
- ^ develop immunity (doi.org)
- ^ three new large studies (doi.org)
- ^ Public Domain (commons.wikimedia.org)
- ^ incorrectly claimed vaccinations cause autism (doi.org)
- ^ Linus Pauling (www.nobelprize.org)
- ^ incorrect structure of DNA (doi.org)
- ^ Watson and Crick (doi.org)
- ^ Only 14% of drugs and therapies (doi.org)
- ^ paying close attention (www.journalism.org)
Authors: Mark R. O'Brian, Professor and Chair of Biochemistry, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, The State University of New York
