When authoritative sources hold onto bad data: A legal scholar explains the need for government databases to retract information
- Written by Janet Freilich, Associate Professor of Law, Fordham University
In 2004, Hwang Woo-suk was celebrated for his breakthrough discovery creating cloned human embryos[1], and his work was published in the prestigious journal Science. But the discovery was too good to be true[2]; Dr. Hwang had fabricated the data. Science publicly retracted the article and assembled a team to investigate what went wrong[3].
Retractions are frequently in the news. The high-profile discovery of a room-temperature superconductor was retracted[4] on Nov. 7, 2023. A series of retractions toppled the president[5] of Stanford University on July 19, 2023. Major early studies on COVID-19 were found to have serious data problems[6] and retracted on June 4, 2020.
Retractions are generally framed as a negative: as science not working properly, as an embarrassment for the institutions involved, or as a flaw in the peer review process. They can be all those things. But they can also be part of a story of science working the right way: finding and correcting errors, and publicly acknowledging when information turns out to be incorrect.
A far more pernicious problem occurs when information is not, and cannot, be retracted. There are many apparently authoritative sources that contain flawed information. Sometimes the flawed information is deliberate, but sometimes it isn’t – after all, to err is human. Often, there is no correction or retraction mechanism, meaning that information known to be wrong remains on the books without any indication of its flaws.
As a patent and intellectual property legal scholar[7], I’ve found that this is a particularly harmful problem with government information, which is often considered a source of trustworthy data but is prone to error[8] and often lacking any means to retract the information.
Patent fictions and fraud
Consider patents, documents that contain many technical details that can be useful to scientists[9]. There is no way to retract a patent[10]. And patents contain frequent errors[11]: Although patents are reviewed by an expert examiner before being granted, examiners do not check[12] whether the scientific data in the patent is correct.
In fact, the U.S. Patent and Trademark Office permits patentees to include fictional experiments and data[13] in patents. This practice, called prophetic examples[14], is common; about 25% of life sciences patents contain fictional experiments[15]. The patent office requires that prophetic examples be written in the present or future tense while real experiments can be written in the past tense. But this is confusing to nonspecialists, including scientists, who tend to assume that a phrase like “X and Y are mixed at 300 degrees to achieve a 95% yield rate” indicates a real experiment.
Almost a decade after Science retracted the journal article claiming cloned human cells, Dr. Hwang received a U.S patent[16] on his retracted discovery. Unlike the journal article, this patent has not been retracted. The patent office did not investigate the accuracy of the data – indeed, it granted the patent long after the data’s inaccuracy had been publicly acknowledged – and there is no indication on the face of the patent that it contains information that has been retracted elsewhere.
This is no anomaly. In a similar example, Elizabeth Holmes, the former – now imprisoned – CEO of Theranos, holds patents[17] on her thoroughly discredited claims for a small device that could rapidly run many tests on a small blood sample. Some of those patents were granted long after Theranos’ fraud headlined major newspapers.
Long-lived bad information
This sort of under-the-radar wrong data can be deeply misleading to readers. The system of retractions in scientific journals is not without its critics, but it compares favorably to the alternative of no retractions. Without retractions, readers don’t know when they are looking at incorrect information.
My colleague Soomi Kim[19] and I conducted a study of patent-paper pairs. We looked at cases where the same information was published in a journal article and in a patent by the same scientists, and the journal paper had subsequently been retracted. We found that while citations to papers dropped steeply after the paper was retracted, there was no reduction in citations to patents[20] with the very same incorrect information.
This probably happened because scientific journals paint a big red “retracted” notice on retracted articles online, informing the reader that the information is wrong. By contrast, patents have no retraction mechanism, so incorrect information continues to spread.
There are many other instances where authoritative-looking information is known to be wrong[21]. The Environmental Protection Agency publishes emissions data supplied by companies but not reviewed by the agency. Similarly, the Food and Drug Administration disseminates official-looking information about drugs that is generated by drug manufacturers and posted without an evaluation by the FDA.
Retractions play an important role in science.Consequences of nonretractions
There are also economic consequences when incorrect information can’t be easily corrected. The Food and Drug Administration publishes a list of patents[22] that cover brand-name drugs. The FDA won’t approve a generic drug unless the generic manufacturer has shown that each patent that covers the drug in question is expired, not infringed or invalid.
The problem is that the list of patents is generated by the brand-name drug manufacturers[23], who have an incentive to list patents that don’t actually cover their drugs[24]. Doing so increases the burden on generic drug manufacturers. The list is not checked by the FDA or anyone else, and there are few mechanisms for anyone other than the brand-name manufacturer to tell the FDA to remove a patent from the list.
Even when retractions are possible, they are effective only when readers pay attention to them. Financial data is sometimes retracted and corrected, but the revisions are not timely. “Markets don’t tend to react to revisions[25],” Paul Donovan, chief economist of UBS Global Wealth Management, told the Wall Street Journal, referring to governments revising gross domestic product figures.
Misinformation is a growing problem. There are no easy answers to solve it. But there are steps that would almost certainly help. One relatively straightforward one is for trusted data sources like those from the government to follow the lead of scientific journals and create a mechanism to retract erroneous information.
References
- ^ cloned human embryos (doi.org)
- ^ was too good to be true (www.nytimes.com)
- ^ investigate what went wrong (doi.org)
- ^ was retracted (www.wsj.com)
- ^ toppled the president (www.washingtonpost.com)
- ^ serious data problems (doi.org)
- ^ patent and intellectual property legal scholar (scholar.google.com)
- ^ source of trustworthy data but is prone to error (dx.doi.org)
- ^ useful to scientists (doi.org)
- ^ no way to retract a patent (doi.org)
- ^ frequent errors (dx.doi.org)
- ^ examiners do not check (ir.lawnet.fordham.edu)
- ^ fictional experiments and data (doi.org)
- ^ prophetic examples (doi.org)
- ^ 25% of life sciences patents contain fictional experiments (dx.doi.org)
- ^ Dr. Hwang received a U.S patent (www.nytimes.com)
- ^ holds patents (doi.org)
- ^ U.S. Patent and Trademark Office (image-ppubs.uspto.gov)
- ^ Soomi Kim (scholar.google.com)
- ^ no reduction in citations to patents (doi.org)
- ^ authoritative-looking information is known to be wrong (dx.doi.org)
- ^ a list of patents (www.fda.gov)
- ^ generated by the brand-name drug manufacturers (dx.doi.org)
- ^ don’t actually cover their drugs (www.ftc.gov)
- ^ Markets don’t tend to react to revisions (www.wsj.com)
Authors: Janet Freilich, Associate Professor of Law, Fordham University
