Imagine that a paper is published in 2018. Over the next eight years it is cited hundreds of times. Some authors mention it only briefly, while others use it to support an argument, adopt its methodology, use its data or build new experiments upon its findings. It may also find its way into literature reviews, systematic reviews or meta-analyses.
Then, in 2026, the paper is retracted. We know what happens to the original article. The publisher should clearly identify it as retracted, publish a retraction notice and ensure that its changed status is reflected in the scholarly record. But what happens to everything that came afterwards?
Retraction Does Not Necessarily Stop at the Retracted Paper
We tend to think about retractions as events affecting individual publications. A problem is identified with a paper, an investigation takes place and, if the problem is sufficiently serious, the paper is retracted. In that sense, the system is largely document based.
Research, however, is cumulative. Researchers build on previous findings, cite earlier studies as evidence, reuse methods and data, incorporate published results into reviews and use previous work to decide which questions should be investigated next. That is, a paper sits within a network of other research.
If one part of that network is subsequently found to be unreliable, simply changing the status of the original paper does not necessarily address everything that has been built upon it. Retracted papers also continue to be cited, sometimes for many years after their retraction.
The important question, though, is not simply whether a retracted paper has been cited. It is whether later research actually depends upon it.
Citation Is Not the Same as Dependency
Suppose my paper contains 50 references and one of them is subsequently retracted. That does not mean that my paper should also be retracted. I may have cited the paper simply to describe the history of a subject, to acknowledge an alternative position or even to criticise its findings. The retracted paper might also support a relatively minor statement that has no bearing on my own results.
At the other extreme, imagine that my central hypothesis was based upon the findings of the retracted paper. Perhaps I used its dataset, adopted an experimental procedure that is subsequently shown to be unreliable, or included its results as an important component of a meta-analysis. In those circumstances, the position is quite different.
There is an important distinction between citation and dependency. Rather than asking whether a paper cites something that has subsequently been retracted, perhaps the more useful question is whether its conclusions would be materially different if the retracted research could no longer be relied upon.
That distinction matters because otherwise we risk two opposite mistakes. We could treat every citation of a retracted paper as evidence that the citing paper is somehow contaminated, which would plainly be unreasonable. Alternatively, we could assume that marking the original paper as retracted is sufficient, even where later research has materially depended upon it.
What Should Happen to Downstream Research?
Once we recognise that distinction, another question follows. What should happen when a paper genuinely does depend upon research that is later retracted?
There is unlikely to be a single answer. In some cases, no action may be necessary. In others, a correction might be appropriate so that readers understand that one of the sources upon which the paper relied has subsequently been retracted. More serious cases might justify reassessing the validity of the paper itself.
Consider a systematic review containing 40 studies. If one relatively small study is subsequently retracted, removing it from the analysis might make almost no difference to the overall result. But what if the retracted paper was one of the largest studies in the review, or contributed heavily to the effect being reported? What if several of the underlying studies were later retracted because their data could not be trusted?
At some point, a problem affecting the source literature may become a problem affecting the later publication. Depending on the circumstances, the appropriate response could range from doing nothing, through publishing a correction or notice, to reconsidering the validity of the downstream paper itself.
The difficulty is that somebody first has to notice.
Who Is Responsible for Noticing?
When a journal retracts a paper, should its responsibility end once the retraction notice has been published? Or should somebody identify the publications that have subsequently cited it?
The technology already exists to do at least part of this. Citation databases know which papers cite which other papers, while scholarly metadata can record that an article has been corrected or retracted. That makes it possible, at least in principle, to identify the authors and journals whose publications cite a paper whose status has subsequently changed.
Imagine a relatively simple system. When a paper is retracted, authors of papers that cite it receive an automated notification stating that one of the publications referenced in their article has been retracted and asking them to consider whether this affects their work. The journal that published the citing article could receive the same notification.
No accusation would be made and no automatic correction would be required. Nobody would assume that the later paper was unreliable simply because it contained the citation. The purpose would be to put the information in front of the people best placed to decide whether the retraction matters.
That seems preferable to hoping that somebody eventually notices.
Perhaps We Need to Think Beyond Citations
There is another complication. A citation tells us that one paper refers to another, but it does not tell us why.
Two papers may each cite the same retracted publication, yet their exposure to the problem could be completely different. One might contain a sentence explaining that the now-retracted study once reported a particular finding. The other might have based an entire experimental programme upon that finding.
Both appear as citations in a citation database, but intellectually they represent very different relationships. If we want to understand how unreliable research propagates through the literature, simply counting citations may therefore tell us only part of the story.
Perhaps scholarly publishing needs to think more seriously about dependencies between publications. Which earlier studies merely provide context and which underpin the data, methods or conclusions of the later research? That would be difficult to capture systematically, but the distinction is important if we are serious about correcting the scholarly record.
Retraction Is a Network Problem
There has been considerable effort to make retractions more visible. That is important. A reader accessing a retracted paper should be left in no doubt about its status, and bibliographic databases and other services should reflect that status accurately.
But perhaps the next stage is to move beyond asking whether the retracted paper itself has been clearly marked. We should also ask what happened after it was published. Which papers cited it? Which relied upon it? Were its data reused? Was it incorporated into systematic reviews or meta-analyses? Did it influence subsequent experiments?
Most importantly, we should ask whether any of those later publications need to be reconsidered now that the original research has been retracted.
A retraction changes the status of one paper, but science is built through connections between papers. Correcting the scholarly record may therefore require us to follow those connections.
About the Author
Graham Kendall is Vice-Chancellor of GlobalNxt University and an Emeritus Professor at the University of Nottingham. He has published more than 300 peer-reviewed papers and has held senior editorial roles, including Editor-in-Chief of the IEEE Transactions on Computational Intelligence and AI in Games and Associate Editor of the Journal of the Operational Research Society.
If You Liked This Edition
I publish two free LinkedIn newsletters:
Governance, Leadership & Research
Thoughts on leadership, universities, research, AI and organisational life.
Publishing with Integrity
Exploring research integrity, academic publishing and the systems surrounding scholarly communication. https://buff.ly/fNxYrjq