Authors trust editors to select appropriate reviewers. Editors trust reviewers to provide independent and informed assessments. Journals trust that reviewers will protect the confidentiality of manuscripts, declare conflicts of interest and write the reports themselves.
Readers, in turn, trust that publication following peer review results in a manuscript that has received meaningful scrutiny.
But how much of that trust is verified?
When authors receive a peer-review report, they are normally expected to take it seriously. They may be required to conduct additional analysis, rewrite substantial sections of the manuscript, gather new data or reconsider conclusions.
Yet authors are usually given remarkably little information about the report itself. They may not know who wrote it, what expertise that person has, whether artificial intelligence was used, whether the reviewer has an undeclared interest or even whether the manuscript was read carefully.
We authenticate authors, manuscripts, ethics declarations and funding statements. But the intellectual process that supposedly validates the research often remains largely hidden.
Was the report written by the reviewer?
Most journals assume that the person invited to review a manuscript is the person who completes the review. That assumption may often be correct, but it is not guaranteed.
Senior academics sometimes involve doctoral students, postdoctoral researchers or junior colleagues in reviewing manuscripts. There can be educational value in this, particularly when the review is discussed and supervised. However, problems arise when the editor is not informed and the additional reviewer is not identified.
The person invited by the journal may have accepted obligations relating to confidentiality, conflicts of interest and the responsible handling of the manuscript. A colleague who is subsequently given access to the paper may not have accepted any of those conditions. Indeed, the original obligation may already have been violated if somebody else is invited to help with the review without the editor’s knowledge or permission.
There are more troubling possibilities. Reviewer identities can be fabricated, compromised or manipulated. Reports may be written by people other than the individual named in the journal’s system. In the worst cases, the supposed review may be part of a coordinated attempt to influence publication decisions.
From the author’s perspective, the report arrives with the authority of peer review. But the author may have no way of knowing who produced it or whether that person possessed the necessary expertise.
Was it written by a human?
Generative artificial intelligence has introduced another layer of uncertainty.
A reviewer might use AI to improve the language of a report, organise comments or check whether an explanation is clear. Some journals may regard limited assistance of this kind as acceptable, provided confidentiality is protected and the reviewer remains responsible for the final assessment.
But AI could also be used to summarise the manuscript, identify alleged weaknesses or generate the entire report. The reviewer might do little more than upload the paper, copy the output and submit it to the journal.
This creates several problems. Uploading an unpublished manuscript to an external system may breach confidentiality. The generated comments may contain errors, overlook important details or confidently criticise something that the paper does not actually say. Most importantly, the reviewer may not have exercised the expert judgement for which they were selected.
There is a considerable difference between using AI to help express an independently formed assessment and asking AI to perform the assessment itself. That distinction is rarely visible to the author.
If a report has been substantially generated by AI, should the author be told? Should the editor be told? And should the report still carry the authority associated with expert peer review?
Journals will need clearer answers. A policy that simply tells reviewers to act responsibly is unlikely to be sufficient when authors cannot see how that responsibility was exercised.
here is another consequence of AI-generated reviews that I have not seen discussed very much. A large language model can generate substantial amounts of text with very little effort. Authors may therefore receive much longer reports and face a considerably greater burden when preparing their responses. Responding to reviewers can already take a significant amount of time, but what happens when reports become four times as long without becoming four times as useful?
Was the reviewer independent?
Peer review is expected to provide an independent assessment, but independence is not always straightforward.
Researchers work within professional networks. They collaborate, compete for funding, disagree over methods, review one another’s work and sometimes pursue similar findings at the same time. A reviewer may recognise the authors from the subject, citations, dataset or writing style even when the journal uses anonymous review.
A conflict does not have to be financial. It may arise from a close collaboration, institutional relationship, personal dispute, intellectual rivalry or direct competition to publish similar work. Reviewers may also have an interest in encouraging citations to their own publications or promoting a preferred theoretical or methodological position.
Most journals ask reviewers to declare conflicts, but the process largely depends on self-disclosure. Editors may have little evidence on which to judge whether a declaration is complete.
The author may suspect a conflict when faced with an unusual concentration on one body of work, repeated demands for particular citations or criticism that appears personal rather than scholarly. But raising that concern can be difficult. Authors may fear appearing defensive, especially when the reviewer has recommended rejection.
The problem is not that every critical report conceals a conflict. Strong criticism can be entirely legitimate and valuable. The problem is that authors are usually expected to trust the reviewer’s independence without being given enough information to evaluate it.
Was there any meaningful review?
Not every inadequate review is fabricated, conflicted or produced by AI. Some are simply poor.
Most experienced researchers have received reports containing vague statements, generic recommendations or comments that appear unrelated to the manuscript. Others have encountered reviewers who demand that an entirely different study should have been conducted, contradict one another or criticise material that is clearly explained in the paper.
A brief review is not necessarily a bad review. A reviewer may identify a decisive problem in a few carefully written paragraphs. Equally, a long report may create an impression of thoroughness while offering little that helps either the editor or the author.
The more useful question is whether the report demonstrates serious engagement with the manuscript. Does it identify specific issues? Are its conclusions supported? Does it distinguish between major weaknesses and matters of preference? Does it help the editor reach an informed decision and the author improve the work?
When a report fails these basic tests, it can still have significant consequences. It may contribute to rejection, delay publication or require weeks of unnecessary revision.
Who assesses the reviewer?
Many journal systems allow editors to assess reviewers. An editor may be asked whether the report was useful, sufficiently detailed and submitted on time. That information can then influence whether the reviewer is invited again.
The author, who must interpret and respond to the report, is rarely offered the same opportunity.
This is a curious omission. Editors can judge whether a report helped them make a decision, but authors are often better placed to identify whether it meaningfully engaged with the manuscript. They can see whether the reviewer overlooked explanations, misunderstood the methods, requested impossible changes or made claims that are already explained in the paper.
Of course, authors should not be asked whether they liked the report. A rigorous negative review may be excellent, while an enthusiastic recommendation may be superficial. Nor should an author’s dissatisfaction with a rejection be treated as evidence that the review was poor.
But authors could be invited to assess more specific features. Was the report relevant to the manuscript? Was it sufficiently detailed to understand the concerns? Was it internally consistent? Was it professionally expressed? Did it appear to be based on a careful reading of the paper?
These are questions about the quality of the review, not whether the author agrees with its recommendation.
Should authors rate peer-review reports?
A system of author feedback would need to be designed carefully.
The assessment should probably take place after the editorial decision so that it cannot be used to pressure reviewers or influence the immediate outcome. The feedback should remain confidential and should not be treated as a simple popularity score.
One poor rating would tell an editor very little. Authors may respond emotionally to criticism, and disagreements over methods or interpretation are part of scholarly debate. But repeated feedback across several manuscripts could reveal a pattern.
If different authors repeatedly report that a reviewer’s comments are vague, irrelevant, contradictory or unprofessional, an editor should want to know this. Similarly, consistent feedback that a reviewer produces thoughtful and constructive reports could help journals recognise and retain their strongest reviewers.
Author feedback could also identify problems that editorial ratings miss. An editor may consider a report useful because it supports a clear decision. The author may be able to show that the report misread the analysis, cited irrelevant literature or demanded changes that the methodology cannot support.
The two assessments answer different questions. Editors judge whether the report helped the editorial process. Authors can help judge whether it engaged responsibly with the research.
Neither perspective is sufficient on its own.
Greater transparency without abandoning anonymity
Improving accountability does not necessarily require every reviewer to be named.
Open peer review can make parts of the process more transparent, but it also brings legitimate concerns. Junior researchers may be reluctant to criticise powerful scholars publicly. Reviewers in small fields may fear professional consequences. Some disciplines and research cultures may be more comfortable with openness than others.
There are still steps that journals could take while preserving reviewer anonymity.
They could tell authors whether reviewer identities were independently verified, whether the reviewers declared any conflicts and whether the journal permits AI assistance. They could require reviewers to disclose substantial AI use to the editor and confirm that unpublished manuscripts have not been uploaded to unauthorised systems.
Journals could also give authors a confidential mechanism for commenting on the quality of each report. Editors would retain responsibility for interpreting that feedback, recognising that an unfavourable review is not necessarily a poor one.
These measures would not eliminate weak, biased or fabricated reviews. They would, however, acknowledge that trust should be supported by some degree of accountability.
Trust has to work in both directions
Authors are asked to make extensive declarations when submitting a manuscript. They identify their institutions, disclose funding, describe conflicts of interest, confirm authorship and accept responsibility for the work.
Those requirements are justified because journals need confidence in the provenance and integrity of the research.
The peer-review report can profoundly influence whether that research enters the scholarly archive, how it is revised and how long publication takes. Yet the report itself may come with little assurance about its authorship, independence or intellectual quality.
Peer review cannot function without trust, but trust should not mean that authors must accept every report as authentic and authoritative simply because it arrived through a journal’s submission system.
If journals want authors to respond seriously to peer review, they should be able to offer reasonable confidence that the review was produced by an appropriate person, handled confidentially, written independently and based on meaningful engagement with the manuscript.
And if editors routinely judge the performance of reviewers, perhaps the researchers who receive and act upon those reviews should also be given a voice.
The purpose would not be to let authors rate whether they agreed with the verdict. It would be to help journals answer a much more fundamental question: is their peer-review system producing reports that deserve to be trusted?
If authors cannot know who produced a review, whether AI was used, whether conflicts existed or whether the manuscript was read carefully, on what basis are they expected to trust it?
Did you find this edition useful?
I publish two free weekly newsletters exploring different aspects of higher education, research and scholarly publishing. If you enjoyed this edition, you may also find value in the following.
Governance & Research in HE
A weekly brief for higher education leaders and researchers examining governance, leadership and research in practice.
Publishing with Integrity
Examining scholarly publishing through an ethical lens, challenging assumptions and supporting academic career development.
About the Author
Professor Graham Kendall is Vice-Chancellor of GlobalNxt University, Malaysia, and an Emeritus Professor at the University of Nottingham. Over the past 25 years, he has published more than 300 peer-reviewed papers and has served as Editor-in-Chief and Associate Editor of several international journals. Through Publishing with Integrity, he examines the ethical, governance and practical challenges facing scholarly publishing, encouraging greater transparency, integrity and informed debate across the global research community.