How to fix the academic peer review system

Peer review has long been a holy cow in the academic publication process. Before new research is published, academic journals send the work to other experts in the field (peers), who may advise against publication, or (usually) demand revisions before recommending publication. The idea of peer review is to hold academics’ feet to the proverbial fire, ensuring that we publish only work of reasonable quality.

But in fact, what the global academy has been clinging to for more than a century is largely anonymous, off-the-record, pre-publication peer review. Every academic knows the frustration of trying to satisfy or rebut reviewers who contradict each other or who demand revisions that miss the point and dilute the results.

The original intention and lifeblood of peer review — opening the doors of scholarly journals beyond the old boys’ clubs — has been squeezed out by the forces of over-commitment, financial gain, careerism and raw jealousy.

Some might argue that in an era of fake news and dubious “scientific findings”, a vibrant pre-publication editorial process is crucial. Won’t it provide some assurance that what we read is likely to be scientifically and ethically sound, and that it draws valid conclusions?

Indeed. But the pretence that peer review ensures this has led to a once unimaginable proliferation of “scholarly” journals. These journals require little more than an online content management system to create. “Peer-reviewed” journals are no longer meaningful filters.

Most academics don’t seriously “read” journals to keep abreast of developments in their field. At most, one might read a few wide-ranging journals, hoping to stumble across interesting ideas outside one’s own narrow speciality. Instead, search engines are used to find relevant articles; academics then use their own sifting processes to decide which ones to take seriously.

There is an expanding field of bibliometrics, which uses statistical indicators, like frequency and forum of citation, to give clues to whose work is making an impact. In any broad field (physics, mathematics, biomedical research, social science) or niche area (semiconductor materials, HIV surveillance, etc.) there is probably room for a handful of serious journals, e.g. Nature and Science, that people would actually read.

Beyond that, we may as well skip ahead to the inevitable outcome: a world in which the default route for academic work is self-publication on research repositories. Physicists and mathematicians are already routinely posting their work on arXiv, one of the more prominent scholarly repositories, before submitting it to peer-reviewed journals. Around these repositories vibrant communities emerge, characterised by serious engagement with new ideas and results, the pointing out of flaws and the identification of valuable work.

These repositories conduct no prior peer review. Editorial intervention is limited to checking that the work appears to report research findings and that it has been correctly categorised by field and subfield. After being posted, the papers are subjected to the usual critical reading and commentary by other scientists in the field. We can think of this as on-the-record, open and robust post-publication peer review, and it is much better than commentary from unaccountable and anonymous reviewers, free to ride their hobby horses or shoot down the work of their competitors.

Nothing would stop the few journals that survive from embracing the self-publication model, and haggling with the authors whose work they wish to include. They can do so on the strength of post-publication peer review, or on any ad-hoc basis they wish, to create curated collections of important work. We should quietly allow the other hundred thousand or so journals to die.

In the present system we feel obliged, having committed time and energy to journal-specific formatting, including arbitrary length constraints (often for costless online space), and the whole process of waiting for initial reviews, to tough it out and see the process through by investing even more time and energy. We “gratefully” take on board the reviewers’ comments to the limit of our pain threshold, and politely rebut only the truly intolerable – rather than cut our losses and walk away.

In many fields with high stakes, big budgets and serious impact, there are very few articles written by one or two lone gun authors. If groups of authors aren’t capable of critical self-appraisal before putting out their work, we probably need not bother to read it. Small groups and single authors can surely solicit feedback from colleagues, or revise their work following comments received after posting the article online.

The primary archives all allow formal revisions without undue complications. Once a work is “dropped” into the public domain, you have to be savvy about how you circulate notice of its existence, but eventually the cream will rise to the top.

Funders and academic employers, groaning under the weight of the modern knowledge edifice, place great emphasis on such metrics as how many “peer reviewed” papers you have published. But increasingly they also consider more nuanced evaluations of where you have published, how many citations you receive and from whom. Well-designed self-publication forums can (and do) provide metrics that are just as useful for judging quality and impact.

Deep down we all know where this is headed. Once an article comes up on Google Scholar, Pubmed, or the arXiv alerts, we consider who wrote it, what it is about, etc., in order to decide whether to read any further. Where it was published is still a consideration, but only one of many.

And so we come back to the conclusion that only a tiny number of serious journals deserves to survive the coming collapse.

Pre-publication peer review, then, is a relic from another era when it served to expand the pool of authors. Today it is a needless detour on the road to producing meaningful research and creatively disseminating our findings, free from the shackles of journal styles and our competitors’ whims.

Thanks to technological advancement, we all have access to the tools we need to create beautiful expositions, demonstrations and presentations. And we have all the tools to make post-publication peer review – the heart and soul of scientific conversation – richer and more efficient.

The authors are scientists at Stellenbosch University.

Views expressed are not necessarily GroundUp’s.

Donate using SnapScan.