Peer review and hot cross buns

Ben Martin is going to explain how academics get their work published in peer-reviewed journals.

He is Associate Professor of Mathematics at the University of Auckland. He is also an associate editor for the Journal of the Australian Mathematical Society and has himself published a number of peer-reviewed papers.

Rachel: Let’s say, you’ve written a paper. You send it to a journal and it lands on the editor’s desk. What do they do with it?

Ben: Well, first they’ll have a quick look at it just to make sure it’s within the scope of the journal and see whether it’s written in a satisfactory sort way and not riddled with typographical mistakes or in terrible English. Generally the more prestigious the journal is, the fussier it is about matters of presentation. The editor has to make an initial decision about whether the paper is worth being sent for refereeing. Sometimes the paper will be rejected very quickly by the editor without being sent for refereeing on the grounds that it’s poorly written, or not in the right subject area for this particular journal, or the results are not considered significant enough for that journal.

Rachel: The editor has decided this paper is worth being sent off to referees. How many referees are there and how are they chosen?

 Ben: It depends on the journal. In my discipline, pure maths, it’s more common to just have one referee and referees are expected to do a very thorough job of reading the paper and checking it for correctness.  In other subjects, often things are less clear-cut than in maths: there can be several competing theories and no-one’s really sure which is correct.  So the paper might then be sent to two or three referees and the editor might choose referees with a range of different theoretical viewpoints. In particular, if the paper is debunking a paper written by somebody else, it might be sent to that other person – or someone in the same theoretical camp – for comment.

 Rachel: So the people who receive a paper for review, do they get paid anything for their review?

 Ben: Generally, no. They don’t get paid anything.

 Rachel: Does the editor get paid anything?

 Ben: A common sort of journal structure is you have an editor-in-chief, you maybe have a managing editor as well who deals with the mechanics of running the journal, printing, distribution, etc., and then there’ll be an editorial board, usually composed of academics from various Universities. The people on the editorial board don’t get paid, but the managing editor will often get paid a salary. The editor-in-chief may or may not receive some money for work they do but usually this person is an academic at some university and they’ll be receiving a salary from their University.

Rachel: How do the editors get selected?

Ben: Journals like to make sure they have a range of editors to cover all of the different topics that that journal deals with. Typically they look for people who are experts in the field and have themselves published a lot. This way they should be better at making judgements on other people’s work.

Rachel: If you become editor of a journal, is this a role for life?

 Ben: It depends on the journal. Usually this is something you do for a few years. Sometimes people will stay on editoral boards for ten or fifteen years, but probably more like five.

Rachel: The paper is now in the hands of a referee. What exactly does the referee do?

Ben: Something I should probably say is the referee is anonymous. The author who submitted the paper does not know who the referee is. Their identity is not disclosed. In some disciplines it’s common to have double blind refereeing which means the referee does not know who the author is, either. So the paper is submitted and the name of the author is blanked out which is supposed to make the process more impartial and objective.

Back to the question: the referee reads the paper, they’re supposed to decide whether the paper is suitable for publication in that journal and the kind of thing they look for is the suitability of the subject matter, the correctness of the results, the originality of the results – in the sense that they’re not the same as someone else’s results – which will involve reading other papers in the area. They also make a judgement on the significance of the results – how important are they? Are they important enough to be published in this particular journal?  The more prestigious the journal is, the higher the bar, and the more important the results have to be.  If the results are showing some established theory is wrong, for instance, then that tends to be quite important and more likely to be published than a paper that is providing just a little more evidence in support of some established theory.

The referee reads the paper and writes a report and they make a recommendation about whether the paper should be published or not. But the decision is not the referee’s: the paper goes back to the editor and the editor makes the final decision.

Rachel: How long does this referee process take from the moment the referee gets the paper to the moment they send back a report?

 Ben: It depends on the discipline. In some disciplines the referee only has a few weeks. In pure maths referees tend to spend a lot of time going through papers quite carefully and checking correctness. In other disciplines correctness might not be so easy to check – a referee can’t easily reproduce someone’s experiment – and this is one of the reasons why these other disciplines have more referees: to get a range of opinions.

Rachel: How many hours would you spend refereeing a paper?

 Ben: I’ve had papers before that I’ve easily spent 20 or 30 hours looking at. It all depends on the complexity and various other factors.

When the editor gets the recommendations they have to decide whether the paper should be published or get rejected. The options are: you reject or you can recommend publication with or without revisions. If the referee’s recommendation is very clear-cut then that makes it easier for the editor, but even then there are wider constraints. If you have a very prestigious journal, their acceptance rate is small. I don’t know what the acceptance rate for a journal like Nature is but it’s probably well under 10%. So even a paper recommended by referees might not be accepted because it’s competing with other papers also recommended by their referees.

If there’s more than one referee, they may well say different things and the editor might ask for one or more additional referee reports.

If the decision is to reject the paper the author can send it off to a different journal and the process starts from the beginning.

Rachel: What happens if a paper gets published and someone finds a mistake after peer review and publication?

 Ben: It depends on the nature of the mistake. So you might have a paper in physics for instance where the author is putting forward some new theory which matches all of the existing experimental evidence. Then somoeone goes out and conducts an experiment and gets data which contradicts the theory. In this case there’s no requirement or expectation that the author of the original paper will withdraw or retract or change anything they wrote, because what was in the paper was correct to the best of everyone’s knowledge and consistent with the existing facts and data. So what would happen then is just that the people who did the experiment would write their own paper.

It’s not uncommon for people to make mistakes in their papers. In pure maths maybe they’ll realise that one of their proofs is wrong. What you do then typically is publish an erratum: this is just a short note which will go in the same journal saying, in this paper, that was published on such and such a date, this part here is wrong and here’s how you fix it. In my experience in pure maths, these errata are not serious. It’s quite rare that you’ll get an erratum that says the whole main result is wrong. As long as the mistake was made in good faith and was reasonably minor,  you would publish an erratum. I guess if the mistake was so great that it invalidated the main result of the paper then you might withdraw it. Withdrawal or retraction is more common in cases where there’s some suggestion of malpractice – that the author has deliberately falsified the data.

Thanks very much, Ben. Ben is also an accomplished maker of hot cross buns, something he does annually on Good Friday, which is today:


The referee’s report on the hot cross buns was favourable.