The role of reviewing is to help the conference Chairs decide on whether to accept or reject papers or abstracts. Up to now, the most common forms of reviewing has been the single or double-blind review, where the authors don’t know who the reviewers are. However, the idea of using an open peer review system has grown. The quality of the review is paramount, whatever way it is reviewed. But could an open review system be more effective than what is currently in place?
An open reviewing system can lead to very tame reviews being given. If the reviewer knows that their opinions and recommendations will be seen by everyone, they may not be as critical as if their review was blind. Similarly, if a reviewer is reviewing their colleague’s paper, they may be more inclined to give a good general review than be too critical and cause unnecessary tension in the workplace. The review in this way can have a bias towards not offending someone they know. Good practice in reviewing is to avoid a member of a certain organisation review a paper from an author within the same organisation.
One of the better aspects of an open review system is it highlights who gives the not-so-constructive reviews. It is natural that work can build up. The deadline is getting closer and abstracts need to be reviewed. Sometimes to get all the reviews done in time, a reviewer may not be as thorough with an abstract as they need be. By making it publicly known the review giving a specific critique, it will force them to be more thorough in the critique they give, as their reputation is on the line with whatever they say.
Do you think that authors are entitled to know who is reviewing their paper? They have put a lot of time and effort into writing their abstract, and if given a very critical review, should they be allowed to know by whom to ask them explain certain parts of a critique? While it may be beneficial to the research (as an author can get a better understanding of what is the right path and why), it may not be feasible. Reviewers do not want to be “hassled” for the critique they have given, and may lead to less reviewers offering their services down the line.
In a study carried out by BMJ, it was discovered that the quality of the review did not diminish whether an abstract was reviewed openly or blind. However, the time taken to review one abstract significantly increased when reviewers knew it was an open peer review. When reviewed openly, reviewers on average took 25 more minutes to critique an abstract than when peer-review was blind. The case can be made that it takes reviewers long enough to review (voluntarily, may I add) as it is without increasing their workload when it doesn’t affect quality. What was even more ground-breaking from this study was that 55% of reviewers declined to take part when they knew it was an open review system. This shows that the reviewers are reluctant to publicise their comments.
In a lot of research disciplines, teams are small and close knit. If one reviewer is openly very critical of an abstract, it may harm future relationships within a sector and stop reviewers putting their reputation on the line.
There are a lot of reasons for and against an open-review system. What has to be decided is – do the benefits of having an open review system really outweigh the costs?