Home About Me My Calendar My Diagrams My Garden Going Green My Research Activities My Research Publications

Hypermedia Online Publishing: the Transformation of the Scholarly Journal

5.4.2 Problems with refereeing

The most rigorous model of peer review (or refereeing) is the so-called 'double-blind' peer review. Submitted articles are sent to two or more reviewers, and neither the reviewers or the author are aware of each other's identity. This is the traditional form of quality control in p-journals and provides a filtering/gatekeeping/censorship function, depending on one's perspective. There are a number of criticisms of this model:

It is essentially a binary process. The articles are either accepted or rejected (acceptance with changes can be considered ultimately in the same way as acceptance). The journal does not attempt to rank the quality of the articles published. There is an implied ranking between journals but not within (except between refereed and un-refereed sections).
It is anonymous. There is no way for the rest of the scholarly community to observe the process and its affect on the final product. There is also no way to effectively challenge a negative referee's report that may have been motivated by malice or professional envy. There is also no overt reward to referees for their efforts and often significant contributions [Gaines, 1993].
It may involve either errors in the judgement of the referees or conscious or unconscious bias towards the article or author [Meadows, 1977].
It may well involve referees in a conflict of interest. Fjällbrant has argued that because of today's narrow specialisations, referees will almost certainly know those whose work they are refereeing. Given the competitive nature of modern research, "the temptation to delay or downgrade information from a parallel research team is obvious" [Fjällbrant, 1997]
It stops once the article has been accepted. There is no real opportunity for ongoing discussion of an article within a scholarly community. Letters to the editor often appear months after the original article and the discussion is limited to one round of comment and response.
It is reliant on individuals who are usually eminent in their fields and hence have very busy schedules. It is therefore frequently prone to delays. An analysis in suggests that under reasonable circumstances authors of articles will end up 'owing' the journal many more reviews of other articles than most authors are prepared to accept [Gaines, 1993, p. 151].
Articles that challenge the accepted wisdom have real difficulties in getting published because of the conservatism of the referees (or because they require extra space to expound their non-standard assumptions [Gaines, 1993]). It has been suggested in fact that the refereeing system may actually suppress new ideas [Zuckerman&Merton1979] rather than only allow the best ideas through.

Last modified: Monday, 11-Dec-2017 14:38:18 AEDT