The genre of research article reviews

In order to respond to reviews of research articles (RAs) submitted for publication, researchers have to be astute readers of those reviews.  Plus, novice researchers are usually asked to write their own reviews of research during their doctoral studies–often beginning with papers (or abstracts) submitted to conferences. We developed a rhetorical move structure (see definition #2 of the about.com link) for describing the content+structure of journal submission reviews last spring in my doctoral seminar. I thought it was worth sharing as I get ready to lead that seminar for a new group of students.

I’ve written several posts about the genre of RAs. (If you need a brief introduction to what I mean by genre, read Pros have contextualized knowledge.) My approach to describing a genre builds on the ESP (English for Specific Purposes) body of work. Basically, a document people recognize as a particular genre will include a set of rhetorical moves, each of which can often be broken down into more detailed steps. You’ll see an example below. Or you can check out any of my posts filed under Crafting a Genre.

Brief Background on Journal Submission Review (for novices)

Peer review has been the cornerstone of research dissemination for at least a couple of hundred years. Here’s the reasoning. Research quality should be judged by experts (that is, peers) before it is published. The practice of peer review has been much criticized because of some truly abominable behavior by a small number of individuals. And there are ways to “publish” research without having it reviewed by peers. But there’s little to suggest traditional peer review will end any time in the near future. Publication of quality research is a core job requirement for university professors across the globe in every academic discipline (from accounting to zoology).

So how does peer review work? For anyone outside a university setting who is still reading, it’s important to understand that much of the journal publication process is completed by volunteers. Reviewers (also called “referees”) earn no money from their work on behalf of journals. Often editors of those journal also earn little to nothing. (They do sometimes get a small stipend from a professional organization that sponsors the journal or some release time from other duties from their employer.) And authors are not paid for work that is published; in some disciplines, they must actually pay a fee. So all of this labor is actually paid for by the employers of experts: mostly universities, with some stand-alone research labs or organizations. Once an RA is accepted for publication, there are paid staff in a publishing company who handle production and delivery of the journal in several issues per year.

Here’s how the peer review process happens. An author submits an RA to a journal. The journal’s editor reads the RA to determine whether it is appropriate to send it to referees. If the editor believes the RA is inappropriate, he or she sends a “desk rejection” letter to the author, and that’s the end of the process. If the editor believes the RA is potentially publishable, he or she chooses referees and invites two or more of them to review the submission. In many disciplines, the standard practice is called “double blind” review so that the identities of both author and referee are unknown to everyone except the editor. (In practice, the blindness isn’t always complete.)

Once referees return their reviews, the editor makes a publication decision and communicates it to the author. The editor’s letter will include the referees’ comments–either to help the author revise the RA for publication in the editor’s journal (that’s a revise & resubmit–R&R–decision) or in another journal (that’s a rejection decision). The likelihood of the editor sending a letter accepting an RA for publication as it was submitted is just about zero. The editor of the Quarterly Journal of Economics provided an interesting explanation of their specific process.

The review process involves occluded genres. That is, editor letters and referee reports are private rather than public artifacts. That makes it challenging to learn about them. Editors are the lynchpin. If you’re interested, the Committee on Publication Ethics (COPE) produces ethical standards for journal editors.

Rhetorical Moves (Structure + Content) in Editor R&R Letters

Although journal editors write several types of letters, my interest here is in the R&R letter because authors must use it as the basis for revising their RA before it can be resubmitted to the journal for potential publication. Basically, a document people recognize as an editor’s R&R letter includes five rhetorical moves: (1) expressing gratitude, (2) stating the publication decision, (3) justifying the decision, (4) providing directions for revision, and (5) maintaining goodwill. The table below breaks down those rhetorical moves into more detailed steps and provides examples from two actual editor letters. (Identifying information has been removed. Accounting Journal and Marketing Journal are pseudonyms rather than titles of actual publications.)

Rhetorical Moves (Structure + Content) in Referee Comments

Step 2b of the editor letter refers to the referees’ comments. Wise authors read these as carefully as the editor letter. Referee reports include at least three moves: (1) summarizing the submitted RA, (2) providing positive feedback, and (3) commenting on negative quality issues. Two previous studies of referee comments support a similar move structure. An optional fourth move involves maintaining goodwill. See the examples below, which were sent along with the editor letters above.

 

A Comparison of Textual Elements

The inclusion of organizational signals (headings and numbering or bullet lists) is more common in referee reports than in editor letters. Tone elements demonstrate that sensitivity to author feelings is different for editors than for reviewers, but some reviewers’ tone is more sensitive than others. Details are described below.

	Tone Elements				Organizational Elements
Sub-genres	Pronouns	Hedging	Directness (requests)	Presuppositions (reader truths)	Headings	Numbering or Bullets
Editor Letter	1st person (editor) = high 2nd person (author) = high 3rd person (reviewers) = mid	Moderate	Moderate	Minimal conflict	Low	Low
Referee Reports	1st person (reviewer) = high 2nd person (author) = low 3rd person (author) = high	Low to moderate	Moderate to High	Minimal to Some conflict	Mid	Mid

One tone comparison can be made with hedging the meaning of a claim. (Red text represents less sensitive and green more sensitive tone choices.) (1a) an Accounting Journal reviewer writes less sensitively, “I feel that the paper has been reasonably well-written.” (2a) the Marketing Journal editor writes more sensitively, “Considering her/his comments may enhance the insights . . .” (2b) a Marketing Journal reviewer writes less sensitively, “Isn’t the finding then a mere confirmation?”, while another Marketing Journal reviewer writes more sensitively, “Some of the covariates that you use could be better used as grouping factors.”

Another comparison can be made with directness of a heavily weighted request: (1a) the Accounting Journal editor writes sensitively, “One way to address this latter issue, as Reviewer A notes, is to perform the test using only those…”; (1b) an Accounting Journal reviewer writes sensitively, “your discussion may address this line of thinking by discussing …” or “It would have been useful to have a dependent variable the explored the nature of the evidence sought…” but less sensitively, “The authors need to provide convincing arguments as to why this alternative explanation is not feasible.” (2a) the Marketing Journal editor writes sensitively, “please concentrate on the following in the Revision …” (2b) a Marketing Journal reviewer writes less sensitively, “I therefore strongly suggest that the authors specify …” while another Marketing Journal reviewer writes sensitively, “you may wish to suggest …” and “you can compare O&A and J&C more directly …”

A final comparison can be made with presupposition that involves conflict with the author: (1a) an Accounting Journal reviewer writes sensitively, “The only effect on the hypotheses would be to add …” but less sensitively, “The authors claim that all four criteria listed in the footnote can be found…” or “ The theoretical development … ignoring all the prior research demonstrating the unique nature of auditing” and “there is a valid counterargument.” (2a) the Marketing Journal editor writes sensitively, “If you decide to accept my invitation to revise and resubmit, …” (2b) a Marketing Journal reviewer writes less sensitively, “The manuscript claims to present three “intriguing“ findings:” and “The authors should always report both measures of internal consistency reliability …”

It seems obvious that the anonymity of reviewers may influence their desire to manage author feelings.  Looks like an area ripe for research.

Research Sources

Disclaimer: The information in this post is based on limited data and does not offer conclusions based on published research.  (I just need to find time!)

Fortanet (2008). Evaluative language in peer review referee reports. Journal of English for Academic Purposes, 7, pp. 27-37.

Gosden (2003). ‘Why not give us the full story?’: Functions of referees’ comments in peer reviews of scientific research papers. Journal of English for Academic Purposes, 2, pp. 87-101.

Swales & Feak (1994). Academic Writing for Graduate Students. Ann Arbor: University of Michigan Press.