Making Decisions

Decision Types

Accept - No further revision required. The manuscript is publishable in its current form. The majority of articles require revision before reaching this stage.

Minor Revision - A limited number of changes are required. Implies that the editors and reviewers feel the paper is publishable once their comments have been addressed.

Examples:

• Some re-writing is needed to address specific areas where the manuscript is ambiguous and requires clarity.
• Citations are appropriate but need revision i.e., these could be limited or excessive.
• Simple factual or numerical errors, which are easily resolved.
• Presentational issues with tables and figures i.e., incorrect labelling, missing arrows etc.
• Ethics and consent statements are present and appropriate, but need rewording for clarity.
• Minor language edits required i.e., repetitive statements, typos, spelling errors.

Major Revision – Substantial work is required prior to resubmission.

Examples:

• Substantial rewriting is required. Key elements are missing from the paper or not described in adequate detail, meaning the study is unreproducible and difficult to interpret.
• The reviewers have highlighted significant issues with the data and additional analysis, or reanalysis is needed.
• The reviewers have raised publication ethics concerns which need explanation i.e., if similar work has been published without the authors appropriately acknowledging or citing it.
• Lack of clarity regarding ethics approval or patient consent. We should have this information prior to review, but sometimes the reviewers raise concerns which warrant investigation.
• Substantial language edits are required.
• Extensive problems with figures & tables.

Reject – The manuscript is of insufficient quality, novelty or significance to warrant publication. Even when rejecting a paper, editors are encouraged to share suggestions for improvement in the decision letter.

Examples:

• If issues of quality, novelty and/or contribution* cannot be addressed through revision
• Revisions made are insufficient.

*Dependent on the journal’s Aims and Scope for requirements with regard to levels of novelty and/or contribution.

Making a Decision

• Ideally, we should aim for no more than three rounds of revisions in total, although there may be exceptions to this, and editors should use their discretion.
• For transparency, the comments of the reviewers should not be edited or deleted, unless it is to remove offensive language.
• If the reviewers are quite critical of the paper, consider whether they have built a credible case to support their argument i.e., have they included references to support their concerns?
• Only offer revisions if you feel the concerns raised by the reviewers are addressable. For example, if the reviewers identify fundamental issues with the design of the study, the authors will not be able to address these with a revision and it would be better to reject the paper. It’s worth keeping in mind that sometimes reviewers recommend major revisions, but actually suggest changes that the authors will not be able to make (as outlined above), so in this case, it would be better to reject the paper.
• The opposite is also true and sometimes reviewers recommend reject, when their concerns are actually addressable.
• While the reviewers may suggest additional analysis, keep in mind that it is not the reviewer’s job to expand the scope of the article.
• Not all feedback is helpful and there may be cases when you need to disregard some of the reviewer comments. For example:
  • You can disregard comments about novelty and importance if your journal does not select for this. Instead, focus on the comments regarding the soundness of the science presented and how the study adds to literature.
  • Be wary of reviewers who ask authors to cite papers on which they are listed as an author. The recommendation to cite their own papers is against best practice and can be seen as an attempt to manipulate the citation record. Any suggested citations relayed to the authors must be accompanied by a clear explanation of why these specific references are the most relevant to the submitted manuscript, and how they can assist in advancing the arguments made by the authors.
  • Hostile or inflammatory comments should be disregarded.

Making a Decision Based on One Report

• You may be able to reject a paper based on one report. For example, you may have one report which identifies a fatal flaw in the study design or indicates a lack of novelty or significance.
• If the paper is scientifically sound but lacks novelty, editors should consider referring the paper to Sage Path (if available) so it can be sent to a less selective Sage journal.
• Some examples of unfixable flaws include problems with the number or selection of research participants, the use of incorrect or unvalidated instruments, inappropriate treatments, the incomplete or absent measurement of important study variables, incorrect blinding and lack of a control group.

Re-review and When to Seek a Third Reviewer

• Not all manuscripts will require re-review and editors should use their discretion to decide when to send a paper back to the original reviewers for assessment.
• If the comments are minor, and the changes are easy to assess, consider reviewing the changes in house, without the need for further review.
• If the authors have made substantial changes to the manuscript, re-analysed their data or added additional analysis, re-review is required. If the original reviewers have declined to see the revised paper, seek new reviewers to assess the changes.
• If the reviewers have not provided a comprehensive assessment of the manuscript, or if you have two conflicting reports, it can be helpful to ask for a third reviewer.
• If the reviewer’s expertise does not match the topic of the paper and/or methodology of the paper, and you’re not confident that they were able to properly assess this, seek a third reviewer.
• You might find that one, or both reviewers, ask for the paper to be reviewed by a dedicated statistical reviewer.  In this situation, an additional statistical reviewer should be sought before a decision is made.

What makes a good or bad review?

Good Review

• Reviewer comments are substantial for the first review (even if Minor Revision recommended) and any Major Revision recommendations.
• Reject recommendations don’t necessarily have to be substantial if there are fundamental errors/issues which have been noted by the reviewer.
• It provides an overview of the paper’s suitability for publication, followed by more detailed feedback.
• Ideally, the review should be easy to read and written in a logical order.
• The reviewers should list any specific edits (e.g. spelling) with a page and paragraph/line number.
• Will include the good points as well as bad, especially for Major Revisions, as authors will have a better idea of the aspects of the paper which are strong.
• Comments to the Editor don’t include any useful material for the authors that isn’t already in the Comments to the Authors.
• The reviewer agrees to review again….

Bad Review

• Vast majority of the time – any review that recommends Acceptance at first round review.
• Lacking detail at first review stage – even if the recommendation is for Minor Revisions, the reviewer should be able to justify why the paper is already suitable.
• Any major revisions without substantial detail and or/justification.
• When the overall recommendation doesn’t reflect the reviewer’s comments.
• Purely descriptive with no evaluation of content
• Reviewer asks authors to cite own papers with no justification
• Only grammatical/spelling changes requested.

Handling Poor Quality Reviews

Blank and poor quality reviews should not be used to inform a recommendation or decision.

There are two options for dealing with insufficient reviews on Sage Track:

1. Rescind and unassign the poor quality review and unassign the reviewer
   • The benefits of this option are that the reviewer is informed if their review is not used, and it's very clear to the handling editor that poor quality reviews cannot be used.
   • The disadvantages of this option are it's more time consuming, the reviewer may dispute their removal from the manuscript, and there is a chance that the editor will re-invite the same reviewer, which would add unnecessary time to the process and they will likely decline or submit another blank/poor review.
2. Keep the poor quality review on the system and increase the number of reviews required by one (e.g. if set to '2', increase to '3') 

• The benefits of this option are that the poor quality review remains on the system, so can be used as evidence if needed in future, there are fewer steps involved on the system, and the reviewer cannot be invited back to review that manuscript again.
• The disadvantages of this option are that the editor may keep trying to make a recommendation or decision using the poor quality review, and the reviewer is not aware that their review is not used to inform the recommendation/decision.

In general, especially for larger journals, the most effective option is to keep the review on the system and increase the required number of reviews by one. Rescinding the review may be preferable in certain circumstances.

Reviewer Selection FAQ

When I am selecting reviewers, why do some say "opt out", some say "opt in", and others don't say either?

When reviewers create their accounts, they are asked whether they “opt in“ or “opt out” to receive emails form Sage and its affiliates about their products and services. This is an “attribute” associated with their accounts like other keywords, which is why it shows up in the search results.

You don’t need to worry about this, however. It’s due to an unusual system setup that the opt in/opt out option appears next to other keywords. Additionally, you are free to invite any reviewer no matter if they’ve answered opt in, opt out, or nothing at all.

Why can't I add a reviewer to the Reviewer List? Why is the "Add" check box greyed out?

If a user does not check that they wish to be invited for peer review in their profile, then they will be marked as an Excluded Reviewer.

What is an R score?

The R-score is the average score a reviewer receives from editors. At the bottom of a completed review, you will see two brief rating scales – one for timeliness and one for quality assessment, on a scale from 1 to 3. Simply give your score for the review and click Save.  Each score a reviewer receives is collected and averaged to produce the overall R-score. The R-score is attached to a person’s account, and can be used in reviewer searches and reports to find and determine the best reviewers in your site. You will get the most accurate R-score results if every editor in the site rates reviewers once they complete their reviews.