CRAFTING HIGH-QUALITY REVIEWS: GUIDELINES, EXAMPLES AND FEEDBACK
为期刊评审人提供撰写高质量评审的指南，包括建立积极关系、提供建设性批评和具体改进建议，并举例说明好与差的评审，旨在提升评审质量和期刊发展。
During JSCM's last two annual Associate Editor (AE) meetings, our AEs have raised the question of how we can provide feedback to reviewers concerning the quality of their reviews, with the goal of developing reviewers and further improving the journal's review process. These are certainly valid requests. A primary responsibility of a journal editor and indeed a cornerstone of the scientific review process is to ensure a high-quality, objective review process. Good reviews benefit authors and the AEs who must integrate reviewers' evaluations with their own assessments to ultimately make a recommendation about a manuscript's disposition — rejection or invitation to revise and resubmit. Unfortunately, the peer review process is far from perfect, and can suffer from unnecessarily harsh critiques, lack of consensus among reviewers and misunderstanding concerning the expectations of what constitutes a high-quality review (Miller 2006; Rynes 2006). The purpose of this essay is to further refine JSCM's review process — to provide guidance to reviewers and to continue to develop reviewers' ability to provide high-quality reviews. As a starting point, we have “dusted off” the JSCM guidelines for reviewers, which we crafted when we first began our editorship. We have augmented these guidelines with explicit examples, based on both the best, and the worst, reviews which we have seen over the past 3 years. These examples are an amalgam of the many hundreds of reviews which we have read during this time period; the exact details have been excluded or substantively changed to protect the innocent (and the not so innocent!). The remainder of this essay is largely written in the second person, and is directed toward reviewers. However, the advice and direction applies to our AEs who must not only integrate these reviews, but provide their own reviews and assessments of the manuscripts which they shepherd through the review process at JSCM. Your activities as a reviewer impact the careers of authors, the editorial team who has limited journal space to publish articles, and thus ultimately the quality of research and the knowledge base in our field. Thus, your job as a reviewer is twofold: (1) evaluative — is the quality level sufficient to invite a revision and perhaps ultimately an acceptance of the paper in the Journal, and is the paper's topic neither overly narrow nor overly broad to warrant publication? and (2) developmental — can you provide constructive comments for improving the quality of the paper? While a reviewer should identify a manuscript's deficiencies, a good reviewer should also provide suggestions for how the deficiencies can be addressed. To be effective, these roles must also be performed in a timely manner. An author's tenure and promotion decision might rest on the timeliness of the reviewer's response. A paper that is delayed by a lengthy review process does not contribute to the discipline as it sits on the reviewer's desk, and the paper's data and even ideas may become increasingly less valuable over time. There are a number of benefits to being a reviewer. First, you remain current with emerging topics and research methods and can build your critical thinking skills, both of which are useful when writing your own papers. Second, the editors track the performance of reviewers in terms of both quality and timeliness. If a reviewer is performing multiple, high-quality reviews, then s/he may be asked to serve the Journal as a member of the Review Board (or, in the case of reviewers who are currently members of the Review Board, the Associate Editor Board). Service as an editorial reviewer or as an Associate Editor results in external professional recognition and is often recognized internally during the merit review process and tenure and promotion process. Conversely, poor performance, in terms of timeliness and/or quality of reviews, may result in a reviewer's removal as an editorial reviewer or Associate Editor. Finally, your high-quality and timely reviews contribute to the development of the Journal, and thus our field. The best manuscripts are likely to be submitted to journals that provide fast turnaround and high-quality reviews, and that are open to innovative ideas and methodologies. First, you can create a positive rapport and demonstrate an understanding of the paper by providing a brief summary of the paper at the beginning of the review, followed by a list of the paper's strengths. This shows that you have read and understand the paper, and it acknowledges the often extensive effort of the authors. The summary should also include a description of the paper's problem areas. Here are some brief excerpts, which provide examples of how to create such a positive rapport: First — thanks very much for submitting your best work to JSCM. As a member of the field — we need your best work going to the best SCM journals. And, this is a quality study. The paper is well written, and shows a strong level of familiarity with the literature in … The authors have obviously placed a considerable amount of time and effort in collecting the study's data. Second, you should convey a constructive attitude. One potential failure in the peer review process is that we, as reviewers, can forget that we are supposed to evaluate and provide feedback to our peers. This is a common courtesy, as most papers consist of what the authors considered to be good work. For example, consider the following review, which recommended conditional acceptance of a manuscript during the first round of the review process: This paper follows a sound scientific approach to examine an interesting research phenomenon. The authors have done an admirable job of integrating the relevant literature. And, the authors have clearly explained the research methodology and results. The manuscript makes a very strong contribution to supply chain management theory and practice. This review does a reasonably good job of exemplifying the first characteristic of a high-quality review — creating a positive rapport. Unfortunately, this was the entire review, rather than what should have been the first paragraph of a high-quality review. Thus, this review falls short achieving the second characteristic of a high-quality review — providing constructive criticisms and suggestions for improving the quality of a manuscript. Additionally, a conditional acceptance of a manuscript should be the exception, rather than the rule. Even the best papers can be improved! Third (and admittedly this is closely related to the first two characteristics), you should provide a list of specific comments regarding weaknesses and concerns about the manuscript. These comments should not simply highlight the problem areas of the paper, but should also provide recommendations for how they might be improved. In order to truly understand the full range of these potential improvements that can be made to a manuscript, you may need to read the paper more than once, as opposed to a single, “aggressive skim.” This applies both to papers which you recommend for revision and resubmission, as well as papers which you recommend for rejection. Further, authors can better build on positive comments like the following: A clearer argument is needed prior to the introduction of Hypothesis 2. The authors may want to integrate the work of Krause et al. (2006) and related work which has examined negotiations from the economic approach — see for example Nash 1950, 1953 and Tversky and Kahneman 1991, as opposed to negative and/or vague remarks, such as: The discussion in the Hypothesis section makes no sense to me. You should, however, be careful to avoid being overly kind and giving false hope to the authors. If you feel that a manuscript has potential, but that the outcome of a revision is very uncertain, you could qualify an invitation of a “revise and resubmit” along the following lines: It was difficult to make a decision with this paper as it has substantial potential but it lacked several key elements. I feel that a major revision is not out of the question though the requested changes will present a challenge for the author(s). I would therefore classify this as a high-risk revision. If you believe that a manuscript cannot viably be revised and ultimately published in the Journal, or if a submitted paper suffers from a fatal flaw that is not correctable or unlikely to be corrected by the authors (for example a defect with the data collection), you need to state why the paper cannot make a contribution to the Journal. Even if you recommend that a manuscript be rejected, it is still extremely helpful to provide suggestions for improvement as noted above, along with suggestions for potential, alternative outlets for the authors' work. Fourth, you should ensure that an article that reports the collection and analysis of data (whether quantitative or qualitative) takes either an inductive (e.g., building grounded theory using accepted techniques or conceptual theory development) or deductive approach (e.g., using theory to develop hypotheses which are tested through an analysis of the data). Articles that simply collect quantitative data and then report summary, descriptive statistics or which use inferential statistics without the testing of theoretically derived hypotheses, probably cannot be accepted for publication. In addition, for most empirical manuscripts, whether quantitative or qualitative, you must determine whether the authors have adequately assessed validity, the “sine qua non” of empirical research. Have the authors addressed both reliability and the various dimensions of validity? Fifth and finally, you should assess the level of theoretical development in the paper. Have the authors provided logical, well-integrated and compelling arguments for the introduction of their propositions or hypotheses? Does the paper correctly integrate existing theories in building these arguments? In addition and better yet, does the article test theory for the first time or in unique and novel ways, challenge existing theory, or synthesize recent advances and concepts to generate new theory? And finally, is there pragmatic relevance — does the theory make managerial sense? Can the authors provide realistic examples, which can improve both theoretical soundness and richness and demonstrate managerial relevance? We evaluate reviewers each year as potential candidates for JSCM's Best Reviewer Award, and as part of our on going process of growing and maintaining a high-quality Review Board. However, we recognize that this does not allow for comprehensive or timely feedback. While we will continue this review process, we have recently implemented a feedback process through Manuscript Central, our online platform, which asks AEs to provide more immediate and detailed feedback to reviewers. One of our goals as editors is to share with the reviewers both praise and opportunities for improvement in the review process. This serves the dual purpose of guiding the reviewers to refine their skills, and giving those who submit to JSCM a positive and valuable review experience, whether or not their paper is ultimately accepted for publication by the Journal. One of the hallmarks of the premiere journals in the field of business is the provision of timely, constructive and well-thought-out feedback. The developmental feedback shared with authors as part of the review process should challenge the authors to think about their work more deeply, and improve their current submission and possibly their future research. If relevant, the authors should be directed to critical work with which they may be unfamiliar, but which could improve their current submission. The quality of the Journal is highly dependent on the quality of the reviews that are provided to authors. We as Co-Editors-in-Chief are deeply indebted to the reviewers and the Associate Editors for their fine work in identifying the pieces with the greatest potential and presenting the authors with value-adding ideas for refining their work. We are also very grateful for the constructive feedback offered to those authors whose work does not make it into the Journal. This helps the authors improve their future work, and makes it more likely that they will submit their best work to the Journal of Supply Chain Management in the future.