Public Policy Involvement

Vol. 20, No. 1
ISSN: 1546-2676

Editor:
Dorothy Mitstifer (Posthumous)

Guest Editor:
Holly Roseski

Publication date:
Compiled 2022

Kappa Omicron Nu FORUM, Vol. 20, No. 1.
ISSN: 1546-2676. Editor: Holly Roseski. Official publication of Kappa Omicron Nu National Honor Society. Member, Association of College Honor Societies. Copyright © 2022. Kappa Omicron Nu FORUM is a refereed, semi-annual publication serving the profession of family and consumer sciences. The opinions expressed by the authors are their own and do not necessarily reflect the policies of the society. Further information: Kappa Omicron Nu, PO Box 798, Okemos, MI 48805-0798. Telephone: (727) 940-2658 ext. 2003

Interested in submitting an article to KON FORUM? Papers are now being accepted for review.

Kappa Omicron Nu
FORUM

Peer Review: A Filter for Quality

Dorothy I. Mitstifer

Dr. Mitstifer is Executive Director of Kappa Omicron Nu Honor Society.

Contact: [email protected]

There is little dispute that peer review will remain as a means of quality control and certification of higher education scholarship and research. Harnad (2000, p. 1) refers to peer review as the “invisible hand” that maintains quality. Alternatives have been discussed, but no system has gained enough support for change in the foreseeable future. Thus, it is in the best interest of the human sciences to ensure that the peer review system serves the interests of all the parties involved.

Criticism of peer review indicates that neither the editor nor the referees are infallible. Poor selection of specialists for specific articles and misinterpretation of referee advice are potentially problematic. Failures of referees include careless review, unfair criticism, and inadequate feedback. In addition, attention to technical details instead of substantive issues can sometimes diminish the effectiveness of peer review. Authors share in the problem; a good many papers have had insufficient editing before submission. And authors are not always conscientious in using the feedback to make changes.

Because little attention is given to the intricacies of peer review and reviewers receive little or no training in how to conduct a review, this paper will explore the editorial objectives and the elements of the review that could enhance the process. An explicit peer review process and objective criteria can defend against unethical or unfair critiques. In addition to benefiting reviewers and the peer review process, these guidelines will enable authors to refine their research manuscripts before submission.

The editorial objectives of peer-reviewed publications include validity, originality, and significance. Validity requires that reliable research methods and techniques have been used and that conclusions are suitably qualified. Originality requires that the study is new or reports findings that have never been published. Significance requires that the two previous criteria are met and that the research findings add something new: an understanding, a perspective, an observation, or a potentially important piece of information (Gordon, 1983). The problem with meeting these criteria is that “the highly original paper which reports unusual research methods, unconventional forms of argument, unexpected findings, or a radically new perspective is far more likely to be considered of suspect validity” (p. 8). Thus the danger is that the work of a future Nobel Laureate could be rejected. The editor has to balance the risks of publishing a manuscript with the least errors and least novel and original findings against one that might have errors but might be breaking new ground.

The peer review process should give particular attention to the following questions (adapted from Morgan, 2002) when reviewing a manuscript:

Does the title reflect the content of the paper in a short, interesting form?
Does the introduction review the literature and objective of the paper?
Does the review of the literature integrate the outside resources smoothly and efficiently?
Is the “objective” (or hypothesis) statement thorough and specific?
Is the method section logically arranged and complete (including research design, the sample, data collection, procedure, and approach to data analyses)?
Could you repeat the study in its entirety by following the procedures?
Are the results presented in a thorough and simple way?
Are the figures and tables well prepared, easy to understand, and necessary?
Does the discussion section really discuss the results?
Was the objective met?
Were the references taken from “good” sources (i.e., peer reviewed and recent)?

Once the reviewer has explored the above questions, made notes, and “digested” the manuscript, a review should be written to include:

A brief summary of the major points and message of the manuscript;
Perceived strengths of the paper;
Weaknesses or changes that need to be made, including source of comments;
Quality and clarity of writing;
Benefits to the literature; and
The action to be taken on the manuscript.

Jawaid and Jafary (2002) discuss the ethical considerations from an Evidence-Based Peer Review perspective. A reviewer has an obligation to the publication but also to the author to focus on ways to strengthen manuscripts; authors do not gain from harsh and abusive criticism. Conflicts of interest should be communicated to the editor to determine whether there is adequate reason to withdraw. It is not expected that a reviewer comment on every aspect, but reviewers should note the areas of a manuscript that are beyond their expertise. If consultation is required, there must be acknowledgement of that fact in the review report. Reviewers must maintain confidentiality in regard to the manuscript. Authors have the right to expect that an article sent to a journal (or shared with colleagues) will be treated as their exclusive property. Reviewers will neither disclose the contents nor make personal use of them. In an open peer review process, explicit written permission of the author should be obtained to cite or use the information in any way. Lest too much emphasis is placed on reviewers, the ethical perspective of peer review requires an inclusive approach to the rights and responsibilities of the publisher, editor, reviewer, and author.

The role of institutions in mentoring researchers is as important as the role of publishers in assuring quality of scholarship and research. Instead of a competitive climate, an intentional program to mentor researchers and authors will create a collegial atmosphere for communication about content, ethical, and technical standards of excellence. The educational outcomes of an institution depend upon the quality of the faculty, which is measured in part by the continuing commitment to learning and to extending the boundary of knowledge. As important as institutional health in this domain is that of the fields that faculty represent. The human sciences require both institutional support and quality publications to sustain the field and nourish future leaders.

This article has described the context for exploration of the peer review process, described the editorial objectives of peer-reviewed publications, posed questions for reviewers in reading a manuscript, described the elements of a review report, and explored the rights and responsibilities of the publisher, editor, reviewer, and author. At its best, peer review benefits researchers and consumers of research by providing a detailed, constructive, and well-founded critique. Although the editor makes the final decision, it is in the best interest of publications to have effective reviewers and concrete information upon which to make publishing decisions and to ensure that peer review is a filter for quality.

References

Gordon, M. (1983). Running a refereeing system. Leicester, UK: Primary Communications Research Centre, University of Leicester.

Harnad, S. (2000, April). The invisible hand of peer review. Exploit Interactive 5, 1-5.

Jawaid, S. A., & Jafary, M. H. (2002, July-September). Peer review to improve quality of manuscripts. Pakistan Journal of Medical Sciences, 1-9.Retrieved December 16, 2002 from http://www.pulsepakistan.com/pakmedsciences/page2.html.

Morgan, E. L. (2002). Peer review of a scientific journal article. Retrieved December 16, 2002 from http://iweb.tntech.edu/elmorgan/392/392_Peer_Review_Journal.htm.

********

Note: Unauthorized citation of this paper is discouraged. Instead please contact the author prior to citing the work.