Article Summary and Critique

Checklist for Critiquing Research Articles

Title and Abstract

• Does the title accurately reflect the content of the article?
• If an abstract is provided, does the abstract properly summarize the content of the article?

Introduction

• Is pertinent background information presented in sufficient detail for the reader to understand the context of the research?
• Are important citations missing and is the salient literature reviewed?
• Is the background clearly and concisely written, or is unnecessary detail included?
• Does the article include clear statements regarding the purpose of research, its objectives and hypotheses?
• Is the stated hypothesis clearly derived from the rationale? Are other hypotheses consistent with the model or theory being tested?
• Are the main variables clearly defined in both positive and operationalized terms?

Participant Selection (Sampling Procedures)

• Does the article include clear statements regarding the sampling procedure used (probabilistic vs. non-probabilistic)?
• Are sample characteristics reported?
• Is the sample appropriate for the research question?
• Does the article include a statement of ethics, informed consent, or debriefing of participants?
• If deception was involved in the instructions to the participants, how were participants debriefed?
• If some participants withdrew from the study, does the article include information on possible differential attrition?
• Is the sample size adequate for the study?
• Can results be generalized from this sample to the intended population?

Procedure Section

• In general, are the procedures appropriate to the research question?
• Are there possible confounds in the study?
• Are the procedures described in sufficient detail to allow replication?
• Is the training of raters/clinicians reported?
• If more than one measure was used, was the administration order counterbalanced? Were possible order effects addresses?

Measures Section

• Is there an explicit rationale for the measures selected?
• Is method selection derived logically from the hypotheses?
• Are the psychometric properties of the measures (i.e., reliability and validity) reported and are they adequate?
• If variables are scored by observers using categories or codes, is the inter-rater reliability reported?
• Are possible effect of testing and instrumentation decay addressed and/or ruled out?

Statistical Analysis and Results

• Are the statistical analyses appropriate for the research design and the types of variables involved?
• Are the assumptions of parametric tests addressed and satisfied?
• Is the sample size sufficient for the chosen statistical tests?
• Are tests of statistical significance properly used and reported (e.g., risk of type-I and type-II errors, Bonferroni adjustment, etc.)?
• Are practical significance (magnitude of effect size) and clinical significance addressed?
• Are standard error of measurement and confidence intervals reported?
• Are the results clearly explained and displayed? Are tables and figures used appropriately?

Discussion and Interpretation

• Are the results discussed fully?
• Is the discussion focused on findings, or is it digressive or speculative?
• Are limitations of the study and/or methodological biases addressed?
• Are the results generalizable?
• Are directions for future research suggested?
• Are discrepancies from previous findings in the research literature explained?
• Are possible alternative interpretations considered?

General Issues

• Is the writing style understandable, clear, and appropriate to the intended audience?
• Are the various sections of the report integrated in a coherent manner?
• Do sentences and paragraphs follow one another logically?
• Are references (citations and quotations) used and identified appropriately?

Additional Resources:

Holosko, M. J. (2006a). A suggested authors’ checklist for submitting manuscripts to Research on Social Work Practice. Research on Social Work Practice, 16(4), 449-454.

Holosko, M. J. (2006b). Primer for critiquing social research: A student guide. Belmont, CA: Thomson Brooks/Cole.

Hyman, R. (1995). How to critique a published article. Psychological Bulletin, 118(2), 178-182.

Oxman, A. D. (1994). Checklists for review articles. British Medical Journal, 309(6955), 648-651.

Some useful guidelines for writing article critiques can also be found at the Writing Center of Saint Mary’s University of Minnesota (http://www2.smumn.edu/deptpages/tcwritingcenter/forms_of_writing/summ_crit.php): “Critiques” by Susan Katz and Jennie Skerl.