Systematic Reviews

Types of Reviews

Systematic Review Definition

This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions. In particular, it requires the use of robust techniques for:

searching for and identifying primary studies,
selecting the studies to be included in the review,
extracting the data from the studies, and
appraising the quality of these studies,
synthesizing the findings narratively and/or through pooling suitable quantitative data in meta-analysis

Dixon-Woods, M., & Sutton, A. (2004). Systematic Review. In Michael S. Lewis-Beck, A. Bryman, & Tim Futing Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 1111-1112). Thousand Oaks, CA: Sage Publications

Systematic Reviews / Traditional Reviews

Feature	Systematic Review	Narrative / Traditional Reviews
Question	Often focused (clinical) question.	Sometimes broad in scope.
Sources & Search	Explicit search strategy of multiple databases. Comprehensive sources.	Not usually specified.
Selection	Criterion-based selection; uniformly applied	Not usually specified.
Appraisal	Rigorous critical appraisal.	Variable.
Synthesis	Quantitative summary. Also qualitative/narrative.	Often qualitative summary.
Inferences	Based on all available evidence.	Based on a sample of the evidence.
Grading	All evidence is graded (quality)	May or may not be graded.

Adapted from: Cook, D., Mulrow, C. & Haydens, R. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5):376-380.

Systematic Review Myths

Systematic reviews:

are the same as ordinary literature reviews, only bigger
include only randomized controlled trials
require the adoption of a biomedical model of health
are of no relevance outside of health / medicine
must involve statistical analysis / synthesis
must be conducted by experts
can be done without experienced librarian support
are not really research

Adapted from: Petticrew, M. (2001) Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 322:98-101.

Reducing Bias

Systematic reviews attempt to minimize bias and error throughout the review process

Uses systematic process that is transparent and replicable
Clearly document transparent and replicable search strategies
Inter-rater reliability on key data
Assessment of bias in studies
Assessment of study quality
Meta-analysis (when possible) to statistically synthesize results across studies

Tanner-Smith, E. & SJ Wilson. (2013) Systematic reveiwing for evidence-based practice: An introductory workshop. The Campbell Collaboration.

Libraries & Cultural Resources

Research Guides