Systematic reviews and Meta-analysis in Business/Management
Systematic Reviews and Meta-analysis
A Systematic Review is a form of Evidence Synthesis; Its purpose is to summarize all available relevant evidence to answer a pre-defined question. It aims to remove bias by using systematic and explicit methods.
A meta-analysis is the statistical analysis of the results of included studies from a systematic review. Meta-analysis should be based on systematic review methodology, with the addition of the statistical analysis of the results.
Evidence synthesis reviews including, systematic reviews and meta-analyses, must be transparent and reproducible, therefore all aspects of the review must be reported in sufficient detail so that it can be replicated.
Note: The term systematic review might be used synonymously with the term evidence synthesis in this guide.
How are they different from traditional reviews?
Traditional literature reviews differ from evidence synthesis reviews in many ways, and especially in their methods. The table below highlights some of the differences between a traditional narrative review and a systematic review.
| Traditional Literature Review | Systematic Reviews | |
|---|---|---|
| Review protocol | Not required | Requires a protocol/plan for the entire review to be created a priori |
| Review Question | Can be broad; highlight only particular pieces of literature, or support a particular viewpoint. | Requires the question to be well-defined and focused so it is possible to answer. |
| Methods to locate studies | No expectations regarding where and how the studies were found. Searches do not need to be comprehensive. | Explicit and transparent search methods should be used to attempt to locate all available research. Methods should be well-documented, well-reported, comprehensive in scope, and search a variety of sources. This requires in-depth knowledge of information systems; the researcher should consult a librarian. |
| Study selection | The researcher can choose what to include, with no justification required. | Eligibility criteria are pre-determined and applied systematically. The process is transparent, and attempts to minimize bias (by using 2 screeners where possible). Reasons for inclusion or exclusion are reported. Study selection is reported in a flow diagram. |
| Data extraction | No consistency; researcher can choose what parts of the individual studies to include. | Standardized data extraction forms/templates should be used to extract relevant information from each included study, according to the pre-defined categories specified in the review protocol. |
| Risk of Bias assessment | None required | Risk of bias assessment is expected. This can be done using validated checklists or appraisal tools, or using pre-defined criteria. |
| Reporting | None required | A well-conducted systematic review or meta-analysis should be reported according to an established reporting standard. This ensures that relevant aspects of the review are reported in detail such that it can be replicated. |
Overview of the systematic review process
The typical steps of a systematic review are shown below
Cochrane (n.d.) What systematic review authors do [Image]. Retrieved from: https://cccrg.cochrane.org/Infographics
Evidence syntheses are time-consuming given the explicit methodological steps that need to be followed. In one study (Borah et al., 2017) that analyzed the time taken to conduct and publish systematic reviews, based on data published in a protocol registry and the final published manuscripts, the mean estimated time was found to be approximately 67 weeks.
Due to the explicit and rigorous methods expected in an evidence synthesis project, a variety of skills and competencies are required:
- Content expertise
- Information retrieval expertise
- Statistical expertise (if you are including a meta-analysis)
- methodological expertise (an understanding of the methodological implications and expectations at every step)
Therefore, evidence syntheses are often done by teams which may include a combination of content experts, an information expert (librarian), a statistician (or individual with statistical expertise), and an evidence synthesis methodology expert.
It is important that these experts be engaged in the evidence synthesis process from the beginning, at the protocol development stage.
In one study (Borah et al., 2017), the mean number of authors on a review was analyzed, and found to be 5.
- Animated storyboard: What are systematic reviewsThis animated storyboard explains what systematic reviews are and describes the process briefly. It was prepared by the Cochrane Consumers and Communication Group, La Trobe University with support from Cochrane Australia. It was written by Jack Nunn and Sophie Hill, and graphics produced by Shakira Moss.