Skip to Main Content

Systematic reviews and Meta-analysis in Business/Management

This guide is available for researchers and students undertaking systematic reviews or meta-analysis on Management or Business-related topics. It highlights best practices to improve reproducibility.

Systematic Reviews and Meta-analysis

A Systematic Review is a form of Evidence Synthesis; Its purpose is to summarize all available relevant evidence to answer a pre-defined question. It aims to remove bias by using systematic and explicit methods.

A meta-analysis is the statistical analysis of the results of included studies from a systematic review. Meta-analysis should be based on systematic review methodology, with the addition of the statistical analysis of the results.

Evidence synthesis reviews including, systematic reviews and meta-analyses, must be transparent and reproducible, therefore all aspects of the review must be reported in sufficient detail so that it can be replicated.

Note: The term systematic review might be used synonymously with the term evidence synthesis in this guide. 

How are they different from traditional reviews?

Traditional literature reviews differ from evidence synthesis reviews in many ways, and especially in their methods. The table below highlights some of the differences between a traditional narrative review and a systematic review.

                     Traditional Literature Review Systematic Reviews         
Review protocol Not required Requires a protocol/plan for the entire review to be created a priori
Review Question Can be broad; highlight only particular pieces of literature, or support a particular viewpoint.  Requires the question to be well-defined and focused so it is possible to answer. 
Methods to locate studies                                                    No expectations regarding where and how the studies were found. Searches do not need to be comprehensive.                                                                                                                                       Explicit and transparent search methods should be used to attempt to locate all available research. Methods should be well-documented, well-reported, comprehensive in scope, and search a variety of sources. This requires in-depth knowledge of information systems; the researcher should consult a librarian.
Study selection        The researcher can choose what to include, with no justification required.                                                                      Eligibility criteria are pre-determined and applied systematically. The process is transparent, and attempts to minimize bias (by using 2 screeners where possible). Reasons for inclusion or exclusion are reported. Study selection is reported in a flow diagram.
Data extraction No consistency; researcher can choose what parts of the individual studies to include. Standardized data extraction forms/templates should be used to extract relevant information from each included study, according to the pre-defined categories specified in the review protocol.
Risk of Bias assessment  None required                                                                                                        Risk of bias assessment is expected. This can be done using validated checklists or appraisal tools, or using pre-defined criteria.
Reporting None required                                                                                                                                    A well-conducted systematic review or meta-analysis should be reported according to an established reporting standard. This ensures that relevant aspects of the review are reported in detail such that it can be replicated.

 

Overview of the systematic review process

The typical steps of a systematic review are shown below

 

Steps of a systematic review - Cochrane

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Cochrane (n.d.) What systematic review authors do [Image]. Retrieved from: https://cccrg.cochrane.org/Infographics

Evidence syntheses are time-consuming given the explicit methodological steps that need to be followed. In one study (Borah et al., 2017) that analyzed the time taken to conduct and publish systematic reviews, based on data published in a protocol registry and the final published manuscripts, the mean estimated time was found to be approximately 67 weeks. 

 

Due to the explicit and rigorous methods expected in an evidence synthesis project, a variety of skills and competencies are required:

  • Content expertise
  • Information retrieval expertise
  • Statistical expertise (if you are including a meta-analysis)
  • methodological expertise (an understanding of the methodological implications and expectations at every step)

Therefore, evidence syntheses are often done by teams which may include a combination of content experts, an information expert (librarian), a statistician (or individual with statistical expertise), and an evidence synthesis methodology expert.

It is important that these experts be engaged in the evidence synthesis process from the beginning, at the protocol development stage.

In one study (Borah et al., 2017), the mean number of authors on a review was analyzed, and found to be 5.