Systematic Reviews in the Health Sciences

• AMSTAR checklist
• Cochrane risk-of-bias tool for randomized trials
• Centre for Evidence Based Medicine critical appraisal tools
• Critical Appraisal Skills Programme
• Joanna Briggs Institute critical appraisal tools
• McGill mixed methods appraisal
• Newcastle – Ottawa Scale
• STROBE checklists for observational studies

Before moving on reporting your findings, you’ll have to evaluate your studies for data synthesis. If your studies are sufficiently homogenous, you may be able to synthesize them statistically into a meta-analysis. If they are heterogeneous or qualitative, then you can perform a narrative or descriptive synthesis. The only problem is that there is no standard process for this.

Before you even start your review, you should check to see if there are any instructions for authors in the journals you’re hoping to publish in. If there aren’t, you can check the format of other systematic reviews published in the journals you’re interested in.

If you’re not sure what journal to publish in, JANE is a good resource. All you have to do is copy and paste your title or abstract and they will make suggestions on appropriate journals to publish in. You can also view the University of Calgary's Scholarly Communications Research Guide for information on publication. 

Another thing to consider when reporting your findings is whether you’re going to publish in an open access journal. Open access journals make their articles free for readers to access without subscriptions and licensing restrictions. Article processing charges can be discounted by certain publishers through library licensing agreements. 

Keep in mind that even the best of us can fall prey to predatory journals which have become all too prevalent. Predatory journals are defined as “entities that prioritize self-interest at the expense of scholarship[. They] are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices.” They try to appear reputable, but charge publication fees to authors without checking articles for quality or legitimacy. Check out these tools to help avoid these journals

• Think. Check. Submit.
• Journal Evaluation Tool

You may also consider reporting your findings is archiving your review in PRISM, UCalgary’s institutional repository.

You may also need to update your review, whether prior to submission for publication because of the delay between when you conducted the literature searches and when the review was accepted for publication, or as a requirement of the publishing body, or as new evidence is published.

Typically, in your protocol, you’ll have identified a reporting guideline that you want to follow. If you’re unsure of what guideline you should use, you may want to check out the EQUATOR Network. The EQUATOR Network stands for Enhancing the Quality and Transparency of Health Research and it provides links to key reporting guidelines applicable to different types of research. CONSORT, for example, is a guideline for reporting randomized controlled trials, and STROBE is for reporting observational studies.

PRISMA or the Preferred Reporting Items for Systematic Reviews and Meta-Analyses is typically what you’ll want to use if you’re doing a systematic review. It identifies the minimum information you need to report for transparency in your review. It consists of a 27-item checklist and a flow diagram that you should complete by the end of your review. Prisma also has a flowchart, both for seaches within databases and for searches within databases and grey literature. 

The second chart looks significantly more confusing, but works on the same principle: you want to report the number of records found through other methods, identify how many you were actually able to retrieve, and report the number of records assessed for eligibility, along with your reasons for excluding any.

These charts are why you’ll need to keep track of how many records you identify through database and register searching, how many duplicates or other records are removed prior to the screening process, the number of records that you screen based on title and abstract (and how many you exclude at that point), the number of full-text articles you’re able to retrieve as well as those not retrieved, the number of full-text articles you actually assess (along with reasons for excluding any at this point), and the final number of studies you include in your systematic review. These studies could be represented in multiple reports, which is why there’s a separate number for the reports of included studies.

In addition to your PRISMA flow chart, you'll need to report your search strategy in the Methods section of your review. There is a checklist called PRISMA-S which is helpful for making sure you include everything. The article linked below includes helpful examples of what each item on the checklist is referring to. 

•   Rethlefsen, Melissa L, Kirtley, Shona, Waffenschmidt, Siw, Ayala, Ana Patricia, Moher, David, Page, Matthew J, & Koffel, Jonathan B. (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10(1), 39–39. https://doi.org/10.1186/s13643-020-01542-z 

For example, listing the name of each database searched is not as simple as saying, “We searched MEDLINE, EMBASE, CINAHL, and Cochrane Central.” You’ll actually need to specify which platform you searched those databases on. So, more specifically, “We searched MEDLINE (Ovid), EMBASE (Ovid), CINAHL (EBSCOhost), and Cochrane Central Register of Controlled Trials (Ovid).”