How to do a systematic review: the PRISMA 2020 steps

What are the steps of a systematic review?

Six steps are there, and you have to do them in this order only.

1. 1

   Frame the question

   Write one narrow question using PICO, PEO or SPIDER. From that question, write down clearly which studies come in and which stay out.

2. 2

   Protocol and registration

   Decide the methods first, before any result is in front of you. After that, register on PROSPERO, OSF or INPLASY.

3. 3

   Search

   Run the search in more than one database, plus registers and grey literature. Keep a log of each search you do.

4. 4

   Screen

   First titles and abstracts, then full texts. Two people do the screening, and neither one looks at the other’s answers till both are done.

5. 5

   Extract and appraise

   Test the extraction form on a few studies before using it. Pick a risk-of-bias tool suited to the design of each study.

6. 6

   Synthesis and reporting

   Combine the findings by meta-analysis or by narrative synthesis. Report as per the PRISMA 2020 checklist and flow diagram.

Most scholars skip the second step. Suppose you changed the criteria after seeing what the searches gave. Then it is just a narrative review with extra paperwork. Many times an examiner can see this from the dates itself.

How do you frame the question and register the protocol?

The question decides everything after it, from the search terms and criteria up to the appraisal tool. In health research PICO is the standard choice. Outside health it does not fit so well.

SPIDER was proposed by Cooke, Smith and Booth (Qualitative Health Research, 2012). As per them, the intervention-versus-comparison shape of PICO bends qualitative questions out of shape. When your question is not fitting cleanly into any framework, mostly it is still too broad. To narrow it down, use our guide to the research problem statement.

Look at one made-up PICO question. Say you want to know whether SMS or app reminders improve medication adherence for adults with type 2 diabetes in low- and middle-income countries, compared to usual care. The criteria come straight from this question. The population is adults with type 2 diabetes. The intervention is any SMS or app reminder, and usual care is the comparison. Adherence can be by any validated measure. Only randomised trials from 2010 onwards are included. In the viva-voce somebody will ask why 2010, so keep a proper reason ready.

A protocol is many times five to ten pages long, and it is laid out using the PRISMA-P checklist. In it you fix the question, the criteria, the databases and a draft search string. You also fix the screening method, what data you will extract, the appraisal tool and how the results will be combined. After that comes registration. PROSPERO’s eligibility page takes only reviews whose outcomes have direct relevance to human health. If your review has already reached data extraction, it is not eligible (please read the current wording on that page before registering). Topics in management, education or engineering can go to OSF Registries, which gives a free systematic review template for any discipline. INPLASY takes scoping reviews also, but there is a fee, so check what they are charging now. The trade-offs are explained on our systematic review support page.

How do you build and document the search strategy?

Search a minimum of two or three databases that suit your subject. If you used only Google Scholar, a reviewer will not accept it.

• Health and nursing: PubMed/MEDLINE, Embase, CINAHL, Cochrane CENTRAL, and also trial registers like ClinicalTrials.gov and the Clinical Trials Registry – India (CTRI).
• Psychology and education: PsycINFO and ERIC, along with Scopus or Web of Science.
• Management, engineering, computing: Scopus and Web of Science, and then IEEE Xplore, ACM Digital Library or a business database, as per what your library subscribes to.

One string cannot be pasted as it is everywhere, so adapt it per database. Where a database has controlled vocabulary, use it (MeSH in PubMed, Emtree in Embase). Taking the diabetes example, the PubMed string could start like this:

("Diabetes Mellitus, Type 2"[Mesh] OR "type 2 diabetes"[tiab]) AND ("Text Messaging"[Mesh] OR SMS[tiab] OR mHealth[tiab]) AND ("Medication Adherence"[Mesh] OR adherence[tiab])

Don’t trust the string till you have run it. Take five studies you are sure belong in the review and check the string picks up all of them.

Grey literature means registers, conference abstracts, reports and theses, and it picks up what journals don’t carry. For topics on India, go through Shodhganga. Plenty of useful researches are lying in theses that never got published. Also do citation chasing from the studies you have included.

For every search, note down the database, platform, date, full string and number of records. PRISMA 2020 wants the full strategy for each database, register and website, and the date each was last searched. Remove duplicates before screening with a reference manager (our Zotero vs Mendeley comparison helps here).

How does screening by two reviewers work?

Screening happens in two rounds. In round one you read titles and abstracts and remove the clearly off-topic records. In round two each full text is checked against every criterion.

As per chapter 4 of the Cochrane Handbook, two people working independently is mandatory for eligibility decisions. At the title-and-abstract stage it is only desirable. Still, journal reviewers are mostly expecting two people in both rounds. One reviewer does not see the other’s decisions till both have finished. Where they disagree, they discuss about it or bring in a third person, many times the guide. Do a pilot on 50 to 100 records first, and tighten whichever criterion caused disagreement.

The bigger difficulty for most scholars is that no review team exists. Request a fellow scholar from your department to screen with you, and you screen their review in return. Mention their name in the methods. Writing that dual screening was done when it was not is misreporting, and one viva question is enough to expose it.

Note one reason for every full text you throw out, as the flow diagram needs those counts.

How do you extract data, assess risk of bias and synthesise?

Make the extraction form before anything else. Keep basic study details at the top. Then add setting and sample, and the intervention or exposure with what it was compared to. Then add each outcome, how that outcome was measured and the results you plan to combine. Test the form on two or three studies, and only after that use it for the remaining ones.

With every risk-of-bias judgement, copy one line from the paper which supports it. Then in front of the panel you can defend the rating without digging through PDFs.

In a meta-analysis the effect sizes are pooled together. This is sensible only when the studies are similar in population, intervention and outcome measure. When they are not similar, do a narrative synthesis. Group the studies by outcome or by theme and explain what patterns you see. For reviews that don’t pool, the SWiM guideline (Campbell and colleagues, BMJ, 2020) lists what should be reported. My suggestion is to plan for narrative synthesis from the start, and see the meta-analysis as something the data may or may not allow. Pooling five studies that don’t match gives very little.

What goes in the PRISMA 2020 flow diagram and checklist?

PRISMA 2020 (Page and colleagues, BMJ, March 2021) has replaced the 2009 statement. Its flow diagram is in three phases. Phase one is Identification. Here you show the records from databases, registers and other sources, and subtract those removed before screening (mostly duplicates). Phase two is Screening. It shows records screened and excluded, reports sought, reports not retrieved, reports assessed for eligibility, and the ones excluded with reasons. Phase three is Included, which gives the number of studies in the review. The older four-phase diagram, with its separate “Eligibility” band, is from 2009. On the PRISMA website there are four templates, for a new or updated review, with or without other sources like citation searching.

Here is an invented example to show how the sums work. Four databases give 2,640 records and registers give 48, so the total is 2,688. Minus 1,012 duplicates, 1,676 are left to screen. Out of these, 1,540 are excluded and 136 reports are sought. Six cannot be retrieved. So 130 get assessed, and 112 of them are excluded (41 wrong population, 29 wrong intervention, 34 not randomised, 8 no adherence outcome). Finally 18 studies remain. Examiners do check whether these sums add up.

The 27-item checklist covers seven sections, which are title, abstract, introduction, methods, results, discussion and other information. That last section is for registration, funding, competing interests and data availability. A few items have sub-items. There is also a separate 12-item abstract checklist. Fill the page numbers only against final version of the manuscript.

A scoping review is different. It maps what research is available on a topic. Answering one focused question is the work of a systematic review. It is reported using PRISMA-ScR (Tricco and colleagues, 2018), with 20 essential items and 2 optional items.

How long does a systematic review take, and where does it fit in a PhD?

Some supervisors are expecting it “by Monday”, but really it takes months. Borah and colleagues (BMJ Open, 2017) studied 195 PROSPERO reviews, where the teams had five authors on average. The mean they found was 67.3 weeks from registration to publication. If you are doing most of the work alone, don’t assume you will finish quicker.

I keep around one year in the plan for a focused review. Question and protocol take two months, and searching takes one month. Screening needs three or four months, extraction and appraisal two or three, and synthesis and writing another two or three. All this changes completely if the search brings 8,000 records and not 1,500.

A systematic review works well as a first paper. The reading has to be done anyway, and now it becomes a publication. Your literature chapter also gets a method which you can defend. Plus you can complete it before data collection starts. In the thesis it normally becomes chapter two, sometimes along with a shorter chapter on theory and context. See our guide to literature review formats.

• Before registering, put the question and protocol in front of your DC or RAC meeting. Then the question is fixed before screening begins.
• Maintain only one master screening file. Whatever numbers you show in the flow diagram should match with that file.
• Keep writing the methods while you work. After six months nobody remembers why Embase was dropped.
• Suppose submission happens more than about a year after the search. Run the searches again and give the new date.