Contents

Systematic Review and Meta-analysis

Fang-fang Zeng
Systematic Review and Meta-analysis

Contents
01 Introduction to Systematic Review

02 Essential Steps of Systematic Review

03 Introduction to Meta-analysis

04 Reporting Guidelines of Systematic Review

 Evidence Based Medicine
“the conscientious use, explicit, and judicious
use of current best evidence in making
decisions about the care of individual
patients.”

Sackett DL, et al. Evidence-based medicine: what it is and it isn’t. BMJ

1996;312:71-2.
Evidence Based Medicine
Patient
Values Medical Decision

Clinical Best
Expertise Research
Evidence
Evidence Pyramid

Goal: Use best available evidence

1. Introduction to Systematic Review

 Basic Concept of Systematic Review***

A systematic review is essentially a systematic investigation of existing

research data identified via a reproducible systematic search leading to data
abstraction, appraisal of methodological quality, clinical relevance and
consistency of published evidence on a specific clinical topic in order to provide
clear suggestions for a specific health care problem.
 Good Systematic Reviews Have Several Strengths
(1) Comprehensive search strategy.
(2) Explicit methodology.
(3) Emphasis on all clinically important outcomes.
(4) Limiting errors.
 Classification of Systematic Review

 A Qualitative Systematic Review

 The results of primary studies are summarized but not statistically
combined.
 A Quantitative Systematic Review
 Meta-analysis
 Uses statistical methods to combine the results of two or more studies.
 Difference between Systematic Review and Traditional Review

Feature Systematic review Traditional review

Review Focused, well-defined clinical question Question is usually broad and not
question formulated with PICO framework well defined

Protocol A-priori protocol is developed and No protocol

(optional) published
Methods Usually very well-defined and explicitly Usually not well-defined
stated with study inclusion and exclusion
criteria
Literature A good systematic review includes a well- Search strategy is usually not stated
search defined comprehensive search, without and the review is confined to well-
language or other restrictions known articles often supporting the
authors’ views
 Difference between Systematic Review and Traditional Review
Feature Systematic review Traditional review
Critical Internal validity of the individual studies Critical appraisal is usually not
appraisal included in a systematic review is vetted performed
by various tools such as Cochrane risk of
bias assessment tool

Synthesis Qualitative (sometimes quantitative with Usually qualitative summary

Meta-analysis) may answer a clinical
question which may not be answerable
by individual studies

Findings/conc Findings are reproducible Findings are not reproducible.

lusion Author's personal belief may
influence the overall conclusion of a
traditional review
Review

Systematic review

Meta-
analysis
Contents
01 Introduction to Systematic Review

02 Essential Steps of Systematic Review

03 Introduction to Meta-analysis

04 Reporting Guidelines of Systematic Review

2. Essential Steps of Systematic Review

 Research Question and a Protocol

 Comprehensive Search
 Selection of Studies
 Data Extraction
 Risk of Bias Assessment
 Data Synthesis
 Presenting Results
 Update the Systematic Review
2. Essential Steps of Systematic Review

 Research Question and a Protocol

PICOs Model
 Patient/Person: who does this relate to?
 Intervention (or cause, prognosis): what is the intervention or cause?
 Comparison (Is there something to compare the intervention to?)
 Outcome (What outcome are you interested in?).
 Study design
 Does Tamiflu help to prevent the flu?
 P: People with the flu
 I: Tamiflu
 C: No Tamiflu
 O: Reduction in flu symptoms and/or duration
 S: RCT
Whole Grain Intake and Mortality From All Causes,
Cardiovascular Disease, and Cancer: A Meta-Analysis of
Prospective Cohort Studies
I:（1）：wholegrain OR "whole grain" OR "whole-grain" OR "whole-
grains" OR "whole grains" OR bran OR germ OR oat OR barley OR
"brown rice" OR wheat OR maize OR millet OR quinoa OR amaranth
OR triticale* OR teff OR buckwheat
O:（2）：mortality OR "cause of death"
S:（3）：case reports[pt] OR comment[pt] OR editorial[pt] OR letter[pt]
OR review[pt]
（1）AND（2）NOT （3）
 Comprehensive Search

The electronic databases for medical literature

Chinese database
 MEDLINE/ PubMed 万方
 EMBASE CNKI
维普
 Cochrane Library Etc.
 etc.
 Selection of Studies

 Reviewers use predefined inclusion (and exclusion) criteria to select

studies for the review. The criteria refer to the study design, specific
population, intervention, comparison and outcome relevant for the
research question based on the PICO.
 Two researchers independently, and disputes settled between you, or
with discussion with a third person.
Example:
Candidate articles were included if they met all of the following
criteria: （1）They applied a prospective cohort design; （2）the
exposure included intakes of WG ingredients (with specified methods
for calculation) or WG foods (with specified food items); and （3）the
outcome included mortality from all causes, CVD, or cancer.
For multiple sets of results published from the same study population,
we used results based on the longest duration of follow-up.
 Data Extraction
Data extraction form Information to consider in data extraction
field
Reviewer Review author ID; date
identification
Study identification Study ID; report ID; citation; author contact details;
publication year; country; source of data

Methods Study design; setting; enrolment period; no. of

centers
Participant Total no.; age; sex; co-morbidity; ethnicity; no. lost
characteristics to follow-up
Disease characteristics Staging; severity; biological behavior; surgical
complexity; method of diagnosis

Interventionsa Surgical technique; surgeon experience/volume;

drug dose, route of delivery and length
Data extraction Information to consider in data extraction
form field
Diagnostic test Description of the reference standard, index test, comparator,
characteristicsb manufacturer; interpreter of diagnostic test
Prognostic factor Dose, level, duration of exposure; method of measurement
characteristicsc
Outcome Outcome definition (including unit of measurement, scale,
assessor, time point of measurement)
Results to Dichotomous outcomes: no. of events/no. of participants
include in a Continuous outcomes: mean value and SD in each
Meta-analysis intervention group
Time-to-event outcomes: HR (with 95% CI)
Diagnostic test performance outcomes: TP, FP, TN, FN
Risk of bias Cochrane risk of bias tool for RCTs or other tools such as
QUIPS and QUADAS-2
 Risk of Bias Assessment

 Newcastle-Ottawa for observational study

 Cochrane Collaboration's RoB Tool is used for RCTs
 QUADAS-2 is used for assessing RoB in diagnostic test
accuracy systematic reviews
 QUIPS is used for prognostic factor systematic reviews.
 Data Synthesis (only for meta)

 The data from the studies can be presented narratively and/or

statistically (a Meta-analysis)
 If studies are very heterogenous it may be most appropriate to
summaries the data narratively and not attempt a statistical (meta-
analytic) summary.
 Presenting Results

The PRISMA statement and this related article provides very

clear guidance on reporting of systematic reviews, including a
flow chart of studies included.
 Update the Systematic Review

 Updating systematic reviews is, in general, more efficient than

starting afresh when new evidence emerges.
 Decisions about whether and when to update a systematic review
are judgments made for individual reviews at a particular time.

 The decision needs to take into account whether the review

addresses a current question, uses valid methods, and is well
conducted; and whether there are new relevant methods, new
studies, or new information on existing included studies.
Contents
01 Introduction to Systematic Review

02 Essential Steps of Systematic Review

03 Introduction to Meta-analysis

04 Reporting Guidelines of Systematic Review

 3. Introduction to Meta-analysis
 Origin of ‘Meta-analysis’

“Meta”—— “more comprehensive”

1904： Pearson’s first use of data collection
1932： Sir Ronald Fisher reported on combining P values
Gene Glass, created original and groundbreaking contributions
1955： The first meta-analysis in medicine was published
 What is Meta-analysis***

A “quantitative method of combining the results of independent

studies (usually drawn from the published literature) and
synthesizing summaries and conclusions which may be used to
evaluate therapeutic effectiveness, plan new studies, etc, with
application chiefly in the areas of research and medicine.”
——The National Library of Medicine (NLM)
 Basic Process of Meta-analysis
 Formulation of the research question
 Searching for studies
 Selection of studies Same with systematic review
 Appraisal (quality assessment) of studies
 Data collection and extraction
 Data and analyses

 Result analyses and discussion

 Heterogeneity
Similarity ——‘homogeneity’
Dissimilarity —— ‘heterogeneity’

 Types of Heterogeneity：
Heterogeneity：

• Clinical heterogeneity

• Methodological heterogeneity

• Statistical heterogeneity
 Test of Homogeneity or Heterogeneity:

• Q-test : P<0.1 heterogeneous

P>0.1 similar

Q： the chi-squared statistic df ：its degrees of freedom

 A rough guide to interpret I2:

0% - 40%: probably not important

30% - 60%: moderate heterogeneity

50% - 90%: substantial heterogeneity

75% - 100%： considerable heterogeneity

 Statistical Models of Meta-analysis

Similar • Fixed effects model

Heterogeneous • Random effects model

 Pooling Data
 Statistical methods in meta-analysis
Type of data Effect measure Models Calculation methods
Dichotomous Odds ratio (OR) Fixed-effect models Peto
Fixed-effect models Mantel-Haenszel (M-H)
Random-effects models DerSimonian and Laird (D-L)
Risk ratio (RR) Fixed-effect models Mantel-Haenszel (M-H)
Random-effects models DerSimonian and Laird (D-L)
Risk difference (RD) Fixed-effect models Mantel-Haenszel (M-H)
Random-effects models DerSimonian and Laird (D-L)
Continuous Mean difference (MD) Fixed-effect models Inverse variance (IV)
Random-effects models DerSimonian and Laird (D-L)
Standardized mean Fixed-effect models Inverse variance (IV)
difference (SMD)
Random-effects models DerSimonian and Laird (D-L)
O-E and Odds ratio (OR) Fixed-effect models Peto
Variance
 Forest Plots

An example of forest plots of using results from a review of compression

stockings to prevent deep vein thrombosis (DVT) in airline passengers
 Reporting Biases
Funnel Plots

Symmetrical plot in the absence of bias Asymmetrical plot in the presence of bias
Contents
01 Introduction to Systematic Review

02 Essential Steps of Systematic Review

03 Introduction to Meta-analysis

04 Reporting Guidelines of Systematic Review

4. Reporting Guidelines of Systematic Review
PRISMA：Preferred Reporting Items of Systematic reviews and Meta-Analyses

PRISMA Flow Diagram

 PRISMA Checklist：

The 27 checklist items include：

 the title
 abstract
 methods
 results
 discussion
 funding

(http://www.prisma-statement.org/)