Meta Analysis

SAIRA TARIQ
DEPT OF COMMUNITY MEDICINE, KEMU
Learning Outcomes
By the end of this presentation, the participants shall be able to understand :
- methods used to produce a rigorous meta-analysis
- aspects of presentation and interpretation of meta-analysis
- reporting of Systematic Reviews and Meta Analysis using PRISMA Checklist
INTRODUCTION
 Evidence-based medicine may be defined as the systematic, quantitative,
preferentially experimental approach to obtaining and using medical
information.
 Systematic reviews and meta-analyses are a critical foundation of evidence-
based medicine
 are considered essential tools for synthesizing evidence needed to inform
clinical decision making and policy.
 Systematic reviews are an objective, reproducible method to find answers to a
research question by collecting all available studies related to that question and
reviewing and analyzing their results.
 Systematic reviews sometimes, use statistical techniques to combine data
from the examined individual research studies, and use the pooled data to come
to new statistical conclusions.
 This is called meta-analysis, and it represents a specialized subset of
systematic reviews.
Meta-analysis is a quantitative, formal, epidemiological study design used to
systematically assess the results of previous research to derive conclusions about that
body of research. Typically, but not necessarily, the study is based on randomized,
controlled clinical trials.

Conclusions produced by meta-analysis are statistically stronger than the analysis of any
single study, due to increased numbers of subjects, greater diversity among subjects, or
accumulated effects and results.

Meta-analyses are the most frequently cited form of clinical research

History
Karl Pearson was the first researcher to use formal techniques to pool data from
different studies in 1904

He synthesized data from several studies on typhoid vaccination

The rationale provided was :

“ Many of the groups are far too small to allow a definitive opinion being formed
at all , having regard to the size of the probable error involved”
In 1952 Hans Eysenck concluded that there were no favorable effects of
psychotherapy , initiating a raging debate which 25 years of evaluation research
and hundreds of studies failed to resolve
In 1978 to prove Eysenck wrong, Gene Glass statistically aggregated the findings
of 375 psychotherapy outcome studies

Glass coined the term ‘Meta-analysis”

Hierarchy of Evidence in Quantitative
Studies
Significance
Systematic Reviews and meta analysis are considered the highest level of
evidence for informing clinical decisions.

In the hierarchy of evidence , Meta analysis, Systematic reviews and RCTs are at
the highest levels in the evidence pyramid.

Recently the exponential increase in the number of meta analysis studies has
raised the issue of quality and reliability of publications.
 Rationale
Meta-analysis are used for the following purposes:
• To establish statistical significance with studies that have conflicting results
• Increase statistical power and precision to detect an effect
• Identify heterogeneity in effects among multiple studies and where appropriate, provide summary
measures
• Develop , refine and test hypothesis
• Reduce the subjectivity of study comparisons by using systematic and explicit comparison procedure
• Identify data gaps in knowledge base and suggest direction for future research
• Calculate sample size for future studies
Meta-analyses have become common in the social and biomedical sciences.
However, some challenge the validity of meta-analysis, arguing that combining data from
disparate studies produces misleading or unreliable results.

Meta-analyses can be a challenging undertaking, requiring tedious screening and statistical

understanding.

Software packages supporting clinical meta-analyses include the Excel plugins MetaXL (
Barendregt and Doi, 2009) and Mix 2.0 (Bax, 2016), Revman (Cochrane Collaboration, 2011),
Comprehensive Meta-Analysis Software [CMA (Borenstein et al., 2005)], JASP (
JASP Team, 2018) and MetaFOR library for R (Viechtbauer, 2010).
Steps in Meta Analysis
- Define research question and specific hypothesis
According to the PRISMA statement, an explicit statement of questions being addressed with
reference to participants, interventions, comparisons, outcomes and study design (PICOS) should be
provided
- Define criteria for inclusion and exclusion of studies
Inclusion criteria are ideally defined at the stage of initial development of the study protocol. The
rationale for the criteria for study selection used should be clearly stated.
- Locate research studies
Published papers and abstracts are identified by a computerized literature search of electronic
databases that can include PubMed , ScienceDirect , Scirus , ISI Web of Knowledge , Google Scholar
and CENTRAL (Cochrane Central Register of Controlled Trials, PRISMA statement recommends that a
full electronic search strategy for at least one major database to be presented
Steps in Meta Analysis
- Determine study eligibility for inclusion
Before study selection , quality assessment protocol and data forms should be developed. The goal
of this process is to reduce the risk of bias in the estimate of effect.
- Classify and code important study characteristics ( eg. Sample size, length of follow up, definition of
outcome, drug brand or dose )
- Select or translate results from each study using a common metric
- Aggregate findings across studies , generating weighted pooled estimates of effect size
The most common measures of effect used for dichotomous data are the risk ratio (also called
relative risk) and the odds ratio.
The dominant method used for continuous data are standardized mean difference (SMD) estimation.
Methods used in meta-analysis for post hoc analysis of findings are relatively specific to meta-analysis
and include heterogeneity analysis, sensitivity analysis, and evaluation of publication bias.
- Evaluate statistical homogeneity of pooled studies
The greatest benefit of conducting meta analysis is to examine sources of heterogeneity, if
present, among studies.
If heterogeneity is present, the summary measure must be interpreted with caution . When
heterogeneity is present, one should question whether and how to generalize the results.
- Perform sensitivity analysis to assess the impact of excluding or down weighting unpublished
studies or studies of lower quality
Sensitivity analyses are used to examine the effects of studies identified as being aberrant
concerning conduct or result, or being highly influential in the analysis.
Presentation of Results
The flow-diagram depicts the flow of information through the different phases
of a systematic review or meta-analysis.
It maps out the number of records identified, included and excluded, and the
reasons for exclusions.
The results of meta-analyses are often presented in a forest plot, where each study is shown
with its effect size and the corresponding 95% confidence interval
Biases in Meta Analysis

Publication bias
- The most common type of reporting bias in meta-analyses.
- This refers to the distortion of meta-analysis outcomes due to the higher likelihood of
publication of statistically significant studies rather than non-significant studies.
- In order to test the presence or absence of publication bias, first, a funnel plot can be used
Studies are plotted on a scatter plot with effect size on the x-axis and precision or total sample
size on the y-axis.
- If the points form an upside-down funnel shape, with a broad base that narrows towards the
top of the plot, this indicates the absence of a publication bias .
- On the other hand, if the plot shows an asymmetric shape, with no points on one side of the
graph, then publication bias can be suspected
Guidelines for Reporting Meta Analysis
Since 1999, various papers have presented guidelines for reporting meta-analyses of RCTs.
Following the Quality of Reporting of Meta-analyses (QUORUM) statement , and the
appearance of registers such as Cochrane Library’s Methodology Register, a large number of
systematic literature reviews have been registered.
MOOSE Guidelines : Meta Analysis of Observational Studies in Epidemiology
In 2009, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)
statement was published, and it greatly helped standardize and improve the quality of
systematic reviews and meta-analyses
http://prisma-statement.org/PRISMAStatement/Checklist
The PRISMA checklist
Title (1) Identify the report as a systematic review, meta-analysis, or both.
Eg. Mortality in randomized trials of antioxidant supplements for primary and secondary
prevention: systematic review and meta-analysis
Structured summary (2) Provide a structured summary including, as applicable: background;
objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal
and synthesis methods; results; limitations; conclusions and implications of key findings; funding
for the systematic review; and systematic review registration number
Introduction (3): Rationale - Describe the rationale for the review in the context of what is
already known
(4) Objectives - Provide an explicit statement of questions being addressed with reference to
participants, interventions, comparisons, outcomes, and study design (PICOS)
Checklist contd.
Methods
(5) : Protocol and registration : Indicate if a review protocol exists, if and where it can be
accessed (e.g., Web address) and, if available, provide registration information including the
registration number
(6) : Eligibility Criteria: Specify study characteristics (e.g., PICOS, length of follow-up) and report
characteristics (e.g., years considered, language, publication status) used as criteria for eligibility,
giving rationale.
(7): Information Sources : Describe all information sources in the search (e.g., databases with
dates of coverage, contact with study authors to identify additional studies) and date last
searched.
(8) : Search : Present the full electronic search strategy for at least one major database, including
any limits used, such that it could be repeated.
Checklist contd.
(9) : Study Selection: State the process for selecting studies (i.e., for screening, for determining
eligibility, for inclusion in the systematic review, and, if applicable, for inclusion in the meta-analysis).
(10): Data Collection Process: Describe the method of data extraction from reports (e.g., piloted
forms, independently by two reviewers) and any processes for obtaining and confirming data from
investigators.
(11): Data items : List and define all variables for which data were sought (e.g., PICOS, funding
sources), and any assumptions and simplifications made.
(12): Risk of bias in individual studies: Describe methods used for assessing risk of bias in individual
studies (including specification of whether this was done at the study or outcome level, or both),
and how this information is to be used in any data synthesis
(13): Summary Measures: State the principal summary measures (e.g., risk ratio, difference in
means).
Checklist contd.
(14): Planned methods of analysis: Describe the methods of handling data and combining results of studies, if
done, including measures of consistency (e.g., Higgins I2 ) for each meta-analysis.
(15) : Risk of bias across studies: Specify any assessment of risk of bias that may affect the cumulative
evidence (e.g., publication bias, selective reporting within studies )
(16): Additional analyses: Describe methods of additional analyses (e.g., sensitivity or subgroup analyses,
meta-regression), if done, indicating which were pre-specified.
Results
(17): Study selection: Give numbers of studies screened, assessed for eligibility, and included in the review,
with reasons for exclusions at each stage, ideally with a flow diagram
(18): Study characteristics: For each study, present characteristics for which data were extracted (e.g., study
size, PICOS, follow-up period) and provide the citation.
(19) : Risk of bias within studies Present data on risk of bias of each study and, if available, any outcome-level
assessment (see Item 12).
Checklist contd.
(20): Results of individual studies: For all outcomes considered (benefits and harms), present, for each
study: (a) simple summary data for each intervention group and (b) effect estimates and confidence
intervals, ideally with a forest plot
(21): Syntheses of results: Present the main results of the review. If meta-analyses are done, include for
each, confidence intervals and measures of consistency
(22): Risk of bias across studies: Present results of any assessment of risk of bias across studies (see Item
15)
(23) : Additional analyses: Give results of additional analyses, if done (e.g., sensitivity or subgroup
analyses)
Discussion
(24): Summary of evidence: Summarize the main findings, including the strength of evidence for each
main outcome; consider their relevance to key groups (e.g., health care providers, users, and policy
makers).
Checklist contd.
(25): Limitations: Discuss limitations at study and outcome level (e.g., risk of bias), and at review
level (e.g., incomplete retrieval of identified research, reporting bias)
(26) : Conclusions: Provide a general interpretation of the results in the context of other
evidence, and implications for future research.
(27): Funding: Describe sources of funding or other support (e.g., supply of data) for the
systematic review; role of funders for the systematic review
CONCLUSION
Meta-analysis can be a powerful tool to combine results from studies with similar design and
patient populations that are too small or underpowered individually to demonstrate a
statistically significant association.
As with clinical trials, having an appropriate study question and design are essential when
performing a meta-analysis to ensure that there is internal validity and that the results are
clinically meaningful
As our understanding of evidence-based medicine increases and its importance is better
appreciated, the number of systematic reviews and meta-analyses will keep increasing.
However, indiscriminate acceptance of the results of all these meta-analyses can be dangerous,
and hence, we recommend that their results be received critically on the basis of a more
accurate understanding.
Bibliography
- Haidich A B. Meta-analysis in medical research. Hippokratia. 2010 Dec; 14(Suppl 1): 29–37.
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for
systematic reviews and meta-analyses: the PRISMA. J Clin Epidemiol. 2009 Oct; 62(10):1006-12.