CLINICAL PERSPECTIVE

A Practical Guide to Meta-Analysis

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD
From the Section of Plastic Surgery, Department of Surgery, University of Michigan Health System,
Ann Arbor, MI; and the Center for Statistical Consultation and Research, University of Michigan, Ann Arbor, MI.

The wealth of medical research published on a yearly basis demands methods to summarize the
literature. Narrative or expert reviews were the traditional method to provide this summary;
however, biases associated with narrative reviews raise questions regarding whether this process
provides sufficient evidence to guide practice. Meta-analysis is becoming increasingly popular
because it can combine results from similar studies to calculate an overall estimate of a treatment
effect. Although meta-analysis has its limitations, a thoughtful and well-planned analysis is a
valuable tool in providing a high level of evidence. We discuss the steps needed to perform a
meta-analysis as a guideline for those interested in pursuing this type of research. (J Hand Surg
2006;31A:1671–1678. Copyright © 2006 by the American Society for Surgery of the Hand.)
Key words: Guide, meta-analysis, clinical trials, hand surgery, outcomes.

iven the large amount of medical literature alone, it has too small a sample size to have undeniable

G published yearly, physicians and researchers

alike need methods to summarize the infor-
mation presented. Narrative reviews of the literature,
statistical significance. Because of small sample sizes,
some RCTs are unable to detect a statistically signifi-
cant treatment effect when in fact a clinically meaning-
usually performed by experts in the field, are the ful effect exists. Meta-analysis is conducted when run-
initial approach to summarizing research. The biases ning large expensive RCTs is not possible. For
associated with narrative reviews have been well example, a clinical trial to evaluate antibiotics in elec-
documented, which led to the development of a more tive hand surgery may be prohibitively expensive be-
systematic method of summarizing research. cause the infection rate is quite low and an extremely
A systematic review is a more scientific method of large sample size would be needed to show an effect.
summarizing literature because specific protocols are Meta-analysis also is useful when performing certain
used to determine which studies will be included in the RCTs would be considered unethical. A clinical trial to
review. In a systematic review, a research question is compare the effectiveness of thumb replantation versus
identified and a comprehensive search of the literature amputation may be unethical because of the entrenched
is performed. Studies are included or excluded accord- teaching that the thumb is indispensable. Although the
ing to predetermined metrics and then assessed for information provided by this trial would be im-
biases and quality of research. Because a specific pro- mensely important to provide a high level of evi-
tocol is used and reported in a systematic review, it is dence to support this labor-intensive procedure,
possible for others to reproduce the process. When data many surgeons will refuse to participate in this trial
from a systematic review are summarized using statis- for ethical reasons. An observational study may be an
tical techniques, it is considered a meta-analysis. alternative design for such a study.
Meta-analysis is a quantitative method of combining Although ideally a meta-analysis should include only
results from multiple studies to obtain a more precise RCTs, a random sample of meta-analysis studies iden-
estimate of the effect of interest. A recent study found tified through a MEDLINE search found that half were
that meta-analyses are more likely to be cited than based on epidemiologic or observational studies.3
randomized clinical trials (RCTs) or other study de- Meta-analysis can be performed using data from obser-
signs.1 In addition, meta-analyses and systematic re- vational studies, but the inherent biases in these studies
views of RCTs are considered to have the highest level make meta-analysis complicated. RCTs are considered
of evidence in a grading system for evidence-based to give unbiased estimates of treatment effects because
guidelines.2 Traditionally, meta-analysis was restricted of the strict protocols for inclusion of study subjects and
to RCTs. In many instances, when each trial stands the randomization of treatments. On the other hand,

The Journal of Hand Surgery 1671

1672 The Journal of Hand Surgery / Vol. 31A No. 10 December 2006

observational studies are prone to give biased effect ity of Reports of Meta-analysis [QUOROM])15 and
estimates because of confounding factors such as age, observational studies (Meta-analysis of Observational
race, gender, and smoking status. Confounding factors Studies in Epidemiology [MOOSE]).16 These guide-
are factors that are associated with the outcome of lines were developed to provide proper procedures for
interest and the exposure of interest (such as types of conducting a meta-analysis and to standardize the meth-
surgical procedures). If confounding factors are not ods of reporting a meta-analysis. By using these 2
controlled for in the analysis, it will result in a biased protocols as our guide, we discuss the steps necessary to
estimate. In observational studies these confounding perform a meta-analysis. These steps include the fol-
factors usually are controlled for in the analysis, but lowing: (1) define the research question, (2) perform the
residual confounding factors still can be present. Com- literature search, (3) select the studies, (4) extract the
bining the results from biased observational studies in a data, (5) analyze the data, and (6) report the results.
meta-analysis will produce a more precise but biased
estimate.3 Because of these problems, some investiga- Define the Research Question
tors4 have suggested that meta-analysis should be aban- A meta-analysis begins with a question. Common
doned for observational studies. Others3 have suggested questions addressed in meta-analyses are whether
that meta-analysis in observational studies can be useful to one treatment is more effective than another or if
explore sources of heterogeneity among study results. exposure to a certain agent will result in disease.
The limitations of meta-analysis have been discussed Before beginning an analysis, the investigators need
extensively.4,5– 8 Some of the criticisms include incom- to define the problem or question of interest. For
plete search of the literature, bias in selection and in- example, does one surgical technique result in a
clusion of studies, incorrectly combining heterogeneous lower nonunion rate and fewer complications as
data, use of meta-analysis for observational studies, compared with another technique?
difficulty in extrapolating results to the clinical setting, The investigators also should have a good under-
and lack of knowledge of the subject material. Despite standing of the problem and the subject matter.5 This
these concerns, the push to summarize research means avoids the issue of uninformed researchers churning out
this statistical method is here to stay. The number of meta-analyses in a mechanized way.17 The study pop-
meta-analyses reported in the literature has been in- ulation baseline data (eg, age, race, gender, diagnosis,
creasing over time. We performed a MEDLINE search length of illness), the study outcomes (eg, grip strength,
using the key word “meta-analysis.” In 1988, 62 reports key pinch, range of motion, complication rate), treat-
were identified. By 2000, the number of meta-analyses ment or intervention (eg, surgical technique), and type
identified had increased to 490. A further review of the of study designs to be used (eg, restricted to RCTs or
reports published in 2000 found that only 6 of the include observational studies such as prospective or
meta-analyses were in the surgical field. The majority retrospective studies) also should be defined.16 For ex-
of meta-analyses were concentrated in the fields of ample, a meta-analysis was conducted to address the
cancer and cardiovascular disease research; however, a question of whether 1 of 2 surgical techniques (external
MEDLINE search of the Journal of Hand Surgery fixation vs plate osteosynthesis) is preferable to treat
(American and British volumes) identified 9 articles unstable distal radius fractures.13 In this analysis, RCTs
with the key word “meta-analysis.” After further re- and observational studies were included. A clinical def-
search, 3 studies were found to be systematic reviews inition of unstable distal radius fractures was given.
and 1 was a commentary article, leaving only 5 meta- Outcomes of interest included grip strength, range of mo-
analysis studies.9 –13 Because of the increasing interest in tion data (eg, wrist flexion, wrist extension, supination, and
this method, this article offers a practical guide to meta- so forth), radiographic outcomes, and complications.
analysis to help investigators plan such a study and to
inform readers in interpreting this type of publication. Perform the Literature Search
Once the research question has been defined, a system-
Getting Started atic search of the literature can begin. This is a critical
Meta-analysis is not just a simple method of combining step in the meta-analysis and often the most difficult
data. As Berman and Parker14 stated, “meta-analysis is part. The initial search of the literature should be broad
neither quick nor easy.” For the results of a meta- so that as many studies as possible are gathered. During
analysis to be meaningful, a great deal of thought and the selection phase, some of the initial studies will be
planning are needed. Protocols for the reporting of weeded out using the inclusion criteria. The literature
meta-analysis results were developed for RCTs (Qual- search begins with searching electronic databases of
 Chung, Burns, and Kim / Guide to Meta-Analysis 1673

published studies such as MEDLINE, EMBASE, and ies are poorly designed and therefore should be ex-
CINAHL. MEDLINE is maintained by the National cluded from the analysis. Easterbrook et al24 found no
Library of Medicine and contains more than 13 million evidence that studies with positive results were better
citations dating back to 1966.18 EMBASE is a database designed than studies with null results.
produced by the publisher Elsevier BV and contains The majority of meta-analyses reported in English-
data from 1974 to the present.19 Although EMBASE language journals restrict the analysis to reports printed
and MEDLINE overlap in their coverage of the litera- in English. The restriction of the literature search to
ture, EMBASE has better coverage of European jour- articles published in the English language is another
nals.20 CINAHL covers literature related to nursing and possible source of bias in a meta-analysis. For example,
allied health from 1982 to the present.21 The research- an article reported that RCTs with significant results
ers should search more than just MEDLINE to ensure a were published more often in English-language versus
comprehensive search. For example, a report found that German-language journals.30 Studies with nonsignifi-
approximately only half of all RCTs presented as ab- cant results are more likely to be reported in a local
stracts are subsequently published on MEDLINE.22 It is journal. If non-English articles are excluded, which are
necessary to use other sources to access many of these more likely to have nonsignificant results, the summary
unpublished studies. A good source for unpublished estimate will be exaggerated erroneously. In addition to
clinical trials is the Cochrane Central Register of Con- the difficulty and expense of translating non–English-
trolled Trials, which is a database of controlled trials. language reports, it is assumed that the quality of stud-
The database was set up to provide a source of data for ies printed in English is better; however, a study com-
systematic reviews and contains more than 300,000 paring English versus non-English RCTs found no
reference to RCTs.23 Other suggestions for locating differences in the completeness of reporting.31
studies include searching reference lists from the gath- Another form of bias in meta-analysis is citation
ered reports, manually searching journals with lists of bias. Although it is recommended that reference lists
abstracts presented at meetings, or searching on the be used to identify studies for a meta-analysis, it also
Internet. Contacting experts in the field or networking could introduce bias. For example, studies showing
with colleagues also could be a source of studies, al- significant results are more likely to be cited and thus
though this mode of data gathering is seldom done. more easily identified than studies with nonsignifi-
Publication bias is a well-documented problem with cant results.27 Studies with significant results are
meta-analysis. Publication bias occurs because some cited more frequently, increasing the probability of
RCTs and observational studies will not result in a being identified. If this citation bias occurs, the over-
publication and therefore are not discovered in a liter- all estimate will be exaggerated.
ature search. One of the issues is that studies with
positive results are more likely to be published.24 –27 Select the Studies
Studies with positive results also take less time to be Once the literature search is complete, it is time to select
published and therefore are more likely to be found in which studies to include in the meta-analysis. The in-
a literature search.26 Easterbrook et al24 found that clusion and exclusion criteria for studies needs to be
observational studies are even more likely than RCTs to defined at the beginning, during the design stage of the
be published if they have positive results. It is expected meta-analysis. Factors determining inclusion in the
that the inclusion of more studies with positive results analysis are study design, population characteristics,
will erroneously inflate the summary estimate. This was type of treatment or exposure, and outcome measures.14
shown in a study that examined meta-analyses that The inclusion and exclusion criteria should be part of
originally included unpublished studies (also known as the meta-analysis protocol. One should keep track of
grey literature).28 McAuley et al. found that when they the studies included and excluded at each step of the
repeated the meta-analysis after excluding the grey lit- selection process to document the selection process.
erature, the summary estimate was inflated by an aver- The QUOROM guidelines for reporting a meta-analysis
age of 12%. Furthermore, a report by Sutton et al29 requests that investigators provide a flow diagram of the
estimated that half of the meta-analyses in the Cochrane selection process.15 The flow diagram lists the number
Database of Systematic Reviews had some level of of studies excluded and included at each stage of the
publication bias.29 If journals are more likely to publish selection process and the reasons for exclusion. An
positive results, investigators may be less inclined to example of a flow diagram is shown in Figure 1. The
submit articles for publication if they have negative selection process involves reviewing the titles and ab-
findings.24,25 It often is assumed that unpublished stud- stracts of all articles identified through the literature search.
1674 The Journal of Hand Surgery / Vol. 31A No. 10 December 2006

Primary search quality scores.16 Sensitivity or subgroup analysis

(N= )
allows comparisons between studies of different
quality.35 For example, studies can be separated into
Secondary search
(N= ) high versus low quality and then the meta-analysis
can be repeated for each group. Results then can be
Title search
(N= )
compared between the 2 groups. A method that is
being used increasingly is meta-regression. Quality
Abstract search
scores or some measure of study quality (eg, assign-
(N= ) ment to a treatment group) are entered into a regres-
sion model as an explanatory variable.32 This method
Papers read for content allows the researchers to estimate the effect of qual-
(N= )
ity on the results of the meta-analysis.
Number of studies
rejected
and reasons for
rejection Extract the Data
(N= )
The type of data to be extracted from each study should
Studies with usable information be determined in the design phase and a standardized
(N= )
form is constructed to record the data. Examples of data
Figure 1. Example of a flow diagram of the selection process commonly extracted include study design, descriptions
for studies to be included in the analysis. of study groups (eg, number in each group, age, gen-
der), diagnostic information, treatments, length of fol-
Many of the studies will be excluded at this stage based on low-up evaluation, and outcome measures. Two inde-
the exclusion criteria. The remaining studies will be read to pendent reviewers will be instructed on the appropriate
determine their suitability for inclusion. data to collect. For example, how will age be recorded
The validity of a meta-analysis depends on the on the abstract form? Will the standard deviation or
quality of the studies included, and an assessment of standard error be used in the analysis? If data are
quality is a necessary part of the process. The re- missing, they should be recorded on the form. If too
searcher wants to include as many studies as possi- much data are missing, the study may need to be ex-
ble, but reduce the number of studies with low- cluded. It is recommended that the reviewers be blinded
quality data; however, restricting the meta-analysis to the investigators’ names but it is not essential.14,36
to only perfect studies may leave the researcher with Data to be extracted are identified before beginning the
little data.32 A variety of checklists and scales have meta-analysis to avoid data dredging or a fishing expe-
been developed to assess quality in RCTs.33 Check- dition.
lists provide guidelines as to what should be reported The difficulty with data extraction is that studies
in an RCT, whereas scales are a way of quantifying often use different outcome metrics, which make com-
the level of bias in an RCT. For example, a scale will bining the data awkward. The data should be converted
assign a score based on a specific characteristic of an to a uniform metric for pooling. For example, data
RCT (eg, presence of adequate concealment of patient reported may be continuous (eg, blood pressure) or
assignment to treatment groups), but a checklist does binary (eg, high blood pressure vs low blood pressure).
not assign scores. Although quality needs to be assessed A meta-analysis estimating the effect of a medication
in some way, caution should be used when using these on blood pressure may find some studies reporting
scales or checklists.33,34 The reasons for the inclusion blood pressure as a continuous outcome whereas in
of items in a scale or checklist often are not given and other studies the outcome is reported only as high or
the score assigned to scales can be arbitrary.33 The low blood pressure. In this case it would be necessary to
relatively imprecise scoring scheme in some of the convert continuous blood pressure measurements into
scales may change the results of a meta-analysis.34 categories of high or low blood pressure to standardize
There are several options available to deal with the data into one format. Similarly, some studies report
study quality once it has been ascertained. A cut-off the standard deviation and others report standard error.
value for the quality score can be used to exclude or Again, it is necessary to convert one into the other to
include studies.32,35 Another choice is to use the make the data uniform.
quality scores to weight study results in the analysis. Although it is difficult to resist combining the
MOOSE reporting guidelines, however, recommend data,3 if combining data is not possible because dif-
using a sensitivity analysis rather than weighting for ferent metrics are used then it is best to leave the
 Chung, Burns, and Kim / Guide to Meta-Analysis 1675

analysis as a systematic review. For example, Ger- heterogeneity exists. For example, the studies used in
ritsen et al37 described their attempt to perform a the meta-analysis may in reality vary considerably, but
meta-analysis of carpal tunnel syndrome. The inves- the low power makes the heterogeneity test nonsignif-
tigators decided to report their results as a systematic icant. This would lead the researcher to the incorrect
review because of a lack of common outcomes and conclusion that the amount of variation among the
inadequate reporting of data in the individual studies. studies is low. The best choice may be to always use the
random-effects model or to use both models and com-
Analyze the Data pare the results. Statistical packages are available to
A statistician who is familiar with meta-analysis calculate summary estimates using either model. If het-
should be consulted to help plan this type of project erogeneity can be explained, then it should be included
and to participate in analyzing the data. We briefly in the model. For instance, we may observe that some
discuss aspects of data analysis for a meta-analysis of the variation in studies can be explained by gender.
but more detailed instructions can be found else- In that case, separate summary estimates can be calcu-
where.32,38 – 40 A meta-analysis calculates a weighted lated for males and for females. Or, meta-regression
average of the study effect that is pooled from the models can be used to explain heterogeneity, but a large
selected studies. The weight is directly proportional number of studies are needed when investigating mul-
to the precision of the effect estimate and usually the tiple effects.
inverse of the variance (square of the standard error)
of the effect estimate.14,32 Therefore, larger studies Report the Results
will have more influence over the summary estimate Detailed guidelines for the reporting of meta-analyses
than smaller studies.36 A summary estimate is calcu- for RCTs were described in the QUOROM statement.15
lated by multiplying each study’s weight by its effect Similar guidelines were developed for observational
estimate and adding these values together. This sum studies by the MOOSE group.16 These articles should
then is divided by the sum of the study weights. For be consulted during the design phase to ensure that
example, in the meta-analysis of distal radius frac- these reporting procedures are used and that proper data
tures, summary means for grip strength and range- are collected and presented in the report. We briefly
of-motion data were calculated by averaging the raw describe some of the items that should be reported.
or individual study means weighted by the inverse of Similar to a research report, a meta-analysis report
the standard error of the raw means.13 should include a title, abstract, and introduction, and
There are 2 statistical models used in a meta-analy- methods, results, and discussion sections. The title
sis: fixed effects and random effects. The fixed-effects should identify the report as a meta-analysis. The
model assumes that the true effect of treatment is the introduction should indicate the clinical question of
same for every study. Because there is no heterogeneity interest, the hypothesis being tested, the types of
between study results, only within-study variability is treatment or exposure being studied, the study de-
taken into account. Given the degree of variation or signs to be included, and a description of the study
heterogeneity among studies, this assumption may be population. The methods section should describe the
unreasonable. The random-effects model is often more literature search, specifically the databases used, and
realistic because it assumes that the true effect estimate if the search was restricted in any way (eg, English
for each study does vary. Sources of variation may language only). The selection process for articles,
include differences in patient population or treatment quality assessment, methods of data abstraction, and
methods. The random-effects model will produce an synthesis also should be described in this section.
estimate with wider confidence intervals, but the sum- The results section should include a flow chart of
mary estimates for both models will be similar if there studies included in each step of the selection process,
is not a great deal of heterogeneity among studies. A a figure displaying the results from each individual
statistical test for heterogeneity can be used, but this test study such as a forest plot, results of heterogeneity
has low statistical power in most cases.32 Power refers testing, overall summary statistic and its 95% confi-
to the ability of a statistical test to reject the hypothesis dence interval, and results of a sensitivity analysis
being tested (null hypothesis) when it is false. The null and meta-regression, if performed. A forest plot
hypothesis states that there is no heterogeneity or vari- shows each individual study and the summary esti-
ation among the studies. Low power for the heteroge- mate in a single graph. For sensitivity analysis, sev-
neity test means that we are unable to reject the null eral features of a meta-analysis can be altered to
hypothesis of no heterogeneity even when important assess the robustness of the results, such as excluding
1676 The Journal of Hand Surgery / Vol. 31A No. 10 December 2006

Internal Fixation diagram is the results from the individual studies

(N = 14 Studies) used in the meta-analysis of unstable distal radius
Keating 1994
Trumble 1994 fractures (Fig. 2).13 The mean grip strength for those
Putnam 1997
Fitoussi 1997 with internal fixation as compared with those with
Carter 1998
Doi 1999
external fixation is shown. In this figure, the mean
Jakob 2000
Campbell 2000
grip strength for each study and the overall estimate
Schneeberger 2001 are plotted for internal and external fixation sepa-
Jupiter 2002
Orbay 2002 rately. The reader can see that external fixation stud-
Ring 2004
Beharrie 2004
ies have more heterogeneity and wider confidence
Orbay 2004
intervals than internal fixation studies.
Combined A funnel plot is used as a way to assess publication
A .2 .3 .4 .5 .6 .7 .8 .9 1.0 bias in a meta-analysis. The funnel plot is a scatter
plot of each study’s effect estimate (eg, odds ratio or
External Fixation
(N = 16 Studies) mean difference) on the x-axis against a measure of
Cooney 1980 the study’s precision on the y-axis.32,41 The overall
Riis 1989
Edwards 1991 sample size can be used on the y-axis but often an
Sanders 1991
Steffen 1994 inverse of the standard error is used.29 If a publication
Trumble 1994
Porter 1994 bias is not present, the plot will resemble an inverted
McQueen 1996
Trumble 1998 funnel. Large studies should have smaller variation and
McQueen 1998
Rikli 1998 therefore a more precise effect estimate, whereas small
Doi 1999
Herrera 1999
Widman 2002
studies should have larger variation and therefore a
Krishnan 2003
Harley 2004
less-precise estimate. We expect the effect estimates for
Combined
small studies will have wider scatter at the bottom of the
.2 .3 .4 .5 .6 .7 .8 .9 1.0
plot and larger studies will have less scatter at the top of
B Grip Strength (Fraction of Uninjured Side) the plot. If small studies with negative or null results
tend not to be published, we would see asymmetry in
Figure 2. Example of a forest plot. Results of mean grip
strength and associated confidence intervals for individual the funnel plot from the left bottom of the plot contain-
(A) internal and (B) external fixation studies included in the ing few or no data points. Figure 3 shows examples of
meta-analysis of unstable distal radius fractures by Margaliot funnel plots from hypothetic meta-analyses. In these
et al.13 (B) External fixation studies have more heterogeneity examples, the x-axis contains the odds ratio from each
and wider confidence intervals than (A) internal fixation study included in the meta-analysis, plotted against the
studies.
inverse of the standard error on the y-axis. When the
shape of the funnel plot is symmetric, the literature
questionable or unpublished studies. The sensitivity search is considered unbiased (Fig. 3A). On the other
analysis may include an analysis weighted by a qual- hand, if fewer studies with nonstatistically significant
ity score for each study. The discussion section odds ratios were included in the literature search, it
should summarize the key findings and identify pos- would result in an asymmetric funnel plot (Fig. 3B).
sible sources of bias and heterogeneity. Funnel plots can be inspected visually but interpretation
A forest plot, the figure with the effect estimate can differ from person to person. Statistical tests such as
from each study and their associated confidence in- the rank correlation test developed by Begg and Ma-
tervals along with the summary estimate, is an im- zumdar42 are available to assess the symmetry of the
portant part of the report. Studies can be grouped by plot. The correlation test, however, should be used with
size or by other study characteristics such as year of caution in small meta-analyses because the power of the
publication. It allows the reader to observe the het- test depends on the number of studies included.42 In a
erogeneity of the studies included. If the confidence small meta-analysis (⬍ 25 studies), the correlation test
intervals for effect estimates are not overlapping, will have low statistical power so a nonsignificant test
indicating a great deal of study variation, a meta- will not rule out bias in the literature search.
analysis may not be appropriate. In this case, it is A meta-analysis is a statistical method of combining
necessary to explore the reasons for the variation results from multiple studies to determine the overall
among the studies, which may lead to the discovery impact of a treatment or exposure. If performed prop-
of associations between the study design or patient erly, using the steps outlined in this article, a meta-
groups and the study outcome. An example of such a analysis can be a powerful research tool. Although
 Chung, Burns, and Kim / Guide to Meta-Analysis 1677

Figure 3. Example of funnel plots indicating (A) no publication bias and (B) publication bias. In these examples, effect is
estimated using odds ratios in each study (x-axis) and precision is estimated using the inverse of the standard error (1/standard
error, y-axis).

meta-analyses are considered to have the highest level References

of evidence and are cited more often than other study 1. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative cita-
designs, there are still lingering questions regarding its tion impact of various study designs in the health sciences.
validity when compared with well-conducted clinical JAMA 2005;293:2362–2366.
trials.8,17 The results from a meta-analysis can be used 2. Harbour R, Miller J. A new system for grading recommenda-
tions in evidence based guidelines. BMJ 2001;323:334 –336.
to plan a large RCT to test a treatment effect; however,
3. Egger M, Schneider M, Davey Smith G. Spurious precision?
a report comparing the results from meta-analyses and Meta-analysis of observational studies. BMJ 1998;316:140–144.
subsequent RCTs found only fair agreement.43 The 4. Shapiro S. Meta-analysis/shmeta-analysis. Am J Epidemiol
distrust of meta-analyses and the lack of agreement 1994;140:771–778.
with RCTs do not imply that the meta-analysis should 5. Bailar JC III. The practice of meta-analysis. J Clin Epide-
be abandoned. It does, however, point out the limita- miol 1995;48:149 –157.
6. Meinert CL. Meta-analysis: science or religion? Control Clin
tions and biases involved with the meta-analysis and
Trials 1989;10(suppl):257S–263S.
shows the need for conducting a thoughtful, well- 7. Thompson SG, Pocock SJ. Can meta-analysis be trusted?
planned meta-analysis with minimal bias. Lancet 1991;338:1127–1130.
We provide this article as a practical guideline for 8. Horwitz RI. Large-scale randomized evidence: large, simple
those interested in conducting or interpreting a meta- trials and overviews of trials: discussion. A clinician’s per-
analysis publication. The number of meta-analyses spective on meta-analyses. J Clin Epidemiol 1995;48:41– 44.
9. Merrel GA, Wolfe SW, Slade JF III. Treatment of scaphoid
conducted in hand surgery has been modest to date. nonunions: quantitative meta-analysis of the literature. J Hand
This may be the result of a lack of common outcomes Surg 2002;27A:685– 691.
or incompatible data, as found by Gerritsen et al.37 In 10. Merrell GA, Barrie KA, Katz DL, Wolfe SW. Results of
the hand surgery field, meta-analysis is still in the nerve transfer techniques for restoration of shoulder and
infancy stage because high-quality clinical trials that elbow function in the context of a meta-analysis of the
provide robust data that can be pooled are still lack- English literature. J Hand Surg 2001;26A:303–314.
11. Boeckstyns ME, Sorensen AI. Does endoscopic carpal tun-
ing. When performed appropriately, however, a nel release have a higher rate of complications than open
meta-analysis can lend evidence to many of the dif- carpal tunnel release? An analysis of published series.
ficult decisions we make in our daily practice. J Hand Surg 1999;24B:9 –15.
12. Larsen CF, Jacoby RA, McCabe SJ. Nonunion rates of
Received for publication August 18, 2006; accepted in revised form limited carpal arthrodesis: a meta-analysis of the literature.
September 7, 2006. J Hand Surg 1997;22A:66 –73.
No benefits in any form have been received or will be received from a 13. Margaliot Z, Haase SC, Kotsis SV, Kim HM, Chung KC. A
commercial party related directly or indirectly to the subject of this article. meta-analysis of outcomes of external fixation versus plate
Corresponding author: Kevin C. Chung, MD, Section of Plastic Sur-
osteosynthesis for unstable distal radius fractures. J Hand
gery, University of Michigan Health System, 2130 Taubman Center,
Surg 2005;30A:1185–1199.
1500 E. Medical Center Dr, Ann Arbor, MI 48109-0340; e-mail:
[email protected]. 14. Berman NG, Parker RA. Meta-analysis: neither quick or
Copyright © 2006 by the American Society for Surgery of the Hand easy. BMC Med Res Method 2002;2:10 –18.
0363-5023/06/31A10-0016$32.00/0 15. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF.
doi:10.1016/j.jhsa.2006.09.002 Improving the quality of reports of meta-analyses of random-
1678 The Journal of Hand Surgery / Vol. 31A No. 10 December 2006

ized controlled trials: the QUOROM statement. Quality of 30. Egger M, Zellweger-Zahner T, Schneider M, Junker C,
reporting the meta-analyses. Lancet 1999;354:1896 –1900. Lengeler C, Antes G. Language bias in randomised con-
16. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, trolled trials published in English and German. Lancet 1997;
Rennie D, et al. Meta-analysis of observational studies in 350:326 –329.
epidemiology: a proposal for reporting. Meta-analysis of 31. Moher D, Fortin P, Jadad AR, Juni P, Klassen T, Le Lorier
Observational Studies in Epidemiology (MOOSE) group. J, et al. Completeness of reporting of trials published in
JAMA 2000;283:2008 –2012. languages other than English: implications for conduct and
17. Davey Smith G, Egger M. Meta-analysis. Unresolved issues reporting of systematic reviews. Lancet 1996;347:363–366.
and future developments. BMJ 1998;316:221–225. 32. Sutton AJ, Abrams KR, Jones DR. An illustrated guide to the
18. National Library of Medicine. MEDLINE fact sheet. methods of meta-analysis. J Eval Clin Pract 2001;7:135–148.
Available at: http://www.nlm.nih.gov/pubs/factsheet/ 33. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P,
medline.html. Accessed August 2, 2006. Walsh S. Assessing the quality of randomized controlled
19. Science and Technical Information Network (STN) data- trials: an annotated bibliography of scales and checklists.
base summary sheet EMBASE. Available at: http:// Control Clin Trials 1995;16:62–73.
www.cas.org/ONLINE/DBSS/embasess.html. Accessed 34. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring
August 2, 2006. the quality of clinical trials for meta-analysis. JAMA 1999;
20. Egger M, Davey Smith G. Principles of and procedures for 282:1054 –1060.
systematic reviews. In: Egger M, Davey Smith G, Altman
35. Jadad AR, Moher D, Klassen TP. Guides for reading and
DG, eds. Systematic reviews in health care: meta-analysis in
interpreting systematic reviews: II. How did the authors find
context. 2nd ed. London: BMJ Publishing Group, 2001:26.
the studies and assess their quality? Arch Pediatr Adolesc
21. CINAHL Information Systems. Available at: www.cinahl.
Med 1998;152:812– 817.
com. Accessed August 2, 2006.
36. Egger M, Smith GD, Phillips AN. Meta-analysis: principles
22. Scherer RW, Dickersin K, Langenberg P. Full publication of
and procedures. BMJ 1997;315:1533–1537.
results initially presented in abstracts. A meta-analysis
37. Gerritsen AA, de Vet HC, Scholten RJ, van Tulder MW,
[erratum: JAMA 1994;272:1410]. JAMA 1994;272:158 –162.
23. United States Cochrane Center. Available at: http://www. Bouter LM. Enabling meta-analysis in systematic reviews on
cochrane.us/faq.htm. Accessed August 2, 2006. carpal tunnel syndrome. J Hand Surg 2002;27A:828 – 832.
24. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Pub- 38. Hedges LV, Olkin I. Statistical methods for meta-analysis.
lication bias in clinical research. Lancet 1991;337:867– 872. London: Academic Press, 1985.
25. Dickersin K, Min YI. NIH clinical trials and publication 39. Wolf FM. Meta-analysis: quantitative methods for research
bias. Online J Curr Clin Trials 1993 Apr 28;doc. no. 50. synthesis. Newbury Park: Sage Publications, 1986.
26. Stern JM, Simes RJ. Publication bias: evidence of delayed 40. Fleiss JL. The statistical basis of meta-analysis. Stat Meth-
publication in a cohort study of clinical research projects. ods Med Res 1993;2:121–145.
BMJ 1997;315:640 – 645. 41. Egger M, Davey Smith G, Schneider M, Minder CE. Bias in
27. Egger M, Smith GD. Bias in location and selection of meta-analysis detected by a simple graphical test. BMJ
studies. BMJ 1998;316:61– 66. 1997;315:629 – 634.
28. McAuley L, Pham B, Tugwell P, Moher D. Does the inclu- 42. Begg CB, Mazumdar M. Operating characteristics of a rank
sion of grey literature influence estimates of intervention correlation test for publication bias. Biometrics 1994;50:
effectiveness reported in meta-analyses? Lancet 2000;356: 1088 –1101.
1228 –1231. 43. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian
29. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. F. Discrepancies between meta-analyses and subsequent
Empirical assessment of effect of publication bias on meta- large randomized, controlled trials. N Engl J Med 1997;337:
analyses. BMJ 2000;320:1574 –1577. 536 –542.