Department of Internal Medicine

Guidelines on Presenting at Journal Club

September 2006
Aims

Journal clubs are educational interventions that can improve reading habits, knowledge of
clinical epidemiology and biostatistics, and the use of medical literature in clinical
practice for postgraduate physicians in training (Medical Teacher, Vol. 23, No. 5, 2001).

During the course of the program, each third-year internal medicine resident (PGY-3)
will be responsible for presenting at least three to four Evidence-Based Medicine Journal
Club meetings on a topic related to internal medicine or its subspecialties. This
experience fulfils several goals of the research curriculum such that at its conclusion each
resident will be able to:

1. Conduct structured critical appraisal

2. Understand the limitations of the application of evidence
3. Recognize and understand basic study design, distinguishing weak from strong
methodology
4. Gain familiarity with basic statistical tests
5. Gain insight into a specific clinical problem
6. Hone skills related to oral and written presentations

Expectations

Presenting Resident

You will be assigned an article, a date and a preceptor. Except under extraordinary
circumstances, these will not be changed. Any problems with either assignment (article,
date or preceptor) should be brought to the attention of the Chief Resident or Program
Director as early as possible. The primary responsibility for the EBM Journal Club rests
with the presenting resident and the faculty preceptor.

Responsibilities

1. You will be assigned a primary article and a faculty preceptor 6 weeks prior to
your presentation (see criteria for articles below).
2. Once you have read and reviewed the article, ensure that the article is distributed
to the residents and faculty at least 2 weeks prior to the date of your presentation.
This can be done electronically via email to the Residents mailing list from your
registered address. (PDF's are preferred but hard copies may be placed in the
Chief Resident’s office).
3. Pick 1-3 supporting articles to go with the primary article. Supporting articles
might include up-to-date reviews, classic articles on the subject or studies that
support or refute the results of the primary article.
4. Read the article and decide what type it is (Therapy or Harm, Prognosis,
Diagnosis, Symptom prevalence, Economic or Decision Analysis). Most articles
will fall into the first three categories. Worksheets are best used to analyze and
evaluate the article content (see Resources).
5. Develop your presentation and produce a final critique.
Process

A recent systematic review identified 3 "best practices" for journal clubs: (1) use of a
structured checklist, (2) explicit written learning objectives, and (3) a formalized meeting
structure and process.

Article Selection

Articles will be selected by the Chief Resident in conjunction with the Program Director
and will comprise core internal medicine articles published within the prior 2 years from
one of the five frequently cited medicine journals: Annals of Internal Medicine, BMJ,
JAMA, The Lancet, and the New England Journal of Medicine. Relevant articles from
internal medicine subspecialties may be chosen from the respective literature when the
contribution to general medicine practice is beneficial. The articles chosen should provide
a level of evidence of grade 1a or 1b (see Appendix)

Review

Typically, review of the article will address the following 8 domains (Annals of Family
Medicine 4:196-197 (2006)):

1. Issues addressed by the article—What is the research question? Why does it

matter? How does it fit with what already is known? How can it help solve
important problems for practice or policy?
2. Design of the study—Is the study design appropriate for the question and what
already is known about the question?
3. Study methods—To what degree can the findings be accounted for by:
o How participants were selected?
o How key variables were defined and measured?
o Confounding (false attribution of causality because two variables
discovered to be associated actually are associated with a third factor)?
o How information was interpreted?
o Chance (as indicated by inferential statistics)?
4. Main findings—Does this study advance current knowledge?
5. Generalizability—How transportable are the findings to other settings, particularly
to my patients, practice and community?
6. Implications—How can the information be used to change practice, policy or
training?
7. Constituencies—Who are the constituencies for the findings, including patients,
and how might they be engaged in interpreting or using the findings?
8. Next steps/new questions—What are the next steps in interpreting or applying the
findings? What new questions arise and how might they be best answered?

Presentation

Your presentation will be done in PowerPoint® and should be roughly organized as

follows:

• Topic Background (state why this is important) 3 min

• Outline a hypothetical (or real) case (one slide, keep it brief) 2 min
 • Present the paper (make use of figures and tables) 10 min
o Methods
o Results
• Present your critical appraisal (this is best done with a slide for each of the
questions outlined above and your answers for each). 10 min
• Sum up. Derive your conclusion and how these results will affect YOUR practice.
5 min

Aim to complete your presentation within the 30 allotted minutes, as there shall be two
articles presented per session and there should be time for a few questions from the
attendees. Try not to consume excess time or get too over-involved in one aspect of the
article (e.g., application of statistical methods). If this cannot be resolved quickly, it's
your duty to indicate this and move on.

Resources

Critical Appraisal Worksheets

Learning Tools for Evidence Based Practice
http://www.ebmtips.net/

Douglas G. Altman, Kenneth F. Schulz, David Moher, Matthias Egger, Frank Davidoff,
Diana Elbourne, Peter C. Gøtzsche, Thomas Lang for the CONSORT Group
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and
Elaboration
Ann Intern Med, Apr 2001; 134: 663 - 694.
http://www.annals.org/cgi/reprint/134/8/663.pdf

The CONSORT statement: revised recommendations for improving the quality of reports
of parallel group randomized trials.
BMC Medical Research Methodology 2001, 1:2
The electronic version of this article is the complete one and can be found online at:
http://www.biomedcentral.com/1471-2288/1/2

Being Smarter About Clinical Trials: A Report of the NIH Workshop

Moving From Observational Studies to Clinical Trials: Why Do We Sometimes Get It
Wrong?
January 11–12, 2005
http://www.meetinglink.org/omar/ct/agenda.htm#agenda

The Cochrane Handbook for Systematic Reviews of Interventions

http://www.cochrane.org/resources/handbook/

How to assess epidemiological studies. J H Zaccai.

Postgrad. Med. J. 2004;80;140-147
http://pmj.bmjjournals.com/cgi/reprint/80/941/140

How to Read a Paper

Trisha Greenhalgh
BMJ Publishing Group
http://bmj.bmjjournals.com/collections/read.shtml
 Appendix: Levels of Evidence

Grade of Therapy: Whether a

Therapy: Whether a drug is superior Differential diagnosis/symptom
recommendat Level of Evidence treatment is efficacious/ Prognosis Diagnosis Economic and decision analysis
to another drug in its same class prevalence study
ion effective/harmful

SR (with homogeneity*) of SR (with homogeneity**) of head-to- SR (with homogeneity*) of inception SR (with homogeneity*) of Level 1 SR (with homogeneity*) of prospective SR (with homogeneity*) of Level 1
1a RCTs head RCTs cohort studies; CDR† validated in diagnostic studies; CDR† with 1b studies cohort studies economic studies
different populations from different clinical centres

Individual RCT (with narrow Within a head-to-head RCT with Individual inception cohort study with > Validating** cohort study with good††† Prospective cohort study with good Analysis based on clinically sensible
Confidence Interval‡) clinically important outcomes 80% follow-up; CDR† validated in a reference standards; or CDR† tested follow-up**** costs or alternatives; systematic
A
1b single population within one clinical centre review(s) of the evidence; and
including multi-way sensitivity
analyses

All or none§ All or none case-series Absolute SpPins and SnNouts†† All or none case-series Absolute better-value or worse-value
1c
analyses‡‡

SR (with homogeneity*) of Within a head-to-head RCT with SR (with homogeneity*) of either SR (with homogeneity*) of Level >2 SR (with homogeneity*) of 2b and SR (with homogeneity*) of Level >2
2a cohort studies validated surrogate outcomes ‡‡‡ retrospective cohort studies or untreated diagnostic studies better studies economic studies
control groups in RCTs

Individual cohort study Across RCTs of different drugs v. Retrospective cohort study or follow-up Exploratory** cohort study with Retrospective cohort study, or poor Analysis based on clinically sensible
(including low quality RCT; placebo in similar or different patients of untreated control patients in an RCT; good†††reference standards; CDR† after follow-up costs or alternatives; limited
2b e.g., <80% follow-up) with clinically important or validated Derivation of CDR† or validated on derivation, or validated only on split- review(s) of the evidence, or single
surrogate outcomes split-sample§§§ only sample§§§ or databases studies; and including multi-way
sensitivity analyses

"Outcomes" Research; "Outcomes" Research Ecological studies Audit or outcomes research

2c
Ecological studies
B

SR (with homogeneity*) of Across subgroup analyses from RCTs of SR (with homogeneity*) of 3b and better SR (with homogeneity*) of 3b and SR (with homogeneity*) of 3b and
case-control studies different drugs v. placebo in similar or studies better studies better studies
3a different patients, with clinically
important or validated surrogate
outcome

Individual Case-Control Study Across RCTs of different drugs v. Non-consecutive study; or without Non-consecutive cohort study, or very Analysis based on limited
placebo in similar or different patients consistently applied reference standards limited population alternatives or costs, poor quality
3b but with unvalidated surrogate outcomes estimates of data, but including
sensitivity analyses incorporating
clinically sensible variations.

Case-series (and poor quality Between non-randomised studies Case-series (and poor quality prognostic Case-control study, poor or non- Case-series or superseded reference Analysis with no sensitivity analysis
cohort and case-control (observational studies and studies ***) independent reference standard standards
C 4
studies§§ ) administrative database research) with
clinically important outcomes

Expert opinion without Expert opinion without explicit critical Expert opinion without explicit critical Expert opinion without explicit critical Expert opinion without explicit critical Expert opinion without explicit
explicit critical appraisal, or appraisal, or based on physiology, appraisal, or based on physiology, appraisal, or based on physiology, bench appraisal, or based on physiology, critical appraisal, or based on
D 5 based on physiology, bench bench research or "first principles"; or bench research or "first principles" research or "first principles" bench research or "first principles" economic theory or "first principles"
research or "first principles" non-randomised studies with
unvalidated surrogate outcomes
 Appendix: Levels of Evidence

1. These levels were generated in a series of iterations among members of the NHS
R&D Centre for Evidence-Based Medicine (Bob Phillips, Chris Ball, Dave
Sackett, Brian Haynes, Sharon Straus and Finlay McAlister).
2. Users can add a minus-sign "-" to denote the level of that fails to provide a
conclusive answer because of:
EITHER a single result with a wide Confidence Interval (such that, for
example, an ARR in an RCT is not statistically significant but whose
confidence intervals fail to exclude clinically important benefit or harm)
OR a Systematic Review with troublesome (and statistically significant)
heterogeneity.
3. Grades of recommendation are shown as linked directly to a level of evidence.
However levels speak only of the validity of a study not its clinical applicability.
Other factors need to be taken into account (such as cost, easy of implementation,
importance of the disease) before determining a grade. Grades that are currently
in the guides link closely to the validity of the evidence - these will change over
time to reflect better concerns that we highlight in the text of the guide or related
CATs.

Notes
By homogeneity we mean a systematic review that is free of worrisome variations
(heterogeneity) in the directions and degrees of results between individual studies. Not all
* systematic reviews with statistically significant heterogeneity need be worrisome, and not all
worrisome heterogeneity need be statistically significant. As noted above, studies displaying
worrisome heterogeneity should be tagged with a "-" at the end of their designated level.
Clinical Decision Rule. (These are algorithms or scoring systems which lead to a prognostic
†
estimation or a diagnostic category)
See comment #2 for advice on how to understand, rate and use trials or other studies with wide
‡
confidence intervals.
Met when all patients died before the Rx became available, but some now survive on it; or
§
when some patients died before the Rx became available, but none now die on it.
By poor quality cohort study we mean one that failed to clearly define comparison groups
and/or failed to measure exposures and outcomes in the same (preferably blinded), objective
way in both exposed and non-exposed individuals and/or failed to identify or appropriately
§ control known confounders and/or failed to carry out a sufficiently long and complete follow-
§ up of patients. By poor quality case-control study we mean one that failed to clearly define
comparison groups and/or failed to measure exposures and outcomes in the same (preferably
blinded), objective way in both cases and controls and/or failed to identify or appropriately
control known confounders.
§
Split-sample validation is achieved by collecting all the information in a single tranche, then
§
artificially dividing this into "derivation" and "validation" samples.
§
An "Absolute SpPin" is a diagnotic finding whose Specificity is so high that a Positive result
†
rules-in the diagnosis. An "Absolute SnNout" is a diagnostic finding whose Sensitivity is so
†
high that a Negative result rules-out the diagnosis.
Better-value treatments are clearly as good but cheaper, or better at the same or reduced cost.
‡
Worse-value treatments are as good and more expensive, or worse and equally or more
‡
expensive.
Good reference standards are independent of the test, and applied blindly or objectively to
†
applied to all patients. Poor reference standards are haphazardly applied, but still independent
†
of the test. Use of a non-independent reference standard (where the 'test' is included in the
†
'reference', or where the 'testing' affects the 'reference') implies a level 4 study.
Validating studies test the quality of a specific diagnostic test, based on prior evidence. An
*
exploratory study collects information and trawls the data (e.g. using a regression analysis) to
*
find which factors are 'significant'.
By poor quality prognostic cohort study we mean one in which sampling was biased in favour
*
of patients who already had the target outcome, or the measurement of outcomes was
*
accomplished in <80% of study patients, or outcomes were determined in an unblinded, non-
*
objective way, or there was no correction for confounding factors.
*
* Good follow-up in a differential diagnosis study is >80%, with adequate time for alternative
* diagnoses to emerge (eg 1-6 months acute, 1 - 5 years chronic)
*
‡
Surrogate outcomes are considered validated only when the relationship between the surrogate
‡
outcome and the clinically important outcomes has been established in long-term RCTs.
‡

References
1. Canadian Task Force on the Periodic Health Examination: The periodic health
examination. CMAJ 1979;121:1193-1254.
2. Sackett DL. Rules of evidence and clinical recommendations on use of
antithrombotic agents. Chest 1986 Feb; 89 (2 suppl.):2S-3S.
3. Cook DJ, Guyatt GH, Laupacis A, Sackett DL, Goldberg RJ. Clinical
recommendations using levels of evidence for antithrombotic agents. Chest 1995
Oct; 108(4 Suppl):227S-230S.
4. Yusuf S, Cairns JA, Camm AJ, Fallen EL, Gersh BJ. Evidence-Based Cardiology.
London: BMJ Publishing Group, 1998.