MM16-P

Vol. 25 No. 19

Use of External RNA Controls in Gene

Expression Assays; Proposed Guideline
PLEASE
TTT
TT
T
This proposed document is published for wide and thorough review in the new, accelerated
Clinical and Laboratory Standards Institute (CLSI) consensus-review process. The
document will undergo concurrent consensus review, Board review, and delegate voting
(i.e., candidate for advancement) for 90 days.

Please send your comments on scope, approach, and technical and editorial content to
CLSI.

Comment period ends

11 October 2005

The subcommittee responsible for this document will assess all comments received by the
end of the comment period. Based on this assessment, a new version of the document will
be issued. Readers are encouraged to send their comments to Clinical and Laboratory
Standards Institute, 940 West Valley Road, Suite 1400, Wayne, PA 19087-1898 USA; Fax:
+610.688.0700, or to the following e-mail address: [email protected]

S
SS
SSS

COMMENT
This document provides protocols supporting the use of external RNA controls in
microarray and QRT-PCR-based gene expression experiments, including preparation of
control transcripts, design of primers and amplicons, quality control, use in final
experimental or clinical test application, and analysis and interpretation of data obtained.
A guideline for global application developed through the Clinical and Laboratory
Standards Institute consensus process.
Clinical and Laboratory Standards Institute
Providing NCCLS standards and guidelines, ISO/TC 212 standards, and ISO/TC 76 standards

Clinical and Laboratory Standards Institute (CLSI, Most documents are subject to two levels of consensus—
formerly NCCLS) is an international, interdisciplinary, “proposed” and “approved.” Depending on the need for
nonprofit, standards-developing, and educational field evaluation or data collection, documents may also be
organization that promotes the development and use of made available for review at an intermediate consensus
voluntary consensus standards and guidelines within the level.
healthcare community. It is recognized worldwide for the
Proposed A consensus document undergoes the first stage
application of its unique consensus process in the
of review by the healthcare community as a proposed
development of standards and guidelines for patient
standard or guideline. The document should receive a wide
testing and related healthcare issues. Our process is
and thorough technical review, including an overall review
based on the principle that consensus is an effective and
of its scope, approach, and utility, and a line-by-line review
cost-effective way to improve patient testing and
of its technical and editorial content.
healthcare services.
Approved An approved standard or guideline has achieved
In addition to developing and promoting the use of
consensus within the healthcare community. It should be
voluntary consensus standards and guidelines, we
reviewed to assess the utility of the final document, to
provide an open and unbiased forum to address critical
ensure attainment of consensus (i.e., that comments on
issues affecting the quality of patient testing and health
earlier versions have been satisfactorily addressed), and to
care.
identify the need for additional consensus documents.
PUBLICATIONS
Our standards and guidelines represent a consensus opinion
A document is published as a standard, guideline, or on good practices and reflect the substantial agreement by
committee report. materially affected, competent, and interested parties
obtained by following CLSI’s established consensus
Standard A document developed through the consensus
procedures. Provisions in CLSI standards and guidelines
process that clearly identifies specific, essential
may be more or less stringent than applicable regulations.
requirements for materials, methods, or practices for use
Consequently, conformance to this voluntary consensus
in an unmodified form. A standard may, in addition,
document does not relieve the user of responsibility for
contain discretionary elements, which are clearly
compliance with applicable regulations.
identified.
COMMENTS
Guideline A document developed through the
consensus process describing criteria for a general The comments of users are essential to the consensus
operating practice, procedure, or material for voluntary process. Anyone may submit a comment, and all comments
use. A guideline may be used as written or modified by are addressed, according to the consensus process, by the
the user to fit specific needs. committee that wrote the document. All comments,
including those that result in a change to the document when
Report A document that has not been subjected to
published at the next consensus level and those that do not
consensus review and is released by the Board of
result in a change, are responded to by the committee in an
Directors.
appendix to the document. Readers are strongly encouraged
CONSENSUS PROCESS to comment in any form and at any time on any document.
Address comments to the Clinical and Laboratory Standards
The CLSI voluntary consensus process is a protocol Institute, 940 West Valley Road, Suite 1400, Wayne, PA
establishing formal criteria for: 19087, USA.
• the authorization of a project VOLUNTEER PARTICIPATION
• the development and open review of documents Healthcare professionals in all specialties are urged to
• the revision of documents in response to comments volunteer for participation in CLSI projects. Please contact
by users us at [email protected] or +610.688.0100 for
additional information on committee participation.
• the acceptance of a document as a consensus
standard or guideline.
 MM16-P
ISBN 1-56238-579-8
Volume 25 Number 19 ISSN 0273-3099
Use of External RNA Controls in Gene Expression Assays; Proposed
Guideline
Janet A. Warrington, PhD
Philippe Corbisier, PhD
Harriet Feilotter, PhD, FCCMG
Joseph L. Hackett, PhD
Laura H. Reid, PhD
Marc L. Salit, PhD
Elizabeth A. Wagar, MD
P. Mickey Williams, PhD
Paul Wolber, PhD

Abstract
Clinical and Laboratory Standards Institute (CLSI) document MM16-P—Use of External RNA Controls in Gene Expression
Assays; Proposed Guideline provides protocols supporting the use of external RNA controls in microarray and QRT-PCR-based
gene expression experiments. This guideline addresses important issues associated with the use of external RNA controls as a
tool for verification of technical performance, and in support of the evaluation of qualitative results for a specific clinical analyte
including preparation of control transcripts, design of primers and amplicons, quality control, use in final experimental or clinical
test application, and the analysis and interpretation of data obtained. The guideline will facilitate research and clinical
laboratories, regulatory agencies, accrediting agencies, reference laboratories, as well as test, microarray, and reagent
manufacturers in measuring the performance of expression assays.

Clinical and Laboratory Standards Institute (CLSI). Use of External RNA Controls in Gene Expression Assays; Proposed
Guideline. CLSI document MM16-P (ISBN 1-56238-579-8). Clinical and Laboratory Standards Institute, 940 West Valley Road,
Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2005.

The Clinical and Laboratory Standards Institute consensus process, which is the mechanism for moving a document through
two or more levels of review by the healthcare community, is an ongoing process. Users should expect revised editions of any
given document. Because rapid changes in technology may affect the procedures, methods, and protocols in a standard or
guideline, users should replace outdated editions with the current editions of CLSI/NCCLS documents. Current editions are
listed in the CLSI catalog, which is distributed to member organizations, and to nonmembers on request. If your organization is
not a member and would like to become one, and to request a copy of the catalog, contact us at: Telephone: 610.688.0100; Fax:
610.688.0700; E-Mail: [email protected]; Website: www.clsi.org
Number 19 MM16-P

This publication is protected by copyright. No part of it may be reproduced, stored in a retrieval system,
transmitted, or made available in any form or by any means (electronic, mechanical, photocopying,
recording, or otherwise) without prior written permission from Clinical and Laboratory Standards
Institute, except as stated below.

Clinical and Laboratory Standards Institute hereby grants permission to reproduce limited portions of this
publication for use in laboratory procedure manuals at a single site, for interlibrary loan, or for use in
educational programs provided that multiple copies of such reproduction shall include the following
notice, be distributed without charge, and, in no event, contain more than 20% of the document’s text.

Reproduced with permission, from CLSI publication MM16-P—Use of External RNA

Controls in Gene Expression Assays; Proposed Guideline (ISBN 1-56238-579-8).
Copies of the current edition may be obtained from Clinical and Laboratory Standards
Institute, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898, USA.

Permission to reproduce or otherwise use the text of this document to an extent that exceeds the
exemptions granted here or under the Copyright Law must be obtained from Clinical and Laboratory
Standards Institute by written request. To request such permission, address inquiries to the Executive Vice
President, Clinical and Laboratory Standards Institute, 940 West Valley Road, Suite 1400, Wayne,
Pennsylvania 19087-1898, USA.

Copyright ©2005. Clinical and Laboratory Standards Institute.

Suggested Citation

(Clinical and Laboratory Standards Institute. Use of External RNA Controls in Gene Expression Assays;
Proposed Guideline. CLSI document MM16-P [ISBN 1-56238-579-8]. Clinical and Laboratory Standards
Institute, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2005.)

Proposed Guideline
July 2005

ISBN 1-56238-579-8
ISSN 0273-3099

ii
Volume 25 MM16-P

Committee Membership

Area Committee on Molecular Methods

Roberta M. Madej, MS, MT Uwe Scherf, PhD Leslie Hall, MMSc
Chairholder FDA Center for Devices and Mayo Clinic
Roche Molecular Systems, Inc. Radiological Health Rochester, Minnesota
Pleasanton, California Rockville, Maryland
Robert B. Jenkins, MD, PhD
Zhimin Cao, MD, PhD Michael A. Zoccoli, PhD Mayo Clinic
New York State Dept. of Health Celera Diagnostics Rochester, Minnesota
Albany, New York Alameda, California
Alan L. Landay, PhD
Maurizio Ferrari, MD Advisors Rush-Presby.-St. Lukes Medical
International Federation of Clinical Center
Chemistry Dale H. Altmiller, PhD Chicago, Illinois
Milan, Italy O.U. Medical Center
Edmond, Oklahoma Mario Pazzagli, PhD
Frederick S. Nolte, PhD University of Florence
Emory University Hospital Lee Ann Baxter-Lowe, PhD Florence, Italy
Atlanta, Georgia University of California, San
Francisco Richard S. Schifreen, PhD, DABCC
Timothy J. O’Leary, MD, PhD San Francisco, California Mirus Bio Corp
Biomedical Laboratory Research Madison, Wisconsin
and Development Service Mark Evans, PhD
Washington, District of Columbia American Medical Association Laurina O. Williams, PhD, MPH
Chicago, Illinois Centers for Disease Control and
Carolyn Sue Richards, PhD, Prevention
FACMG Cristina Gianchetti, PhD Atlanta, Georgia
Oregon Health Sciences University Gen-Probe
Portland, Oregon San Diego, California Janet L. Wood, MT(ASCP)
BD Diagnostic Systems
Sparks, Maryland

Subcommittee on External RNA Controls

Janet A. Warrington, PhD Marc L. Salit, PhD Debra J. Rasmussen, MBA
Chairholder National Institute of Standards and Veridex, LLC
Affymetrix Technology Warren, New Jersey
Santa Clara, California Gaithersburg, Maryland
Staff
Philippe Corbisier, PhD Elizabeth A. Wagar, MD
Institute for Reference Materials & UCLA Medical Center Clinical and Laboratory Standards
Measurements Los Angeles, California Institute
Geel, Belgium Wayne, Pennsylvania
P. Mickey Williams, PhD
Harriet Feilotter, PhD, FCCMG Roche Molecular Systems
Lois M. Schmidt, DA
Queen’s University Pleasanton, California
Staff Liaison
Kingston, Ontario, Canada
Paul K. Wolber, PhD
Donna M. Wilhelm
Joseph L. Hackett, PhD Agilent Technologies
Editor
FDA Center for Devices and Santa Clara, California
Radiological Health
Rockville, Maryland Advisors Melissa A. Lewis
Assistant Editor
Laura H. Reid, PhD Steven R. Bauer, PhD
Expression Analysis, Inc. FDA Center for Biologics John J. Zlockie, MBA
Durham, North Carolina Evaluation and Research Vice President, Standards
Bethesda, Maryland

iii
Number 19 MM16-P

Acknowledgement

This guideline was prepared by CLSI, as part of a cooperative effort with IFCC to work toward the
advancement and dissemination of laboratory standards on a worldwide basis. CLSI gratefully
acknowledges the participation of IFCC in this project.

iv
Volume 25 MM16-P

Contents

Abstract ....................................................................................................................................................i

Committee Membership........................................................................................................................ iii

Foreword .............................................................................................................................................. vii

1 Scope..........................................................................................................................................1

2 Introduction................................................................................................................................1

3 Standard Precautions..................................................................................................................3

4 Terminology...............................................................................................................................3
4.1 Definitions ....................................................................................................................3
4.2 Acronyms/Abbreviations ..............................................................................................5
5 Overview....................................................................................................................................5
5.1 Gene Expression Profiling Technology ........................................................................5
5.2 Gene Expression Profiling in Clinical Applications .....................................................6
5.3 A Note on Technical Performance and Clinical Performance ......................................8
6 External RNA Controls and Their Use in Expression Assays ...................................................8
6.1 Internal and External RNA Controls in Gene Expression Assays ................................9
6.2 Characteristics of External RNA Controls....................................................................9
6.3 Using Control Materials to Assess Technical and Clinical Performance ...................10
6.4 Limitations of External Controls ................................................................................13
6.5 Metrics for Assessing Technical Performance of the Expression Platform................14
7 Recommended Practices/Protocols ..........................................................................................16
7.1 Overview.....................................................................................................................16
7.2 Protocols for Preparation and Assessment of External RNA Controls.......................17
7.3 External RNA Control Protocols for Microarrays ......................................................19
7.4 External RNA Control Protocols for QRT-PCR.........................................................25
References.............................................................................................................................................31

The Quality System Approach..............................................................................................................36

Related CLSI/NCCLS Publications ......................................................................................................37

v
Number 19 MM16-P

vi
Volume 25 MM16-P

Foreword
Few inventions in the past twenty years have had as much impact on the research and clinical
communities as the polymerase chain reaction (PCR) and microarray technology. These technologies
have captured the imaginations of clinical and laboratory researchers and made possible studies
heretofore unachievable. The scope and breadth of the studies enabled by these technologies have
dramatically changed the practice of laboratory and clinical research, most notably impacting
experimental design and data analyses. Whole genome analyses integrating DNA, RNA, and protein
information promise deeper insight into biological processes and the discovery of new biomarkers. The
volume of data generated by these methods pressures all investigators to design experimental and
analytical strategies that will leverage the integration of clinical and molecular information.

This new era of clinical and laboratory research places increased emphasis on collaboration and
teamwork, including a significant focus on well-defined clinical database structures, data management,
and data access issues. At the very center of this paradigm shift is a requirement for the fundamental tools
necessary for measuring technical performance of the platforms used for data collection. Acceptance of
microarray and quantitative reverse transcriptase real-time polymerase chain reaction (QRT-PCR) data in
the regulatory environment will require standards that assess reliability and quality. The ability to report
reliable gene expression results of known quality is key to the successful employment of microarrays and
QRT-PCR as tools in toxicogenomics, pharmacogenetics, pharmacogenomics, and as diagnostic devices
in clinical medicine.

In this guideline, we present protocols supporting the use of external RNA controls in microarray and
QRT-PCR-based gene expression experiments. External RNA controls refer to RNA species that are
distinct from the RNAs in the sample that is to be analyzed. The external RNA controls are added prior to
enzymatic manipulations such as reverse transcription and labeling. The protocols enable research and
clinical laboratories, regulatory agencies, accrediting agencies, reference laboratories, as well as test,
microarray, and reagent manufacturers to assess the performance of these expression assays.

Minority Opinion

Please note that during committee voting, the timeliness of this document was raised as a
concern. A minority opinion contends that MM16-P is ahead of the technology curve and
that clinical applications have not yet been established based upon complex gene-expression
technologies on microarray platforms.

“My major reservation is that this document is too closely tied to a single set of controls that
are not yet available, and for which there is no experience. Although I agree that there is a
need for standardization and controls for this emerging diagnostic technology, the document
does not adequately address assay process controls, which ultimately will provide the best
quality control for these complex technologies. In my opinion, the quality control concepts
and reference materials are not sufficiently well-developed for a CLSI document. An
important first step in this process is missing, a publication in the peer-reviewed literature
using these concepts and control materials (see “Standardizing global gene expression
analysis between laboratories and across platforms.” Nature Methods. 2005;2:351).”

We invite further comments on this opinion, for committee consideration in advancing the
MM16-P guideline to “approved” status in the CLSI consensus process.

vii
Number 19 MM16-P

Invitation for Participation in the Consensus Process

An important aspect of the development of this and all Clinical and Laboratory Standards Institute (CLSI)
documents should be emphasized, and that is the consensus process. Within the context and operation of
CLSI, the term “consensus” means more than agreement. In the context of document development,
“consensus” is a process by which CLSI, its members, and interested parties (1) have the opportunity to
review and to comment on any CLSI publication; and (2) are assured that their comments will be given
serious, competent consideration. Any CLSI document will evolve as will technology affecting laboratory
or healthcare procedures, methods, and protocols, and therefore, is expected to undergo cycles of
evaluation and modification.

The Area Committee on Molecular Methods has attempted to engage the broadest possible worldwide
representation in committee deliberations. Consequently, it is reasonable to expect that issues remain
unresolved at the time of publication at the proposed level. The review and comment process is the
mechanism for resolving such issues.

The CLSI voluntary consensus process is dependent upon the expertise of worldwide reviewers whose
comments add value to the effort. At the end of a 90-day comment period, each subcommittee is obligated
to review all comments and to respond in writing to all which are substantive. Where appropriate,
modifications will be made to the document, and all comments along with the subcommittee’s responses
will be included as an appendix to the document when it is published at the next consensus level.

A Note on Terminology

Clinical and Laboratory Standards Institute (CLSI), as a global leader in standardization, is firmly
committed to achieving global harmonization wherever possible. Harmonization is a process of
recognizing, understanding, and explaining differences while taking steps to achieve worldwide
uniformity. CLSI recognizes that medical conventions in the global metrological community have
evolved differently in the United States, Europe, and elsewhere; that these differences are reflected in
CLSI, ISO, and CEN documents; and that legally required use of terms, regional usage, and different
consensus timelines are all challenges to harmonization. Despite these challenges, CLSI recognizes that
harmonization of terms facilitates the global application of standards and is an area that needs immediate
attention. Implementation of this policy must be an evolutionary and educational process that begins with
new projects and revisions of existing documents.

In keeping with CLSI’s commitment to harmonize terminology with that of ISO, the term “sensitivity”
will not be used in this document, due to the existence of several alternative common uses of that term.
The preferred term “limit of detection” will be used, due to its more precise definition and common use.
In many clinical laboratories and diagnostic applications, “sensitivity” or “analytical sensitivity” are used
interchangeably with “limit of detection,” “lower limit of detection,” or “detection limit.” However,
“sensitivity” is also used in several other ways, some of which are preferred uses in their areas. In
different applications, “sensitivity” might be used alone or with modifiers.

Key Words

External RNA controls, gene expression, microarray, molecular methods, QRT-PCR, RNA controls

viii
Volume 25 MM16-P

Use of External RNA Controls in Gene Expression Assays;

Proposed Guideline

1 Scope
The recommended external RNA control protocols described in this guideline are for the preparation of
individual transcripts, pooling of transcripts, design of primers and amplicons, quality control, use in final
experimental or clinical test application, and analysis and interpretation of data obtained using external
RNA controls with and without complex background. The protocols will address use on one- and two-
color microarray platforms and multiple QRT-PCR systems.

The external RNA controls are a tool for verification of technical performance, but will only support
evaluation of qualitative results for a particular clinical sample (see Section 5.3). Technical performance
verification does not control for or manage errors arising from human technical error or laboratory error.

This guideline has been developed to provide useful recommendations and protocols for:

• quality requirements for external assay controls;

• statement of control characteristics, performance metrics; and
• explanation of performance specifications, acceptable performance.

The following protocols are not within the scope of this guideline:

• quantitative controls for clinical sample RNA quality;

• controls for BAC, SNP, genotyping, resequencing, and protein arrays;
• preparation of molecular biology reagents;
• fabrication of cloned control materials;
• array manufacture;
• calibration, calibrators for absolute quantification;
• probe design for PCR;
• preanalytic sample handling;
• endogenous controls; and
• prelabeled hybridization controls.

2 Introduction
Microarray and QRT-PCR technologies are emerging as vital components of genomic, evidence-based
medicine and are already having an impact.1 Standard controls, best practice guidelines, and reference
materials are expected and required for acceptance of microarray and QRT-PCR results in the regulatory
environment, and are necessary for assessing reliability and quality results from these assay platforms.
Industry-recognized controls are key to successful use of microarrays and QRT-PCR in toxicogenomics,
pharmacogenomics, and as diagnostic devices in clinical medicine.

External RNA controls have been used to validate and interpret data from microarray hybridization
experiments since the techniques were first reported in the scientific literature.2-5 These controls, which
are added before labeling the RNA sample, continue to be valuable tools for assessing the limit of
detection, linear range, nonspecific background, and reproducibility of microarray platforms and could
easily be extended to QRT-PCR assays. Control of these characteristics is essential for reliably
implementing gene expression measurements in the clinic.

©
Clinical and Laboratory Standards Institute. All rights reserved. 1
Number 19 MM16-P

A set of external RNA controls is being developed in parallel with this guidance by an industry-led
consortium, the External RNA Control Consortium (ERCC). The ERCC is working to make these
materials readily accessible. They are intended to have broad acceptance within the research and clinical
communities. The controls will be made commercially available as individual plasmid clones for the
synthesis of polyadenylated transcripts (see Section 6.2). This set of external RNA controls is consistent
with recommendations made at the 2003 meeting hosted by National Institute of Standards and
Technology (NIST) at Stanford, “Metrology and Standards Needs for Gene Expression Technologies:
Universal RNA Standards.”6 This CLSI guidance document is intended to describe a set of protocols
recommended for use of the controls to validate gene expression experiments. These controls can be
applied in an expression experiment as depicted in Figure 1. This application will control for technical
performance, but not sample integrity. Valid clinical results depend on validated platform performance,
but clinical results cannot be validated solely with these controls (see Section 5.3).

Patient sample

Extracted total RNA Add external RNA control

Label, hybridize cRNA QRT-PCR

Scan Relative quantification

Image analysis

Interpretation

Figure 1. Flowchart Illustrating the Use of External RNA Controls

This document addresses the need among the research and clinical communities to develop standardized
external RNA controls for expression analysis by microarray and QRT-PCR assays. It is intended to be
useful for assuring assay performance and enabling comparisons of gene expression results. With these
goals in mind, stakeholders in regulatory agencies, diagnostics, pharmaceutical, reagent, and microarray
manufacturing companies, as well as the research and clinical community have participated in writing
and/or reviewing the document, including members of the Association for Molecular Pathology, College
of American Pathologists, American College of Medical Genetics, American Society of Human Genetics,
Canadian College of Medical Genetics, International Committee on Harmonization, ERCC, Microarray
Gene Expression Data Society, American Society of Hematology, American Association for Clinical
Chemistry, International Federation of Clinical Chemistry, American Association of Cancer Research,
National Institute of Standards and Technology, and the Food and Drug Administration.

MM16 is the second CLSI document presenting guidelines relevant to microarray assays, joining
MM12—Diagnostic Nucleic Acid Microarrays in that focus. MM16 is the third CLSI document

©
2 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

presenting guidelines related to QRT-PCR with MM5—Nucleic Acid Amplification Assays for Molecular
Hematopathology and MM6—Quantitative Molecular Methods for Infectious Diseases. This document
provides an overview of gene expression profiling methods with an emphasis on clinical applications. The
main focus of the guidelines in MM16 is the appropriate use of the external RNA controls developed by
the External RNA Controls Consortium (available at www.cstl.nist.gov/biotech/workshops/ERCC2003).
The chosen protocols and recommendations are not absolute or immutable. The MM16 guidance
document addresses the incorporation of the control materials in an assay, including experimental design
considerations, and analysis and interpretation of control results.

3 Standard Precautions
Because it is often impossible to know what might be infectious, all patient and laboratory specimens are
treated as infectious and handled according to “standard precautions.” Standard precautions are guidelines
that combine the major features of “universal precautions and body substance isolation” practices.
Standard precautions cover the transmission of all infectious agents and thus are more comprehensive
than universal precautions which are intended to apply only to transmission of blood-borne pathogens.
Standard and universal precaution guidelines are available from the U.S. Centers for Disease Control and
Prevention (Guideline for Isolation Precautions in Hospitals. Infection Control and Hospital
Epidemiology. CDC. 1996;17(1):53-80 and MMWR 1988;37:377-388). For specific precautions for
preventing the laboratory transmission of all infectious agents from laboratory instruments and materials
and for recommendations for the management of exposure to all infectious disease, refer to the most
current edition of CLSI document M29—Protection of Laboratory Workers From Occupationally
Acquired Infections.

4 Terminology

4.1 Definitions

accuracy (of measurement) – closeness of the agreement between the result of a measurement and a true
value of the measurand (VIM93).7

amplicon – an amplicon is the product of PCR. It is a fragment of DNA that has been synthesized using
amplification techniques.

analyte – component represented in the name of a measurable quantity (ISO 17511).8

analytical cross-reactivity – evaluation of the level of nonspecific binding of control and/or test probes
to nontargeted analytes that may be present in samples.

cycle threshold (Ct) – the Ct value of each RT-PCR reaction depends on the initial template amount
(copy number) of the target sequence, and it is inversely proportional to the log of this copy number. In an
experiment where all PCR reactions have similar efficiency, the Ct value will be the lowest for reactions,
where the initial template copy number was highest.

endogenous control – endogenous controls are used in gene expression studies to normalize gene
expression values in the total RNA relative to expression levels of genes whose expression levels are
invariant in a given tissue or tissues; NOTE: Endogenous controls are not within the scope of this
guideline.

external RNA control – the RNA species that are added to an RNA sample prior to enzymatic
manipulations.

©
Clinical and Laboratory Standards Institute. All rights reserved. 3
Number 19 MM16-P

invariant gene – a gene with an expression level that is unchanging in a given tissue under experimental
conditions or in the normal and diseased state.

limit of detection – the lowest amount of analyte in a sample that can be detected with (stated)
probability, although perhaps not quantified as an exact value (modified from WHO BS/95.1703).9

linearity//linear range – the ability (within a given range) to provide results that are directly proportional
to the concentration (amount) of the analyte in the test sample (WHO-BS/95.1793)10; NOTE 1: Linearity
typically refers to overall system response (i.e., the final analytical answer rather than the raw instrument
output); NOTE 2: The linearity of a system is measured by testing levels of an analyte which are known
by formulation or known relative to each other (not necessarily known absolutely); when the system
results are plotted against these values, the degree to which the plotted curve conforms to a straight line is
a measure of system linearity.

measurand – particular quantity subject to measurement (VIM93).7

normalization – array’s normalization is the operation of removing experimental artifacts of non-

biological origin such as variations in the sample preparation, variation in the manufacture of arrays, and
in the processing of the arrays (labeling efficiency, hybridization, and scanning).

pharmacogenetics – the study of inherited differences (variation) in drug metabolism and response.

pharmacogenomics – the general study of the many different genes that determine drug behavior.

precision (of measurement) – the closeness of agreement between independent test results obtained
under stipulated conditions (ISO 3534-1).11

primer (QRT-PCR) – a single-stranded nucleic acid that can “prime” replication of a template. More
specifically, a single-stranded nucleic acid capable of hybridizing to a template single-stranded nucleic
acid in such a way as to leave part of the template to the 3´ end of the primer single-stranded. DNA
polymerase can then synthesize a new strand starting from the 3´ end of the primer and adding
nucleotides to the growing strand by base complementarity to the template; NOTE: Primers used for
quantitative reverse transcriptase real-time PCR are generally designed by specialized software taking
into account a number of parameters (melting temperature, primers-dimers formation, GC percentages) to
optimize the polymerase chain reaction.

probe (microarray) – defined piece of single-stranded nucleic acid used to identify specific DNA or
RNA molecules bearing the complementary sequence, which is immobilized on the microarray substrate.

probe (QRT-PCR) – a single-stranded nucleic acid that binds to amplicon sequences between QRT-PCR
primers with a fluorescent marker and enhances the specificity of sequence detection and quantification;
NOTE: One example of this type of probe is in the 5´ nuclease reaction, where fluorescent markers with
different excitation/emission profiles can be used on different probes to multiplex QRT-PCR for the
simultaneous detection of target sequence and control sequence. The 5´ nuclease reaction signal is
generated as free fluorescent molecules are released from the 5´ end of a probe where they had been
quenched by a quencher group attached to the 3´ end of the probe.

scaling – the operation of adjusting the total or average intensity of each array to be approximately the
same to an arbitrary constant; NOTE: Scaling can be done by multiplying each value by a scaling factor
(a ratio) which can be determined by comparing the total average intensity of each array to an arbitrary
constant.

©
4 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

signal – a quantity that represents the measurand and which is functionally related to it (VIM93)7;
NOTE: In microarray analyses, signal is the fluorescence intensity information captured by the detection
system.

target – in expression analysis, the RNA sample or gene of interest that is being assayed; NOTE: For
microarrays, the target is labeled and hybridized to the array. In QRT-PCR, the target RNA is labeled and
hybridized.

trueness (of measurement) – the closeness of agreement between the average value obtained from a
large series of test results and an accepted reference value (ISO 3534-1).11

4.2 Acronyms/Abbreviations

BAC Bacterial Artificial Chromosome

BCR/ABL The translocated ABL gene from chromosome 9 fuses with the remaining
portion of the reciprocally translocated BCR gene to form a fusion
oncogene BCR/ABL.
cDNA complementary deoxyribonucleic acid
CLIA Clinical Laboratory Improvement Amendments
Ct cycle threshold
DNA deoxyribonucleic acid
ERCC External RNA Control Consortium
GLP good laboratory practice
GMP good manufacturing practice
ICH International Conference on Harmonization
ISO International Organization for Standardization
mRNA messenger ribonucleic acid
NIST National Institute of Standards and Technology
OD260/280 ratio of the absorbance measured at 260 nm and 280 nm
PCR polymerase chain reaction
Poly dT sequence repeat containing several thymine bases
QRT-PCR quantitative reverse transcriptase real-time PCR
RNA ribonucleic acid
rRNA ribosomal RNA
SNP single nucleotide polymorphism
VIM International Vocabulary of Basic and General Terms in Metrology

5 Overview

5.1 Gene Expression Profiling Technology

5.1.1 Gene Expression Overview

Broadly speaking, the term “gene expression profiling” is used to describe experiments that measure
changes in mRNA expression levels between different samples or within the same sample over time.
Generally, such protocols are carried out under the assumption that measured changes in mRNA
expression can be correlated with changes in physiological state or in disease status. The development of
high-throughput microarrays and QRT-PCR technologies has provided rapid, accurate methods for
measuring such changes. These new tools have become instrumental for basic scientific discovery, for
development of new clinical diagnostic tools, and for development and characterization of new drugs and
biological therapeutics.

©
Clinical and Laboratory Standards Institute. All rights reserved. 5
Number 19 MM16-P

5.1.2 Technology Platforms

5.1.2.1 QRT-PCR

Since the original descriptions of monitoring quantitative PCR in real time, a multitude of instruments
and detection approaches have come onto the market place.12-16 The technology is based on the fact that
there is a quantitative relationship between the starting amount of target message and the amount of
corresponding PCR product within the linear phase of the reaction for that message. Detection of the
target molecules is usually accomplished by measuring changes in fluorescence proportional to the
increase in the product of interest. Available instruments range from rapid cycling to high sample density
plates and use a variety of detection schemes in order to quantify input target material. Higuchi et al12
describe the measurement of fluorescence from ethidium bromide. Multiple subsequent detection methods
have been developed, applied, and commercialized. The specificity of QRT-PCR, as well as the
application of endogenous and exogenous standards for the method, are dramatically improved with the
application of fluorescent probes.17-19

5.1.2.2 Microarrays

Microarray-based hybridization techniques for peptide and nucleotide applications were first described in
the early 1990s.2,3 With advances in the technology, microarray platforms became more comprehensive in
their ability to monitor many genes simultaneously.4,5 The arrays themselves are solid surfaces onto which
single-stranded DNA has been chemically linked either following robotic spotting or via in situ synthesis
techniques. Each spot of single-stranded DNA molecules represents a single gene and a single array may
carry many thousands of spots. Each individual spot acts as a hybridization target for labeled cDNA
added in solution to the solid matrix.

Most microarray platforms utilize either one-color or two-color technologies. In two-color approaches,
the first sample of interest is labeled with a fluorescent tag of one emission spectrum, while a second
sample or reference material is labeled with a distinct and discernable second fluorescent tag. The two
samples are mixed in a competitive hybridization reaction, and the final analysis depends on the ratio of
one color to the other for each spot on the array. In one-color detection systems, each sample of interest
is labeled with a single color and hybridized to its own microarray. Final analysis depends on comparing
signal intensities from one sample on one array to corresponding values from a second sample on a
second array. This approach relies on manufacturing precision whereby each microarray must reliably
compare to each subsequent microarray. In both platforms, changes in the abundance of a particular
transcript in the input samples are detected by changes in the fluorescent intensities at its corresponding
spot on the microarray(s).

5.2 Gene Expression Profiling in Clinical Applications

The use of sensitive assays that detect changes in gene expression will increase our ability to perform
discriminating diagnostic and prognostic molecular tests in the clinical setting. Such technologies are
already in use in some contexts, and many more applications are expected soon. For instance, the use of
QRT-PCR has become standard of care for some somatic cancers, such as chronic myelogenous
leukemia, where monitoring of levels of the aberrant BCR/ABL transcript is used to evaluate response to
imatinib therapy and evaluate for presence of residual disease. The presence of this transcript has also
been extended to evaluation of some acute leukemias.20 Other markers, such as the IgH/BCL-2 fusion and
NPM-ALK are associated with follicular lymphoma and anaplastic large cell lymphoma respectively,
extending the possibility of using QRT-PCR markers for solid tumors.21,22

Gene expression measurements by QRT-PCR for genomic markers of immune system function have been
used for the assessment of tissue rejection in kidney transplants.23 Clinical microbiology applications
include the detection and characterization of drug resistance in multiple viral strains24,25 and bacterial,26
©
6 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

protozoan parasite,27 and fungal28 species. Accurate quantification of gene expression by QRT-PCR has
been applied to the detection of microorganisms summarized in Table 1.

Table 1. Clinical Microbiology Applications of QRT-PCR

MICROORGANISM REFERENCE
Francisella tularensis McAvin et al 200429
Specific genetic lineages of Anaplasma Polin et al 200430
phagocytophilum in Ixodes ricinus ticks
Periodontopathic bacteria Tannarella forsythensis Suzuki et al 200431
and Fusobacterium spp. in periodontal pockets
Candida albicans in concentrated oral rinse White et al 200432
cultures
Shigella Vu et al 200433
Human herpesvirus 8 and Epstein-Barr virus Friedrichs et al 200434
Parvovirus mutant Hokynar et al 200435
Severe acute respiratory syndrome (SARS) Hourfar et al 200436; Drosten et al, 200437; Hui et
coronavirus al, 200438

Adoption of QRT-PCR in clinical applications has followed platform developments from a tool for
confirmatory assays for hybridization experiments,39 to a stand-alone technology driven by the universal
guidelines for assay development in QRT-PCR,40 integration of robotic instruments into the assembly of
96- and 384-well qPCR formats,41 and the development of miniaturized matrices preloaded with primers
and probes for simultaneous QRT-PCR measurements of hundreds of gene targets.42

For microarrays, the long-term vision includes the identification of well-characterized molecular disease
subgroups to which diagnostic and/or prognostic information is attached. Currently, promising results
suggest that consistent changes in the expression patterns of a limited number of genes may be sufficient
to differentiate important molecular disease subgroups. Microarray-based studies have demonstrated the
potential utility of this approach in cancer diagnostics. For example, target genes have been identified that
may provide important clinical information about breast cancer outcomes43-46 and about diagnosis and
prognosis in hematologic cancers47-55 to highlight a few. Although validation and further analysis of these
studies is still required, the groundwork has been set for gene expression patterns to become an integral
part of patient clinical information in some situations.

In addition to molecular profiling, gene expression-based assays are beginning to play a role in clinical
microbiology, as established and novel pathogens are increasingly specifically identified by their gene
signatures.56-60 The long-term vision for such studies would include the generation of microarrays or
RNA-based assays that can identify multiple pathogens from a single sample with a rapid turnaround time
and high degree of specificity.

RNA-based assays have great potential in clinical tests. However, stringent controls that permit the
assessment of limit of detection, specificity, and accuracy of gene expression measurements for any given
test are required. Controls are also necessary to compare results between technologies, samples, and
laboratories. A number of recent studies examining within-platform performance have demonstrated good
reproducibility and repeatability when all aspects of the system are well-controlled.61-64

Currently, effective controls for RNA-based assays that span platforms are lacking, limiting the clinical
utility of these assays. Without them, comparability of the quality and reproducibility of data produced
intralaboratory and interlaboratory, as well as results between platforms, have been misleading or
uninformative.65-67 Sources of variability affecting interpretation of data include operator performance, the
characteristics of the sequences being queried, annotation differences, sample integrity, labeling
methodology, length of probe and hybridization dynamics (for microarray applications), and identity of
reference RNA.68,69
©
Clinical and Laboratory Standards Institute. All rights reserved. 7
Number 19 MM16-P

The introduction of shared, well-characterized external RNA controls should provide an additional tool
for assessing cross-platform comparability.

Because of the diverse nature of the parameters affecting test performance, along with the variation
intrinsic in RNA measurements and the potentially subtle changes that may be important clinical
parameters, it is critical that methods be established to effectively allow such quality assessments.
Without these protocols, interpretation of molecular tests becomes difficult and potentially misleading.70
Therefore, it is critical that appropriate external RNA controls are implemented to verify technical
performance as the first step toward quality management in gene expression profiling for clinical
applications.

5.3 A Note on Technical Performance and Clinical Performance

This document refers to two types of performance evaluation: quantitative technical performance and
qualitative clinical performance.

Technical performance refers to instrument or system verification and monitoring. It is independent of the
RNA tested and does not require the presence of a clinical sample. Technical performance as measured
by external RNA controls plays a role in measuring variability introduced by the instrument or system
and in establishing acceptable instrument performance criteria. For example, one could verify the
equivalence of different lots of reagents or reagents provided by different suppliers by monitoring
technical performance with the same external RNA controls. Several quantitative measures of technical
performance of an assay platform, such as limit of detection, precision, and linearity, can be established
by measuring external RNA controls.

Clinical performance measures require the presence of a clinical RNA sample, because they refer to the
specific gene or analyte being tested. External RNA enables the detection of systematic errors of different
analytes on the same platform; however, external RNA controls do not give specific metrics for the
clinical analyte. External RNA controls can provide valuable qualitative information about the test, such
as the presence of enzyme inhibitors in the clinical sample or catastrophic assay failure. While clinical
applications will naturally require additional, more comprehensive strategies for validation, traceability,
and quality management, the external RNA controls and protocols described in this document are useful
in both clinical and research settings.

6 External RNA Controls and Their Use in Expression Assays

External RNA control materials will play an essential role in developing, validating, and monitoring gene
expression assays. Results of an assay where the performance established with external RNA controls is
consistent with expectations are likely to be far more reliable than assays run without analytical controls.
However, performance assessments made with RNA controls are not intended to extrapolate to
performance assessments about particular transcripts in a sample. For this reason, external RNA controls
should not be used to predict measures of selectivity/specificity or signal-to-RNA concentration
calibration for particular transcripts.

The protocols developed in this document are part of a larger effort to establish standardized methods for
gene expression assays. These protocols can be used to validate technical performance when analyzing
clinical samples. An example of this use might be establishing performance limits for the signal that
arises from a given concentration of a particular external RNA control, and monitoring that metric in
patient samples.

©
8 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

6.1 Internal and External RNA Controls in Gene Expression Assays

Two types of external RNA controls are currently available to monitor technical performance in
microarray assays.

• Unlabeled, polyadenylated transcripts are combined with RNA samples before cDNA synthesis to
monitor the efficiency of target preparation. Often, the controls are a collection of transcripts with
different lengths that are mixed in at different concentrations in order to reflect the complex RNA
population in a true biological sample.

• Hybridization controls are labeled targets that are prepared separately and then added to the
hybridization cocktail for each sample. They are generally used as a set of genes at staggered
concentrations, in order to determine the limit of detection, as well as verify the quality of each
hybridization.

The external RNA controls discussed in this document refer only to labeling controls. Hybridization
controls are not included, as they are not germane or pertinent to QRT-PCR.

Although several sets of labeling controls are commercially available, each set uses different sequences
that are microarray specific. For example, one microarray provider uses bacterial sequences for external
RNA controls, while another manufacturer includes controls based on Arabidopsis sequences. Therefore,
the resources and the results from these external controls are not interchangeable between platforms.

QRT-PCR assays generally rely on an internal RNA control, such as a well-characterized invariant gene.
Currently, the choice of invariant gene, as well as the corresponding primers and amplicons, varies
between laboratories and assays. A frequent choice of endogenous control for QRT-PCR is 18S RNA,
because its quantification is an excellent approximation of the total RNA in the sample. In the specific
case of the 5´ nuclease reaction fluorescent primers and probes, reagents are available to multiplex
detection of 18S RNA with reproducible and high-PCR efficiency across multiple eukaryotes from yeast
to human.71 The use of external RNA controls will enable standardization by providing a common
reference material. It is optimal to use both internal and external RNA controls to assess both clinical and
technical parameters. Internal RNA controls aid in the normalization of analyte mass in a clinical sample,
whereas external controls provide a standardized reference material for the assessment of technical
performance.

6.2 Characteristics of External RNA Controls

The ERCC, an ad-hoc consortium, is developing a set of external RNA controls (available at
www.nist.gov). This effort will provide agreement on which external RNA controls can be used on all
microarray platforms as well as in QRT-PCR assays. The protocols and applications in this document are
based upon a standardized set of external RNA controls, such as those being developed by the ERCC.
These control characteristics are intended to be useful across different expression technologies and
platforms. The ERCC is working to make these materials readily accessible. They are intended to have
broad acceptance within the research and clinical communities.

The external RNA control set being developed by the ERCC will consist of approximately 96 well-
characterized polyadenylated transcripts, comprised of random unique and nonmammalian sequences, as
would be appropriate to ensure minimal cross-reactivity in mammalian clinical applications. All relevant
information regarding the process of sequence selection, verification, quality control, characterization,
nucleic acid sequences, recommended handling and storage, stability data, and observed performance
characteristics will be published in the open, archival literature and made publicly available by the ERCC.
Public availability of this information will facilitate adaptation and implementation by any interested user.

©
Clinical and Laboratory Standards Institute. All rights reserved. 9
Number 19 MM16-P

The ERCC is an international organization, and it is expected that the ERCC controls will be
internationally recognized.

External RNA Controls

• 96 different RNA transcripts

• 500-2000 nucleotides long
• 20 nucleotide 3´ polyadenylation
• Well-characterized for:
o sequence
ƒ integrity
o concentration
o source
ƒ random unique sequences
ƒ plant
ƒ bacterial
• Data describing controls will be publicly
available in the open literature:
o sequence
o unique identifier
o test and characterization methods
o test results

Figure 2. Characteristics of ERCC Controls

6.3 Using Control Materials to Assess Technical and Clinical Performance

External RNA controls can be used in both microarray and QRT-PCR experiments. Microarrays allow
simultaneous assessment of technical performance of the instrument and quality of the clinical sample.
Multiplexed QRT-PCR also can be used to detect external RNA controls with specific primers and probes
designed for this purpose. This application is particularly valuable for the identification of factors in the
chemical matrix that may affect overall PCR efficiency.

6.3.1 External RNA Controls in Microarray Assays

Microarray applications may use two types of pools containing external RNA controls: calibration curve
pools and differential comparison pool pairs. The selection of the pool type is dependent upon the final
work product of the assay. If the assay produces values related to the concentrations of mRNA species in
a given sample, then the calibration curve approach may be most appropriate. If, alternatively, the assay
compares the levels of mRNA species in two different samples, then a differential comparison pool pair
may be more appropriate. The formulation and use of these pools are described in Section 7.3.2.

6.3.1.1 Calibration Curve Pools

Calibration curve pools are used to verify the linear range of an assay. For example, a pool of external
RNA controls can be added to a clinical RNA sample before target preparation. If the controls were
present at different concentrations in the pool, then the expression measure (e.g., signal) for each control
could be plotted against its input concentration to generate a standard curve.

©
10 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

1.4

1.2

Background (lower limit of

quantification) Chemical or transducer
Response

0.8
saturation (upper limit of
quantification)

0.6

0.4 "Linear" measurement

regime

0.2

0
0 2 4 6 8 10 12

Amount of mRNA

Figure 3. Example of Calibration Curve Based on External RNA Controls

As shown in Figure 3, the expected form of a calibration curve is sigmoid.

• At low concentrations, background signal dominates, so that the constant target signal does not reflect
the input mRNA concentration. If background is subtracted aggressively, the signal observed at low
concentration mimics the no-signal noise distribution.

• At moderate concentrations, the signal responds linearly to increases in target concentration.

• At high concentrations, saturation phenomena (either of the binding feature or the detection device)
cause the signal to flatten at some maximum value.

Thus, a well-constructed calibration series can be used to determine the maximum linearity of a
microarray system (see Section 6.5.1.3, and CLSI/NCCLS document EP6—Evaluation of the Linearity of
Quantitative Measurement Procedures: A Statistical Approach). Limits of the assay will be identified as
the nonlinear sections. Poor performing assays will be apparent by compression or other changes in the
standard curve. Unlike QRT-PCR calibration assays (see Section 6.3.2.1), each concentration in a
microarray calibration pool is represented by a different external RNA control. This approach efficiently
generates the multiple signals necessary to construct a calibration curve in a single hybridization, but it
relies on different probes, with unique sequences and hybridization characteristics, to recognize each
external RNA target.

6.3.1.2 Pairs of Differential Comparison Pools

Microarrays are often used to perform differential expression measurements, either by comparing two
arrays hybridized to different samples (one-color experiments) or by comparing two samples hybridized
to one array (two-color experiments). Differential comparison pools are used to determine the limits of
differential detection in either platform. These experiments use two different pools containing the same
external RNA controls at different concentrations. For example, one sample is added with a pool that
contains a known concentration of an external RNA control. The second sample is added with a different

©
Clinical and Laboratory Standards Institute. All rights reserved. 11
Number 19 MM16-P

pool that contains a higher concentration of the same external RNA control. By comparing the results
generated with these two samples, one can confirm that the known difference in transcript abundance
between the pools is detected.

5
Number of Labs

0
0 1 2 3 4
Fold Change Detected

Figure 4. Example of Differential Comparisons Using External RNA Controls

An example of the use of differential comparison pools is presented in Figure 4. In this experiment, three
samples with a high concentration, 1:50 000 copies, and three samples with a low concentration, 1:100 000,
of the same external RNA control were distributed to multiple laboratories for target generation and
hybridization. Signal values for each hybridization were extracted, scaled, and then averaged for the three
replicates at each concentration. Figure 4 illustrates the distribution of the change between the high and
low concentrations observed in each laboratory. The median value of all laboratories is close to the
expected twofold change. It is important to note that in order to compare signals from different arrays or
different color channels, comparable linearity should be observed.

6.3.2 External RNA Controls in QRT-PCR Assays

External RNA controls can be used to monitor many facets of technical performance for QRT-PCR
assays. Operator competence, general assay precision, accuracy, limit of detection, assay range as well as
reagent and PCR platform comparisons can be assessed with standardized RNA controls. These uses of
external RNA controls do not require use of a clinical analyte but may also be multiplexed for detection in
a clinical analyte.72 An additional advantage of using a well-defined external RNA control, rather than an
internal invariant gene, is that technical performance can be compared across many independent sites and
QRT-PCR platforms. External RNA controls in QRT-PCR assays will be used for the evaluation of
technical performance in the absence of a clinical analyte, as well as for measuring qualitative clinical
performance with an analyte.

6.3.2.1 Standard Curves for Calibration

Individual external RNA controls can be serially diluted in order to provide a standard curve for
instrument calibration. This protocol would be carried out using enzyme and buffer reagents compatible
with the intended clinical assays, but without including actual clinical samples. As with microarray
calibration curves (see Section 6.3.1.1), the expression measure (e.g., Ct) for each sample would be
plotted against its input concentration. The calibration curves generated could be used for verifying
instrument and reagent performance, as well as competency testing and training.

©
12 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

6.3.2.2 Clinical Performance Evaluation

Individual external RNA controls are selected at a concentration relevant to the concentration of the
clinical analyte. This external RNA control is added to each clinical sample and serves to monitor assay
performance. The efficiency of amplification of the external RNA control will be similar to that of the
clinical analyte. The use of external RNA controls for the assessment of QRT-PCR clinical performance
generally utilizes a single control molecule added at a constant concentration to all samples. Diminished
amplification efficiency of an external RNA control in a particular sample is a useful indicator of
inhibitory effects. In the cases of multiplex instrumentation and multiplex assay design, it is feasible to
consider the possible use of at least two different external RNA controls (see Section 7.4.4).

6.4 Limitations of External Controls

Microarrays are complicated by the multiplex nature of the assay, as thousands of transcript levels can be
measured simultaneously. Random noise will result in inaccuracies in the individual measurements for
some transcripts. The external controls will provide information on the overall quality of the sample
labeling and hybridization for that microarray, but not for each individual transcript on the microarray.

In QRT-PCR experiments, external controls added to input RNA do not reflect gene expression levels of
that sample. Therefore, quantifying gene levels in clinical samples will still require the use of internal
controls specific for the assay design. External controls do not reflect internal gene expression levels of a
sample. External controls are used in QRT-PCR to identify matrix effects capable of reducing PCR
efficiency and assay limit of detection.

6.4.1 Evaluating Gene Specificity

The ratio of homologous-to-heterologous hybridization for a hybridizing nucleic acid with, for example,
10% mismatches, reveals the level of gene specificity inherent in that probe sequence. Gene specificity is
usually tested by hybridizing either a series of gene family members, or a series of oligonucleotides with
progressive sequence changes, to an array bearing probe(s) representing that transcript. Gene specificity
performance of individual probes is described as the ratio of homologous to heterologous hybridization
for a series of nucleic acids with progressively larger percent mismatches. The set of ERCC RNA
molecules do not systematically address specificity related to any one gene sequence.

6.4.2 Normalization or Scaling of Microarray Data

Datasets from two or more microarrays, or from two channels in two-color microarray formats are usually
adjusted to account for differences in limit of detection between arrays or channels. This normalization or
scaling can be done by multiplying expression levels from one array or channel by an adjustment factor
such that the mean or median level of hybridization for all arrays or channels is the same. While it is
possible to normalize array data by matching the mean or median of a set of RNA controls, this approach
is problematic for several reasons. First, the overall mRNA level in eukaryotic tissues is subject to a very
robust homeostasis—the level of mRNA per cell does not change perceptibly when expression of subsets
of genes are induced and repressed. It is likely to be much less true to assume that the levels of 10 to 100
external RNA controls are identical between samples. Second, the normalization to external RNA
controls would not necessarily reflect differences in labeling efficiencies between samples, for example,
errors due to inexact pipetting. Third, normalization using hundreds or thousands of genes within the
linear range of response of the assay is mathematically more robust than using a small number of external
RNA controls. For these reasons, it is recommended that normalization or scaling of microarray data be
performed based on many genes intrinsic to the biological samples.

©
Clinical and Laboratory Standards Institute. All rights reserved. 13
Number 19 MM16-P

6.4.3 Determining Absolute mRNA Levels

External RNA controls are quite useful to determine linearity of hybridization response, and an
approximate limit of detection for hybridization for a given platform. It is not reasonable, however, to
extrapolate from a standard curve of 10 to 100 external RNA controls to determine absolute abundances
of other intrinsic gene transcripts. This limitation is because probes hybridize to cognate sequences with
great variability and unpredictability. While physical chemists have generated a long list of rules to
predict which DNA or RNA sequences will not hybridize effectively, our ability to use these predictions
at the level of individual oligonucleotide is not well-established. Determination of absolute transcript
levels is best done by adding exogenous RNA from the gene in question into hybridizations, or into
quantitative QRT-PCR reactions in a standard curve.

6.5 Metrics for Assessing Technical Performance of the Expression Platform

Gene expression technologies were initially used for research purposes, and are now beginning to be used
in clinical applications. Because clinical laboratories require a high degree of rigor and validated
performance, this transition brings up many questions on the performance of expression assays and
whether they are appropriate for clinical use.

Well-designed experiments that include appropriate external controls will help alleviate fears and
demonstrate the quality of expression measurements. External RNA controls can be used in a number of
ways to verify the technical performance of an assay. In a clean system (buffer), the limit of detection,
precision, and linearity of the measurement of the external controls can be determined by replicate assays
under defined assay conditions. These results can be compared to similar experiments conducted with
external RNA controls added into typical samples. The degree to which the results in the clean system are
similar to the results in the typical samples is an indicator of the assay’s resistance to sample interference
effects or the matrix effects of the test material.

6.5.1 Metrics

Explanation of the following technical performance metrics is helpful in order to best assess the use of
external RNA controls in gene expression assays.

6.5.1.1 Precision

Precision is the closeness of agreement between independent test results obtained under stipulated
conditions.11 Intermediate precision refers to the closeness of agreement between interlaboratory results.
Note that particular sets of extremes in variation of conditions are termed repeatability (low-varying) and
reproducibility (high-varying). The replicate measures can address intralaboratory variability under
repeatability conditions, for instance the same RNA sample assayed in the same laboratory on different
days, and interlaboratory variability under reproducibility conditions, for instance, the same RNA sample
assayed in different laboratories (see the most current edition of CLSI/NCCLS document EP5—
Evaluation of Precision Performance of Quantitative Measurement Methods).

The external controls can be used as large pools of consistent, known composition that can be repeatedly
analyzed in one or many laboratories. The precision of these results can be evaluated as the standard
deviation or the coefficient of variation in Ct values.

For microarrays, an intra-array precision metric may also be generated when multiple spots of the same
probe are placed at different locations on the same array. Quantitative measures of precision depend
critically on the stipulated conditions. In an ideal situation, precision can be assessed by testing replicates
of a panel of patient samples. External RNA controls are valuable in evaluating precision in situations

©
14 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

where relevant patient samples are limited in quantity or number and cannot be pooled (i.e., lung biopsy
specimens).

The precision of external RNA control amplification and detection may be affected by the QRT-PCR
conditions and cycling parameters used for the target RNA. Universal primer and probe design guidelines,
amplification (thermocycling) protocols, primer and probe concentrations, and master mix formulations
recommended by developers of QRT-PCR platforms are essential to obtain precision values according to
the specifications of the platform.

6.5.1.2 Accuracy

Accuracy is the closeness of agreement between the average value obtained from a test result and an
accepted value.

NOTE: The term accuracy, when applied to a set of test results, involves a combination of random
components and a common systematic error or bias component.11

External RNA controls cannot be used to measure accuracy for a particular clinical analyte. Technical
performance for a particular assay might be established by comparing the signal for a single control
transcript to that established as a reference for that transcript when a method is commissioned and
validated in a laboratory.

Demonstration of accuracy is a definitive requirement for quantitative diagnostic tests in which an actual
analyte level is determined by the assay. For nonquantitative assays, the term “accuracy” as defined
above is inappropriate. For RNA expression assays that are based on pattern information, the positive and
negative predictive values of expression patterns will serve as measures of how effectively they detect
disease occurrence and susceptibility. The accuracy of the estimated change or Log Ratio expression
measures can be evaluated by introducing different differential comparison pools of external RNA
controls into samples being compared. Latin square assays have demonstrated that multiple microarray
platforms generate accurate results within a defined linear region.

For QRT-PCR, if there is a standard curve or an algorithm for calculating concentration for the assay,
then the value obtained for the external RNA control in the QRT-PCR can be compared to the assigned
value of the external RNA control to assess accuracy. This result is valid for the control and can also
extend to sample RNAs by multiplex detection of external controls spiked throughout multiple steps in
the sample isolation, purification, and conversion to cDNA.

6.5.1.3 Linearity

Linearity is the ability within a given range to provide results that are directly proportional to the
concentration of the analyte in the test sample; NOTES: a) Linearity typically refers to overall system
response (i.e., the final analytical answer rather than the raw instrument output); b) The linearity of a
system is measured by testing levels of an analyte which are known by formulation or known relative to
each other (not necessarily known absolutely); when the system results are plotted against these values,
the degree to which the plotted curve conforms to a straight line is a measure of system linearity (see the
most current edition of CLSI/NCCLS document EP6—Evaluation of the Linearity of Quantitative
Measurement Procedures: A Statistical Approach).

In microarray analysis, linearity refers to the relationship between hybridization signals and known or
relative RNA concentrations whereby the log-log plot of the signal generated from a serial dilution of
well-characterized and quantified control samples correlates linearly with the known concentrations of
those samples.5 The linear range of an assay is the set of detectable signal intensities that correlate in a
linear fashion with known concentrations of the controls and/or analytes. This metric is generally reported
©
Clinical and Laboratory Standards Institute. All rights reserved. 15
Number 19 MM16-P

as the slope and intercept of a least squares regression that compares observed amounts to observed signal
intensities of known transcripts in the test samples.

The external controls and their corresponding microarray probe or QRT-PCR primer sets will be selected
based on their ability to maintain linearity within a specified range of concentrations. An external control
RNA pool would be designed to contain multiple controls each at a different concentration within that
defined linear range. Although different, the linearity of results for each expression experiment can be
confirmed using the collection of relative Ct values for given sets of external controls.

6.5.1.4 Analytical Cross-Reactivity

Measurement of cross-reactivity refers to evaluation of the level of nonspecific binding of control and/or
test probes to nontargeted analytes that may be present in samples. Before external RNA controls are
chosen for a specific gene expression assay, they should be screened for cross-reactivity to the test probes
of the assay (see Section 7.2).

6.5.1.5 Limit of Detection

Limit of detection, is defined as the lowest amount of analyte in a sample that can be detected with
(stated) probability, although perhaps not quantified as an exact value.9 NOTE: Also called “lower limit
of detection,” “minimum detectable concentration” (or dose or value), and sometimes used to indicate
“sensitivity.”

Typically in diagnostic assays, the limit of detection is determined for each analyte being assayed.
However, in the case of expression microarrays, it is not realistic to determine the limit of detection of
each analyte being probed. In these cases, external RNA controls could be used to determine the limit of
detection of a series of representative RNA transcripts on a given platform.

An external RNA control is not useful in defining the limit of detection for any RNA other than itself.
Since the RNA control value will be tied to a NIST standard, it can be diluted serially and, for a given set
of QRT-PCR conditions, a limit of detection can be determined for the control.

7 Recommended Practices/Protocols

7.1 Overview

The following protocols address the use of external RNA controls in gene expression assays. Diagnostic
kit manufacturers may supply external RNA controls that have equivalent performance with the controls
described in this document. The use of a set of controls developed by the diagnostic manufacturer
could be used in place of the external controls described in this guideline provided the controls are
verified and allow technical performance assessment as described here. Section 7.2 includes protocols
that apply to both microarrays and QRT-PCR. Sections 7.3 and 7.4 deal with protocols specific to
microarrays and QRT-PCR, respectively.

As described in Section 5.3, the external RNA controls can provide quantitative technical performance
information and qualitative clinical performance information in gene expression assays. It is important
to note that the external RNA controls cannot be used to extrapolate quantitative information about
specific clinical analytes.

7.1.1 Scope and Purpose

The full benefits from use of high-quality, well-characterized external RNA controls will be obtained by
taking steps to ensure that these controls are used in a well-controlled, well-documented, reproducible
©
16 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

fashion. Several important steps should be taken. Perhaps the most important step is to utilize complete
protocols and to record any changes to the protocol during performance of any specific experiment.
Written records including protocols and any deviations from the routine protocols are important to ensure
that reliability of specific data sets can be assessed. Another important step would be to ensure that any
equipment required for making measurements with controls is operating correctly. This can be achieved
by performing routine maintenance and calibration of equipment, such as arranged under service
contracts. A third important step is to make sure that equipment operators are properly trained and that
they routinely keep accurate records regarding instrument service, calibration, and performance, in
addition to the experimental data. Finally, algorithms and software used in the analysis and interpretation
of data should be controlled and validated. These general principles form the foundation for generating
reliable data. More complete discussions of these practices can be found in applicable GLP, GMP, ISO,
ICH, and CLIA documents.73-80

7.2 Protocols for Preparation and Assessment of External RNA Controls

7.2.1 Synthesis of the External RNA Controls

External RNA controls are generally prepared by in vitro transcription of a suitable DNA template, using
a phage RNA polymerase, such as T7, T3, or SP6. Optimized kits for performing such transcriptions are
available from several manufacturers. The DNA template is produced from either a linearized plasmid
construct, or from a template generated by PCR. Most microarray labeling systems take advantage of the
presence of a polyadenylated tail on eukaryotic mRNAs. Therefore, external RNA controls are designed
to contain artificial polyadenylated tails. In the following discussion of DNA template sources, it is
assumed that the template will be produced with a 3´ polyadenylated tail.

Plasmid constructs can be built using any of a number of base plasmids that place a polycloning site
between a phage promoter and a poly-A/T region (e.g., pSP64polyA+). Templates can also be built via
PCR, by adding 5´ extensions to the primers that result in properly located phage promoter and poly-A/T
regions in the final construct. In either case, care must be taken to ensure that the correct sequence
direction (sense or antisense) is achieved after in vitro transcription. After preparation, external RNA
controls should be cleaned according to the kit manufacturer’s specifications and assessed for purity and
quantity via spectrophotometric methods (see Section 7.2.2).

7.2.2 Quantification and Integrity Confirmation

External RNA controls are best quantified via UV-visible spectroscopy. Mass concentrations of RNA can
be estimated using the formula81:

Yield of RNA (µg/ml)=(OD260)(44)(dilution factor),

and purity can be estimated using:

(OD260)/(OD280) ratio (~2.0 for pure RNA).

The ERCC external RNA controls vary in length between 500 and 2000 nucleotides. After quantification,
it is important to account for differences in molecular weights of the external RNA controls, because most
RNA quantification platforms measure the molar concentration of the mRNA species. The mass
concentration produced by the Maniatis formula can be converted into a molar concentration by dividing
by the mRNA molecular weight. Alternatively (and more accurately), the molar extinction coefficient for
a particular RNA species can be calculated, using standard procedures. Several web-based
implementations of these procedures are available, such as that hosted by the Dana-Farber Cancer
Institute, found at http://mbcf.dfci.harvard.edu/docs/oligocalc.html.

©
Clinical and Laboratory Standards Institute. All rights reserved. 17
Number 19 MM16-P

It is optimal to use full-length external mRNA controls as determined semiquantitatively via gel
electrophoresis under RNA-denaturing conditions, or quantitatively via capillary electrophoresis.82-84

7.2.3 Range of Concentrations of External RNA Controls

The amount of external RNA controls added to a clinical sample should be guided by the expected range
of mRNA concentrations in the sample itself. Generally, the linear range detected by microarray assays is
104 and for QRT-PCR assays is ~109. For microarray assays, the amount of external RNA control sample
added to a clinical sample should be adjusted to span a nominal range from less than 1 mRNA copy per
cell to greater than 1000 per cell. This range should be expanded by approximately two orders of
magnitude at each end for QRT-PCR methods (i.e., 0.01 mRNA copy per cell to 100 000 mRNAs per
cell).

Given generally accepted estimates of the amount of total RNA per cell and the fraction of total RNA that
is mRNA, this becomes a calculation of the ratio of external RNA control to total RNA, based on the
following assumptions:

The amount of RNA per mammalian cell varies somewhat, but is approximately 26 pg RNA/cell.85

• The mRNA/total RNA ratio varies considerably by cell or tissue. Generally reported estimates of
mRNA content vary from 1% to 4%. If we rely on an average of 2%, then each cell contains
approximately 520 fg mRNA (2% of 26 pg).

• An average mRNA molecule is 1.7 kb (5.6 x 105 g/mol).

Based on these estimates, an average mammalian cell contains 560 000 mRNA molecules. A detailed
discussion of these points can be found in the ERCC draft specification available at:
www.cstl.nist.gov/biotech/workshops/ERCC2003/031030%20ERCC%20Spec.pdf.

Calculations in this guidance document assume a clinical sample contains 105 cells (5.6 x 1010 mRNA
molecules). Calculations must be scaled to account for the actual number of cells in a particular clinical
sample type. Alternatively (and preferably), the quantity of RNA isolated should be measured prior to
addition of external RNA controls, and a constant ratio of added external RNA control to isolated RNA
should be maintained.

If the external RNA controls are chosen to have molecular weights near the mammalian average, then this
implies a spiking ratio of about 1:500 000 (for polyadenylated RNA) or 1:25 000 000 (for total RNA) to
achieve a level equivalent to ~1 copy/cell. The rest of the external RNA control concentration series
should then be expanded about this level, to enclose the range of limit of detection of the method used as
well as the expected signal range from the clinical sample. Based on these assumptions, the copy number
of external RNA controls added to a microarray sample will vary from 104 to 108, and for QRT-PCR up to
109.

In cases where samples consist of cell suspensions, direct counts of cells can be used to determine a
spiking ratio.

7.2.4 Testing Controls for Analytical Cross-Reactivity

It is critical to understand the level of nonspecific binding of a control and/or test probes to nontargeted
analytes that may be present in samples. Each control can be individually hybridized to the microarray (or
equivalent) at 10 times the concentration required in the assay to detect and measure cross-reactivity.
Once control probes are selected, they can be tested for cross-reactivity to representative patient samples,

©
18 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

including each tissue type to be assayed. Similarly, external RNA controls should not amplify with the
primers and probes used for the sample RNA(s) to be measured in a QRT-PCR assay.

7.2.5 Performance Testing

A solution of external RNA controls may contain contaminants that inhibit efficient amplification and/or
labeling. It is therefore important that some assessment of performance be made. This performance
testing should be completed before formulation of pools because a labeling inhibitor present in one
control RNA can potentially compromise the performance of all of the species present in a mixture.
Quality control testing of external RNA controls can be performed via QRT-PCR or on a microarray
designed to detect the external controls. One effective method for testing performance is to prepare serial
dilutions and evaluate hybridization signal intensity linearity (see Section 6.5.1.3 and CLSI/NCCLS
document EP6—Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical
Approach).

7.2.6 Storage

For optimal preservation, minimize freeze-thaw cycles and keep polyadenylated transcripts of the external
RNA controls stored in concentrated aliquots at -80 °C or in liquid nitrogen. Stability of the stored
external RNA controls should be evaluated prior to use.

7.3 External RNA Control Protocols for Microarrays

Microarrays allow simultaneous analysis of thousands of transcripts in one hybridization. Pools of

external RNA controls, rather than individual control transcripts, can be used in microarray assays. This
feature enables monitoring of technical performance and qualitative clinical performance in clinical
assays. For optimal benefit, these pooled controls should be included in every assay, in multiple
concentrations.

7.3.1 A Comment on Probe Design

Probe design will be performed by the array manufacturer, using their best methods.86 Access to probe
sequence information enables unambiguous interpretation of data. The ERCC will publish information
regarding probes used in performance testing of external RNA controls.

7.3.2 Pooling External RNA Controls

After synthesis, quantification, and performance testing of the controls, the individual external RNA
controls are combined into two different kinds of pools. Below is a general method for calculating the
necessary volume of each external RNA control in a pool or mixture, followed by examples of pool
compositions for the two different applications.

7.3.2.1 General Pooling Methods

During the creation of the pools, it is important to remember the following:

• All RNA transcripts must pass integrity and stability QC before mixing.

• Stocks of each RNA transcript should be 1000 to 500 ng/µL.

• A260 measurements should be done on RNA that is relatively similar in stock concentration (i.e., do
not compare 1200 ng/µL with 23 ng/µL).

©
Clinical and Laboratory Standards Institute. All rights reserved. 19
Number 19 MM16-P

• The total RNA concentrations of the pooled RNA should be ~100 ng/µL.

• At concentrations of 100 ng/µL and below, RNA binding to plastic tubes can be problematic;
therefore inclusion of carrier nucleic acid or use of silanized tubes is recommended. Yeast tRNA has
been successfully used as a carrier at 10 ng/µL.

• Stocks should be dissolved in one of the storage buffers listed below. Long-term stocks should be
stored at -80 °C. In citrate buffer, -20 °C is adequate.

• Stocks should not be freeze/thawed more than ten times. Create stocks useful to avoid multiple
freeze/thaws. Aliquot into several tubes.

• To ensure that the external RNA controls are completely dissolved, the solution should be thoroughly
vortexed and warmed to 25 to 37 °C.

• Because all external RNA controls are stored as concentrated stocks, dilution will be required. It is
best to keep serial dilutions to a minimum of three to four steps. During dilutions, transfers of more
than 2 µL should be used. Dilutions should not be stored. Working concentration should be used
immediately.

• Good record keeping during pooling is essential for identification and correction of errors. Label all
tubes with names, dates, concentrations, and user ID. Make sure notebook or production logs are in
agreement with all labels.

• Confirm RNA concentrations after long-term storage. This should not fluctuate more than 10%. Be
sure to vortex and heat prior to measurement.

• For quality control, the composition and integrity of each pool can be checked by capillary
electrophoresis, or microfluidic methods are recommended for QC of RNA spikes and mixtures.
Functional QC should be performed with QRT-PCR, or hybridization to a microarray.

Recommended Storage Buffers

• 1 mM sodium citrate, pH 6.4 +/- 0.2

• 0.1 mM EDTA: in nuclease-free, ultrapure water

• TE Buffer: 10 mM Tris-HCl, 1 mM EDTA, pH 7.0

7.3.2.2 Calculating Transcript Volumes

Proper laboratory procedure dictates that RNA pools should be constructed via a series of prepools in
order to minimize pipetting errors. However, it is also important to place consistency checks on the pre-
pooling scheme in order to ensure that the proper amount of each external RNA control is in the final
pool. The scheme below outlines a method for directly calculating the volume of each RNA stock present
in the final pool, given the initial stock concentration and the desired final concentration in the pool. The
actual procedure for pool construction should be cross-checked against this calculation as a quality
assurance step.

The final concentration of the ith of N control RNA species in a mixture, Ci(f), can be calculated from the
formula:

©
20 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

ciVi
C i( f ) = N
,
VW + ∑ V j
j =1

where ci is the stock concentration of a given mRNA species, Vj is the volume of that stock added to the
final formulation, and VW is the extra volume of water or buffer added to the formulation. Note that this
formula works for either mole- or mass-based concentrations.

Generally, the formulator is interested in the mole- or mass-fraction χi of the ith mRNA species in the final
mixture. This is given by the relationship

ciVi
χi = N
.
∑c V
j =1
j j

The above formula can be rearranged into a system of N simultaneous equations for the volumes Vj:

⎡⎛ χ ⎞ N ⎤
0 = ⎢⎜⎜ i ⎟⎟ ∑ c jV j ⎥ − Vi .
⎣⎝ ci ⎠ j =1 ⎦

This system is generally solved by volumes of the form

Aχ i
Vi = ,
ci

where A is an arbitrary constant that is related to the total volume (VT) of the final mixture by the
equation

VT − VW
A= .
N χj
∑c
j =1 j

Consider, for example, the set of mole fractions shown in Table 2 (corresponding to the “copies” in Table
3; the mole fractions were calculated by dividing each value of “copies” by the sum of all values of
“copies”):

©
Clinical and Laboratory Standards Institute. All rights reserved. 21
Number 19 MM16-P

Table 2. Example of Formal Pool Composition

Control ID Fraction (χ) Stock Concentration (pM) Volume (µL)
Control 1 0.00 100 0.00
Control 2 6.84 x 10-5 100 0.68
Control 3 2.16 x 10-4 100 2.16
Control 4 6.84 x 10-4 100 6.84
Control 5 2.16 x 10-3 100 21.64
-3
Control 6 6.84 x 10 1000 6.84
Control 7 2.16 x 10-2 1000 21.64
Control 8 6.84 x 10-2 1000 68.37
Control 9 2.16 x 10-1 1000 216.35
-1
Control 10 6.84 x 10 1000 683.68

In the above example, the constant A is 1 pmole-1, and VW is zero. In practice, one might choose to form
several subpools, and mix the subpools to achieve the desired final concentrations. However, a subsequent
formal calculation of the volume of each stock in the final mixture should correspond to Table 2.

7.3.2.3 Example Composition of a Calibration Pool

Calibration curves are usually presented on a logarithmic scale, so series based on successive doublings or
half-log increases (3.16 X) of concentration are most effective. The following table illustrates a series of
ten controls that span a detection range for microarray experiments from 105 to 108 copies per cell (as
described in Section 7.2.3).

Table 3. Example of Calibration Pool Composition

External RNA Pool A
Control (# copies)
Control 1 0
Control 2 3.16 x 104
Control 3 105
Control 4 3.16 x 105
Control 5 106
Control 6 3.16 x 106
Control 7 107
Control 8 3.16 x 107
Control 9 108
Control 10 3.16 x 108

7.3.2.4 Example Composition of a Differential Comparison Pool

Complete calibration of a differential expression system should evaluate spans of ratios of the same
external control between samples, as well as ratios of different external controls within the same sample.
This requires a minimum of two samples, with multiple external RNA controls at each concentration, in
order to span a range of differential expression ratios. The following table illustrates the composition of
two pools that could be used to detect differential expression. Note that the same ratio is evaluated using
controls at different amounts. Depending upon application, a larger linear range in ratio space may be
required. For example, some sets of differential pools may examine expression ratios representing 1:2,
1:10, 1:50, 1:100, and so on. It is left to the user to determine the most informative range of ratios for any
specific intended use.

©
22 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

Table 4. Example of Differential Comparison Pool Composition

External RNA Pool B Pool C Ratio
Control (# copies) (# copies) B:C
Control 11 0 0 ---
Control 12 3.16 x 104 3.16 x 104 1:1
Control 13 105 105 1:1
Control 14 3.16 x 105 3.16 x 105 1:1
Control 15 106 106 1:1
Control 16 3.16 x 106 3.16 x 106 1:1
Control 17 107 107 1:1
Control 18 3.16 x 107 3.16 x 107 1:1
Control 19 108 108 1:1
Control 20 3.16 x 108 3.16 x 108 1:1
Control 21 3.16 x 104 1.05 x 104 3:1
Control 22 105 3.33 x 104 3:1
5 5
Control 23 3.16 x 10 1.05 x 10 3:1
Control 24 106 3.33 x 105 3:1
Control 25 3.16 x 106 1.05 x 106 3:1
Control 26 107 3.33 x 106 3:1
7 7
Control 27 3.16 x 10 1.05 x 10 3:1
Control 28 108 3.33 x 107 3:1
Control 29 3.16 x 108 1.05 x 108 3:1
4 4
Control 30 1.05 x 10 3.16 x 10 1:3
Control 31 3.33 x 104 105 1:3
Control 32 1.05 x 105 3.16 x 105 1:3
Control 33 3.33 x 105 106 1:3
Control 34 1.05 x 106 3.16 x 106 1:3
Control 35 3.33 x 106 107 1:3
Control 36 1.05 x 107 3.16 x 107 1:3
7 8
Control 37 3.33 x 10 10 1:3
Control 38 1.05 x 108 3.16 x 108 1:3

7.3.3 Experimental Design

The external RNA controls are added to the samples prior to labeling (see Figure 1). A variety of options
exist for optimizing the use of external RNA controls in one- and two-color assays (see Table 5). As
shown in Table 5 below, multiple pools can be used in the same sample, provided they contain different
sets of external RNA controls.

©
Clinical and Laboratory Standards Institute. All rights reserved. 23
Number 19 MM16-P

Table 5. Experimental Design Scenarios

Platform Sample External RNA Control Application
Tumor 1 Pool A Calibration/linear range
One-color Tumor 2 Pool A Technical performance
Tumor 3 Pool A Qualitative clinical performance
Differential expression range
Tumor 1 Pool B
One-color Technical performance
Normal 1 Pool C
Qualitative clinical performance
Calibration/linear range
Tumor 1 Pool A + Pool B Differential expression range
One-color
Tumor 2 Pool A + Pool C Technical performance
Qualitative clinical performance
Calibration/linear range
Tumor 1 Pool A + Pool B Differential expression range
One-color
Normal 1 Pool A + Pool C Technical performance
Qualitative clinical performance
Calibration/linear range
Tumor 1-fluor 1 Pool A
Two-color Technical performance
Reference-fluor 2 Pool A
Qualitative clinical performance
Differential expression range
Tumor 1-fluor 1 Pool B
Two-color Technical performance
Reference-fluor 2 Pool C
Qualitative clinical performance
Differential expression range
Tumor 1-fluor 1 Pool B
Two-color Technical performance
Normal 1-fluor 2 Pool C
Qualitative clinical performance
Calibration/linear range
Tumor 1-fluor 1 Pool A + Pool B Differential expression range
Two-color
Tumor 2-fluor 2 Pool A + Pool C Technical performance
Qualitative clinical performance
Calibration/linear range
Tumor 1-fluor 1 Pool A + Pool B Differential expression range
Two-color
Reference-fluor 2 Pool A + Pool C Technical performance
Qualitative clinical performance

7.3.4 Data Preprocessing

7.3.4.1 A Comment on Scaling and Normalization Models

Comparisons between different one-color arrays or between channels of two-color arrays are usually
preceded by data processing steps called scaling and normalization. Scaling is the operation of centering
each data set to an arbitrary constant. Normalization is the operation of removing experimental artifacts
of nonbiological origin such as minor differences in labeling efficiency or scanner settings. A variety of
scaling and normalization methods exist.87-89

Also, another useful resource for normalization is the Normalization Working Group of the Microarray
Gene Expression Data (MGED) organization (http://www.mged.org), which is attempting to define such a
standard90 and have been described in the most current edition of CLSI document MM12—Diagnostic
Nucleic Acid Microarrays.

An important potential use of external RNA controls is to test the appropriateness of the assumptions
underlying scaling and normalization algorithms. If the assumptions are not valid, then the signal
intensity differences observed using a pair of differential comparison pools will be skewed. Likewise, the
normalized level of signal will vary between samples containing calibration curve pools.
©
24 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

7.4 External RNA Control Protocols for QRT-PCR

QRT-PCR enables quantitative analysis of template concentration. External RNA controls can be used for
assessment of technical and clinical assay performance. Generally, technological limitations dictate that
one or a few external RNA controls would be used for these purposes. QRT-PCR instrumentation is
limited to detection of one or a few (e.g., four) simultaneous targets per reaction tube.

7.4.1 General Use

Two types of QRT-PCR detection technologies exist:

• Single-color DNA-binding dyes (e.g., SYBR Green probe)

This method permits detection of a single target per assay tube.

• Target-specific, fluorescent hybridization probe(s)

This method permits detection of several targets per assay tube.

7.4.2 Primer and Amplicon Design and Optimization

As with microarray probes, QRT-PCR primer design and assay optimization are the responsibility of
individual investigators. The term amplicon refers to the product sequence that is specifically amplified
during PCR.

Primer and probe design are some of the most critical assay components necessary for optimal
performance of an assay. Multiple commercial and shareware software are available to assist in primer
and probe design. Instrument providers can also provide a source of recommendations for effective
primer and probe design. Briefly, several factors to consider for primer, probe, and amplicon design
include:

• Amplicon lengths should not exceed 150 bp.

• Dissociation temperatures (TM) for primers and probes must correspond to specifications associated
with annealing temperatures and the chemical matrix of the master mix. In the specific case of the
master mix, recommended parameters are:

Annealing Temperature: 60 oC
TM for primers: 58 to 60 oC
TM for 5´ nuclease reaction fluorescent probes: 65 to 67 oC

• Primers and probes should be designed to minimize secondary structures.91

PCR efficiency can be assessed by analysis of serial dilutions of the various analyte targets. Such
dilutions should produce similar slopes for all intended analytes when plotted as semilog plots (cycle
threshold (Ct) versus analyte concentration). During assay development, the acceptable variance of
slopes should be determined using sound statistical practices and replicate assays.92-96

The specific external RNA control(s), primers, and amplicons for the control should be designed
following appropriate specifications in order to match PCR efficiencies with intended assay targets.

©
Clinical and Laboratory Standards Institute. All rights reserved. 25
Number 19 MM16-P

7.4.3 External RNA Controls to Assess Technical Performance

7.4.3.1 Use of a Single Concentration of an External RNA Control

A large master reaction mix containing all reaction components and an external RNA control at a single
concentration can be made. Select a concentration of the external RNA control so that it will be detected
within the known linear assay range. If the control concentration is too low, the variance within replicates
will be unacceptable. This master mix can be equally distributed to all wells/tubes of an instrument and
tested for replicate precision. This type of analysis yields valuable insight into the performance of the
instrument and assay. Calculations can be made to determine the precision of replicate measurements and
the associated variance. Additionally, the measured variance can help assess realistic limits of
discrimination. If, for example, replicate variance of an optimized external RNA control assay approaches
twofold within such a replicate assay, twofold discrimination of clinical analytes would not be achievable.
Such error could be cause for requesting instrument calibration from the appropriate service organization.

7.4.3.2 Use of a Serial Dilution of External RNA Controls

An optimized QRT-PCR assay designed to detect and measure an external RNA control can be used to
train and test assay operators. Replicate serial dilutions can be periodically performed and assessment of
replicate slopes and individual control dilutions can be used to determine operator and assay performance
levels. Instruments can also be compared using such an experimental design scheme. The minimal
molecular amounts represented in dilutions that are successfully discriminated can serve as a general
guide to technical capabilities of change discrimination. It is important to avoid over-interpretation of
quantitative information obtained from the external controls. External RNA controls serve as a general
guide but not an absolute measure of the quantities of clinical target analytes.

7.4.3.3 Use of a Serial Dilution of an External RNA Control for a QRT-PCR Calibration Standard

An appropriate serial dilution of a selected RNA control can be made that reflects the molecular range of
intended clinical measurements (see Section 7.3.2 for details on molecular-based formulation). Dilutions
should be made fresh from a concentrated stock of external RNA controls. Dilutions should not be used
after extended periods of storage. Ideally, concentrations will be selected that range from less than single
molecule detection to beyond the upper suspected limit of clinical analyte intended to be measured. The
actual dilution factor can be determined by the requirements for assay accuracy, instrument platform (96-
well versus 384-well), and of the laboratory. Minimally, at least four dilutions should be made in order to
generate adequate data for linear regression analysis.

7.4.4 Use of External RNA Controls to Assess QRT-PCR Clinical Performance

As described above (see Section 6.3), the use of external RNA controls for assessment of QRT-PCR
clinical performance will generally utilize a single control molecule added at a constant concentration to
all samples. In the case that multiplex instrumentation and multiplex assay design are used, it is feasible
to consider the possible use of at least two different external RNA controls.

7.4.4.1 Use of External RNA Controls to Assess Effects of Complex Biological Matrices and Methods
of Clinical RNA Extraction

Use of a satisfactory quality control result as an indicator of a valid patient test result requires that the
external RNA control be tested in the same mixture as the patient sample. Adding the external RNA
control ensures that the control is subjected to at least some of the variables potentially affecting the
patient sample during assay analysis. The expected concentration means and variances of the external
RNA control can be established by repetitive analysis of this control in buffer (see the most current
edition of CLSI/NCCLS document C24—Statistical Quality Control for Quantitative Measurements:
©
26 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

Principles and Definitions). The effect of adding the external RNA control to clinical RNA purified from
biological matrices (e.g., tissues, cells, body fluids) can be tested next. If significant differences in
measurements exist between the buffer and biologically derived RNA experiments, it could suggest
reason to examine a different means of RNA purification.

7.4.4.2 Use of External RNA Controls for Assessment of Clinical Assay Results

Protocols supporting scenarios that cover the simplest single color detection systems and multicolor
applications are included in this section and referred to as protocols A, B, and C. In a single color system,
replicates of a sample are recommended to fully assess the relative level of expression of the target RNA
(Protocol A). When multicolor chemistries are used, there are two possible scenarios for external RNA
control use. The first scenario uses only a single external RNA control at a constant concentration
(Protocol B). If more than three color channels are available, the choice is available to include two
different external RNA controls or to use a single constant amount of an external RNA control and
increase the number of target genes of interest (Protocol C).

The following protocols assume that basic assay development and assay validation are being performed
for each target molecule. Parameters such as assay precision, limit of detection, accuracy, and specificity
should be measured for target molecules, including the external RNA controls. External RNA controls
will have been selected during assay development and validation, such that they represent a concentration
with a reporting cycle threshold of detection (i.e., Ct) at or near the target genes of interest. If two
controls are used, they could be selected to be within high and low assay ranges. During assay
development, the efficiency of external RNA control amplification should be similar to the amplification
efficiency of the targets of interest. Amplification efficiency is measured using serial dilutions of samples
and controls.94-96 Amplification efficiency can be determined from the slope of an amplification curve.
The amount of product (Nc) generated in PCR after c cycles is given by Nc=Ni (1+f)c where Ni is the
initial number of template molecules and f is the reaction efficiency. In a graph of log Nc versus c, the
slope is log(1+f).97

In addition to an external RNA control, an invariant gene is selected and verified for normalization of
clinical sample RNA mass loads, during the development and validation of QRT-PCR assays.

©
Clinical and Laboratory Standards Institute. All rights reserved. 27
Number 19 MM16-P

Single Color 3 Color > 3 Color

Protocol A Protocol B Protocol C

Relative Quantification

Interpretation

Figure 5. QRT-PCR Protocol Selection for External RNA Controls

CAUTION: Since the polymerase chain reaction yields excessive amounts of final product, it is
imperative that good laboratory practice include physical separation of pre-PCR activities, reagents, and
equipment from post-PCR materials. Cross-over prevention methods should be employed and negative
controls should frequently be used to ensure cleanliness and accurate results. In all cases, note that QRT-
PCR conditions, master mix composition, and cycling conditions have been previously developed and
verified during assay development.

Protocol A. Single Color System, Replicates

A constant amount of a single external RNA control is chosen based on amplification efficiency match to
target genes of interest in the sample. This external RNA control is added to every sample at constant
amounts. Replicates of each sample are analyzed in different tubes. Depending on development criteria,
multiple replicates may need to be analyzed for each target sequence.

1. The total number of replicates of each of the following reactions will be determined during assay
development and governed by availability of clinical sample RNA: Reaction 1, external RNA
control; Reaction 2, target gene of interest; Reaction 3, invariant gene endogenous to sample RNA.

Begin by making appropriate master mixes of primer and probe combinations for each gene to be
analyzed. If an external RNA control, one gene of interest, and one internal invariant gene are to be
analyzed, this requires three separate mixes of primer and probes, as listed above. Next, make a basic
reaction master mix. In a separate tube, pipette the appropriate amount of external RNA control for
all assay replicates and add the appropriate aliquot of clinical sample RNA to the external RNA
control tube. Add RNA mixture directly to basic reaction master mix. Aliquot appropriate volume of
RNA and basic reaction master mix solution to each of the assay reaction replicate tubes. Using
separate pipette tips, add the appropriate amount of primer and probe mixture to each assay reaction
tube.

2. Perform assay and determine Ct values for each of the three reactions.
©
28 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

3. Compare Ct values for each set of reaction replicates.

4. Comparison of external RNA controls across different clinical samples implies that QRT-PCR
reactions are valid and within limits determined during validation.

5. From target gene of interest and endogenous invariant gene, the relative amount of target RNA can be
inferred.

Protocol B. Multicolor chemistry, single external RNA control at a constant concentration (three-
color QRT-PCR)

1. Add the chosen amount of external RNA control into the extracted clinical sample RNA. Volumes
will be chosen based on the total number of replicates to be analyzed.

2. Add the RNA mixture to a basic reaction master mix containing primer and probes for the external
RNA control, target gene of interest, invariant gene, and other reaction components. Add appropriate
amounts of this mixture to assay reaction tubes.

3. Perform assay and determine Ct values for each of the three above reactions.

4. Compare Ct values for each set of sample replicates.

5. Comparison of external RNA controls implies that QRT-PCR reactions are valid and within limits
determined during validation.

6. From target gene of interest and endogenous invariant gene, the relative amount of target RNA can be
inferred.

NOTE: This is a multiplex primer and probe assay and adequate attention should be given to optimizing
such multiplex assays to prevent inaccurate results.

If the capabilities exist to use more than three colors, Protocol B can be followed and the additional color
used for additional target genes of interest. Alternatively, Protocol C can be followed with the additional
colors being used to assess additional external RNA controls.

Protocol C. Multicolor chemistry, two different external RNA controls or a single constant amount
of external RNA control with multiple target genes (more than three-color QRT-PCR)

1. Add the chosen amount of two or more different external RNA controls into the extracted clinical
sample RNA. The concentration of the different external RNA controls should be chosen to lie
within the predetermined assay range (based on expected Ct of the clinical analyte), but may be
chosen to detect near the lower and higher limits of this range. If a third external RNA control is
used, it could be added at a midrange level.

2. Add the RNA mixture into a basic reaction master mix containing primer and probes for external
RNA control, target gene of interest, invariant gene, and other reaction components. Add aliquots of
this mixture into appropriate reaction replicate tubes.

3. Perform assay and determine Ct values for each of the three above reactions.

4. Compare Ct values for each set of sample replicates.

©
Clinical and Laboratory Standards Institute. All rights reserved. 29
Number 19 MM16-P

5. Comparison of external RNA controls implies that QRT-PCR reactions are valid and within limits
determined during validation.

If more than one external RNA control has been used and has been added at high, low, and
intermediate predetermined amounts, the analysis of these external RNA controls in each sample can
permit one to infer that the test performed within the expected linear range of the assay.

6. From target gene of interest and endogenous invariant gene, the relative amount of target RNA can be
inferred.

©
30 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

References
1
Gibson UEM, Heid CA, Williams PM. A novel method for real time quantitative RT-PCR. Genome Res. 1996;6:995-1003.
2
Fodor SPA, Read JL, Pirrung MC, Stryer L, Lu AT, Solas D. Light-directed, spatially addressable parallel chemical synthesis. Science.
1991;251:767-773.
3
Fodor SPA, Rava RP, Huang XC, Pease AC, Holmes CP, Adams CL. Multiplexed biochemical assays with biological chips. Nature.
1993;364:555-556.
4
Schena M, Shalon D, Davis RW, Brown PO. Quantitative monitoring of gene expression patterns with a complementary DNA microarray.
Science. 1995;270:467-470.
5
Lockhart DJ, Dong H, Byrne MC, et al. Expression monitoring by hybridization to high-density oligonucleotide arrays. Nature Biotechnology.
1996;14:1675-1680.
6
Cronin M, Ghosh K, Sistare F, Quackenbush J, Vilker V, O’Connell C. Universal RNA reference materials for gene expression. Clin Chem.
2004;50(8):1464-1471.
7
ISO. International Vocabulary of Basic and General Terms in Metrology. Geneva: International Organization for Standardization; 1993.
8
ISO. In Vitro Diagnostic Medical Devices – Measurement of Quantities in Biological Samples – Metrological Traceability of Values Assigned
to Calibrators and Control Materials. ISO 17511. Geneva: International Organization for Standardization; 2003.
9
WHO Expert Committee on Biological Standardization. Glossary of Terms for Biological Substances Used for Texts of the Requirements.
WHO unpublished document BS/95.1703. Geneva: World Health Organization; 1995.
10
WHO Expert Committee on Biological Standardization. Glossary of Terms for Biological Substances Used for Texts of the Requirements.
WHO unpublished document BS/95.1793. Geneva: World Health Organization; 1995.
11
ISO. Statistics – Vocabulary and Symbols – Part 1: Probability and General Statistical Terms. ISO 3534-1. Geneva: International
Organization for Standardization; 1993.
12
Higuchi R, Fockler C, Dollinger G, Watson R. Kinetic PCR analysis: real-time monitoring of DNA amplification reactions. Biotechnology
(N.Y.). 1993;11(9):1026-1030.
13
Heid CA, Stevens J, Livak KJ, Williams PM. Real time quantitative PCR. Genome Res. 1996;610:986-984.
14
Wittwer C, Herrmann M, Moss A, Rasmussen R. Continuous fluorescence monitoring of rapid cycle DNA amplification. Biotechniques.
1997;1:130-1,130-8.
15
Myers TW, Gelfand DH. Reverse transcription and DNA amplification by a Thermus thermophilus DNA polymerase. Biochemistry.
1991;30(31):7661-7666.
16
Gibson KM, Mori J, Clewly JP. Detection of HIV-1 in serum using reverse transcriptase and the polymerase chain reaction (RT-PCR). J Virol
Methods. 1993;43(1):101-109.
17
Koch WH. Technology platforms for pharmacogenomic diagnostic assays. Nat Rev Drug Discovery. 2004;3(9):749-761.
18
Tan BH, Lim EA, Liaw JC, Seah SG, Yap EP. Diagnostic value of the real-time capillary thermal cycler in viral detection. Expert Rev Mol
Diagn. 2004;4(2):219-230.
19
Burkardt HJ. Standardization and quality control of PCR analyses. Clin Chem Lab Med. 2000;38(2):87-91.
20
Nashed AL, Rao KW, Gulley ML. Clinical applications of BCR-ABL molecular testing in acute leukemia. J Molecular Diagn. 2003;5(2):63-
72.
21
Devassa SS, Fears T. Non-Hodgkin’s lymphoma time trends: United States and international data. Cancer Res. 1992;52(Suppl):5432s-5440s.
22
Pirc-Danoewinata H, Chott A, Onderka E, et al. Karyotype and prognosis in non-Hodgkin lymphoma. Leukemia. 1994;8:1929-1939.
23
Hoffmann SC, Hale DA, Kleiner DE, et al. Functionally significant renal allograft rejection is defined by transcriptional criteria. Am J
Transplant. 2005;5(3):573-581.
24
Josefsson A, Livak K, Gyllensten U. Detection and quantitation of human papillomavirus by using the fluorescent 5′ exonuclease assay. J Clin
Microbiol. 1999;37(3):490-496.
25
Niesters HG. Clinical virology in real time. J Clin Virol. 2002;25:Suppl 3:S3-12.
26
Matsuki T, Watanabe K, Tanaka R. Genus- and species-specific PCR primers for the detection and identification of bifidobacteria. Curr Issues
Intest Microbiol. 2003;4(2):61-69.

©
Clinical and Laboratory Standards Institute. All rights reserved. 31
Number 19 MM16-P

27
Bell AS, Ranford-Cartwright LC. A real-time PCR assay for quantifying Plasmodium falciparum infections in the mosquito vector. Int J
Parasitol. 2004;34(7):795-802.
28
Atkins SD, Clark IM. Fungal molecular diagnostics: a mini review. J Appl Genet. 2004;45(1):3-15.
29
McAvin JC, Morton MM, Roudabush RM, Atchley DH, Hickman JR. Identification of Francisella tularensis using real-time fluorescence
polymerase chain reaction. Mil Med. 2004;169(4):330-333.
30
Polin H, Hufnagl P, Haunschmid R, Gruber F, Ladurner G. Molecular evidence of Anaplasma phagocytophilum in Ixodes ricinus ticks and
wild animals in Austria. J Clin Microbiol. 2004;42(5):2285-2286.
31
Suzuki N, Yoshida A, Saito T, Kawada M, Nakano Y. Quantitative microbiological study of subgingival plaque by real-time PCR shows
correlation between levels of Tannerella forsythensis and Fusobacterium spp. J Clin Microbiol. 2004;42(5):2255-2257.
32
White PL, Williams DW, Kuriyama T, Samad SA, Lewis MA, Barnes RA. Detection of Candida in concentrated oral rinse cultures by real-
time PCR. J Clin Microbiol. 2004;42(5):2101-2107.
33
Vu DT, Sethabutr O, Von Seidlein L et al. Detection of Shigella by a PCR assay targeting the ipaH gene suggests increased prevalence of
shigellosis in Nha Trang, Vietnam. J Clin Microbiol. 2004;42(5):2031-2035.
34
Friedrichs C, Neyts J, Gaspar G, De Clercq E, Wutzler P. Evaluation of antiviral activity against human herpesvirus 8 (HHV-8) and Epstein-
Barr virus (EBV) by a quantitative real-time PCR assay. Antiviral Res. 2004;62(3):121-123.
35
Hokynar K, Norja P, Laitinen H et al. Detection and differentiation of human parvovirus variants by commercial quantitative real-time PCR
tests. J Clin Microbiol. 2004;42(5):2013-2019.
36
Hourfar MK, Roth WK, Seifried E, Schmidt M. Comparison of two real-time quantitative assays for detection of severe acute respiratory
syndrome coronavirus. J Clin Microbiol. 2004;42(5):2094-2100.
37
Drosten C, Doerr HW, Lim W, Stohr K, Niedrig M. SARS molecular detection external quality assurance. Emerg Infect Dis.
2004;10(12):2200-2203.
38
Hui RK, Zeng F, Chan CM, Yuen KY, Peiris JS, Leung FC. Reverse transcriptase PCR diagnostic assay for the coronavirus associated with
severe acute respiratory syndrome. J Clin Microbiol. 2004;42(5):1994-1999.
39
Rajeevan MS, Vernon SD, Taysavang N, Unger ER. Validation of array-based gene expression profiles by real-time (kinetic) RT-PCR. J Mol
Diagn. 2001;3(1):26-31.
40
Livak KJ, Schmittgen TD. Analysis of relative gene expression data using real-time quantitative PCR and the 2(-Delta Delta C(T)) Method.
Methods. 2001;25(4):402-408.
41
Pinhasov A, Mei J, Amaratunga D, et al. Gene expression analysis for high throughput screening applications. Comb Chem High Throughput
Screen. 2004;7(2):133-140.
42
De Mello AJ. DNA amplification: does “small” really mean “efficient”? Lab on a chip. 2001;1:24N-29N.
43
Ramaswamy S, Ross KN, Lander ES, Golub TR. A molecular signature of metastasis in primary solid tumors. Nat Genet. 2003;33:49-54.
44
Huang E, Ishida S, Pittman J, et al. Gene expression phenotypic models that predict the activity of oncogenic pathways. Nat Genet.
2003;34:226-230.
45
van de Vijver MJ, He YD, van’t Veer LJ, et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med.
2002;347:1999-2009.
46
Van’t Veer LJ, Dai H, van de Vijver MJ, et al. Gene expression profiling predicts clinical outcome of breast cancer. Nature. 2002;415:530-
536.
47
Golub TR, Slonim DK, Tamayo P, et al. Molecular classification of cancer: class discovery and class prediction by gene expression
monitoring. Science. 1999;286:531-537.
48
Alizadeh AA, Eisen MB, Davis RE, et al. Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature.
2000;403:503-511.
49
Rosenwald A, Wright G, Chan WC, et al. The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell
lymphoma. N Engl J Med. 2002;346:1937-1947.
50
Shipp MA, Ross KN, Tamayo P, et al. Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised
machine learning. Nat Med. 2002;8:68-74.
51
Rosenwald A, Wright G, Wiestner A, et al. The proliferation gene expression signature is a quantitative integrator of oncogenic events that
predicts survival in mantle cell lymphoma. Cancer Cell. 2003;2:185-197.

©
32 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

52
Bullinger L, Dohner K, Bair E, et al. Use of gene-expression profiling to identify prognostic subclasses in adult acute myeloid leukemia. N
Engl J Med. 2004;350:1605-1616.
53
Valk PJ, Verhaak RG, Beijen MA, et al. Prognostically useful gene-expression profiles in acute myeloid leukemia. N Engl J Med.
2004;350:1617-1628.
54
Ebert BL, Golub TR. Genomic approaches to hematologic malignancies. Blood. 2004;104(4):923-932.
55
Neben K, Schnittger S, Brors B, et al. Distinct gene expression patterns associated with FLT3- and NRAS-activating mutations in acute
myeloid leukemia with normal karyotype. Oncogene. 2005;24(9):1580-1588.
56
Ksiazek TG, Erdman D, Goldsmith CS, et al, and the SARS Working Group. A novel coronavirus associated with severe acute respiratory
syndrome. New Engl J Med. 2003;348:1953-1966.
57
Wang D, Coscoy L, Zylberberg M, et al. Microarray-based detection and genotyping of viral pathogens. Proc Natl Acad Sci USA.
2002;99:15687-15692.
58
Rota PA, Oberste MS, Monroe SS, et al. Characterization of a novel coronavirus associated with severe acute respiratory syndrome. Science.
2003;300:1394-1399.
59
Bodrossy L, Sessitsch A. Oligonucleotide microarrays in microbial diagnostics. Curr Opin Microbiol. 2004;7:245-254.
60
Shieh B, Li C. Multi-faceted, multi-versatile microarray: simultaneous detection of many viruses and their expression profiles. Retrovirology.
2004;1:11.
61
Piper MD, Daran-Lapujade P, Bro C, et al. Reproducibility of oligonucleotide microarray transcriptome analyses. An interlaboratory
comparison using chemostat cultures of Saccharomyces cerevisiae. J Biol Chem. 2002;277(40):37001-37008.
62
Barczak A, Rodriguez MW, Hanspers K, et al. Spotted long oligonucleotide arrays for human gene expression analysis. Genome Res.
2003;13(7):1775-1785.
63
Bakay M, Chen YW, Borup R, Zhao P, Nagaraju K, Hoffman EP. Sources of variability and effect of experimental approach on expression
profiling data interpretation. BMC Bioinformatics. 2002;3(1):4.
64
Hughes TR, Mao M, Jones AR, et al. Expression profiling using microarrays fabricated by an ink-jet oligonucleotide synthesizer. Nat
Biotechnol. 2001;19(4):342-347.
65
Tan PK, Downey TJ, Spitznagel EL, Jr., et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic
Acids Res. 2003;31(19):5676-5684.
66
Waring JF, Ulrich RG, Flint N, et al. Interlaboratory evaluation of rat hepatic gene expression changes induced by methapyrilene. Environ
Health Perspect. 2004;112(4):439-448.
67
Chu TM, Deng S, Wolfinger R, Paules RS, Hamadeh HK. Cross-site comparison of gene expression data reveals high similarity. Environ
Health Perspect. 2004;112(4):449-455.
68
Shippy R, Sendera TJ, Lockner R, et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in
making cross-platform correlations. BMC Genomics. 2004;5(1):61.
69
Shi L, Tong W, Goodsaid F, et al. QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev Mol
Diagn. 2004;4(6):761-777.
70
Ransohoff DF. Rules of evidence for cancer molecular-marker discovery and validation. Nat Rev Cancer. 2004;4:309-314.
71
Goodsaid FM, Palamanda JR, Montgomery D, et al. Assessment of temporal biochemical and gene transcription changes in rat liver
cytochrome P450: utility of real-time quantitative RT-PCR. Pharm Res. 2003;20(9):1373-1380.
72
Meijerink J, Mandigers C, van de Locht L, Tonnissen E, Goodsaid F, Raemaekers J. A novel method to compensate for different amplification
efficiencies between patient DNA samples in quantitative real-time PCR. J Mol Diagn. 2001;3(2):55-61.
73
Medicare, Medicaid, and CLIA programs; regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA).
74
Medicare, Medicaid, and CLIA programs; regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Final
rule. Fed Regist. 1992;57(40):7002-7186.
75
Medicare, Medicaid, and CLIA programs; laboratory requirements relating to quality systems and certain personnel qualifications. Final rule.
Fed Regist. 2003;68(16):3639-3714.
76
Title 21 Code of Federal Regulations, Chapter 1, FDA, 4-1-97, Subchapter A, Part 58, General requirements for the competence of testing and
calibration laboratories.

©
Clinical and Laboratory Standards Institute. All rights reserved. 33
Number 19 MM16-P

77
ISO/IEC 17025:1999 Text on validation of Analytical Procedures (ICH-guideline Q2A); Validation of Analytical Procedures: Methodology
(ICH-Guideline Q2B).
78
ISO. Medical laboratories - Particular requirements for quality and competence. ISO 15189. Geneva: International Organization for
Standardization; 2003.
79
21 Code of Federal Regulations Part 58, 2004. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm.
80
42 Code of Federal Regulations, Part 493, 2004. Available at: http://www.access.gpo.gov/nara/cfr/waisidx_ 04/42cfr493_04.html.
81
Sambrook J, Fritsch EF, Maniatis T. Molecular Cloning: A Laboratory Manual. Cold Spring Harbor: Cold Spring Harbor Laboratory Press;
1989.
82
Auer H, Lyianarachchi S, Newsom D, Klisovic MI, Marcucci G, Kornacker K. Chipping away at the chip bias: RNA degradation in
microarray analysis. Nat Genet. 2003;35(4):292-293.
83
Brown T, Mackey K. Analysis of RNA by northern and slot blot hybridization. In: Ausubel FM, Brent R, Kingston RE, et al, eds. Current
Protocols in Molecular Biology. New York: John Wiley & Sons Inc; 2002.
84
Masek T, Vopalensky V, Suchomelova P, Pospisek M. Denaturing RNA electrophoresis in TAE agarose gels. Anal Biochem. 2005;336(1):46-
50.
85
Alberts B, Bray D, Lewis J, Raff M, Roberts K, Watson JD. The Cell Nucleus, Chapter 8, in Molecular Biology of the Cell, 3rd edition. New
York, NY: Garland Publishing, 1994. Pp. 335-399. p. 370, Table 8-3 Selected Data on Amounts of RNA in a Typical Mammalian Cell.
86
Mei R, Hubbell E, Bekiranov S, et al. Probe selection for high-density oligonucleotide arrays. PNAS USA 2003;100(20):11237-42.
87
Microarray data normalization and transformation by J. Quackenbush, Nature Genetics, Vol. 32, Dec. 2002. "Normalization for cDNA
microarray data: a robust composite method addressing single and multiple slide systematic variation" by Yang YH, Dudoit S, Luu P, et al.
Nucleic Acids Res. 2002;30(4):15.
88
Li C, Wong WH. Model-based analysis of oligonucleotide arrays: expression index computation and outlier detection. Proceedings of the
National Academy of Sciences. 2001;98:31-36.
89
Yang YH, Dudoit S, Luu P, et al. Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide
systematic variation. Nucleic Acids Res. 2002 Vol. 30, No. 4e15.
90
Parmigiani G, Garett ES, Irizarry RA, Zeger SL. The Analysis of Gene Expression Data: Methods and Software. New York: Springer; 2003.
91
Heid CA, Stevens J, Livak KJ, Williams PM. Real time quantitative PCR. Genome Res. 1996;6:986-984.
92
Higuchi R, Fockler C, Dollinger G, Watson R. Kinetic PCR analysis: real-time monitoring of DNA amplification reactions. Biotechnology.
1993;11:1026-1030.
93
Peccoud J, Jacob C. Theoretical uncertainty of measurements using quantitative polymerase chain reaction. Biophys J. 1996;71:101-108.
94
Winer J, Jung CKS, Shackel I, Williams PM. Development and validation of real-time quantitative reverse transcriptase-polymerase chain
reaction for monitoring gene expression in cardiac myocytes in vitro. Anal Biochem. 1999;270:41-49.
95
Godfrey TE, Kim S-H, Chavira M, et al. Quantitative mRNA expression analysis from formalin-fixed, paraffin-embedded tissues using 5′
nuclease quantitative reverse transcription-polymerase chain reaction. J Mol Diagn. 2000;2:84-91.
96
Bustin A, Nolan T. Pitfalls of Quantitative Real-Time Reverse-Transcription Polymerase Chain Reaction. J Biomol Tech. 2004;15:155-166.
97
Scheuermann RH, Bauer SR. Polymerase chain reaction-based mRNA quantification using an internal standard: analysis of oncogene
expression. Methods Enzymol. (Recombinant DNA, Part I). 1993;218:446-473.

©
34 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

©
Clinical and Laboratory Standards Institute. All rights reserved. 35
Number 19 MM16-P

The Quality System Approach

Clinical and Laboratory Standards Institute (CLSI) subscribes to a quality management system approach in the
development of standards and guidelines, which facilitates project management; defines a document structure via a
template; and provides a process to identify needed documents. The approach is based on the model presented in the
most current edition of CLSI/NCCLS document HS1—A Quality Management System Model for Health Care. The
quality management system approach applies a core set of “quality system essentials” (QSEs), basic to any
organization, to all operations in any healthcare service’s path of workflow (i.e., operational aspects that define how
a particular product or service is provided). The QSEs provide the framework for delivery of any type of product or
service, serving as a manager’s guide. The quality system essentials (QSEs) are:

Documents & Records Equipment Information Management Process Improvement

Organization Purchasing & Inventory Occurrence Management Service & Satisfaction
Personnel Process Control Assessment Facilities & Safety

MM16-P addresses the quality system essentials (QSEs) indicated by an “X.” For a description of the other
documents listed in the grid, please refer to the Related CLSI/NCCLS Publications section on the following page.
Purchasing &

Improvement
Organization

Management

Management
Information

Satisfaction
Assessment

Facilities &
Occurrence
Documents

Equipment
& Records

Service &
Personnel

Inventory

Control
Process

Process

Safety
X M29
C24
EP5
EP6
Adapted from CLSI/NCCLS document HS1—A Quality Management System Model for Health Care.

Path of Workflow

A path of workflow is the description of the necessary steps to deliver the particular product or service that the
organization or entity provides. For example, CLSI/NCCLS document GP26⎯Application of a Quality
Management System Model for Laboratory Services defines a clinical laboratory path of workflow which consists of
three sequential processes: preexamination, examination, and postexamination. All clinical laboratories follow these
processes to deliver the laboratory’s services, namely quality laboratory information.

MM16-P addresses the clinical laboratory path of workflow steps indicated by an “X.” For a description of the other
documents listed in the grid, please refer to the Related CLSI/NCCLS Publications section on the following page.

Preexamination Examination Postexamination

receipt/processing
Sample collection

Results reporting
Sample transport

Results review
and follow-up

and archiving
Interpretation
Examination

Examination

management
ordering

Sample

Sample

MM12 MM12 MM12 MM12 MM12 MM12 MM12

Adapted from CLSI/NCCLS document HS1—A Quality Management System Model for Health Care.

©
36 Clinical and Laboratory Standards Institute. All rights reserved.
Volume 25 MM16-P

Related CLSI/NCCLS Publications*

C24-A2 Statistical Quality Control for Quantitative Measurements: Principles and Definitions; Approved
Guideline—Second Edition (1999). This guideline provides definitions of analytical intervals, planning of
quality control procedures, and guidance for quality control applications.

EP5-A2 Evaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline—

Second Edition (2004). This document provides guidance for designing an experiment to evaluate the
precision performance of quantitative measurement methods; recommendations on comparing the resulting
precision estimates with manufacturers’ precision performance claims and determining when such
comparisons are valid; as well as manufacturers’ guidelines for establishing claims.

EP6-A Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach;

Approved Guideline (2003). This document provides guidance for characterizing the linearity of a method
during a method evaluation; for checking linearity as part of routine quality assurance; and for determining
and stating a manufacturer’s claim for linear range.

MM12-P Diagnostic Nucleic Acid Microarrays; Proposed Guideline (2005). This guideline provides
recommendations for many aspects of the array process including: a method overview; nucleic acid extraction;
the preparation, handling, and assessment of genetic material; quality control; analytic validation; and
interpretation and reporting of results.

M29-A3 Protection of Laboratory Workers From Occupationally Acquired Infections; Approved Guideline—
Third Edition (2005). Based on U.S. regulations, this document provides guidance on the risk of transmission
of infectious agents by aerosols, droplets, blood, and body substances in a laboratory setting; specific
precautions for preventing the laboratory transmission of microbial infection from laboratory instruments and
materials; and recommendations for the management of exposure to infectious agents.

*
Proposed-level documents are being advanced through the Clinical and Laboratory Standards Institute consensus process;
therefore, readers should refer to the most recent editions.
©
Clinical and Laboratory Standards Institute. All rights reserved. 37
 Active Membership
(as of 1 July 2005)
Sustaining Members FDA Center for Devices and LifeScan, Inc. (a Johnson & Johnson Cambridge Memorial Hospital
Radiological Health Company) (Cambridge, ON, Canada)
Abbott Laboratories FDA Center for Veterinary Medicine Medical Device Consultants, Inc. Canterbury Health Laboratories
American Association for Clinical FDA Division of Anti-Infective Merck & Company, Inc. (New Zealand)
Chemistry Drug Products Micromyx, LLC Cape Breton Healthcare Complex
Bayer Corporation Iowa State Hygienic Laboratory National Pathology Accreditation (Nova Scotia, Canada)
BD Maryland Dept. of Health & Mental Advisory Council (Australia) Carilion Consolidated Laboratory
Beckman Coulter, Inc. Hygiene Nippon Becton Dickinson Co., Ltd. (VA)
bioMérieux, Inc. Massachusetts Department of Public Nissui Pharmaceutical Co., Ltd. Carolinas Medical Center (NC)
CLMA Health Laboratories Novartis Institutes for Biomedical Cathay General Hospital (Taiwan)
College of American Pathologists National Center of Infectious and Research Central Laboratory for Veterinarians
GlaxoSmithKline Parasitic Diseases (Bulgaria) Olympus America, Inc. (BC, Canada)
Ortho-Clinical Diagnostics, Inc. National Health Laboratory Service Optimer Pharmaceuticals, Inc. Central Ohio Primary Care
Pfizer Inc (South Africa) Ortho-Clinical Diagnostics, Inc. Physicians
Roche Diagnostics, Inc. National Institute of Standards and (Rochester, NY) Central Texas Veterans Health Care
Technology Ortho-McNeil Pharmaceutical System
Professional Members National Pathology Accreditation (Raritan, NJ) Centro Diagnostico Italiano (Milano,
Advisory Council (Australia) Oxoid Inc. Italy)
American Academy of Family New York State Department of Paratek Pharmaceuticals Chang Gung Memorial Hospital
Physicians Health Pfizer Animal Health (Taiwan)
American Association for Clinical Ontario Ministry of Health Pfizer Inc Children’s Healthcare of Atlanta
Chemistry Pennsylvania Dept. of Health Powers Consulting Services (GA)
American Association for Saskatchewan Health-Provincial Predicant Biosciences Children’s Hospital (NE)
Respiratory Care Laboratory Procter & Gamble Pharmaceuticals, Children’s Hospital Central
American Chemical Society Scientific Institute of Public Health; Inc. California
American Medical Technologists Belgium Ministry of Social QSE Consulting Children’s Hospital & Clinics (MN)
American Society for Clinical Affairs, Public Health and the Radiometer America, Inc. Childrens Hospital of Wisconsin
Laboratory Science Environment Radiometer Medical A/S Children’s Hospital Medical Center
American Society for Microbiology Reliance Life Sciences (Akron, OH)
American Society of Hematology Industry Members Replidyne Chinese Association of Advanced
American Type Culture Collection, Roche Diagnostics GmbH Blood Bankers (Beijing)
Inc. AB Biodisk Roche Diagnostics, Inc. Christus St. John Hospital (TX)
Assn. of Public Health Laboratories Abbott Diabetes Care Roche Diagnostics Shanghai Ltd. City of Hope National Medical
Assoc. Micro. Clinici Italiani- Abbott Laboratories Roche Laboratories (Div. Hoffmann- Center (CA)
A.M.C.L.I. Acrometrix Corporation La Roche Inc.) Clarian Health - Methodist Hospital
British Society for Antimicrobial Advancis Pharmaceutical Roche Molecular Systems (IN)
Chemotherapy Corporation Sanofi Pasteur CLSI Laboratories (PA)
Canadian Society for Medical Affymetrix, Inc. Sarstedt, Inc. Community College of Rhode Island
Laboratory Science - Société Ammirati Regulatory Consulting Schering Corporation Community Hospital of Lancaster
Canadienne de Science de Anna Longwell, PC Schleicher & Schuell, Inc. (PA)
Laboratoire Médical A/S ROSCO SFBC Anapharm Community Hospital of the
Canadian Standards Association AstraZeneca Pharmaceuticals Streck Laboratories, Inc. Monterey Peninsula (CA)
Clinical Laboratory Management Axis-Shield POC AS SYN X Pharma Inc. CompuNet Clinical Laboratories
Association Bayer Corporation - Elkhart, IN Sysmex Corporation (Japan) (OH)
COLA Bayer Corporation - Tarrytown, NY Sysmex Corporation (Long Grove, Covance Central Laboratory
College of American Pathologists Bayer Corporation - West Haven, IL) Services (IN)
College of Medical Laboratory CT TheraDoc Creighton University Medical Center
Technologists of Ontario BD Theravance Inc. (NE)
College of Physicians and Surgeons BD Diabetes Care Thrombodyne, Inc. Detroit Health Department (MI)
of Saskatchewan BD Diagnostic Systems THYMED GmbH DFS/CLIA Certification (NC)
ESCMID BD VACUTAINER Systems Transasia Engineers Diagnostic Accreditation Program
International Council for Beckman Coulter, Inc. Trek Diagnostic Systems, Inc. (Vancouver, BC, Canada)
Standardization in Haematology Beckman Coulter K.K. (Japan) Vicuron Pharmaceuticals Inc. Diagnósticos da América S/A
International Federation of Bio-Development S.r.l. Wyeth Research (Brazil)
Biomedical Laboratory Science Bio-Inova Life Sciences XDX, Inc. Dianon Systems (OK)
International Federation of Clinical International YD Consultant Dr. Everett Chalmers Hospital (New
Chemistry Biomedia Laboratories SDN BHD YD Diagnostics (Seoul, Korea) Brunswick, Canada)
Italian Society of Clinical bioMérieux, Inc. (MO) Duke University Medical Center
Biochemistry and Clinical Biometrology Consultants Trade Associations (NC)
Molecular Biology Bio-Rad Laboratories, Inc. Dwight David Eisenhower Army
Japanese Committee for Clinical Bio-Rad Laboratories, Inc. – France AdvaMed Medical Center (GA)
Laboratory Standards Bio-Rad Laboratories, Inc. – Plano, Japan Association of Clinical Emory University Hospital (GA)
Joint Commission on Accreditation TX Reagents Industries (Tokyo, Japan) Enzo Clinical Labs (NY)
of Healthcare Organizations Blaine Healthcare Associates, Inc. Florida Hospital East Orlando
National Academy of Clinical Bristol-Myers Squibb Company Associate Active Members Focus Technologies (CA)
Biochemistry Cepheid Focus Technologies (VA)
National Association of Testing Chen & Chen, LLC 82 MDG/SGSCL (Sheppard Foothills Hospital (Calgary, AB,
Authorities - Australia Chiron Corporation AFB,TX) Canada)
National Society for The Clinical Microbiology Institute Academisch Ziekenhuis -VUB Franciscan Shared Laboratory (WI)
Histotechnology, Inc. Comprehensive Cytometric (Belgium) Fresno Community Hospital and
Ontario Medical Association Quality Consulting ACL Laboratories (WI) Medical Center
Management Program-Laboratory Copan Diagnostics Inc. All Children’s Hospital (FL) Gamma Dynacare Medical
Service Cosmetic Ingredient Review Allegheny General Hospital (PA) Laboratories (Ontario, Canada)
RCPA Quality Assurance Programs Cubist Pharmaceuticals Allina Health System (MN) Gateway Medical Center (TN)
PTY Limited Cumbre Inc. American University of Beirut Geisinger Medical Center (PA)
Sociedad Espanola de Bioquimica Dade Behring Inc. - Cupertino, CA Medical Center (NY) General Health System (LA)
Clinica y Patologia Molecular Dade Behring Inc. - Deerfield, IL Anne Arundel Medical Center (MD) Guthrie Clinic Laboratories (PA)
Sociedade Brasileira de Analises Dade Behring Inc. - Glasgow, DE Antwerp University Hospital Hagerstown Medical Laboratory
Clinicas Dade Behring Inc. - Marburg, (Belgium) (MD)
Taiwanese Committee for Clinical Germany Arkansas Department of Health Harris Methodist Fort Worth (TX)
Laboratory Standards (TCCLS) Dade Behring Inc. - Sacramento, CA Associated Regional & University Hartford Hospital (CT)
Turkish Society of Microbiology David G. Rhoads Associates, Inc. Pathologists (UT) Headwaters Health Authority
Diagnostic Products Corporation Atlantic Health System (NJ) (Alberta, Canada)
Government Members Digene Corporation AZ Sint-Jan (Belgium) Health Network Lab (PA)
Eiken Chemical Company, Ltd. Azienda Ospedale Di Lecco (Italy) Health Partners Laboratories (VA)
Armed Forces Institute of Pathology Elanco Animal Health Barnes-Jewish Hospital (MO) High Desert Health System (CA)
Association of Public Health Electa Lab s.r.l. Baxter Regional Medical Center Highlands Regional Medical Center
Laboratories Enterprise Analysis Corporation (AR) (FL)
BC Centre for Disease Control F. Hoffman-La Roche AG Baystate Medical Center (MA) Hoag Memorial Hospital
Caribbean Epidemiology Centre Gavron Group, Inc. Bbaguas Duzen Laboratories Presbyterian (CA)
Centers for Disease Control and Gen-Probe (Turkey) Holy Cross Hospital (MD)
Prevention Genzyme Diagnostics BC Biomedical Laboratories (Surrey, Hôpital Maisonneuve - Rosemont
Centers for Medicare & Medicaid GlaxoSmithKline BC, Canada) (Montreal, Canada)
Services Greiner Bio-One Inc. Bo Ali Hospital (Iran) Hôpital Saint-Luc (Montreal,
Centers for Medicare & Medicaid Immunicon Corporation Bon Secours Hospital (Ireland) Quebec, Canada)
Services/CLIA Program Instrumentation Laboratory Brazosport Memorial Hospital (TX) Hospital Consolidated Laboratories
Chinese Committee for Clinical International Technidyne Broward General Medical Center (MI)
Laboratory Standards Corporation (FL) Hospital de Sousa Martins (Portugal)
Commonwealth of Pennsylvania I-STAT Corporation Cadham Provincial Laboratory Hospital for Sick Children (Toronto,
Bureau of Laboratories Johnson and Johnson Pharmaceutical (Winnipeg, MB, Canada) ON, Canada)
Department of Veterans Affairs Research and Development, L.L.C. Calgary Laboratory Services Hotel Dieu Grace Hospital (Windsor,
Deutsches Institut für Normung K.C.J. Enterprises (Calgary, AB, Canada) ON, Canada)
(DIN) LabNow, Inc. California Pacific Medical Center
Huddinge University Hospital Medical University of South Regions Hospital Tenet Odessa Regional Hospital
(Sweden) Carolina Rex Healthcare (NC) (TX)
Humility of Mary Health Partners Memorial Medical Center Rhode Island Department of Health The Children’s University Hospital
(OH) (Napoleon Avenue, New Orleans, Laboratories (Ireland)
Hunter Area Health Service LA) Robert Wood Johnson University The Permanente Medical Group
(Australia) Methodist Hospital (Houston, TX) Hospital (NJ) (CA)
Hunterdon Medical Center (NJ) Methodist Hospital (San Antonio, Sahlgrenska Universitetssjukhuset Touro Infirmary (LA)
Indiana University TX) (Sweden) Tri-Cities Laboratory (WA)
Innova Fairfax Hospital (VA) Middlesex Hospital (CT) St. Alexius Medical Center (ND) Tripler Army Medical Center (HI)
Institute of Medical and Veterinary Montreal Children’s Hospital St. Agnes Healthcare (MD) Truman Medical Center (MO)
Science (Australia) (Canada) St. Anthony Hospital (CO) Tuen Mun Hospital (Hong Kong)
International Health Management Montreal General Hospital (Canada) St. Anthony’s Hospital (FL) UCLA Medical Center (CA)
Associates, Inc. (IL) National Healthcare Group St. Barnabas Medical Center (NJ) UCSF Medical Center (CA)
Jackson Health System (FL) (Singapore) St. Christopher’s Hospital for UNC Hospitals (NC)
Jacobi Medical Center (NY) National Serology Reference Children (PA) Unidad de Patologia Clinica
John H. Stroger, Jr. Hospital of Cook Laboratory (Australia) St-Eustache Hospital (Quebec, (Mexico)
County (IL) NB Department of Health & Canada) Union Clinical Laboratory (Taiwan)
Johns Hopkins Medical Institutions Wellness (New Brunswick, St. John Hospital and Medical United Laboratories Company
(MD) Canada) Center (MI) (Kuwait)
Kaiser Permanente (MD) The Nebraska Medical Center St. John Regional Hospital Universita Campus Bio-Medico
Kantonsspital (Switzerland) Nevada Cancer Institute (St. John, NB, Canada) (Italy)
Kimball Medical Center (NJ) New Britain General Hospital (CT) St. John’s Hospital & Health Center University of Chicago Hospitals
King Abdulaziz Medical City – New England Fertility Institute (CT) (CA) (IL)
Jeddah (Saudi Arabia) New York City Department of St. Joseph’s Hospital – Marshfield University of Colorado Hospital
King Faisal Specialist Hospital Health & Mental Hygiene Clinic (WI) University of Debrecen Medical
(Saudi Arabia) NorDx (ME) St. Jude Children’s Research Health and Science Center
LabCorp (NC) North Carolina State Laboratory of Hospital (TN) (Hungary)
Laboratoire de Santé Publique du Public Health St. Mary Medical Center (CA) University of Maryland Medical
Quebec (Canada) North Central Medical Center (TX) St. Mary of the Plains Hospital System
Laboratorio Dr. Echevarne (Spain) North Coast Clinical Laboratory (TX) University of Medicine & Dentistry,
Laboratório Fleury S/C Ltda. (OH) St. Michael’s Hospital (Toronto, NJ University Hospital
(Brazil) North Shore - Long Island Jewish ON, Canada) University of MN Medical Center -
Laboratorio Manlab (Argentina) Health System Laboratories (NY) St. Vincent’s University Hospital Fairview
Laboratory Corporation of America North Shore University Hospital (Ireland) University of the Ryukyus (Japan)
(NJ) (NY) Ste. Justine Hospital (Montreal, PQ, The University of the West Indies
Lakeland Regional Medical Center Northern Plains Laboratory (ND) Canada) University of Virginia Medical
(FL) Northwestern Memorial Hospital San Francisco General Hospital Center
Lawrence General Hospital (MA) (IL) (CA) University of Washington
Lewis-Gale Medical Center (VA) Ochsner Clinic Foundation (LA) Santa Clara Valley Medical Center US LABS, Inc. (CA)
L'Hotel-Dieu de Quebec (Canada) Onze Lieve Vrouw Ziekenhuis (CA) USA MEDDAC-AK
Libero Instituto Univ. Campus (Belgium) Seoul Nat’l University Hospital UZ-KUL Medical Center (Belgium)
BioMedico (Italy) Orlando Regional Healthcare System (Korea) VA (Tuskegee) Medical Center
Lindy Boggs Medical Center (LA) (FL) Shands at the University of Florida (AL)
Loma Linda Mercantile (CA) Ospedali Riuniti (Italy) South Bend Medical Foundation Virginia Beach General Hospital
Long Beach Memorial Medical The Ottawa Hospital (IN) (VA)
Center (CA) (Ottawa, ON, Canada) South Western Area Pathology Virginia Department of Health
Long Island Jewish Medical Center Our Lady of the Resurrection Service (Australia) Washington Adventist Hospital
(NY) Medical Center (IL) Southern Maine Medical Center (MD)
Los Angeles County Public Health Pathology and Cytology Specialty Laboratories, Inc. (CA) Washington State Public Health
Lab (CA) Laboratories, Inc. (KY) State of Connecticut Dept. of Public Laboratory
Maimonides Medical Center (NY) Pathology Associates Medical Health Washoe Medical Center
Marion County Health Department Laboratories (WA) State of Washington Department of Laboratory (NV)
(IN) Phoenix College (AZ) Health Wellstar Health Systems (GA)
Martin Luther King/Drew Medical Piedmont Hospital (GA) Stony Brook University Hospital West China Second University
Center (CA) Presbyterian Hospital of Dallas (TX) (NY) Hospital, Sichuan University (P.R.
Massachusetts General Hospital Providence Health Care (Vancouver, Stormont-Vail Regional Medical China)
(Microbiology Laboratory) BC, Canada) Center (KS) West Jefferson Medical Center (LA)
MDS Metro Laboratory Services Provincial Laboratory for Public Sun Health-Boswell Hospital (AZ) Wilford Hall Medical Center (TX)
(Burnaby, BC, Canada) Health (Edmonton, AB, Canada) Sunnybrook Health Science Center William Beaumont Army Medical
Medical Centre Ljubljana (Slovinia) Quest Diagnostics Incorporated (ON, Canada) Center (TX)
Medical College of Virginia (CA) Sunrise Hospital and Medical Center William Beaumont Hospital (MI)
Hospital Quintiles Laboratories, Ltd. (GA) (NV) Winn Army Community Hospital
Medical Research Laboratories Regional Health Authority Four Swedish Medical Center - (GA)
International (KY) (NB, Canada) Providence Campus (WA) Winnipeg Regional Health
Authority (Winnipeg, Canada)
York Hospital (PA)

OFFICERS BOARD OF DIRECTORS

Thomas L. Hearn, PhD, Susan Blonshine, RRT, RPFT, FAARC J. Stephen Kroger, MD, MACP
President TechEd COLA
Centers for Disease Control and Prevention
Maria Carballo Jeannie Miller, RN, MPH
Robert L. Habig, PhD, Health Canada Centers for Medicare & Medicaid Services
President Elect
Abbott Laboratories Kurt H. Davis, FCSMLS, CAE Gary L. Myers, PhD
Canadian Society for Medical Laboratory Science Centers for Disease Control and Prevention
Wayne Brinster,
Secretary Russel K. Enns, PhD Klaus E. Stinshoff, Dr.rer.nat.
BD Cepheid Digene (Switzerland) Sàrl

Gerald A. Hoeltge, MD, Mary Lou Gantzer, PhD James A. Thomas

Treasurer Dade Behring Inc. ASTM International
The Cleveland Clinic Foundation
Lillian J. Gill, DPA Kiyoaki Watanabe, MD
Donna M. Meyer, PhD, FDA Center for Devices and Radiological Health Keio University School of Medicine
Immediate Past President
CHRISTUS Health

Glen Fine, MS, MBA,

Executive Vice President
 940 West Valley Road T Suite 1400 T Wayne, PA 19087 T USA T PHONE 610.688.0100
FAX 610.688.0700 T E-MAIL: [email protected] T WEBSITE: www.clsi.org T ISBN 1-56238-579-8