Precision Statements in UOP Methods

UOP Method 999-13

Scope
This method is for developing precision statements as reported in UOP methods and relative bias
of the same method between different laboratories. The calculation of precision, in terms of
repeatability and intermediate precision (within a laboratory), reproducibility (between laboratories)
and relative bias (between laboratories) is described. Precision statements in methods having a 98 or
later suffix were developed by the procedure described; methods having an 88 through 97 suffix used
UOP 888-88, while methods with suffixes earlier than 88 used UOP 666-82.
The features of UOP 999 are as follows:
• Minimum of 16 analyses required with emphasis on multiple concentration levels
• Better identification of the components of variation and repeatability estimates
• Alignment with ASTM repeatability, intermediate precision, and reproducibility definitions
• Relative bias between laboratories with 95% confidence interval
UOP 999 is the minimum guideline for estimating the initial precision for new or revised methods.
Once a new or revised method is in use, a reference sample, if available, should be run on a regular
basis. When 16 analyses have been completed, individual and range control charts are to be used to
estimate the long-term UOP repeatability and are to be maintained on an ongoing basis. The example
calculations in UOP 999 are made using Minitab, a statistical software package. The details of the
calculations are in UOP 999-97 Supplement.

References
ASTM Practice E177, “Practice for the use of Terms Precision and Bias in ASTM Test Methods,”
www.astm.org
ASTM Practice E178, “Practice for Dealing with Outlying Observations,” www.astm.org
ASTM Practice E691, “Practice for Conducting an Interlaboratory Study to Determine the
Precision of a Test Method,” www.astm.org
“Errors of Routine Analysis,” Student, Biometrika, XIX, 1927, pp. 151-164
“Precision Measurement and Calibration,” NBS Special Publication 300, Vol. 1, Feb. 1969, U.S.
Department of Commerce
Supplement to UOP 999-97
UOP Method 666-82, “Precision Statements in UOP Methods”
UOP Method 888-88, “Precision Statements in UOP Methods”

© COPYRIGHT 1997, 2004, 2005, 2013 UOP LLC. All rights reserved.
Nonconfidential UOP Methods are available from ASTM International, 100 Barr Harbor Drive, PO Box C700, West
Conshohocken, PA 19428-2959, United States. The UOP Methods may be obtained through the ASTM website,
www.astm.org, or by contacting Customer Service at [email protected], 610.832.9555 FAX, or 610.832.9585 PHONE.
 2 of 19

Outline of the Method

Using the specified UOP test method, at least 16 tests are performed at a given laboratory or
laboratories on the same representative sample, or on samples at multiple concentrations, using one
of the nested sampling designs shown in the Procedure section. Exceptions to nested sampling
design can be made at the discretion of the method coordinator. In each laboratory, the analysis is
performed by at least two different analysts on each of two separate days, each analyst performing
two tests per day. The estimated within-laboratory standard deviation (esd) and the estimated
between-laboratory standard deviation (esd) are calculated using a stepwise nested analysis of
variance (ANOVA) procedure (see Appendix 2 of the Supplement to this document).
The stepwise ANOVA procedure for the nested design is used to estimate the within-day test esd,
day-to-day esd, analyst-to-analyst esd and lab-to-lab esd. The estimated components of variability
from the stepwise ANOVA are used to calculate repeatability, intermediate precision,
reproducibility, and relative bias as outlined in Table 1.
Repeatability is the expected maximum difference between two tests by the same analyst on the
same day in one laboratory at the 95% confidence level. Repeatability is calculated using the
Within-Day esd. Intermediate precision is the expected maximum difference between two tests by
different analysts on different days in one laboratory at the 95% confidence level and is calculated
using the Within-Laboratory esd. Reproducibility is the expected maximum difference between two
tests by different analysts on different days in different laboratories at the 95% confidence level and
is calculated using the Within-Laboratory esd plus the Between-Laboratory esd.
When all the tests are conducted in only one laboratory, then repeatability and intermediate
precision may be calculated, but reproducibility cannot. For this case, applicable nested designs are
shown in the Procedure section. When the testing is conducted across two or more laboratories, then
the repeatability, the intermediate precision, the reproducibility and the relative bias can be
calculated. For two or three laboratories, the applicable nested designs are found in the Procedure
section. Table 2 in the Procedure section summarizes standard nested designs and their attributes
with cross-reference to the design layout. If precision data are received from one or two laboratories,
the repeatability and intermediate precision is the only precision information reported.
Calculations may also be performed on pairs of data, when such data are available, and the nested
sampling designs shown in the Procedure section would be difficult or impossible to perform. See
Appendix 3.

Definitions
95% Confidence Interval for Mean Difference, the interval about the average difference value that
is believed to include the “true” population difference for approximately 95 out of 100 such
estimates.
Analysis of Variance (ANOVA), a statistical procedure that divides the total variability for a set of
data into meaningful component parts associated with specific sources of variation. The
technique, in conjunction with the F-ratio, is used to provide a test of significance for component
sources of variation and to obtain estimates of the standard deviations attributable to those
sources.
Bias, deviation from a known or accepted value.
Day, refers to the span of time over which the analyses are performed on the same day by the same
analyst in a laboratory and from which the within-day standard deviation is estimated.

999-13
 3 of 19

Table 1
Precision Statement Definitions
(The description of the terms is in the CALCULATIONS)

Quantity Definition
Repeatability Allowable difference between two tests performed by same analyst in one
lab on same day at 95% confidence level.
Repeatability
t DF 2 s Within-Day
Equation
Intermediate Allowable difference between two tests performed by different analysts in
Precision one lab on different days at 95% confidence level.
Intermediate
t DF 2 s Within-Lab
Precision Equation
Within-Day Standard
t DF 2 s Within-Day
Deviation
Within-Lab
Standard Deviation s Within-Lab = s2Within - Day + sDay
2 2
- to - Day sAnalyst - to - Analyst
+

Allowable difference between two tests performed by different analysts in

Reproducibility different labs on different days at 95% confidence.
Reproducibility
Equation t DF 2 s2 + s2
Within - Lab Between - Lab

Relative Bias For specified sample, the average test difference between labs.

Within-Lab 2 2 2
s Within-Day s Day- to-Day s Analyst- to- Analyst
Mean Standard s X Within-Lab = + +
Deviation n R n Days n Analysts n Days n Analysts n Analysts

DF* = nLabs ( nAnalysts - 1 ) where nDays = No. days testing per analyst nLabs = No. labs nAnalysts = No.
analysts nR = No. reps per day
Note: See SUPPLEMENT Appendix 2 for other than full model.

Relative Bias
x Lab 1 - x Lab 2 ± t DF* 2s X Within-Lab
Confidence
Interval
Degrees of Freedom, the number of independent observations available to estimate the standard
deviation. In general, when r constants for a model have been estimated from n data values, only
(n - r) degrees of freedom remain to estimate the model’s variability.
Duplicate, the false replication of an analysis (i.e. one test immediately followed by another) in
which all the sources of inherent variability are not operational.
Intermediate Precision, the allowable difference between two tests performed by different analysts
in one laboratory on different days. Two tests should not differ by more than the stated
allowable difference more than five percent of the time (for 95% confidence). This definition is
the same as the term “UOP Repeatability” used in past UOP methods describing precision, and
should be used when comparing data generated on other than the same day. Intermediate
Precision may also be called Site Precision.
999-13
 4 of 19

Mean Square (MS), an unbiased estimate of the population variance calculated by dividing the sum
of squares (SS) by its degrees of freedom (DF).
Pooling, two or more variances (standard deviations squared) may be pooled by adding their
variances weighted by ratio of the individual variance degrees of freedom divided by the sum of
the degrees of freedom for all the variances.
Randomization, the operation of assigning a testing sequence in a purely chance manner by using a
list of random numbers. Randomization increases the chance of obtaining a representative
sample from the population and thereby assures and ensures a valid estimate of experimental
error and associated significance tests.
Relative Bias, the average test difference between laboratories for the same sample analyses.
Repeatability, the allowable difference between two tests performed by the same analyst in one
laboratory on the same day. The two tests should not differ by more than the stated allowable
difference more than five percent of the time, thus giving 95% confidence in the repeatability.
This definition more closely mirrors that of ASTM Repeatability, and should allow for closer
comparison to independently reported ASTM repeatability statistics.
Replicate, the random repetition of an analysis over a specified period of time, such as a day, under
identical conditions subject only to (but to all of) the random inherent variability of that time
interval.
Reproducibility, the allowable difference between two tests performed by different analysts in
different laboratories on different days. Two such tests should not differ by more than the stated
allowable difference more than five percent of the time (for 95% confidence). This definition is
the same as the ASTM convention; therefore, independently reported reproducibility statistics
can be directly compared.
Test, the result of a single analysis performed in a laboratory by a specified UOP method.
Determinations performed in duplicate (i.e. one test immediately followed by another) are
discouraged, since little information is gained. However, when duplicates are routinely
performed, a test is defined as the average of the two determinations.

Procedure
The laboratory supervisor under whose jurisdiction the method is performed is responsible for
collecting the necessary nested data using the proper sampling procedure to assure uniformity
between the samples analyzed, following the UOP method exactly as written, and reporting those
data together with a record of the analyst, day, time and test number. Care must be taken to
accurately record, on the form provided, the origin of each result, noting the analyst, test number, day
and time of the test.
All the data collected must be reported and no effort should be made to eliminate data points by
rejecting individual tests. Nor should the data be truncated by rounding. It is better to record too
many digits than not enough. The resultant data are referred to as a “balanced nested” data set.

Choosing a Nested Design

Precision studies requiring no less than the minimum 16 separate analyses are characterized in
Table 2 and shown in Figures 1A and 1B, Within-Laboratory and Within-plus-Between Laboratory
Nested Sampling Designs. To estimate only the within-laboratory repeatability and intermediate
precision for a single sample concentration in one laboratory using four analysts, use Form 1A-1 in

999-13
 5 of 19

Appendix 1. When two sample concentrations in a single laboratory are to be used to estimate only
the within-laboratory repeatability and intermediate precision, use Form 1A-2 in Appendix 1. To
estimate repeatability, intermediate precision, reproducibility, and relative bias between two
laboratories for a single sample concentration using two analysts per laboratory, use Form 1B-1 in
Appendix 1. To estimate repeatability, intermediate precision, reproducibility, and relative bias
between two laboratories at two sample concentrations using two analysts per laboratory, use Form
1B-1 in Appendix 1, at each laboratory. This requires 16 separate analyses at each laboratory for a
total of 32 analyses.

Table 2
Nested Designs for Different Precision Objectives

UOP 999 No. No. Sample No. No. See

Precision Objective Labs Concentrations Analysts Tests Figure
Estimate Only Repeatability
Within Lab at 1 Concentration 1 1 4 16 1A-1

Estimate Only Repeatability

Within Lab at 2 Concentrations 1 2 2 16 1A-2

Estimate Only Repeatability

Within Lab at 3 Concentrations 1 3 2 24 1A-3

Estimate Repeatability,
Reproducibility and Relative 2 1 2 16 1B-1
Bias using 2 Labs
Estimate Repeatability, 1B-1
Reproducibility and Relative 2 2 2 32 at each
Bias using 2 Labs Lab
Estimate Repeatability,
Reproducibility and Relative 2 1 3 24 1B-2
Bias using 2 Labs
Estimate Repeatability,
Reproducibility and Relative 3 1 2 24 1B-3
Bias using 3 Labs

Note: For all the designs given in Table 2, each analyst performs 2 tests on each of 2 days.
Figure 1A-3 shows a nested design requiring 24 analyses. This plan is for estimating only the
within-laboratory repeatability and intermediate precision using three different concentration samples
in a single laboratory with two analysts. Another nested design with 24 analyses is shown in Figure
1B-3, and is useful for estimating repeatability, intermediate precision, reproducibility, and relative
bias between three laboratories for a single concentration sample using two analysts per laboratory.
No corresponding data form was prepared for the nested designs requiring 24 analyses.

999-13
 6 of 19

Within-Laboratory Nested Sampling Designs

• Figure 1A-1: 1 Sample Concentration Level using 4 Analysts

Laboratory 1

Analysts 1 2 3 4

Days 1 2 1 2 1 2 1 2

Within 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
Day
16 Tests per Concentration

• Figure 1A-2: 2 Sample Concentration Levels using 2 Analysts

Concentrations

Analysts

Days

Within
Day
8 Tests per Concentration 8 Tests per Concentration

• Figure 1A-3: 3 Sample Concentration Levels using 2 Analysts

Concentration 1 2 3

Analysts 1 2 1 2 1 2

Days 1 2 1 2 1 2 1 2 1 2 1 2

Within
Day 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
8 Tests per Concentration 8 Tests per Concentration 8 Tests per Concentration

Note: Data forms corresponding to the Figure 1A-1 and Figure 1A-2 designs above are found in
Appendix 1.

999-13
 7 of 19

Within-plus-Between Laboratory Nested Sampling Designs

• Figure 1B-1: Single Sample

Laboratories

Analysts

Days

Within
Day
8 Tests per Laboratory 8 Tests per Laboratory

• Figure 1B-2: Single Sample using 3 Analysts

Laboratory 1 2

Analysts 1 2 3 1 2 3

Days 1 2 1 2 1 2 1 2 1 2 1 2

Within 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
Day
12 Tests per Laboratory 12 Tests per Laboratory

• Figure 1B-3: Single Sample for 3 Laboratories

Laboratory 1 2 3

Analysts 1 2 1 2 1 2

Days 1 2 1 2 1 2 1 2 1 2 1 2

Within
Day 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
8 Tests per Laboratory 8 Tests per Laboratory 8 Tests per Laboratory
Note: The data form corresponding to Figure 1B-1 design above is found in Appendix 1.
When a method claims applicability to a broad concentration range or to different sample types, the
samples analyzed must cover the entire concentration range and/or matrices of interest.

999-13
 8 of 19

Scheduling Representative Tests

To obtain a representative sampling for components of variance estimation, the tests run in each
laboratory should be carried out over a period spanning more than one week. Furthermore, the
testing intervals for the analysts should overlap each other. In addition, the two analyses carried out
each day or shift by the analyst should be performed, if possible, at different times during the day or
shift.
The replicate analyses carried out during the course of a day, to provide a realistic estimate of the
within-day precision, must be capable of being considered to be a random sample of the day for the
statistical estimates to work as intended. For the standard nested design, some replicates should be
run one after another in the AM and/or PM. Other replicates should be run with the maximum time
span between the within-day tests made by the same analyst where possible. See Example
Calculation No. 1 Form 1B-1 in Supplement to this document showing the more-or-less random times
that the within-day replicate analysis were run by the same analyst.

Data Examination
It is conceivable that some of the data may be unusable, which then unbalances the nested design.
When the ANOVA is complicated by missing data, it is best to consult a statistician.
The within-day replicate data should be plotted as a function of the Lab or Concentration, Analyst,
and Day identification, to determine whether the data appears reasonable prior to any statistical
analysis. The within-day replicate plot should be examined for unusual analysis results due possibly
to a recording error or error in carrying out the test. Figure 2 in the Example Calculation No. 1
section of the Supplement shows an example of this type of plot. The decision to reject test data
should be made in consultation with a statistician.
The within-day replicate data should be statistically analyzed using a simple range chart to detect
abnormalities (see Example Calculation No. 1 section, Figure 3 of the Supplement).

Analysis and Components of Variance

The total analytical variability may be separated into specific causes or sources of variability by
statistically analyzing the chemical and/or physical analysis results using the stepwise ANOVA
procedure. The components of variance that must be estimated from the data are as follows:
1. Within-day variance component measuring variation between tests performed on a single day,
by one analyst, in one laboratory. This estimated within-day component (s2Within-Day) is used to
calculate the repeatability.
2. Day-to-day variance component measuring variation among single tests performed on different
days, by one analyst, in one laboratory. This estimated component (s2Day-to-Day) could be zero,
under ideal conditions.
3. Analyst-to-analyst variance component measuring variation between single tests performed on
one day, by different analysts, in one laboratory. This estimated component (s2Analyst-to-Analyst)
could be zero, under ideal conditions.
4. A laboratory-to-laboratory variance component measuring the variation among single tests
performed on one day, by one analyst, in different laboratories. Combined with the three other
components of variation, this is used to develop the reproducibility statement. Similarly, the
laboratory-to-laboratory estimated component (s2Lab-to-Lab) could be zero, under ideal conditions.

999-13
 9 of 19

The first three components are added to estimate the total variation in any of the testing
laboratories, and are used to develop the intermediate precision statement.
The nested design stepwise ANOVA procedure is described in Appendix 2 of the SUPPLEMENT
to this document. In the Supplement the nested design stepwise ANOVA procedure is applied to
copper analysis data and tin analysis data in two separate examples.
Once the components of variance are estimated, they are used to calculate repeatability,
intermediate precision, reproducibility, and relative bias with its confidence interval. When a
method claims applicability to a broad concentration range or to different sample types, the samples
analyzed must cover the entire concentration range and/or matrices of interest. Furthermore, a
separate precision statement is developed for each target concentration or matrix, unless statistical
tests indicate that the components of variance by concentration and/or matrices are not significantly
different and hence, may be combined.
The ANOVA and other computations discussed in this document can be carried out using the
Minitab software. The use of this software is recommended since it is relatively easy to use.

Calculations
The details of the precision calculations are in the Supplement to UOP 999-97.

Repeatability Calculation

Repeatability is defined as the allowable difference between two tests performed by the same
analyst in one lab on the same day at the 95% confidence level. It is calculated as follows:
Repeatability = t DF 2 s Within-Day (1)
where:
tDF = t distribution value with DF degrees of freedom for two-tail 95% confidence, Table 3
DF = (Total number of tests / No. tests by analyst per day) x (No. tests by analyst per day -
1) Usually DF equals 8 where the total number of tests is 16 with each analyst
performing 2 tests during the course of a day or shift.
sWithin-Day = Within-day estimated standard deviation with DF degrees of freedom
2 = Value when multiplied by sWithin-Day estimates the standard deviation for the
difference between two tests made anytime during the day by the same analyst.
Intermediate Precision (Site Precision) is defined as the allowable difference between two tests
performed by different analysts in one lab on different days at the 95% confidence level. It is
calculated as follows:
(2)
Intermediate Precision = t DF 2 s Within-Lab
where:
tDF = t distribution value with DF degrees of freedom for two-tail 95% confidence, Table 3
s Within-Lab = Within-Lab estimated standard deviation with DF degrees of freedom containing the
within-day variability and, if present, the day-to-day and analyst-to-analyst
variability; or,
2 2 2
= s Within-Day + sDay-to-Day + s Analyst-to-Analyst with DF degrees of freedom
DF = Estimated by the Satterthwaite equation as discussed in Appendix 2 of the
Supplement. The Satterthwaite equation weights the degrees of freedom of the
within-day, day-to-day and analyst-to-analyst components of variance. For standard
999-13
 10 of 19

nested design with 16 tests, the Satterthwaite calculated DF for sWithin-Lab is between 2
and 8 depending on the ANOVA MS and degree of freedom values.
2 = Value when multiplied by sWithin-Lab estimates the standard deviation for the
difference between two tests made by different analysts in one lab on different days.

Reproducibility Calculation
Reproducibility is defined as the allowable difference between two tests performed by different
analysts in different labs on different days at the 95% confidence level. It is calculated as follows:
(3)
Reproducibility = t DF 2 s 2Within- Lab + s Between-
2
Labs
where:
tDF = t distribution value with DF degrees of freedom for two-tail 95% confidence, Table 3
s Within-Lab = Within-Lab estimated standard deviation with DF degrees of freedom containing the
within-day variability and, if present, the day-to-day and analyst-to-analyst
variability; or,
2 2 2
= s Within-Day + sDay-to-Day + s Analyst-to-Analyst with DF degrees of freedom
sLab-to-Lab = Lab-to-lab estimated standard deviation accounting for average differences between
the laboratories in the study
DF = Estimated by the Satterthwaite equation as discussed in Appendix 2 of the
Supplement. The Satterthwaite equation weights the degrees of freedom of the
within-day, day-to-day, analyst-to-analyst and lab-to-lab components of variance.
For standard nested design with 8 tests at 2 labs, the Satterthwaite calculated DF for
s2Within-Lab + s2Lab-to-Lab is between 1 and 8 depending on the ANOVA MS and degree
of freedom values.
2 = Value when multiplied by s 2Within-Day + sBetween-Lab
2
estimates the standard
deviation for the difference between two tests made by different analysts in different
laboratories on different days.

999-13
 11 of 19

Two-Tailed Student’s t Values

for 95% Confidence

Table 3
Degrees of Degrees of Degrees of
Freedom t-Value Freedom t-Value Freedom t-Value
1 12.7062 12 2.1788 23 2.0687
2 4.3027 13 2.1604 24 2.0639
3 3.1824 14 2.1448 25 2.0595
4 2.7765 15 2.1315 26 2.0555
5 2.5706 16 2.1199 27 2.0518
6 2.4469 17 2.1098 28 2.0484
7 2.3646 18 2.1009 29 2.0452
8 2.3060 19 2.0930 30 2.0423
9 2.2622 20 2.0860 60 2.0003
10 2.2281 21 2.0796 120 1.9799
11 2.2010 22 2.0739 Infinity 1.9600

Relative Bias and

95% Confidence Interval Calculation
Relative bias is defined as the average test difference between laboratories for the same sample
analyses. For three laboratories the relative biases are calculated as follows:

Bias Lab 1 - Lab 2 = X Lab 1 - X Lab 2 (4)

Bias Lab 1 - Lab 3 = X Lab 1 - X Lab 3 (5)

Bias Lab 2 - Lab 3 = X Lab 2 - X Lab 3 (6)

where:
X Lab 1 = Lab 1 Average of n tests

X Lab 2 = Lab 2 Average of n tests

X Lab 3 = Lab 3 Average of n tests

For the nested design shown in Figure 1B-3, n is 8 at each laboratory.

The uncertainty of the laboratory biases is given by the 95% confidence interval for those
differences:

999-13
 12 of 19

_ _
_ (7)
X X ± t DF * 2 s X W ith in -L a b
Lab 1 Lab 2

_ _
_
X X ± t DF * 2 s X W ith in -L a b (8)
Lab 1 Lab 3

_ _
_
X Lab 2
X Lab 3
± t DF * 2 s X W ith in -L a b (9)

where:
2 2 2
s Within-Day s Day- to-Day s Analyst- to- Analyst
s X Within-Lab = + +
n R n Days n Analysts n Days n Analysts n Analysts

DF* = nLabs(nAnalysts-1) degrees of freedom for full model (see Model 1, Appendix 2
in the SUPPLEMENT)
tDF* = t distribution value with DF* degrees of freedom for two-tail 95%
confidence, Table 3
nLabs = Number of laboratories
nAnalysts = Number of analysts per lab
nDays = Number of days testing per analyst
nR = Number of replicates per day per analyst
2 = Value when multiplied by s X Within-Lab estimates the standard deviation for the
average difference of nAnalystsnDaysnR tests made at each of two laboratories.
In the event that bias exists in the data, efforts are expected to be made to find the source of the
bias and then to eliminate the bias, if possible. Relative Bias is used internally for method
development and is not reported in the published method.

Report
The statements included in the Precision section of a UOP method are dependent upon whether
data were collected from only one or two laboratories, or from three or more laboratories. See
Appendix 2 for deciding how many digits to record based on the pooled within-day esd. A Bias
paragraph may be included after the Reproducibility paragraph only if a comparison can be made
between the results obtained by the method and known or accepted values. Refer to ASTM Practice
E 177, “Practice for the use of Terms Precision and Bias in ASTM Test Methods,” for examples of
possible bias statements.
One or Two Laboratories
The within-day and within-laboratory esd and the repeatability are clearly stated. Then a
reproducibility statement is added to clearly show that there is insufficient data for reporting the
reproducibility. For example:

999-13
 13 of 19

Repeatability and Intermediate Precision

A nested design was carried out for copper analysis with four analysts. Each analyst carried out
tests on two separate days and performed two tests each day. The total number of tests performed
was 16. The average copper concentration of all 16 analyses was 0.392 mass-%. Using a
stepwise analysis of variance procedure the within-day estimated standard deviation (esd) was
0.0009. Two tests performed in one laboratory by the same analyst on the same day should not
differ by more than 0.003 with 95% confidence. Using a stepwise analysis of variance procedure
the within-lab estimated standard deviation (esd) was 0.0014. Two tests performed in one
laboratory by different analysts on different days should not differ by more than 0.005 with 95%
confidence.
Reproducibility
There is insufficient data to report the reproducibility of the test at this time.

Multiple Laboratories (Three or More)

The within-day esd, within-laboratory esd and between-laboratory esd are clearly stated. Then the
repeatability and reproducibility are stated. For example:

Repeatability and Intermediate Precision

A nested design was carried for tin analysis. Forty eight analyses were performed in three
laboratories at two concentrations with two analysts per laboratory, with each analyst carrying out
tests on two separate days and performing two tests each day. The average tin concentration of 24
analyses at each concentration was 0.377 mass-% and 0.596 mass-%, respectively. Using a
stepwise analysis of variance procedure the within-day estimated standard deviation (esd) was
0.0007. Two tests performed in one laboratory by the same analyst on the same day should not
differ by more than 0.002 with 95% confidence. Using a stepwise analysis of variance procedure
the within-lab estimated standard deviation (esd) was 0.0018. Two tests performed in one
laboratory by different analysts, on different days, should not differ by more than 0.005 with 95%
confidence.
Reproducibility
A nested design was carried out for tin analysis. Forty eight analyses were performed in three
laboratories at two concentrations with two analysts per laboratory, with each analyst carrying out
tests on two separate days and performing two tests each day. Using a stepwise analysis of
variance procedure the between-lab estimated standard deviation (esd) was 0.0019. The average
tin concentration of 24 analyses was 0.377 mass-% and 0.596 mass-%, respectively. Two tests
performed in one laboratory by different analysts, on different days, should not differ by more
than 0.008 with 95% confidence.

Suggested Supplier
Minitab Inc., 1829 Pine Hall Road, State College, PA 16801-3008 (814-238-3280)
www.minitab.com

999-13
 14 of 19

Appendix 1
Reporting Precision Data Form 1A-1
For Estimating Repeatability for Laboratory at Single Sample Concentration
Laboratory Location _____________________________
Supervisor ______________________ Component, Units ___________________
Sample Concentration _________________ Sample Identification _________________

Analyst Date Within-Day Test

Initials and Time Test Replicate No. Results

1 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

3 __________ ___________ 1 ___________________

___________ 2 ___________________

3 __________ ___________ 1 ___________________

___________ 2 ___________________

4 __________ ___________ 1 ___________________

___________ 2 ___________________

4 __________ ___________ 1 ___________________

___________ 2 ___________________

999-13
 15 of 19

Appendix 1
Reporting Precision Data Form 1A-2
For Estimating Repeatability for Laboratory at Two Sample Concentrations
Laboratory Location _____________________________
Supervisor ______________________ Component, Units _____________________
Sample Concentrations _________________ Sample Identifications _________________

Sample Analyst Date Within-Day Test

Concentration Initials and Time Test Replicate No. Results

1 __________ 1 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ 1 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ 2 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ 2 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ 1 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ 1 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ 2 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ 2 __________ ___________ 1 ___________________

___________ 2 ___________________

999-13
 16 of 19

Appendix 1
Reporting Precision Data Form 1B-1
For Estimating Repeatability, Reproducibility and Relative Bias between Labs
Laboratory Location ______________________ UOP Method No. ________________
Supervisor ______________________ Component, Units ____________________
Sample Concentration _________________ Sample Identification _________________

Analyst Date Within-Day Test

Initials and Time Test Replicate No. Results

1 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

Laboratory Location ______________________ UOP Method No. ________________

Supervisor ______________________ Component, Units ____________________
Sample Concentration _________________ Sample Identification _________________

Analyst Date Within-Day Test

Initials and Time Test Replicate No. Results

1 __________ ___________ 1 ___________________

___________ 2 ___________________

1 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

2 __________ ___________ 1 ___________________

___________ 2 ___________________

999-13
 17 of 19

Appendix 2
Number of Significant Digits to Record
Given the pooled Within-Day estimated standard deviation, the minimum number of digits to record
the analytical data is shown below:

Within Day esd Round Data to Nearest Example

0.0001 to 0.00099 Ten-thousandth 0.3545
0.001 to 0.0099 Thousandth 0.355
0.01 to 0.099 Hundredth 0.36
0.1 to 0.99 Tenth 0.4
1 to 9.99 Integer 1238
10 to 99.99 Ten 1240

When the Within-Day esd is not known from similar procedures, it is better to record the data using
too many digits than to find out later that more digits are needed.
Copper data from Example Calculation No.1 section of the Supplement is shown below. The data
in the rightmost columns has been intentionally truncated and rounded to the nearest thousandth.
The estimated Within-Day standard deviation is 0.000765 for the data rounded to the nearest ten-
thousandth and 0.000866 for the data rounded to the nearest thousandth. When the data is truncated
and rounded to the nearest thousandth the resulting Within-Day standard deviation is 13% higher.
The table above shows the data should be recorded to the nearest ten-thousandth.

8
2
∑ si Based on
i=1
sp = = 0.00076485 data rounded
8
to thousandth
Based on data rounded to
ten-thousandth sp = 0.000866
sp = 0.000765
13% Higher
due to truncated
rounding

999-13
 18 of 19

Appendix 3
Precision Statements Calculated from Pairs of Data
It is preferred that precision statements be generated using data from the nested sampling designs
shown in the Procedure section of the body of the method. However, there may be some instances in
which such data may be difficult or impossible to obtain. If multiple pairs of data are available, e.g.,
from duplicate analyses, such data may be used to generate an alternative precision statement. The
procedure for calculating precision statements from pairs of data is given hereinafter.

Procedure
Obtain analyses on a minimum of 5 duplicate samples (pairs) of the sample. Six or more pairs of
data are preferable since the calculation will be more precise. When a method claims applicability to
a broad concentration range or to different sample types, several estimated standard deviations
should be determined to fully cover the entire range. Calculate an estimated standard deviation and
an allowable difference at the 95% probability level as shown in Calculations.

Calculations
Estimated Standard Deviation (esd) from Pairs
Calculate the average difference (mean range) using Equation A1:

Σ d2
esd = (A1)
2k
where:
d = difference between two results comprising a pair (absolute value)
k = number of pairs

Allowable Difference from Pairs

Calculate the allowable difference (95% probability) from the esd obtained from pairs using
Equation A2 and the appropriate multiplier from Table A1.

Allowable difference = E M (A2)

where:
E = esd, from Equation A1
M = multiplier, obtained from Table A1

Table A1
Multiplier to Calculate Allowable Difference
Number of Number of
Pairs Multiplier Pairs Multiplier
5 3.75 15 3.03
6 3.55 20 2.95
7 3.41 30 2.89
8 3.32 40 2.86
9 3.25 120 2.80

Report
Two statements are included in the precision statement of the method; the first will state the
calculated esd, and the second the allowable difference at the 95% probability level. For example,
999-13
 19 of 19

“Based on six pairs of analyses, the estimated standard deviation (esd) for
platinum in the range of 0.36 – 0.40 mass-% was calculated to be 0.0011 mass-%.
Duplicate results by the same operator should not differ by more than 0.004
mass-% (95% probability) at the stated level.”
When more than one esd is required, due to different sample type or concentration ranges, the data
should be presented in a table format. For example,
“Based on six replicate determinations, the estimated standard deviation (esd)
was calculated for the impurities listed in the table below. Duplicate results by the
same operator should not differ by more than the allowable difference shown (95%
probability).”
Table A2
Precision, mass-ppm
Concentration Allowable
Component Range esd Difference
Benzene 88 - 91 1.11 4.0
Toluene 27 - 29 0.29 1.1

999-13