APLAC TC 004

METHOD OF STATING TEST AND CALIBRATION

RESULTS AND COMPLIANCE WITH
SPECIFICATION

Issue No. 4 Issue Date: 09/10 Page 1 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

PURPOSE

This document gives guidelines on the method for stating test and calibration results
and compliance with specifications. This document is based on ILAC-G8 1996, and
includes amendments found to be necessary when using ILAC G8. The major
amendments include the addition of:

(a) treatment of non-numerical results (clauses 1.1.3, 1.1.5),

(b) an explanation for the ratio of the uncertainty of measurement to the

specified interval (clause 1.1.6),

(c) reporting of results obtained from tests on samples,

(d) an example in clause 2.3.

This document also requires that the level of confidence of uncertainty for numerical
results be reported (clause 1.1.3). When ILAC-G8 1996 has been revised, the need for
this document will be reviewed.

AUTHORSHIP

This document was produced by the APLAC Technical Committee.

COPYRIGHT

The copyright of this document is held by APLAC. APLAC publications may not be
copied for sale by an individual body other than APLAC member organisations.

FURTHER INFORMATION

For further information about this document, contact the APLAC Secretariat at:

NATA
Level 1
675 Victoria Street
Abbotsford VIC 3067
Australia
Tel: +61 3 9274 8200
Fax: +61 3 9421 0887
email: [email protected]
Website: www.aplac.org

Issue No. 4 Issue Date: 09/10 Page 2 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

1. METHOD OF STATING TEST AND CALIBRATION RESULTS AND

COMPLIANCE WITH SPECIFICATION

1.1 General Approach

1.1.1 The extent of the information given when reporting a test or calibration result
and its uncertainty should be related to the requirements of the client, the
specification, and the intended use of the result. The methods used to calculate
the result and, where appropriate, its uncertainty should be available either in
the report or in the records of the test or calibration and should include:

- sufficient documentation of the steps and calculations in the data

analysis to enable a repeat of the calculation if necessary;

- all corrections and constants used in the analysis, and their sources;

- sufficient documentation to show how the uncertainty is calculated.

1.1.2 When reporting the test or calibration result and its uncertainty, the use of
excessive numbers of digits should be avoided. In most cases the uncertainty
should be expressed to no more than two significant figures (although at least
one more figure should be used during the stages of estimation and
combination of component uncertainties in order to minimise rounding errors).

1.1.3 There are two different types of test or calibration results:

- measured numerical values

- non-numerical results, e.g. pass-fail outcome of a test procedure or the

result of applying a go or no-go gauge

For the first type, the results and measurement uncertainty, where appropriate,
should be reported in accordance with clause 1.1.4.

For the second type, the results and any factors affecting the validity of the
result should be reported in accordance with clause 1.1.5.

1.1.4 Numerical results:

Where the measurement uncertainty is relevant to the validity or application of

the results, when a client's instructions require so, or when the uncertainty
affects compliance with a specification limit, the expanded measurement
uncertainty appropriate to approximately a 95% level of confidence should be
calculated. The numerical result and its expanded uncertainty should be
reported in the following manner:

Measured value 100.1 (units)

Uncertainty of measurement 0.1 (units)
Level of confidence 95%

Notes:
1. Where appropriate, the coverage factor should also be reported.
2. More information on reporting measurement results and uncertainty is
given in Chapter 7 of the "Guide to the Expression of Uncertainty in
Measurement". (See No. 4 in Reference list, page 7 below).

Issue No. 4 Issue Date: 09/10 Page 3 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

The method of assessing and reporting compliance with a specification should

be in accordance with section 2 below.

1.1.5 Non-numerical results

Where the method is unambiguously defined in the test or calibration criteria,

test or calibration specifications, client specifications or codes of practice, and,
in the absence of any client instruction to do otherwise, it can be assumed that
the measurement uncertainty has already been taken into consideration in the
method and the laboratory does not need to estimate it. (This may be
considered as a special case of shared risk, as discussed in clauses 2.3 and
2.4 of this document.) In this case, reporting the non-numerical result alone is
adequate.

Where deviations from the specified method are necessary, the laboratory
should evaluate the extent to which the test validity is affected. In this case,
details of the deviations from the specified method and their effects on the
validity of the result should be recorded and reported.

Note:
The measurement method is considered unambiguously defined if the
measurement method, the measurement instruments and their accuracy, the
measurement environmental conditions and any other factors affecting the
result are specified.

For non-numerical results, compliance with a specification can be deduced

readily from the result in most cases. For example, it may be concluded that
the diameter of a hole is in compliance as it allows the insertion of the go gauge
but not the no-go gauge. A statement of compliance should be included in the
report.

Where the conclusion of compliance depends on both numerical and non-

numerical results, a combination of the approaches discussed in clauses 1.1.4
and 1.1.5 should be taken.

1.1.6 When a specification describes an interval with an upper and lower limit, the
ratio of the uncertainty of measurement to the specified interval should be
reasonably small.

Notes:
1. For an uncertainty of measurement U and a specified interval 2T, where
upper limit lower limit
T= , the ratio U:T is a measure of the ability of
2
the measurement method in distinguishing compliance from non-compliance.

2. As explained in Chart 1, a conclusion of compliance can be made for any

measured value falling within the range from [lower limit + U] to [upper limit
U]. If U:T is 1:3, the interval between the [lower limit + U] and the[ upper
limit U] will be 66.7% of the interval 2T. In such a case, if the value is

measured to be within the specified interval, there will be a 66.7%

probability that a conclusion of compliance can be made. A ratio of 1:3 can

Issue No. 4 Issue Date: 09/10 Page 4 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

thus be considered as a reference value.

CHART 1

Ability to Distinguish Compliance from Non-compliance

upper limit
U upper limit - U
If the measured value falls
2T within this range, a
2T 2U compliance conclusion can
be made
lower limit + U
U
lower limit

U = uncertainty of measurement
upper limit lower limit
T=
2
1
Assume U : T is 1 : 3 or U = T
3
1
Then 2T 2U = 2T 2 T
3
2
= 2 T
3
or 66.7% of 2T

3. If compliance with a specification is determined in accordance with clause

2.5 of this document, a larger U:T ratio can be tolerated. However, it should
be noted that, as this ratio is an indicator of the capability of the
measurement method to distinguish compliance from non-compliance, a
measurement method having a U:T ratio approaching unity will be unable to
confirm compliance nor non-compliance for samples having marginal
properties.

1.1.7 When the property of a batch of product or material is assessed by testing

samples taken from it, details of the sampling scheme, the sampling procedure,
the number of samples tested and how the reported measured value is related
to the measured values obtained from individual samples (e.g. by averaging
sample results) should be included in the report.

1.2 Special Cases

1.2.1 In exceptional cases, where a particular factor or factors can influence the
results but where the magnitude cannot be either measured or reasonably
assessed, the reported statement will need to include reference to that fact.

1.2.2 Any uncertainty that results from the test sample not being fully representative
of the single unit of product should normally be identified separately in the
evaluation of uncertainty. However, there may be insufficient information to
enable this to be done, in which case this should be stated in the report. A
possible remark could be:

Issue No. 4 Issue Date: 09/10 Page 5 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

The test results in this report relate only to the test sample as analysed
and not to the single unit of product from which the test sample was
drawn.

2. ASSESSMENT AND REPORTING OF COMPLIANCE WITH SPECIFICATION

2.1 In harmony with 5.10.3.1 (c) of ISO/IEC 17025:2005, these guidelines require
that, when a test is carried out to a stated specification and the client or the
specification requires a statement of compliance, the report must contain a
statement indicating whether the test results show compliance with that
specification. There are a number of possible cases where the uncertainty has
an influence on the compliance statement, and these are examined below.

2.2 The simplest case is where the specification clearly states that the test result,
extended by the uncertainty at a given level of confidence, shall not fall outside
or within a defined specification limit or limits. In these cases (Case 1, 5, 6 and
10 of Appendix A below), assessment of (non)compliance would be
straightforward.

2.3 More often, the specification requires a statement of compliance in the

certificate or report but makes no reference to taking into account the effect of
uncertainty on the assessment of compliance. In such cases it may be
appropriate for the user to make a judgement of compliance, based on whether
the test result is within the specified limits with no account taken of the
uncertainty.

For example, if the measured result for the diameter of a rod is 0.50 mm while
the specification limit is between 0.45 mm to 0.55 mm, the user may conclude
that the rod meets the requirement without considering the measurement
uncertainty.

This is often referred to as shared risk since the end-user takes some of the
risk that the product may not meet the specification after being tested with an
agreed measurement method. In this case there is an implicit assumption that
the uncertainty of the agreed measurement method is acceptable and it is
important that it can be evaluated when necessary. National regulations can
overrule the shared risk principle and can put the uncertainty risk on one party.

2.4 An agreement between the client and the laboratory or a code of practice or a
specification may state that the accuracy of the method applied is adequate and
the uncertainty does not need to be considered explicitly when judging
compliance. Similar considerations as for shared risk (above) apply in such
circumstances.

2.5 In the absence of any criteria, test specifications, client's requirements,

agreements, or codes of practice, the following approach may be taken:

(a) if the specification limits are not breached by the test result, extended by
half of the expanded uncertainty interval at a level of confidence of 95%,
then compliance with the specification can be stated (Cases 1 and 6 of
Appendix A below);

(b) where an upper specification limit is exceeded by the test result, even
after it has been extended downwards by half of the expanded

Issue No. 4 Issue Date: 09/10 Page 6 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

uncertainty interval, then non-compliance with the specification can be

stated (Case 5 of Appendix A below);

(c) if a lower specification limit is breached, even when the test result is
extended upwards by half of the expanded uncertainty interval, then
non-compliance with the specification can be stated (Case 10 of
Appendix A below);
(d) if the measured single value, without the possibility of testing more
samples from the same unit of product nor repeating the measurement,
falls sufficiently close to a specification limit so that half of the expanded
uncertainty interval overlaps the limit, it is not possible to confirm
compliance or non-compliance at the stated level of confidence. The
result and expanded uncertainty should be reported together with a
statement indicating that neither compliance nor non-compliance was
demonstrated. A suitable statement to cover these situations (Cases 2,
4, 7 and 9 of Appendix A below) would be, for example:

The test result is above (below) the specification limit by a margin

less than the measurement uncertainty; it is therefore not possible
to state compliance/non-compliance based on a 95% level of
confidence. However, where a confidence level of less than 95%
is acceptable, a compliance/non-compliance statement may be
possible.

If the law requires a decision concerning rejection or approval, case 2

and 7 of Appendix A below can be stated as compliance with the
specification limit (with a lower calculated and reported confidence
level). In Cases 4 and 9 of Appendix A below, non-compliance with the
specification limit can be stated (with a lower calculated and reported
confidence level).

If two or more samples of a single unit of product can be tested or the

measurement can be repeated, replicate testing or making more
repeated measurements is advisable. After estimating the average
value for all test results on the same samples or results of all repeated
measurements and the new uncertainty for this average value, the same
judgement as described in 2.5 (a) (d) above should be made.

Note:
For 2.5(a) to (d) are based on the assumption that uncertainty
distribution curve for the measured value is symmetrical above the
average value. In certain cases, this may not be true, e.g. when a
significant correction to a measured value is not corrected but
considered as a contribution to uncertainty, or when a dominant
uncertainty component known to have skew distribution is combined
with other uncertainty component as though it is normally distributed. In
such case, a more accurate calculation for the measured value and the
measurement uncertainty may allow the making of an unambiguous
conclusion.

(e) If the result is exactly on the specification limit, it is not possible to state
compliance or non-compliance at the stated level of confidence. The
result and expanded uncertainty should be reported together with a
statement indicating that neither compliance or non-compliance was
demonstrated at the stated level of confidence. A suitable statement to

Issue No. 4 Issue Date: 09/10 Page 7 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

cover these situations (Cases 3 and 8 of Appendix A below) would be,

for example:

The test result is equal to the specification limit; it is therefore not

possible to state either compliance or non-compliance at any level
of confidence.

If the law requires a statement concerning the assessment in the form of

compliance or non-compliance, regardless of the level of confidence,
taking into account the provision of clause 2.3, the statement depends
on the definition of the specifications:

- if the specification limit is defined as < or >, and the test result
is equal to the specification limit, non-compliance can be stated.

- if the specification limit is defined as or , and the test result

is equal to the specification limit, then compliance can be stated.

If possible, it is also advisable to repeat the testing or measurement as

explained in the last paragraph of 2.5(d) above.

3. REFERENCES

1. LAB 12 (2000), The Expression of Uncertainties in Testing, UKAS, UK.

2. NF E 02-204, Verification des tolerances des produits, Declaration de

conformit, December 1993, AFNOR, Paris, France.

3. ISO/DIS 14253 1: 1998 Geometrical Product Specifications (GPS)

Inspection by measurement of workpieces and measuring equipment Part 1:
Decision rules for proving conformance or non-comformance with
specifications; International Organisation for Standardization, Geneva,
Switzerland.

4. BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML, Guide to the Expression of
Uncertainty in Measurement. International Organisation for Standardization,
Geneva, Switzerland, ISBN 92-67-10188-9, First Edition, 1993.

5. ISO 3534 Part 1, Probability and General Statistical Terms - 1993, Statistics -
Vocabulary and symbols, International Organisation for Standardization,
Geneva, Switzerland.

6. VIM, ISO (1993), International Vocabulary of Basic and General Terms in

Metrology, International Organisation for Standardization, Geneva, Switzerland,
ISBN 92-67-01075, Second Edition.

7. ISO/IEC 17025:2005 General Requirements for the Competence of Testing and

Calibration Laboratories, International Organisation for Standardization,
Geneva, Switzerland.

8. ILAC-G8:1996, Guideline on Assessment and Reporting of Compliance with

Specification, International Laboratory Accreditation Cooperation.

Issue No. 4 Issue Date: 09/10 Page 8 of 9

Method of Stating Test and Calibration Results and
Compliance with Specifications APLAC TC 004

APPENDIX A
Case 1 Case 2 Case 3 Case 4 Case 5
The measured result is The measured result The measured result is The measured result The measured result
under the upper limit, is below the upper on the limit itself; it is is above the upper is beyond the upper
even when extended limit, but by a margin therefore not possible limit, but by a margin limit, even when
upwards by half of the less than half of the to state compliance less than half of the extended downwards
uncertainty interval. uncertainty interval; it nor non-compliance at uncertainty interval; it by half of the
is therefore not any significant level of is therefore not uncertainty interval.
The product therefore possible to state confidence. possible to state non-
complies with the compliance. compliance. The product therefore
specification. However, where a does not comply with
However, where a decision must be made However, where a the specification.
confidence level of regardless of the level confidence level of
less than 95% is of confidence and the less than 95% is
acceptable, a requirement is: acceptable, a non-
compliance statement measured result the compliance statement
may be possible. upper limit, a may be possible.
compliance statement
may be possible.
When the requirement
is : measured value <
the upper limit, a non-
compliance statement
may be possible.
upper limit
Specified
lower limit
Specified

Case 6 Case 7 Case 8 Case 9 Case 10

The measured result is The measured result The measured result is The measured result The measured result
above the lower limit, is above the lower on the limit itself; it is is below the lower is beyond the lower
even when extended limit, but by a margin therefore not possible limit, but by a margin limit, even when
downwards by the half of less than half of the to state compliance less than half of the extended upwards by
the uncertainty interval. uncertainty interval; it nor non-compliance at uncertainty interval; it half of the uncertainty
is therefore not any significant level of is therefore not interval.
The product therefore possible to state confidence. possible to state non-
complies with the compliance. compliance. The product therefore
specification. However, where a does not comply with
However, where a decision must be made However, where a the specification.
confidence level of regardless of the level confidence level of
less than 95% is of confidence and the less than 95% is
acceptable, a requirement is: acceptable, a non-
compliance statement measured result compliance statement
may be possible. lower limit, a may be possible.
compliance statement
= measurement result may be possible.
When the requirement
with agreed method
is: measured result >
lower limit, a non-
I = uncertainty interval compliance statement
of agreed method may be possible.

Issue No. 4 Issue Date: 09/10 Page 9 of 9