Standard Practice For Using Significant Digits in Test Data To Determine Conformance With Specifications
Standard Practice For Using Significant Digits in Test Data To Determine Conformance With Specifications
for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
&RS\ULJKWE\$670,QW
ODOOULJKWVUHVHUYHG)UL'HF*07
1
'RZQORDGHGSULQWHGE\
8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHU8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHUSXUVXDQWWR/LFHQVH$JUHHPHQW1RIXUWKHUUHSURGXFWLRQVDXWKRUL]HG
E29 − 22
numbers represented with a decimal point are significant digits. This will be referred to as the rounding method, which is
For example, the numbers 1270. and 32.00 each have four discussed in Section 7.
significant digits. The significance of trailing zeros for numbers 4.4 Section 8 of this practice gives guidelines for use in
represented without use of a decimal point can only be recording, calculating, and reporting the final result for test
identified from knowledge of the source of the value. For data.
example, a modulus strength, stated as 140 000 Pa, may have
as few as two or as many as six significant digits. 5. Treatment of Test Results
3.1.4.3 Discussion—To eliminate ambiguity, the exponen-
tial notation may be used. Thus, 1.40 × 105 indicates that the 5.1 Bias adjustments to be applied and criteria for accept-
modulus is reported to the nearest 0.01 × 105 or 1000 Pa. ability of the test data should be specified in the test method.
3.1.4.4 Discussion—Use of appropriate SI prefixes is rec- 5.2 Unless permitted by the specification or agreed by
ommended for metric units to reduce the need for trailing zeros supplier and receiver, the observed or calculated value for
of uncertain significance. Thus, 140 kPa (without the decimal comparison to limits shall not be modified in consideration of
point) indicates that the modulus is reported either to the measurement uncertainty prior to or subsequent to use of the
nearest 10 or 1 kPa, which is ambiguous with respect to the absolute or rounding method in this standard. When adjust-
number of significant digits. However, 0.140 MPa clearly ments to the observed or calculated value for comparison to
indicates that the modulus is reported to the nearest 1 kPa, and specification based on measurement uncertainty are permitted,
0.14 MPa clearly indicates that the modulus is reported to the the non-adjusted value, adjusted value, and adjustment method
nearest 10 kPa. shall be reported.
3.1.5 test result, n—the value of a characteristic obtained by 5.3 Unless part of a retest that is compliant with applicable
carrying out a specified test method. E2282 guidelines and is permitted by the specification, test method, or
4. Significance and Use a laboratory procedure, the test result for comparison to
specifications shall not be selected among multiple test results
4.1 A test result is compared to specification limits to of equal validity or repeat testing until a passing value is
determine conformance with the specification. The test result obtained.
may be an observed value, calculated from two or more
property measurements, or a summary value of multiple test 6. Absolute Method
determinations as defined in the test method.
6.1 Where Applicable—The absolute method applies where
4.2 The unqualified statement of a numerical limit, such as it is the intent that all digits in an observed value or a calculated
“2.50 in. max,” cannot, in view of different established value are to be considered significant for purposes of deter-
practices and customs, be regarded as carrying a definite mining conformance with specifications. Under these
operational meaning concerning the number of digits to be conditions, the specified limits are referred to as absolute
retained in an observed or a calculated value for purposes of limits.
determining conformance with specifications.
6.2 How Applied—With the absolute method, an observed
4.3 This practice describes two commonly accepted meth- value or a calculated value is not to be rounded, but is to be
ods to evaluate conformance of a numerical value to compared directly with the specified limiting value. Confor-
specification, identified as the Absolute Method and the mance or nonconformance with the specification is based on
Rounding Method. In the applications of this practice to a this comparison.
specific material or materials it is essential to specify which
method is intended to apply. The choice of method depends 6.3 How Expressed—This intent may be expressed in the
upon the current practice of the particular branch of industry or standard in one of the following forms:
technology concerned, and should therefore be specified in the 6.3.1 If the absolute method is to apply to all specified limits
prime publication. in the standard, this may be indicated by including the
4.3.1 Absolute Method—In some fields, specification limits following sentence in the standard:
of 2.5 in. max, 2.50 in. max, and 2.500 in. max are all taken to For purposes of determining conformance with these specifications, all
imply the same absolute limit of exactly two and a half inches specified limits in this standard are absolute limits, as defined in ASTM Practice
E29, for Using Significant Digits in Test Data to Determine Conformance with
and for purposes of determining conformance with Specifications.
specifications, an observed value or a calculated value is to be 6.3.2 If the absolute method is to apply to all specified limits
compared directly with the specified limit. Thus, any deviation, of some general type in the standard (such as dimensional
however small, outside the specification limit signifies noncon- tolerance limits), this may be indicated by including the
formance with the specifications. This will be referred to as the following sentence in the standard:
absolute method, which is discussed in Section 6.
For purposes of determining conformance with these specifications, all
4.3.2 Rounding Method—In other fields, specification limits specified (dimensional tolerance) limits are absolute limits, as defined in ASTM
of 2.5 in. max, 2.50 in. max, and 2.500 in. max are taken to Practice E29, for Using Significant Digits in Test Data to Determine Confor-
imply that, for the purposes of determining conformance with mance with Specifications.
specifications, an observed value or a calculated value should 6.3.3 If the absolute method is to apply to all specified limits
be rounded to the nearest 0.1 in., 0.01 in., 0.001 in., given in a table, this may be indicated by including a footnote
respectively, and then compared with the specification limit. with the table as follows:
&RS\ULJKWE\$670,QW
ODOOULJKWVUHVHUYHG)UL'HF*07
2
'RZQORDGHGSULQWHGE\
8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHU8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHUSXUVXDQWWR/LFHQVH$JUHHPHQW1RIXUWKHUUHSURGXFWLRQVDXWKRUL]HG
E29 − 22
&RS\ULJKWE\$670,QW
ODOOULJKWVUHVHUYHG)UL'HF*07
3
'RZQORDGHGSULQWHGE\
8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHU8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHUSXUVXDQWWR/LFHQVH$JUHHPHQW1RIXUWKHUUHSURGXFWLRQVDXWKRUL]HG
E29 − 22
number of significant digits, choose that which is nearest. If 7.7.2 Round the result obtained in 7.7.1 to the nearest whole
two choices are possible, as when the digits dropped are number following the conventions in 7.4, 7.4.1, 7.4.2, and 7.4.3
exactly a 5 or a 5 followed only by zeros, choose that ending as appropriate.
in an even digit. Table 1 gives examples of applying this 7.7.3 Multiply the result obtained in 7.7.2 by the desired
rounding procedure. rounding increment or interval.
7.5 The rounded value should be obtained in one step by 7.7.4 For example, in rounding 0.07 to the nearest 0.02,
direct rounding of the most precise value available and not in dividing 0.07 by 0.02 gives a value of 3.5. Rounding this value
two or more successive roundings. For example: 89 490 to the nearest whole number gives a value of 4, based on the
rounded to the nearest 1000 is at once 89 000; it would be information in 7.4.3. Multiplying 4 by 0.02 yields 0.08. In
incorrect to round first to the nearest 100, giving 89 500 and rounding 0.09 to the nearest 0.02, dividing 0.09 by 0.02 gives
then to the nearest 1000, giving 90 000. a value of 4.5. Rounding this value to the nearest whole
number gives a value of 4, based on the information in 7.4.3.
7.6 Special Case, Rounding to the Nearest 50, 5, 0.5, 0.05, Multiplying 4 by 0.02 yields 0.08.
etc.—If in special cases it is desired to specify rounding to the
nearest 50, 5, 0.5, 0.05, etc., this may be done by so indicating
8. Guidelines for Retaining Significant Figures in
in the standard. In order to round to the nearest 50, 5, 0.5, 0.05,
Calculation and Reporting of Test Results
etc., double the observed or calculated value, round to the
nearest 100, 10, 1.0, 0.10, etc., in accordance with the 8.1 General Discussion—Rounding test results avoids a
procedure in 7.4, and divide by 2. For example, in rounding misleading impression of precision while preventing loss of
6025 to the nearest 50, 6025 is doubled giving 12 050 which information due to coarse resolution. Any approach to retention
becomes 12 000 when rounded to the nearest 100 (7.4.3). of significant digits of necessity involves some loss of infor-
When 12 000 is divided by 2, the resulting number, 6000, is the mation; therefore, the level of rounding should be carefully
rounded value of 6025. In rounding 6075 to the nearest 50, selected considering both planned and potential uses for the
6075 is doubled giving 12 150 which becomes 12 200 when data. The number of significant digits must, first, be adequate
rounded to the nearest 100 (7.4.3). When 12 200 is divided by for comparison against specification limits (see 7.2). The
2, the resulting number, 6100, is the rounded value of 6075. following guidelines are intended to preserve the data for
statistical summaries. For certain purposes, such as where
7.7 Special Case, Rounding to the Nearest Interval Not
calculations involve differences of measurements close in
Covered in 7.4 or 7.6—In some test methods, there may be a
magnitude, and for some statistical calculations, such as paired
requirement to round a value to an interval that does not align
t-tests, autocorrelations, and nonparametric tests, reporting
with the specific requirements in 7.4 or 7.6, such as to the
data to a greater number of significant digits may be advisable.
nearest 0.02, 0.25, 0.3, etc. In such cases, the following
procedure can be used for rounding to any interval: 8.2 Recording Observed Values—When recording direct
measurements, as in reading marks on a buret, ruler, or dial, all
NOTE 2—Using a calculation subroutine that has been programmed to
perform the rounding procedure described in 7.7.1, 7.7.2, and 7.7.3 can be digits known exactly, plus one digit which may be uncertain
advantageous in evaluating laboratory data. due to estimation, should be recorded. For example, if a buret
7.7.1 Divide the result by the desired rounding increment or is graduated in units of 0.1 mL, then an observed value would
interval. be recorded as 9.76 mL where it is observed between 9.7 and
9.8 marks on the buret, and estimated about six tenths of the
TABLE 1 ExamplesA of Rounding way between those marks. When the measuring device has a
Rounded
vernier scale, the last digit recorded is the one from the vernier.
Observed
To Be
Value to be Conforms 8.2.1 The number of significant digits given by a digital
Value or Used for with display or printout from an instrument should be greater than or
Specified Limit Rounded
Calculated Purposes of Specified
Value
to Nearest
Determining Limit equal to those given by the rule for reporting test results in 8.4
Conformance below.
Yield point, 36 000 35 940 100 psi 35 900 no
psi, min 100 psi 36 000 yes 8.3 Calculation of Test Result from Observed Values—When
h 35 950
35 960 100 psi 36 000 yes calculating a test result from observed values, avoid rounding
of intermediate quantities. As far as is practicable with the
Nickel, 57 %, mass, 56.4 1% 56 no calculating device or form used, carry out calculations with the
min h 56.5 1% 56 no
1% 57 yes observed values exactly and round only the final result.
56.6
8.4 Reporting Test Results—A suggested rule relates the
Water extract 40.4 1 ms/m 40 yes
conductivity, 40 h 40.5 1 ms/m 40 yes significant digits of the test result to the precision of the
ms/m, max 40.6 1 ms/m 41 no measurement expressed as the standard deviation σ. The
applicable standard deviation is the repeatability standard
Sodium bicarbonate 0.54 0.1 % 0.5 yes deviation. The rounding interval for test results should not be
0.5 %, max, dry h 0.55 0.1 % 0.6 no
mass basis 0.56 0.1 % 0.6 no greater than 0.5 σ nor less than 0.05 σ, but not greater than the
A
unit in the specification (see 7.2). When only an estimate, s, is
These examples are meant to illustrate rounding rules and do not necessarily
reflect the usual number of digits associated with these test methods. available for σ, s may be used in place of σ in the preceding
sentence. An alternative statement of the suggested rule is:
&RS\ULJKWE\$670,QW
ODOOULJKWVUHVHUYHG)UL'HF*07
4
'RZQORDGHGSULQWHGE\
8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHU8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHUSXUVXDQWWR/LFHQVH$JUHHPHQW1RIXUWKHUUHSURGXFWLRQVDXWKRUL]HG
E29 − 22
Write down the standard deviation. Round test results to the 8.4.1.4 The rule for numbers representing exact counts or
place of the first significant digit in the standard deviation if the mathematical constants is that they are to be treated as having
digit is two or higher, to the next place if it is a one. an infinite number of significant digits.
Example: Examples:
A test result is calculated as 1.45729. The standard deviation of the test method (1) 1 − 0.23 ⁄2 = 0.88 where the numbers 1 and 2 are exact and 0.23 is an
is estimated to be, 0.0052. Round to 1.457 or the nearest 0.001 since this approximate quantity.
rounding unit, 0.001, is between 0.05 σ = 0.00026 and 0.5 σ = 0.0026. (2 ) A count of 50 pieces times a measured thickness 0.124 mm is
50 × 0.124 = 6.20 mm, having three significant figures.
NOTE 3—A rationale for this rule is derived from Sheppard’s adjustment (3) A measurement of 1.634 in. to the nearest thousandth, is converted to mm.
for grouping, which represents the standard deviation of a rounded test The result, 1.634 × 25.4 = 41.50 mm, has four significant digits. The
conversion constant, 25.4, is exact.
result by = σ 2 1w 2 /12 where σ is the standard deviation of the
NOTE 4—More extensive discussion of dimensional conversion can be
unrounded test result and w is the rounding interval. The quantity w/ =12 found in IEEE/ASTM SI 10.
is the standard deviation of an error uniformly distributed over the range
w. Rounding so that w is below 0.5 σ ensures that the standard deviation
8.5 Specification Limits—When the rounding method is to
is increased by at most 1 %. apply to given specified limits, it is desirable that the signifi-
cant digits of the specified limits should conform to the
8.4.1 When no estimate of the standard deviation σ is precision of the test following the rule of 8.4. That is, the
known, then rules for retention of significant digits of com- rounding unit for the specification limits should be between
puted quantities may be used to derive a number of significant 0.05 and 0.5 times the standard deviation of the test.
digits to be reported, based on significant digits of test data. 8.6 Averages and Standard Deviations—When reporting the
8.4.1.1 The rule when adding or subtracting test data is that average and standard deviation of replicated measurements or
the result contains no significant digits beyond the place of the repeated samplings of a material, a suggested rule for most
last significant digit of any datum. cases is to round the standard deviation to two significant digits
Examples: and round the average to the same last place of significant
(1) 11.24 + 9.3 + 6.32 = 26.9, since the last significant digit of 9.3 is the first digits. When the number of observations is large (more than 15
following the decimal place, 26.9 is obtained by rounding the exact sum,
26.86, to this place of digits.
when the lead digit of the standard deviation is 1, more than 50
(2) 926 − 923.4 = 3 with lead digit 2, more than 100 in other cases), an additional
(3) 140 000 + 91 460 = 231 000 when the first value was recorded to the digit may be advisable.
nearest thousand.
8.6.1 Alternative approaches for averages include reporting
8.4.1.2 The rule when multiplying or dividing is that the x̄ to within 0.05 to 0.5 times the standard deviation of the
result contains no more significant digits than the value with average σ/ =n , or applying rules for retaining significant digits
the smaller number of significant digits. to the calculation of x̄ . ASTM Manual 7 provides methods for
Examples: reporting x̄ and s for these applications.3
(1) 11.38 × 4.3 = 49, since the factor 4.3 has two significant digits. NOTE 5—A rationale for the suggested rule comes from the uncertainty
(2) (926 − 923.4) ⁄4.3 = 0.6 Only one figure is significant since the numerator of a calculated standard deviation s. The standard deviation of s based on
difference has only one significant digit. sampling from a normal distribution with n observations is approximately
8.4.1.3 The rules for logarithms and exponentials are: Digits σ/ =2n . Reporting s to within 0.05 to 0.5 of this value, following the rule
of 8.4, leads to two significant digits for most values of σ when the number
of ln(x) or log10(x) are significant through the n-th place after
of observations n is 100 or fewer.
the decimal when x has n significant digits. The number of Example: Analyses on six specimens give values of 3.56, 3.88, 3.95,
significant digits of ex or 10x is equal to the place of the last 4.07, 4.21, and 4.47 for a constituent. The average and standard deviation,
significant digit in x after the decimal. unrounded, are x̄ = 4.0233... and s = 0.3089... . The suggested rule would
report x̄ and s as 4.02 and 0.31.
Examples:
(1) ln(3.46) = 1.241 to three places after the decimal, since 3.46 has three 9. Keywords
significant digits.
(2 ) 103.46 = 2900 has two significant digits, since 3.46 is given to two places 9.1 absolute method; conformance; rounding; significant
after the decimal. digits; specifications; test data
ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.
This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.
This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or [email protected] (e-mail); or through the ASTM website
(www.astm.org). Permission rights to photocopy the standard may also be secured from the Copyright Clearance Center, 222
Rosewood Drive, Danvers, MA 01923, Tel: (978) 646-2600; http://www.copyright.com/
&RS\ULJKWE\$670,QW
ODOOULJKWVUHVHUYHG)UL'HF*07
5
'RZQORDGHGSULQWHGE\
8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHU8QLYHUVLGDG,QGXVWULDOGH6DQWDQGHUSXUVXDQWWR/LFHQVH$JUHHPHQW1RIXUWKHUUHSURGXFWLRQVDXWKRUL]HG