Measure Phase

Measurement System Analysis

 Measurement System Analysis

Welcome to Measure

Process Discovery

Six Sigma Statistics

Measurement System Analysis

Basics of MSA

Variables MSA

Attribute MSA

Process Capability

Wrap Up & Action Items

OSSS LSS Green Belt v11.0 XL - Measure Phase 2 © OpenSourceSixSigma,LLC

 Introduction to MSA

So far we have learned that the heart and soul of Six Sigma is that it
is a data-driven methodology.
– How do you know that the data you have used is accurate and precise?
– How do know if a measurement is a repeatable and reproducible?

How good are these?

Measurement System Analysis

or
MSA
OSSS LSS Green Belt v11.0 XL - Measure Phase 3 © OpenSourceSixSigma,LLC
 Measurement System Analysis

MSA is a mathematical procedure to quantify variation introduced to a

process or product by the act of measuring.

Reference
Item to be
Measured Measurement
Operator Measurement Equipment
Process

Procedure
Environment

The item to be measured can be a physical part, document or a scenario for customer service.
Operator can refer to a person or can be different instruments measuring the same products.
Reference is a standard that is used to calibrate the equipment.
Procedure is the method used to perform the test.
Equipment is the device used to measure the product.
Environment is the surroundings where the measures are performed.

OSSS LSS Green Belt v11.0 XL - Measure Phase 4 © OpenSourceSixSigma,LLC

 Measurement Purpose

In order to be worth collecting, measurements must provide value - that is,

they must provide us with information and ultimately, knowledge

The question…

What do I need to know?

…must be answered before we begin to consider issues of measurements, metrics,
statistics, or data collection systems

Too often, organizations build complex data collection and information

management systems without truly understanding how the data collected and
metrics calculated actually benefit the organization.

OSSS LSS Green Belt v11.0 XL - Measure Phase 5 © OpenSourceSixSigma,LLC

 Purpose

The purpose of MSA is to assess the error due to measurement

systems.
The error can be partitioned into specific sources:
– Precision
• Repeatability - within an operator or piece of equipment
• Reproducibility - operator to operator or attribute gage to attribute
gage
– Accuracy
• Stability - accuracy over time
• Linearity- accuracy throughout the measurement range
• Resolution
• Bias – Off-set from true value
– Constant Bias
– Variable Bias – typically seen with electronic equipment,
amount of Bias changes with setting levels

OSSS LSS Green Belt v11.0 XL - Measure Phase 6 © OpenSourceSixSigma,LLC

 Accuracy and Precision

Accurate
Accuratebut butnot
notprecise
precise--On On Precise
Precisebut
butnotnotaccurate
accurate--The
The
average,
average,the
theshots
shotsare
areininthe
thecenter
centerofof average
averageisisnot
notononthe
thecenter,
center,but
butthe
the
the
thetarget
targetbut
butthere
thereisisaalot
lotofof variability
variabilityisissmall
small
variability
variability

OSSS LSS Green Belt v11.0 XL - Measure Phase 7 © OpenSourceSixSigma,LLC

 MSA Uses

MSA can be used to:

Compare internal inspection standards with the standards of your

customer.

Highlight areas where calibration training is required.

Provide a method to evaluate inspector training effectiveness as well as

serves as an excellent training tool.

Provide a great way to:

–Compare existing measurement equipment.
–Qualify new inspection equipment.

OSSS LSS Green Belt v11.0 XL - Measure Phase 8 © OpenSourceSixSigma,LLC

 Why MSA?

Measurement System Analysis is important to:

• Study the % of variation in our process that is caused by our
measurement system.
• Compare measurements between operators.
• Compare measurements between two (or more) measurement devices.
• Provide criteria to accept new measurement systems (consider new
equipment).
• Evaluate a suspect gage.
• Evaluate a gage before and after repair.
• Determine true process variation.
• Evaluate effectiveness of training program.

OSSS LSS Green Belt v11.0 XL - Measure Phase 9 © OpenSourceSixSigma,LLC

 Appropriate Measures

Appropriate Measures are:

• Sufficient – available to be measured regularly

• Relevant –help to understand/isolate the problems

• Representative - of the process across shifts and people

• Contextual – collected with other relevant information that might

explain process variability.

OSSS LSS Green Belt v11.0 XL - Measure Phase 10 © OpenSourceSixSigma,LLC

 Poor Measures

Poor Measures can result from:

• Poor or non-existent operational definitions
• Difficult measures
• Poor sampling
• Lack of understanding of the definitions
• Inaccurate, insufficient or non-calibrated measurement
devices

Measurement Error compromises decisions that affect:

– Customers
– Producers
– Suppliers

OSSS LSS Green Belt v11.0 XL - Measure Phase 11 © OpenSourceSixSigma,LLC

 Examples of What to Measure

Examples of what and when to measure:

• Primary and secondary metrics
• Decision points in Process Maps
• Any and all gauges, measurement devices, instruments, etc
• “X’s” in the process
• Prior to Hypothesis Testing
• Prior to modeling
• Prior to planning designed experiments
• Before and after process changes
• To qualify operators

MSA is a Show Stopper!!!

OSSS LSS Green Belt v11.0 XL - Measure Phase 12 © OpenSourceSixSigma,LLC
 Components of Variation

Whenever you measure anything, the variation that you observe can be
segmented into the following components…

Observed Variation

Unit-to-unit (true) Variation Measurement System Error

Precision Accuracy

Repeatability Reproducibility Stability Bias Linearity

All measurement systems have error. If you don’t know how much of the variation
you observe is contributed by your measurement system, you cannot make confident
decisions.

If you were one speeding ticket away from losing your license, how fast would you
be willing to drive in a school zone?
OSSS LSS Green Belt v11.0 XL - Measure Phase 13 © OpenSourceSixSigma,LLC
 Precision

A precise metric is one that returns the same value of a given

attribute every time an estimate is made.

Precise data are independent of who estimates them or when the

estimate is made.

Precision can be partitioned into two components:

– Repeatability
– Reproducibility

Repeatability and Reproducibility = Gage R+R

OSSS LSS Green Belt v11.0 XL - Measure Phase 14 © OpenSourceSixSigma,LLC

 Repeatability

Repeatability is the variation in measurements obtained with one

measurement instrument used several times by one appraiser while
measuring the identical characteristic on the same part.

Repeatability
For example:
– Manufacturing: One person measures the purity of multiple samples of the
same vial and gets different purity measures.
– Transactional: One person evaluates a contract multiple times (over a
period of time) and makes different determinations of errors.

OSSS LSS Green Belt v11.0 XL - Measure Phase 15 © OpenSourceSixSigma,LLC

 Reproducibility

Reproducibility is the variation in the average of the measurements made

by different appraisers using the same measuring instrument when
measuring the identical characteristic on the same part.

Reproducibility

Y Operator A
Operator B

For example:
– Manufacturing: Different people perform purity test on samples from the
same vial and get different results.
– Transactional: Different people evaluate the same contract and make
different determinations.

OSSS LSS Green Belt v11.0 XL - Measure Phase 16 © OpenSourceSixSigma,LLC

 Time Estimate Exercise

Exercise objective: Demonstrate how well you can

estimate a 10 second time interval.

1. Pair up with an associate.

2. One person will say start and stop to indicate how long
they think the 10 seconds last. Do this 6 times.
3. The other person will have a watch with a second hand
to actually measure the duration of the estimate. Record
the value where your partner can’t see it.
4. Switch tasks with partner and do it 6 times also.
5. Record all estimates, what do you notice?

OSSS LSS Green Belt v11.0 XL - Measure Phase 17 © OpenSourceSixSigma,LLC

 Accuracy

An accurate measurement is the difference between the observed average of the

measurement and a reference value.
– When a metric or measurement system consistently over or under estimates the value
of an attribute, it is said to be “inaccurate”
Accuracy can be assessed in several ways:
– Measurement of a known standard
– Comparison with another known measurement method
– Prediction of a theoretical value
What happens if we don’t have standards, comparisons or theories?
True
Average

Accuracy
Warning, do not assume your metrology
reference is gospel.

Measurement

OSSS LSS Green Belt v11.0 XL - Measure Phase 18 © OpenSourceSixSigma,LLC

 Accuracy Against a Known Standard

In transactional processes, the measurement system can consist of a

database query.
– For example, you may be interested in measuring product returns
where you will want to analyze the details of the returns over some
time period.
– The query will provide you all the transaction details.

However, before you invest a lot of time analyzing the data, you must
ensure the data has integrity.
– The analysis should include a comparison with known reference
points.
– For the example of product returns, the transaction details should
add up to the same number that appears on financial reports, such
as the income statement.

OSSS LSS Green Belt v11.0 XL - Measure Phase 19 © OpenSourceSixSigma,LLC

 Accuracy vs. Precision

ACCURATE PRECISE BOTH

+ =

Accuracy relates to how close the

average of the shots are to the Master
or bull's-eye.

Precision relates to the spread of the

shots or Variance.
NEITHER

OSSS LSS Green Belt v11.0 XL - Measure Phase 20 © OpenSourceSixSigma,LLC

 Bias

Bias is defined as the deviation of the measured value from the actual
value.

Calibration procedures can minimize and control bias within acceptable

limits. Ideally, Bias can never be eliminated due to material wear and
tear!

Bias Bias

OSSS LSS Green Belt v11.0 XL - Measure Phase 21 © OpenSourceSixSigma,LLC

 Stability

Stability of a gauge is defined as error (measured in terms of Standard

Deviation) as a function of time. Environmental conditions such as
cleanliness, noise, vibration, lighting, chemical, wear and tear or other
factors usually influence gauge instability. Ideally, gauges can be
maintained to give a high degree of Stability but can never be eliminated
unlike Reproducibility. Gage Stability studies would be the first exercise
past calibration procedures.
Control Charts are commonly used to track the Stability of a measurement
system over time.
Drift

Stability is Bias characterized as a

function of time!

OSSS LSS Green Belt v11.0 XL - Measure Phase 22 © OpenSourceSixSigma,LLC

 Linearity

Linearity is defined as the difference in Bias values throughout the measurement

range in which the gauge is intended to be used. This tells you how accurate your
measurements are through the expected range of the measurements. It answers the
question, "Does my gage have the same accuracy for all sizes of objects being
measured?"

Linearity = |Slope| * Process Variation

Low Nominal High

% Linearity = |Slope| * 100 +e

B i a s (y)
0.00
*
-e
*
*
Reference Value (x)
y = a + b.x
y: Bias, x: Ref. Value
a: Slope, b: Intercept

OSSS LSS Green Belt v11.0 XL - Measure Phase 23 © OpenSourceSixSigma,LLC

 Types of MSA’s

MSA’s fall into two categories:

– Attribute
– Variable

Attribute Variable
– Pass/Fail – Continuous scale
– Go/No Go – Discrete scale
– Document Preparation – Critical dimensions
– Surface imperfections – Pull strength
– Customer Service Response – Warp

Transactional projects typically have Attribute based measurement systems.

Manufacturing projects generally use Variable studies more often, but do use
Attribute studies to a lesser degree.

OSSS LSS Green Belt v11.0 XL - Measure Phase 24 © OpenSourceSixSigma,LLC

 Variable MSA’s

SigmaXL® calculates a column of variance components (VarComp) which are used to

calculate % Gage R&R using the ANOVA Method.

Measured Value True Value

Estimates for a Gage R&R study are obtained by calculating the variance components for
each term and for error. Repeatability, Operator and Operator*Part components are summed
to obtain a total Variability due to the measuring system.
We use variance components to assess the Variation contributed by each source of
measurement error relative to the total Variation.

OSSS LSS Green Belt v11.0 XL - Measure Phase 25 © OpenSourceSixSigma,LLC

 Cheat Sheet

Contribution of Variation to the total

Variation of the study. Use % Study Var when you are interested in comparing
the measurement system Variation to the total
Variation.
% Study Var is calculated by dividing each value in
% Contribution, based on variance components, Study Var by Total Variation and Multiplying by 100.
is calculated by dividing each value in VarComp
by the Total Variation then multiplying the result Study Var is calculated as 5.15 times the Standard
by 100. Deviation for each source.
(5.15 is used because when data are normally
distributed, 99% of the data fall within 5.15 Standard
Deviations.)

OSSS LSS Green Belt v11.0 XL - Measure Phase 26 © OpenSourceSixSigma,LLC

 Cheat Sheet

SigmaXL® Report:

When the process tolerance is entered in the system,

SigmaXL®® calculates % Tolerance which compares
measurements system Variation to customer
specification. This allows us to determine the
proportion of the process tolerance that is used by
the Variation in the measurement system.

0.186980
Distinct Categories  1.41
0.031861517
 5.8685 1.41
8 (Rounded Down )

OSSS LSS Green Belt v11.0 XL - Measure Phase 27 © OpenSourceSixSigma,LLC

 Number of Distinct Categories

The number of distinct categories tells you how many separate groups of
parts the system is able to distinguish.

Unacceptable for estimating

process parameters and indices
Only indicates whether the
process is producing
conforming or nonconforming
parts
1 Data Category

Generally unacceptable for

estimating process parameters
and indices
Only provides coarse estimates
2 - 4 Categories

Recommended

5 or more Categories

OSSS LSS Green Belt v11.0 XL - Measure Phase 28 © OpenSourceSixSigma,LLC

 AIAG Standards for Gage Acceptance

Here are the Automotive Industry Action Group’s definitions for

Gage acceptance.

% Tolerance
or % Contribution System is…
% Study Variance

10% or less 1% or less Ideal

10% - 20% 1% - 4% Acceptable

20% - 30% 5% - 9% Marginal

30% or greater 10% or greater Poor

OSSS LSS Green Belt v11.0 XL - Measure Phase 29 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

Components of Variation

The
TheSigmaXL
SigmaXL®report
®
reportbreaks
breaksdown
downthe
thevariation
variationininthe
themeasurement
measurementsystem
systeminto
intospecific
specificsources.
sources.The
The
bar chart shown was created using Excel’s Clustered Column Bar Chart to graphically
bar chart shown was created using Excel’s Clustered Column Bar Chart to graphically display display
the
theComponents
ComponentsofofVariation.
Variation. Each
Eachcluster
clusterofofbars
barsrepresents
representsaasource
sourceofofvariation.
variation.

InInaagood
goodmeasurement
measurementsystem,
system,the
thelargest
largestcomponent
componentofofVariation
VariationisisPart-to-Part
Part-to-Partvariation.
variation. IfIfinstead
instead
you have large amounts of variation attributed to Gage R&R, then corrective action is needed.
you have large amounts of variation attributed to Gage R&R, then corrective action is needed.

OSSS LSS Green Belt v11.0 XL - Measure Phase 30 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

SigmaXL® ®provides an R Chart and Xbar Chart by Operator. The R chart consists of the following:
SigmaXL provides an R Chart and Xbar Chart by Operator. The R chart consists of the following:
- The plotted points are the difference between the largest and smallest measurements on each part for each operator. If the
- The plotted points are the difference between the largest and smallest measurements on each part for each operator. If the
measurements are the same then the range = 0.
measurements are the same then the range = 0.
- The Center Line, is the grand average for the process.
- The Center Line, is the grand average for the process.
- The Control Limits represent the amount of variation expected for the subgroup ranges. These limits are calculated using
- The Control Limits represent the amount of variation expected for the subgroup ranges. These limits are calculated using
the variation within subgroups.
the variation within subgroups.
If any of the points on the graph go above the upper Control Limit (UCL), then that operator is having problems consistently
If any of the points on the graph go above the upper Control Limit (UCL), then that operator is having problems consistently
measuring parts. The Upper Control Limit value takes into account the number of measurements by an operator on a part
measuring parts. The Upper Control Limit value takes into account the number of measurements by an operator on a part
and the variability between parts. If the operators are measuring consistently, then these ranges should be small relative to
and the variability between parts. If the operators are measuring consistently, then these ranges should be small relative to
the data and the points should stay in control.
the data and the points should stay in control.

OSSS LSS Green Belt v11.0 XL - Measure Phase 31 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

SigmaXL® ®provides an R Chart and Xbar Chart by Operator. The Xbar Chart compares the part-to-part variation to repeatability. The
SigmaXL provides an R Chart and Xbar Chart by Operator. The Xbar Chart compares the part-to-part variation to repeatability. The
Xbar chart consists of the following:
Xbar chart consists of the following:
- The plotted points are the average measurement on each part for each operator.
- The plotted points are the average measurement on each part for each operator.
- The Center Line is the overall average for all part measurements by all operators.
- The Center Line is the overall average for all part measurements by all operators.
- The Control Limits (UCL and LCL) are based on the variability between parts and the number of measurements in each average.
- The Control Limits (UCL and LCL) are based on the variability between parts and the number of measurements in each average.
Because the parts chosen for a Gage R&R study should represent the entire range of possible parts, this graph should ideally show lack-
Because the parts chosen for a Gage R&R study should represent the entire range of possible parts, this graph should ideally show lack-
of-control. Lack-of-control exists when many points are above the Upper Control Limit and/or below the Lower Control Limit.
of-control. Lack-of-control exists when many points are above the Upper Control Limit and/or below the Lower Control Limit.
In this case there are several points out of control which indicates the measurement system is adequate.
In this case there are several points out of control which indicates the measurement system is adequate.

OSSS LSS Green Belt v11.0 XL - Measure Phase 32 © OpenSourceSixSigma,LLC

 SigmaXL®’s Gage R&R Multi-Vari Output

The Multi-Vari Charts show each Part as a separate graph. Each Operator’s response readings are denoted as a vertical
The Multi-Vari Charts show each Part as a separate graph. Each Operator’s response readings are denoted as a vertical
line with the top tick corresponding to the Maximum value, bottom tick is the Minimum, and the middle tick is the Mean. The
line with the top tick corresponding to the Maximum value, bottom tick is the Minimum, and the middle tick is the Mean. The
horizontal line across each graph is the overall average for each part.
horizontal line across each graph is the overall average for each part.
Ideally the connected means red line should be horizontal (i.e., small reproducibility) and the vertical lines should be short
Ideally the connected means red line should be horizontal (i.e., small reproducibility) and the vertical lines should be short
(small repeatability).
(small repeatability).

OSSS LSS Green Belt v11.0 XL - Measure Phase 33 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

Pattern Means…

Using
Usingthe
theSigmaXL
SigmaXL®two-way
®
two-wayANOVA
ANOVA tool toolcreates
creates
Lines are virtually identical Operators are measuring the
an interaction chart that shows the average
an interaction chart that shows the average
parts the same
measurements
measurementstakentakenbybyeach
eachoperator
operatoron oneach
eachpart
partinin
One line is consistently higher That operator is measuring the
or lower than the others parts consistently higher or thestudy,
study,arranged
arrangedby bypart.
part. Each
Eachline
lineconnects
connectsthe
the
averages for a single operator.
averages for a single operator.
lower than the others
Lines are not parallel or they The operators ability to Ideally,
cross measure a part depends on
Ideally,the
thelines
lineswill
willfollow
followthe
thesame
samepattern
patternand
andthe
the
part averages will vary enough that differences
part averages will vary enough that differences
which part is being measured
between
betweenparts
partsare
areclear.
clear.
(an interaction between
operator and part)

OSSS LSS Green Belt v11.0 XL - Measure Phase 34 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

The “By Part” Multi-Vari Chart allows us to analyze all of the measurements taken in the study arranged by part. The
measurements are represented by dots; the means by the middle bar. The red line connects the average measurements
for each part.

Ideally,
 Multiple measurements for each individual part have little variation (the dots for one part will be close
together)
• Averages will vary enough that differences between parts are clear

OSSS LSS Green Belt v11.0 XL - Measure Phase 35 © OpenSourceSixSigma,LLC

 SigmaXL® Graphic Output Cheat Sheet

The
The“By
“ByOperator”
Operator”Multi-Vari
Multi-VariChart
Chartisiscreated
createdby
bymodifying
modifyingthe
theX’s
X’sfrom
fromparts
partstotooperator.
operator. This
Thishelps
helpsusus
determine
determinewhether
whetherthe
thevariability
variabilityininmeasurements
measurementsare
areconsistent
consistentacross
acrossoperators.
operators.
The
Theby
byoperator
operatorgraph
graphshows
showsall
allthe
thestudy
studymeasurements
measurementsarranged
arrangedbybyoperator.
operator. Dots
Dotsrepresent
representthe
themeasurements;
measurements;
the middle bars represent the means. The red line connects the average measurements for each operator.
the middle bars represent the means. The red line connects the average measurements for each operator.

If the red line is … Then…

Parallel to the x-axis The operators are measuring the parts similarly
Not parallel to the x-axis The operators are measuring the parts differently
You
Youcancanalso
alsoassess
assesswhether
whetherthe
theoverall
overallVariability
Variabilityininpart
partmeasurement
measurementisisthe
thesame
sameusing
usingthis
thisgraph.
graph. IsIsthe
thespread
spread
ininthe measurements similar? Or is one operator more Variable than the others?
the measurements similar? Or is one operator more Variable than the others?

OSSS LSS Green Belt v11.0 XL - Measure Phase 36 © OpenSourceSixSigma,LLC

 Practical Conclusions

For this example, the measuring system contributes little to the overall Variation, as confirmed
by both the Gage R&R table and graphs.
The Variation due to the measurement system, as a percent of study (Total) Variation is
causing 16.80% of the Variation seen in the process.
By AIAG Standards this gage should be used. By all standards, the
data being produced by this gage is acceptable, and valid for analysis.

% Tolerance
or % Contribution System is…
% Study Variance

10% or less 1% or less Ideal

10% - 20% 1% - 4% Acceptable

20% - 30% 5% - 9% Marginal

30% or greater 10% or greater Poor

OSSS LSS Green Belt v11.0 XL - Measure Phase 37 © OpenSourceSixSigma,LLC

 Repeatability and Reproducibility Problems

Repeatability Problems:
• Calibrate or replace gage.
• If only occurring with one operator, re-train.

Reproducibility Problems:
• Measurement machines
– Similar machines
• Ensure all have been calibrated and that the standard measurement method is
being utilized.
– Dissimilar machines
• One machine is superior.
• Operators
– Training and skill level of the operators must be assessed.
– Operators should be observed to ensure that standard procedures are followed.
• Operator/machine by part interactions
– Understand why the operator/machine had problems measuring some parts and not
others.
• Re-measure the problem parts
• Problem could be a result of gage linearity
• Problem could be fixture problem
• Problem could be poor gage design

OSSS LSS Green Belt v11.0 XL - Measure Phase 38 © OpenSourceSixSigma,LLC

 Design Types

Crossed Design
• A Crossed Design is used only in non-destructive testing and assumes that all the parts can be
measured multiple times by either operators or multiple machines.
– Gives the ability to separate part-to-part Variation from measurement system Variation.
– Assesses Repeatability and Reproducibility.
– Assesses the interaction between the operator and the part.

Nested Design
• A Nested Design is used for destructive testing (we will learn about this in MBB training) and
also situations where it is not possible to have all operators or machines measure all the parts
multiple times.
– Destructive testing assumes that all the parts within a single batch are identical enough to
claim they are the same.
– Nested designs are used to test measurement systems where it is not possible (or
desirable) to send operators with parts to different locations.
– Do not include all possible combinations of factors.
– Uses slightly different mathematical model than the Crossed Design.

OSSS LSS Green Belt v11.0 XL - Measure Phase 39 © OpenSourceSixSigma,LLC

 Gage R & R Study

Gage R&R Study

– Is a set of trials conducted to assess the Repeatability and Reproducibility of the
measurement system.
– Multiple people measure the same characteristic of the same set of multiple
units multiple times (a crossed study)
– Example: 10 units are measured by 3 people. These units are then randomized
and a second measure on each unit is taken.

A Blind Study is extremely desirable.

– Best scenario: operator does not know the measurement is a part of a test
– At minimum: operators should not know which of the test parts they are
currently measuring.

NO, not that kind of R&R!

OSSS LSS Green Belt v11.0 XL - Measure Phase 40 © OpenSourceSixSigma,LLC

 Variable Gage R & R Steps

Step 1: Call a team meeting and introduce the concepts of the Gage R&R
Step 2: Select parts for the study across the range of interest
– If the intent is to evaluate the measurement system throughout the process range, select
parts throughout the range
– If only a small improvement is being made to the process, the range of interest is now the
improvement range
Step 3: Identify the inspectors or equipment you plan to use for the analysis
– In the case of inspectors, explain the purpose of the analysis and that the inspection
system is being evaluated not the people
Step 4: Calibrate the gage or gages for the study
– Remember Linearity, Stability and Bias
Step 5: Have the first inspector measure all the samples once in random order
Step 6: Have the second inspector measure all the samples in random order
– Continue this process until all the operators have measured all the parts one time
– This completes the first replicate
Step 7: Repeat steps 5 and 6 for the required number of replicates
– Ensure there is always a delay between the first and second inspection
Step 8: Enter the data into SigmaXL® and analyze your results
Step 9: Draw conclusions and make changes if necessary

OSSS LSS Green Belt v11.0 XL - Measure Phase 41 © OpenSourceSixSigma,LLC

 Gage R & R Study

Part Allocation From Any Population

10 x 3 x 2 Crossed Design is shown

A minimum of two measurements/part/operator is required
Three is better!

Trial 1
Operator 1
Trial 2
P
a
Trial 1
r 1 2 3 4 5 6 7 8 9 10 Operator 2
t Trial 2
s
Trial 1
Operator 3
Trial 2
OSSS LSS Green Belt v11.0 XL - Measure Phase 42 © OpenSourceSixSigma,LLC
 Data Collection Sheet

Create a data collection sheet for:

– 10 parts
– 3 operators
– 2 trials

OSSS LSS Green Belt v11.0 XL - Measure Phase 43 © OpenSourceSixSigma,LLC

 The Data Collection Sheet

OSSS LSS Green Belt v11.0 XL - Measure Phase 44 © OpenSourceSixSigma,LLC

 Gage R & R

Open the worksheet “Gage AIAG2 - SigmaXL Format”.

Variables:
– Part
– Operator
– Response

OSSS LSS Green Belt v11.0 XL - Measure Phase 45 © OpenSourceSixSigma,LLC

 Gage R & R

Use 1.0 for the

tolerance.

OSSS LSS Green Belt v11.0 XL - Measure Phase 46 © OpenSourceSixSigma,LLC

 Graphical Output

Looking at the “Components of Variation” chart, the Part to Part Variation needs to be larger than
Gage Variation.

If in the “Components of Variation” chart the “Gage R&R” bars are larger than the “Part-to-Part”
bars, then all your measurement Variation is in the measuring tool i.e.… “maybe the gage needs to
be replaced”.

Part to Part
Variation needs to
be larger than Gage
Variation

OSSS LSS Green Belt v11.0 XL - Measure Phase 47 © OpenSourceSixSigma,LLC

 Graphical Output

The same concept applies to the “Response by Operator” chart. If there is

extreme Variation within operators, then the training of the operators is
suspect.
Operator
Error

OSSS LSS Green Belt v11.0 XL - Measure Phase 48 © OpenSourceSixSigma,LLC

 Session Window

I can see clearly now!

OSSS LSS Green Belt v11.0 XL - Measure Phase 49 © OpenSourceSixSigma,LLC
 Session Window

If the Variation due to Gage R & R is high, consider:

• Procedures revision?
• Gage update? • 20 % < % Tol GRR < 30%  Gage Unacceptable
• Operator issue? • 10 % < % Tol GRR < 20 %  Gage Acceptable
• Tolerance validation?
• 1 % < % Tol GRR < 10 %  Gage Preferable

OSSS LSS Green Belt v11.0 XL - Measure Phase 50 © OpenSourceSixSigma,LLC

 Signal Averaging

Signal Averaging can be used to reduce Repeatability error when a better

gage is not available.
– Uses average of repeat measurements.
– Uses Central Limit theorem to estimate how many repeat measures
are necessary.

Signal Averaging is a method to

reduce Repeatability error in a
poor gage when a better gage is
not available or when a better
gage is not possible.

OSSS LSS Green Belt v11.0 XL - Measure Phase 51 © OpenSourceSixSigma,LLC

 Signal Averaging Example

Suppose SV/Tolerance is 35%.

SV/Tolerance must be 15% or less to use gage.

Suppose the Standard Deviation for one part measured by one person many
times is 9.5.

Determine what the new reduced Standard Deviation should be.

OSSS LSS Green Belt v11.0 XL - Measure Phase 52 © OpenSourceSixSigma,LLC

 Signal Averaging Example

Determine sample size:

Using
Usingthetheaverage
averageofof66
repeated
repeatedmeasures
measureswill
will
reduce
reducethetheRepeatability
Repeatability
component
componentof ofmeasurement
measurement
error
errortotothe
thedesired
desired15%
15%
level.
level.

This method should be considered temporary!

OSSS LSS Green Belt v11.0 XL - Measure Phase 53 © OpenSourceSixSigma,LLC
 Paper Cutting Exercise

Exercise objective: Perform and Analyze a variable MSA

Study.

1. Cut a piece of paper into 12 different lengths that are all fairly
close to one another but not too uniform. Label the back of the
piece of paper to designate its “part number”
2. Perform a variable gage R&R study as outlined in this module. Use
the following guidelines:
– Number of parts: 12
– Number of inspectors: 3
– Number of trials: 5
3. Create a SigmaXL® data sheet and enter the data into the sheet as
each inspector performs a measurement. If possible, assign one
person to data collection.
4. Analyze the results and discuss with your mentor.

OSSS LSS Green Belt v11.0 XL - Measure Phase 54 © OpenSourceSixSigma,LLC

 Attribute MSA

A methodology used to assess Attribute Measurement Systems.

Attribute
Attribute Gage
Gage Error
Error

Repeatability
Repeatability Reproducibility
Reproducibility Calibration
Calibration
– They are used in situations where a continuous measure cannot be
obtained.
– It requires a minimum of 5x as many samples as a continuous study.
– Disagreements should be used to clarify operational definitions for the
categories.
• Attribute data are usually the result of human judgment (which category
does this item belong in).
• When categorizing items (good/bad; type of call; reason for leaving) you
need a high degree of agreement on which way an item should be
categorized.

OSSS LSS Green Belt v11.0 XL - Measure Phase 55 © OpenSourceSixSigma,LLC

 Attribute MSA Purpose

The purpose of an Attribute MSA is:

– To determine if all inspectors use the same criteria to determine “pass” from “fail”.
– To assess your inspection standards against your customer’s requirements.
– To determine how well inspectors are conforming to themselves.
– To identify how inspectors are conforming to a “known master,” which includes:
• How often operators ship defective product.
• How often operators dispose of acceptable product.
– Discover areas where:
• Training is required.
• Procedures must be developed.
• Standards are not available.

An Attribute MSA is similar in many ways to the continuous MSA, including the purposes. Do
you have any visual inspections in your processes? In your experience how effective have they
been?

OSSS LSS Green Belt v11.0 XL - Measure Phase 56 © OpenSourceSixSigma,LLC

 Visual Inspection Test

Take 60 Seconds and count the number of times “F” appears in this paragraph?

The Necessity of Training Farm Hands for First Class Farms in

the Fatherly Handling of Farm Live Stock is Foremost in the
Eyes of Farm Owners. Since the Forefathers of the Farm
Owners Trained the Farm Hands for First Class Farms in the
Fatherly Handling of Farm Live Stock, the Farm Owners Feel
they should carry on with the Family Tradition of Training
Farm Hands of First Class Farmers in the Fatherly Handling of
Farm Live Stock Because they Believe it is the Basis of Good
Fundamental Farm Management.

OSSS LSS Green Belt v11.0 XL - Measure Phase 57 © OpenSourceSixSigma,LLC

 How can we Improve Visual Inspection?

Visual Inspection can be improved by:

• Operator Training & Certification
• Develop Visual Aids/Boundary Samples
• Establish Standards
• Establish Set-Up Procedures
• Establish Evaluation Procedures
– Evaluation of the same location on each part.
– Each evaluation performed under the same lighting.
– Ensure all evaluations are made with the same standard.

Look closely now!

OSSS LSS Green Belt v11.0 XL - Measure Phase 58 © OpenSourceSixSigma,LLC
 Attribute Agreement Analysis

Attribute MSA (Binary)

Attribute MSA is also known as Attribute Agreement Analysis. The
response must be binary (e.g. Pass/Fail, Good/Bad, G/NG, Yes/No).

1. Open the worksheet Attribute MSA

– AIAG. This is an example from the
AIAG MSA Reference Manual, 3rd
Edition, page 127. Note that the
worksheet data must be in stacked
column format

2. Click SigmaXL > Measurement

Systems Analysis > Attribute MSA
(Binary). Ensure that the entire data
table is selected. Click Next.

3. Select Part, Appraiser, Assessed

Result and Reference as shown.
Check Report Information and enter
AIAG Example, Page 127.

4. Click OK. The results are shown on

the next slide.

OSSS LSS Green Belt v11.0 XL - Measure Phase 59 © OpenSourceSixSigma,LLC

Attribute MSA (Binary)

OSSS LSS Green Belt v11.0 XL - Measure Phase 60 © OpenSourceSixSigma,LLC

 M&M Exercise

Exercise objective: Perform and Analyze an Attribute MSA Study.

• You will need the following to complete the study:

– A bag of M&Ms containing 50 or more “pieces”
– The attribute value for each piece.
– Three or more inspectors.

• Judge each M&M as pass or fail.

Number Part Attribute
– The customer has indicated that they want a bright and shiny M&M and
1 M&M Pass that they like M’s.

2 M&M Fail
• Pick 50 M&Ms out of a package.
3 M&M Pass
• Enter results into SigmaXL®'s Attribute MSA Template and draw
conclusions.

• The instructor will represent the customer for the Attribute score.

OSSS LSS Green Belt v11.0 XL - Measure Phase 61 © OpenSourceSixSigma,LLC

 Summary

At this point, you should be able to:

• Understand Precision & Accuracy

• Understand Bias, Linearity and Stability

• Understand Repeatability & Reproducibility

• Understand the impact of poor gage capability on product quality

• Identify the various components of Variation

• Perform the step by step methodology in Variable and Attribute MSA’s

OSSS LSS Green Belt v11.0 XL - Measure Phase 62 © OpenSourceSixSigma,LLC

 The Certified Lean Six Sigma Green Belt Assessment
The Certified Lean Six Sigma Green Belt (CLSSGB) tests are
useful for assessing Green Belt’s knowledge of Lean Six Sigma.
The CLSSGB can be used in preparation for the ASQ or IASSC
Certified Six Sigma Green Belt (CSSGB) exam or for any number
of other certifications, including private company certifications.

The Lean Six Sigma Green Belt Course Manual

Open Source Six Sigma Course Manuals are professionally

designed and formatted manuals used by Belt’s during training and
for reference guides afterwards. The OSSS manuals complement
the OSSS Training Materials and consist of slide content,
instructional notes data sets
and templates.

OSSS LSS Green Belt v11.0 XL - Measure Phase 63 © OpenSourceSixSigma,LLC