Sie sind auf Seite 1von 70

Measurement

System
Analysis
Welcome to MSA

Process Discovery

Six Sigma Statistics

Measurement System Analysis

Basics of MSA

Variables MSA

Attribute MSA

Process Capability

Wrap Up & Action Items


2
Introduction to MSA
So far we have learned that the heart and soul of Six Sigma is that it is a data-driven
methodology.
– How do you know that the data you have used is accurate and precise?
– How do know if a measurement is a repeatable and reproducible?

How good are these?

Measurement System Analysis


MSA
3
Measurement System Analysis
The item to be measured can be
a physical part, document or a
scenario for customer service.
Item to be Reference Operator can refer to a person
Measured Measurement or can be different instruments
measuring the same products.
Operator Measurement Equipment
Reference is a standard that is
Process used to calibrate the equipment.
Procedure is the method used
Procedure to perform the test.
Environment Equipment is the device used to
measure the product.
Environment is the surroundings
where the measures are
MSA is a mathematical procedure to quantify variation performed.
introduced to a process or product by the act of measuring.

4
Measurement Purpose
In order to be worth collecting, measurements must provide value - that is, they must provide us
with information and ultimately, knowledge

The question…

What do I need to know?


…must be answered before we begin to consider issues of measurements, metrics, statistics, or data
collection systems

Too often, organizations build complex data collection and information management systems
without truly understanding how the data collected and metrics calculated actually benefit the
organization.

5
Purpose
The purpose of MSA is to assess the error due to measurement systems.

The error can be partitioned into specific sources:


• Precision
• Repeatability - within an operator or piece of equipment
• Reproducibility - operator to operator or attribute gage to attribute gage
• Accuracy
• Stability - accuracy over time
• Linearity- accuracy throughout the measurement range
• Resolution
• Bias – Off-set from true value
• Constant Bias
• Variable Bias – typically seen with electronic equipment, amount of Bias changes with setting
levels

6
Accuracy and Precision

7
MSA Uses
MSA can be used to:

Compare internal inspection standards with the standards of your


customer.

Highlight areas where calibration training is required.

Provide a method to evaluate inspector training effectiveness as well


as serves as an excellent training tool.

Provide a great way to:


•Compare existing measurement equipment.
•Qualify new inspection equipment.

8
Why MSA
Measurement System Analysis is important to:
• Study the % of variation in our process that is caused by our
measurement system.
• Compare measurements between operators.
• Compare measurements between two (or more) measurement
devices.
• Provide criteria to accept new measurement systems (consider new
equipment).
• Evaluate a suspect gage.
• Evaluate a gage before and after repair.
• Determine true process variation.
• Evaluate effectiveness of training program.
9
Appropriate Measures

Appropriate Measures are:

• Sufficient – available to be measured regularly

• Relevant –help to understand/isolate the problems

• Representative - of the process across shifts and people

• Contextual – collected with other relevant information that might explain


process variability.

10
Poor Measures
Poor Measures can result from:
• Poor or non-existent operational definitions
• Difficult measures
• Poor sampling
• Lack of understanding of the definitions
• Inaccurate, insufficient or non-calibrated measurement
devices

Measurement Error compromises decisions that affect:


– Customers
– Producers
– Suppliers

11
Examples of What to Measure
Examples of what and when to measure:
• Primary and secondary metrics
• Decision points in Process Maps
• Any and all gauges, measurement devices, instruments, etc
• “X’s” in the process
• Prior to Hypothesis Testing
• Prior to modeling
• Prior to planning designed experiments
• Before and after process changes
• To qualify operators

MSA is a Show Stopper!!! 12


Components of Variation
Whenever you measure anything, the variation that you observe can be
segmented into the following components…

Observed Variation

Unit-to-unit (true) Variation Measurement System Error

Precision Accuracy

Repeatability Reproducibility Stability Bias Linearity

All measurement systems have error. If you don’t know how much of the variation you
observe is contributed by your measurement system, you cannot make confident
decisions.

If you were one speeding ticket away from losing your license, how fast would
you be willing to drive in a school zone?
13
Precision
A precise metric is one that returns the same value of a given
attribute every time an estimate is made.

Precise data are independent of who estimates them or


when the estimate is made.

Precision can be partitioned into two components:


– Repeatability
– Reproducibility

Repeatability and Reproducibility = Gage R+R

14
Repeatability
Repeatability is the variation in measurements obtained with one
measurement instrument used several times by one appraiser while
measuring the identical characteristic on the same part.

Repeatability
For example:
– Manufacturing: One person measures the purity of multiple samples of
the same vial and gets different purity measures.
– Transactional: One person evaluates a contract multiple times (over a
period of time) and makes different determinations of errors.

15
Reproducibility
Reproducibility is the variation in the average of the measurements
made by different appraisers using the same measuring instrument
when measuring the identical characteristic on the same part.

Reproducibility

Y Operator A
Operator B

For example:
– Manufacturing: Different people perform purity test on samples from the
same vial and get different results.
– Transactional: Different people evaluate the same contract and make
different determinations.

16
Time Estimate Exercise
Exercise objective: Demonstrate how well
you can estimate a 10 second time interval.
1. Pair up with an associate.
2. One person will say start and stop to
indicate how long they think the 10 seconds
last. Do this 6 times.
3. The other person will have a watch with a
second hand to actually measure the
duration of the estimate. Record the value
where your partner can’t see it.
4. Switch tasks with partner and do it 6 times
also.
5. Record all estimates, what do you notice?
17
Accuracy
An accurate measurement is the difference between the observed average of the
measurement and a reference value.
– When a metric or measurement system consistently over or under estimates the
value of an attribute, it is said to be “inaccurate”
Accuracy can be assessed in several ways:
– Measurement of a known standard
– Comparison with another known measurement method
– Prediction of a theoretical value
What happens if we don’t have standards, comparisons or theories?

True
Average

Warning, do not assume your


Accuracy
metrology reference is gospel.

Measurement
18
Accuracy Against a Known Standard
In transactional processes, the measurement system can consist of a
database query.
– For example, you may be interested in measuring product returns where you
will want to analyze the details of the returns over some time period.
– The query will provide you all the transaction details.

However, before you invest a lot of time analyzing the data, you must
ensure the data has integrity.
– The analysis should include a comparison with known reference points.
– For the example of product returns, the transaction details should add up to
the same number that appears on financial reports, such as the income
statement.

19
Accuracy vs. Precision

ACCURATE PRECISE BOTH

+ =

Accuracy relates to how close the


average of the shots are to the
Master or bull's-eye.

Precision relates to the spread of


the shots or Variance.
NEITHER

20
Bias
Bias is defined as the deviation of the measured value from the
actual value.

Calibration procedures can minimize and control bias within


acceptable limits. Ideally, Bias can never be eliminated due to
material wear and tear!

Bias Bias

21
Stability
Stability of a gauge is defined as error (measured in terms of Standard
Deviation) as a function of time. Environmental conditions such as
cleanliness, noise, vibration, lighting, chemical, wear and tear or other
factors usually influence gauge instability. Ideally, gauges can be
maintained to give a high degree of Stability but can never be
eliminated unlike Reproducibility. Gage Stability studies would be the
first exercise past calibration procedures.
Control Charts are commonly used to track the Stability of a
measurement system over time.

Drift

Stability is Bias characterized as


a function of time!

22
Linearity

Linearity is defined as the difference in Bias values throughout the


measurement range in which the gauge is intended to be used. This
tells you how accurate your measurements are through the
expected range of the measurements. It answers the question,
"Does my gage have the same accuracy for all sizes of objects
being measured?"

Low Nominal High


Linearity = |Slope| * Process Variation +e

B i a s (y)
% Linearity = |Slope| * 100 0.00
*
-e
*
*
Reference Value (x)
y = a + b.x
y: Bias, x: Ref. Value
a: Slope, b: Intercept
23
Types of MSA’s

MSA’s fall into two categories:


– Attribute
– Variable

Attribute Variable
– Pass/Fail – Continuous scale
– Go/No Go – Discrete scale
– Document Preparation – Critical dimensions
– Surface imperfections – Pull strength
– Customer Service Response – Warp

Transactional projects typically have Attribute based measurement


systems.
Manufacturing projects generally use Variable studies more often, but do
use Attribute studies to a lesser degree.

24
Variable MSA’s
Calculates a column of variance components (VarComp) which are used to calculate % Gage
R&R using the ANOVA Method.

Measured Value True Value

Estimates for a Gage R&R study are obtained by calculating the variance components for
each term and for error. Repeatability, Operator and Operator*Part components are summed
to obtain a total Variability due to the measuring system.
We use variance components to assess the Variation contributed by each source of
measurement error relative to the total Variation.

25
Data Sheet

Contribution of Variation to the total


Variation of the study. Use % Study Var when you are interested in
comparing the measurement system Variation to the
total Variation.
% Study Var is calculated by dividing each value in
Study Var by Total Variation and Multiplying by 100.
% Contribution, based on variance
components, is calculated by dividing each Study Var is calculated as 5.15 times the Standard
value in VarComp by the Total Variation then Deviation for each source.
multiplying the result by 100. (5.15 is used because when data are normally
distributed, 99% of the data fall within 5.15 Standard
Deviations.)

26
Data Sheet

Report:

When the process tolerance is entered in the


system, SigmaXL® calculates % Tolerance
which compares measurements system
Variation to customer specification. This allows
us to determine the proportion of the process
tolerance that is used by the Variation in the
measurement system.

0.186980
Distinct Categories  1.41
0.031861517
 5.8685 1.41
 8 (Rounded Down )

27
Number of Distinct Categories

Unacceptable for estimating process


parameters and indices
Only indicates whether the process is
producing conforming or
nonconforming parts

1 Data Category

Generally unacceptable for estimating


process parameters and indices
Only provides coarse estimates

2 - 4 Categories

Recommended
5 or more Categories

28
AIAG Standards for Gage Acceptance

Here are the Automotive Industry Action Group’s


definitions for Gage acceptance.

% Tolerance
or % Contribution System is…
% Study Variance

10% or less 1% or less Ideal

10% - 20% 1% - 4% Acceptable

20% - 30% 5% - 9% Marginal

30% or greater 10% or greater Poor

29
Graphic Output

Components of Variation

Report breaks down the variation in the measurement system into specific sources. The bar
chart shown was created using Excel’s Clustered Column Bar Chart to graphically display the
Components of Variation. Each cluster of bars represents a source of variation.

In a good measurement system, the largest component of Variation is Part-to-Part variation. If


instead you have large amounts of variation attributed to Gage R&R, then corrective action is
needed.

30
Graphic Output

R Chart and Xbar Chart by Operator. The R chart consists of the following:

- The plotted points are the difference between the largest and smallest measurements on each part for each operator.
If the measurements are the same then the range = 0.
- The Center Line, is the grand average for the process.
- The Control Limits represent the amount of variation expected for the subgroup ranges. These limits are calculated
using the variation within subgroups.

If any of the points on the graph go above the upper Control Limit (UCL), then that operator is having problems
consistently measuring parts. The Upper Control Limit value takes into account the number of measurements by an
operator on a part and the variability between parts. If the operators are measuring consistently, then these ranges
should be small relative to the data and the points should stay in control.

31
Graphic Output

SigmaXL® provides an R Chart and Xbar Chart by Operator. The Xbar Chart compares the part-to-part variation to
repeatability. The Xbar chart consists of the following:

- The plotted points are the average measurement on each part for each operator.
- The Center Line is the overall average for all part measurements by all operators.
- The Control Limits (UCL and LCL) are based on the variability between parts and the number of measurements in each
average.

Because the parts chosen for a Gage R&R study should represent the entire range of possible parts, this graph should
ideally show lack-of-control. Lack-of-control exists when many points are above the Upper Control Limit and/or below the
Lower Control Limit.

In this case there are several points out of control which indicates the measurement system is adequate.

32
Gage R&R Multi-Vari Output

The Multi-Vari Charts show each Part as a separate graph. Each Operator’s response readings are denoted as a vertical
line with the top tick corresponding to the Maximum value, bottom tick is the Minimum, and the middle tick is the
Mean. The horizontal line across each graph is the overall average for each part.

Ideally the connected means red line should be horizontal (i.e., small reproducibility) and the vertical lines should be
short (small repeatability).
33
Graphic Output

Pattern Means…
Two-way ANOVA tool creates an interaction chart that
shows the average measurements taken by each
Lines are virtually identical Operators are measuring the
operator on each part in the study, arranged by part.
parts the same
Each line connects the averages for a single operator.
One line is consistently That operator is measuring
higher or lower than the parts consistently higher or Ideally, the lines will follow the same pattern and the
others lower than the others
part averages will vary enough that differences between
Lines are not parallel or they The operators ability to parts are clear.
cross measure a part depends on
which part is being
measured (an interaction
between operator and part)

34

34
Graphic Output

The “By Part” Multi-Vari Chart allows us to analyze all of the measurements taken in the study arranged by
part. The measurements are represented by dots; the means by the middle bar. The red line connects the
average measurements for each part.

Ideally,
 Multiple measurements for each individual part have little variation (the dots for one part will be close
together)
• Averages will vary enough that differences between parts are clear
35
Graphic Output

The “By Operator” Multi-Vari Chart is created by modifying the X’s from parts to operator. This helps us
determine whether the variability in measurements are consistent across operators.

The by operator graph shows all the study measurements arranged by operator. Dots represent the
measurements; the middle bars represent the means. The red line connects the average measurements
for each operator.

If the red line is … Then…


Parallel to the x-axis The operators are measuring the parts similarly
Not parallel to the x-axis The operators are measuring the parts differently

You can also assess whether the overall Variability in part measurement is the same using this graph. Is
the spread in the measurements similar? Or is one operator more Variable than the others?

36
Practical Conclusions
For this example, the measuring system contributes little to the overall
Variation, as confirmed by both the Gage R&R table and graphs.
The Variation due to the measurement system, as a percent of study (Total)
Variation is causing 16.80% of the Variation seen in the process.
By AIAG Standards this gage should be used. By all standards, the
data being produced by this gage is acceptable, and valid for analysis.

% Tolerance
or % Contribution System is…
% Study Variance

10% or less 1% or less Ideal

10% - 20% 1% - 4% Acceptable

20% - 30% 5% - 9% Marginal

30% or greater 10% or greater Poor

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70