MSA Example:

Attribute or
Categorical Data

MSA Operational Definitions

Accuracy: Overall agreement of the measured value with the true
value (which may be an expert value). Bias plus precision.
Attribute Data: Discrete qualitative data.
Attribute Measurement System: Compares parts to a specific set
of criteria and accepts the item if the criteria are satisfied.
Bias: A systematic difference from the true value. Revealed in the
differences in averages from the true value.
Precision: Variation in the measurement process.
R&R: Repeatability and Reproducibility. Two elements of precision.
Repeatability: The variation observed when the same operator
measures the same item repeatedly with the same device.
Reproducibility: The variation observed when different operators
measure the same parts using the same device, sometimes it can
be the same operator using different devices.

All Rights Reserved, Juran Institute, Inc.

The Fundamental MSA Question

Is the variation (spread) of the measurement system

too large to study the current level of process
variation?

Product Variability
Process Variability

Variation
in the
measurement
process

(Observed Variability)
Total Variability

All Rights Reserved, Juran Institute, Inc.

Bias

True
Value or
Standard

Possible Causes of Bias

Sensor not properly
calibrated

Bias

Improper use of sensor

Unclear procedures

Observed
Average

Human limitations

All Rights Reserved, Juran Institute, Inc.

Repeatability

Possible Causes of Poor

Repeatability
Equipment
Gage instrument needs
maintenance
The gage needs to be more
rigid

Repeatability

People
Environmental conditions
(lighting, noise)
Physical conditions (eyesight)

All Rights Reserved, Juran Institute, Inc.

Reproducibility
Mean of
the measurements
of Operator B
Mean of
the measurements
of Operator A

Possible Causes of Poor

Reproducibility
Measurement procedure is not
clear
Operator is not properly trained
in using and reading gage

Reproducibility

Operational Definitions not

established

All Rights Reserved, Juran Institute, Inc.

Attribute Measurement Systems Study

Discrete qualitative data
Go/no-go basis; or limited data categories
Compares parts to specific criteria for accept/not accept or
to be placed in category
Must screen for effectiveness to discern good parts from
bad
At least two appraisers and two trials each
If available, have Quality Master rate parts first

All Rights Reserved, Juran Institute, Inc.

Attribute MSA Study

Attribute MSA Example

Appraiser A

Master
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

G
G
G
NG
G
G
NG
NG
G
G
G
G
G
G
NG
G
G
G
G
NG

Appraiser B

G
G
NG
NG
G
G
NG
NG
G
G
G
G
G
G
G
G
G
G
G
G

G
G
G
NG
G
G
NG
NG
G
G
G
G
NG
G
G
G
G
G
G
G

G
G
G
NG
G
NG
G
G
G
G
G
G
G
G
G
G
G
G
G
G

G
G
G
NG
G
G
NG
G
G
G
G
G
G
G
G
G
G
G
G
G

G = Good
NG = Not Good

All Rights Reserved, Juran Institute, Inc.

Challenges of Continuous Process MSA

MSA study is an experiment

Requires two or more trials for calculating Repeatability
Needs a way to present the inspection units to the appraiser
multiple times
Is not possible within the continuous process

All Rights Reserved, Juran Institute, Inc.

Case Example: Visual Inspection of Glass

Packers

Catwalk
Cutter

Inspector

Glass

All Rights Reserved, Juran Institute, Inc.

Case Example: Challenges to Overcome

Bias to the standard could be evaluated on-line.

Repeatability and Reproducibility (R & R) could not be
evaluated on-line.
A method had to be devised to allow the inspectors to view
the same pieces of glass repeatedly.
The solution was an off-line conveyor which simulated the
on-line condition as closely as possible.

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Method Employed

20 pieces of glass from the process that included both good

and bad samples were selected.
A team of people well versed in the quality standard
classified each piece of glass as either pass or fail.
All regular inspectors independently evaluated each piece
twice (in random order).
The inspectors used a log sheet to record the data.
Minitab was used to analyze the data.

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Data

Excerpt of
full data
for 20
inspectors

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Results

Date of study:
Reported by:
Name of product:
Misc:

Assessment Agreement

Within Appraisers

Appraiser vs Standard
100

95.0% CI
Percent

90

90

80

80
Percent

Percent

100

70
60

95.0% CI
Percent

70
60

50

50

40

40

Appraiser

9/13 and 9/19/07

Bill Sunderland and Bob Panatieri
Solar
Mainline Marking

Appraiser

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Results (continued)

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Results (continued)

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Results (continued)

All Rights Reserved, Juran Institute, Inc.

Case Example: Attribute MSA Study Conclusions

What could have caused the poor agreement?

What was done to improve consistency?

All Rights Reserved, Juran Institute, Inc.