:~

":'f'~

,
"

...... .

..............

From Dr. Muhammad Asim - TOM

;"

~/'/
"

'';;'

There are Sjx Basjc TechniQues for Presenting Performance Measures..:.

1. Cost of Quality
2. Time Series Graph
3. Control Chart
4. Process Capability
5. Taguchi's Loss Function
6. Malcolm Baldrige National Quality Award
,d

:..
~.-

_T~

.but\on

l'

Pe'yfoY monee ~g~~~e. P~9-enb.ti o.!L_ ..___

"

16

g?

'

12
8

0>

Product A

'E

8c 4

o
Z

ttela\

----.

---,---.-,..--,--.--.,---...,-..---,

2
3
Quarters

'le pre
the o'

Figure 7-1 Time Series Graph for Percent Nonconforming

or e
is e(
nple P
.1t

'j

0.08

2..
~

0.06

s freq'
)'/ usir

0.04

al coni

.r '}

0.Q2

LCL

f~ ,'~

>

t. Tal

) I 1-'
.f" . .,'

2. CJ

"t

3. C:
4. C

5
Subgroup number

figure 7-2 Control Chart for Percent Nonconforming

wh~

y.

LSL

USL

('f

~A

......

:2

.3
o

............ ,
1:~A

'""'!""":

.,.

1:

Figure 7-3 Taguchi's Quadratic loss Function for Nominal-tile-Best

\..

".-

:.t

U~r

speciticmion
(optional location) - - - - - - - ,

Pra.:es.< out of control

veL

I ---

X;-,--I- - - : : -

"~~
LCL
(a) Seven consecutive
poinrs above
or below

~:t=.L

u~c ~==;zs=:

LCL

LCL

~~--~--

(b) Six consecutive

points increasing or
decreasing

Xo

\:..---

.I

Distribwio-

__ ~:fT~yem:~

"

~-----~-

(c) TWo CQnsecutiye

Control

po,nts in outer
quarter

3(7,(

I
!'roc.55

I,
I '

limits

capability
3(7

_l

Figure 18-22 Some Unnatural Runs-Process Out of Control

Di:otribulion of
individuil! values

-t------1~-1l

There are some common q~o ask when investigating an out-of-control process:

Lower specirication
(optional location) - - - - - - - - - - - - - - - - - - - LSL

I, Are there differences in the measurement accuracy of the instruments used?

Figure 1823 Relationship of Limits, Specifications, and Distributions

2_ Are there differences in the methods used by different operators?

3. Is the process affected by the environment? If so, have there been any changes? ,
4. Is the process affected by too! wear?

The true process capability cannot be determined until the X and R charts have
achieved the optimal quality improvement without a substantial investment for new
equipment or equipment modification, When the process is in statistical control. process
capability is equal to 60', where 0' = Rold 2 and d1 is a factor from Appendix Table A. In
the example problem. it is

5. Were any untrained workers involved in the process?

6. Has there been any change in the source of the raw materials'?
7. Is the process affected by operator fatigue?
8. Has there been any change in maintenance procedures?
9. Is the equipment being adjusted too frequently?

60' = 6(Rold2l = 6(0.07912.059) = 0.230

10. Did samples come from different shifts. operators. or mach,ines?

It is frequently necessary to obtain the process capability by a quick method rather

than by using the X and R charts. This method assumes the process is stable or in sta
tistical control, which mayor may not be the ca.e. The procedure is as follows:

It is advisable to develop a checklist for each process using these common questions
as a guide.

1. Take 25 subgroups of size 4, for a total of 100 measurements.

V
--------'

Process Capability

--

2. Calculate the range. R, for each subgroup,

3. Calculate the aVerage range:

Control limits are established as a function of the avera es-in other words. control lim
i;;efor average;;. Sp,!;CI Icauons. on the ot er and. are the permissible variation in the
size of the part and are, therefore. for individual values. The specificallon or tolerance
limits are established by design engineers to meet a particular function. Figure 18-23
shows that the location of the specifications is optional and is not related to any of the
other features in the figure. The control limits, process spread (process capability), dis
tribution of averages, and distribution of individual values are interdependent. They are
determined by the process, whereas the speCifications have an optional location. Control
charts cannot determine if the process is meeting specifications.

R=

2:

4. Calculate the estimate of the popUlation standard deviation:

0'

= Rid,

where d2 is obtained from Appendix Table A and is 2.059 for n

4.

5. The process capability will equal 60'.

J:::=

nD.
of

)'O\J~

l.,

,;5

USl

LSL

20t

Tolerance
Capabiliry

I I
101

STATISTICAL PROCESS CO

---01 Be-t-)

_ Min {(USL

A Cpk value of 1.00 is the de facto standard. with some: orgunizutiolllllsln,ll value of
1.33. Figure 18-25 illustrates Cp and Cpk values for processes thlllnfe Cl!IItCfCI.! lind 0110
off center by Icr.

Case 1 Cp = (USL LSL)16 = 8,,160- a 1.33

o f.-..---t
13.09

13.15

13.21

Hole location

13.27

~mm

Figure 1824 Relationship of Process Capability to Tolerance

LSL

CC]'
~~,
Xo

Cp =

Remember that !his technique does not give the true process capability and should be
used only if circumstances require its use. Also. more than 25 subgroups can be used to
improve accuracy.
The relationship of process capability and specifications is shown in Figure 18-24.
Tolerance is the difference between the upper specification limit (USL) and the lower
specification limit (LSL). Process capability and the tolerance are combined to form a
capability index. defined as
Cp

where

Cf' >1
,(p t.. I

X) or (X - LS,:ll
30

Cr, -

USL - LSL
6cr

,,~

LSL

Xu

1.33

Cpt'" 1.00

'"\1

Case 1I Cp = (USl LSL)/6 = 60-/60- '" 1.00

I-

L/\J
Xo

LSl

USL

Cp '" 1.33

Cp' = 1.33

USL

I-

ro

=,

_
Xn

tSL

Cp '" 1.00
Cp' = 1.00

USL - LSL upper specification - lower specification, or tolerance

Cp = capability index
60' = process capability

USl

~ro\1

USL

Cp '" 1.00
Cpi< '" 0.67

60'-1
~

Ca.e III Cp = (USL - LSL)l6 = 4(1100 = 0.67

If the capability index is greater than 1.00. the process is capable of meeting the spec
ifications; if the index is less than 1.00. the process is not capable of meeting the speci
fications. Because processes are continually shifting back and forth. a Cp value of 1.33
has become a de facto standard, and some organizations are using a 2.00 value. Using
the capability index concept. we can measure quality, provided the process is centered.
The larger the capability index. the better the quality. V(.e should stri ve to make the ca
p.:;?i1ity index as large as possible. This result is accomplished by b..aving realistic spec
ificatiws and continual striving to improve the process capability.
The ;;;;'pability index. does not measure process performance in terms of the nominal
or target value. This measure is accomplished using Cpk' which is

I~

LSI:.

Xo

USL

~6cr

~L/)J

LSL

Cp =0.67

Cpt :0.67

Figure 18-25

Xu

USL

Cp =0.67

C". = 0.33

e" and e... Values for Three Different Situations

~~-"'--J
,.~

a,.

.
Comments concerning CI' and Cpk are as follows:
I. The Cp value does not change as the process center changes.
'2. Cp = Cpk when the process is centered.

3. Cpk is always equal to or less than Cp

4. A Cpk value greater than 1.00 indicates the process conforms t<.} specifications.
5. A Cpk value less than 1.00 indicates that the process does not conform.to speci
fications.
6. A Cp value less than 1.00 indicates that the process is not capable.
7. A Cpk value of zero indicates the average is equal to one of the specification
limits.
8. A negative Cpk value indicates that the average is outside the specifications.
Quality professionals will use these eight items to improve the process. For example,
if a Cp value is less than one, then corrective action must occur. Initially 100% inspec
tion is necessary to eliminate noncomformities. One solution would be to increase the
tolerance of the specifications. Another would be to work on the process to reduce the
standard deviation or variability.

fr:;

ch treatment condition as depicted in

Table 19-9. setiua full factorial de~i and evaluate the si~'ficance and fac
tor effects of thi study for a levels of .10 and 0.05. Based on ur results, com
ment on reduc' g this study to a fra ional factorial experim t.

Replicate

<}

Taguchi's Quality
Engineering

16. Reduce the full fact rial experiment in Exer se 14 to a three-factor f ctional fac
lorial experiment y using your experien and personal judgmen For instance.
eliminate the fa r-factor interaction an some of the two- and ee-factor inter
.ct a table similar to T Ie 19-12.
i;

17. Working' dividually or in a i.1ea design a full facto~'

al xperiment and deter
and one of your own
mine re
nses for one or mo of the following ite
choic . Your experiments .sho d have at least three fa ors at two levels each and
two plicates.
(a) Accuracy of weath forecaster

(examples of fact rs are time of day, loe

(b) Quality of pizz

(examples of actors are type of es

forth)

(c) Computer ouse performance

(exampl s of factors are ergon ics. sensitivity, price, and so forth)
(d) Televi on remote control ch nel switching

(examples of factors are Ii

of day, gender. age. and so forth)

Introduction
Most of the body of knowledge associated with the quality sciences was developed in
the United Kingdom as design of experiments and in the United States as statistical qual
ity control. More recently, Dr. Genichi Taguchi, a mechanical engineer who has won
four Deming Awards, has added to this Jx>dy of knowledge_ In particular, he introduced
the loss function concept. which combines cost, target, and variation into one metric
with specifications being of secondary importance. Furthermore. he developed the con
cept of rob\1stness. which means that noise factors are taken into account to ensure that
the system functions correctly. Noise factors are uncontrollable variables that can cause
significant variability in the process or the product.
f.;.'

Loss Function
Taguchi has defined quality as the loss imparted to society from the time a product is
shipped. Societal losses include failure to meet customer requirements, failure to meet
ideal performance. and harn1ful side effepts. Many practitioners have included the losses
due to produCtion, such as raw material. energy. and labor consumed on unusable prod
uCts or toxic by-products.
1be loss-to-socieiy concept can be illustrated by an example associated with the pro
duction of large vinyl covers to protect materials from the elements. Figure 20-1 shows
three stages in the evolution of vinyl thickness. At (I), the process is just capable of

TAGUCHI'S QUALITY ENGINEERING

------.::..,
Tow loss

Producer's loss

+ Customer's loss

lSL
r----

14

-- luny -llpI"

_.~ ,sOnY-USA

I
LSL

USL

Figure 20-1 loss to Society

Reproduced, with permission, from Taguchi Methods: Introduction to Quality Engineering (Allen Park, Mich.: American Sup
plier Institute, inc., 1991).

distribution was normal and centered .on the target. The distributien of the Sony-USA
was unifQrm between the specifications with no values .outside specificatiQns. It was
clear that custemers perceived quality as meeting the target (Japan) rather than just meet
ing the specificatiens (USA). Ferd MQtor Company had a similar experience with
transmissiQns.
,
Out of specification is the commQn measure of quality less. Although this concept
may be appropriate fer accQunting. it is a poor concept fer all other areas. It implies that
all products that meet specifications are good, whereas thQse that de nQt are bad. From
the customer's PQint .of view, the prQduct that barely meets specificatiQn is as geed (.or
bad) as t.. .le product that is barely .out of specificatiQn. It appears the wrQng measuring
system is being used. The less function CQrrects fQr the deficiency described abQve by
cembining CQst, target. and variatiQn into .one metric. /

meeting the specifications (USL and LSL); however, it is on the target tau, ... 1 After con
siderable effQrt, the productiQn p~ess was imprQved by reducing the variability about
the target, as shown at (2). In an effort to reduce its production costs, the organization
decided to shift the target closer to the LSL, as shown at (3). This action resulted in a
substantial improvement by lowering the cost to the organization; however, the vinyl
covers were not as strong as before. When farmers used the covers to protect wheat from
the elements, they tore and a substantial IQSS occurred tQ the farmers. In additiQn, the
CQst .of wheat increased as a result of supply-and-demand factors, thereby causing an in
crease in wheat prices and a further loss to society. The company's reputation suffered,
which created a JQSS of market share with its unfavorable loss aspects.
Assuming the target is correct, Josses of concern are those caused by a product's crit
ical performance characteristics deviating from the target. The importance .of CQncen
trating .on "hitting the target" is documented by Sony. In spite .of the fact that the design
and specificatiQns were identical, U.S. customers preferred the c.olor density of shipped
TV sets produced by Seny-Japan .over those preduced by SQny-USA. Investigatien .of
this situ'ation revealed that the frequency distributions were' markedly different. as
shewn in Figure 20-2. Even theugh SQny-Japan h~d 0.3% .outside the specificatiQns, the

Taguchi uses the symooi

Color
density

T
5
Figure 20-2 Distribution of Color Density 10r Sony-USA and Sony-Japan
Source: The Asahi. April 17, 1979.

Vinyl thickness

.----~

Nominal-the-Best

-----.-----. - - - .---.- - . " AlthQugh Taguchi developed mQre than 68 loss functiens, many situations are ap
prQximated by the quadratic functien which is called the neininal-the-best type.
Figure 20-3(a) shQWS the step functiQn that describes theSQny-USA situatiQn. When
the value fQr the performance characteristic, y, is within specificatiQns the Joss is $0,
and when it is outside the specificatiQns the loss is $A. The quadratic function is'
shQwn at 20-3(b) and describes the Sony-Japan situatiQn. In this situatIOn lQSS occurs
as soon as the perfermance characteristic, y, departs frQm the target, ...

mfor the target.

TAGUCHI'S QUALITY ENGINEERiNG

EXAMPLE PROBLEM

USL

LSL

If Ihespecifications are 10:!:: 3 for a particular quality ~huncteri~tlc anti Iht Jvtru._ r,.
pair cost is $230, determine the loss function. Determine the lou At.V. 12.

=230/3 2 =25.6

k
Thus, L = 25.6 (y - 10)2 and at y = 12,

----+--.---+ Y

't

't-A

L = 25.6(y - 10)2
=25.6(12
$102.40

t+A

Perfomlance characteristic
(a) Step function (Sony - USA)

~ A ~.

LSL

USL

characteristic is static. In reality. on~

ng due to noise, and the loss function must re
flect the ariation of many pieces rer than just one piece. Noise faCTors are classi
fied as xternal and internal, withfoternal being further classified as unit-to-unit a:1d
dele .oration/
refrig,erator temperaturelonttol will serve as an example to help cI:uify the noise
number of times the
concept. External noise is due to the actions of the user, such as
door is opened and c . sed. amount of food inside, the initial t perature. and so forJ!.
Unit-lo.-unit intern noise is due to variation during produ ton such as seal tightness.
controlsensor v .ations and so forth. Although this type noise is inevitable, every ef
fort should be ade to keep it to a minimum. Noise e to deterioration is caused by
leaIiage of re gerant, mechanical wear of compres r parts, and so forth. This ty~ of
noise is pri
'lya function of the design. Noise f tors cause deviation fr()m the target,
w~ich ca es a loss to society.
'Figu 20-4 shows the nominal-!he-best loss function with the distribution of the
noise factors. An equation can be derived by summing the individual loss ,'alues and di
/tiding by their number to give

'\

3
---t----.,
't

't-A

't+A

Perfonnance charncterislic
(b) Quadratic function (Sony - Japan)

Figure 20-3 St.p and Quadratic Loss Functions

. The quadratic loss function is described by the equation

L=kfy

-li

where L cost incurred a~ quality deviates from the target

y ::= performance characteristic
1: target

k quality loss coeffi~ient

L ::: Ha" + (y - T/l

=
=

where I. = the average 0 xpected loss.

Because the popul on standard deviation, cr, will rar
be known, the sample stan
dard de'fiation, s, w need to be substituted. This acti will make the value somewhat
larger; however, e average loss is a very conserva . e value,
The loss can
lowered by first reducing the v iation, cr, and then adjusting the 3'
erage, y, to b' g it on target. 1:. The loss function 'speaks the language of things," which

The loss cqefficient is determineq. by setting A ::= fy 1:). the deviation from the tar
get. When A is at the USL or LSL, the loss to the customer of repairing or discarding the
product is SA.
.

~=A/(y -

=AltJ,?

.1

PERFORMANCE MEASURES
6. Should operating personnel be responsible for inspection. thus decreasing costs?
7. Is appraisal being used as a substitute for prevention?

6..

Malcolm Baldrige National Quality Award'

-.~- . . ." , . " , '~""'"

" . . . . . . . - - _

'W

'The Malcolm Baldrige National Quality Award (MBNQA) is an annulIl award to recolnilC!
U.S. organizations for performance excellence. It was created by Puhlic Law 100-107 (In
August 20. 1987. 'The award promotes: understanding of the requirements for performnn.:e
excellence and competitiveness improvement. sharing of information on successful ~r.
fonnance strategies, and the benefits deriVed from using these strategies. 'There are five cat.
egories: manufacturing, service, small business, health care. and education. 'Three awards
may be given each year in each category. Competition for the awards is intense, and intermany organizations who are not interested in the award are, nevertheless, using the
categories as a technique to measure their TQM effort on an annual assessment basis.

~~()g r~f!lJ!!lpl_e..m._e._n.t_a__t_io_n_ _____._._..~...

'The first step in the implementation of a quality cost program is to determine if the pro
gram can be beneficial to the organization. Small organizations with limited resources
are cautioned that implementation time and expense are extensive. It might be better to
concentrate on one metric, such as scrap or easily measured external failures. A simple
system using one metric will provide very conservative numbers.
For larger organizations, a review and analysis of the cost data, when made in suffi
cient detail, will usually indicate that the costs are quite large. Before proceeding, it is
necessary to detemline that top management is receptive to new ideas.
A presentation is prepared and given to top management to con vince them of the need
for the program. 'The presentation should describe the results to expect the implemen
tation plan, and the return on in vestment. Emphasis is placed on the organization-wide
aspects of quality costs and n()t just the operational aspects. It should be pointed out to
management that they will need to take an active role in the program.
'The program will have a greater chance of success if a single product line or department
is used on a trial basis. Selection of the trial area should be strongly influenced by its p0.
tential to produce quick and significant results. A full-time leader who has the confidence
of senior management will be needed. 'The steps of the trial program are measurement of
quality costs, determination of the appropriate indexes and ratios, establishment of trend
analysis charts, identification of improvement opportunities, assignment of project teams,
and reporting of results.
All personnel who will be involved with the system should be educated and trained.
This initial training will be limited to those involved with the trial program and key
members of each functional area. Education will concentrate on the purpose of quality
costs: identification of opportunities for quality improvement, justification of corrective
action, and measurement of results.
Concurrent with me progress of the trial program, basic accounting procedures are
revised to accommodate the quality cost system. Each cost element is described. as well
as how the data will be collected or estimated. Also, the methods for treating fringe ben
efits, overhead, and other accounting adjustments are determined. To ensure the integrity
and acceptance of the data, the comptroller approves the procedures.
After the trial program is complete, the quality cost program is ex.panded to include
the entire organization. Because of better reporting, quality costs may actually increase
while the system is in the development period. Data are collected onto appropriate
spreadsheets by department. product, project, and so forth. Trend and Pareto analyses
are conducted to determine opportunities for improvement. Projects are assigned, and an
ongoing system is in operation.

Criteria for Performance Excellence

--------~~---------.~-"----

..

-----.~ ""--".

The criteria for performance excellence are the basis for malejng awards and for giving
feedback to applicants. In addition, they (I) help improve performance practices and ca.
pabilities, (2) facilitate communication and sharing of best practices information among
U.S. organizations of all types, and (3) serve as a Working tool for understanding and
managing performance, planning. training. and assessment. The results-oriented goals
are designed to deliver ever-improving value to customers, resulting in marketplace sue
c~ss, and to improve overall organization performance and capability. 'The criteria are
derived from the set of core values and concepts described in Chapter 2, Leadership.
'The core values and concepts are embodied in seven categories, as shown in Figure 7-9.
'The seven categories shown in the figure are subdivided into 20 ex.amination item.;.
Each examination item consists of sets of areas to address. Information is submitted by
applicants in response to specific requirements of these areas.

Characteristics of the Criteria

I. 'The criteria are directed toward results. 'They focus principally on seven key area;
of business performance. Results are 1 composite of
Customer satisfactiollfretention.
Market share, new market development.
Product and service quality.
Productivity, operational effectiveness, and responsiveness.
Human resource performance/development.
Supplier performance/development.
Public responsibility/corporate citizenship.
1

, . - . , - - -

Adapted from U.S. Dep,artmentofComrnerce, Malcolm Baldrige National Quality Awanll998 Criteria. 1997.

'1
_ _ _'he:

PERFORMANCE MEASURES
Baldrige Criteria for Perfomlanc. Exc.Hence Framework
A System, Perspecti..

oriented learning takes place

c1es of learning.

feedback between processes and results through cy

The learning cycles have four clearly defined stages. similar to Shewharfs P-D-S-A.

~~

~_

Leade",hip
'---_ _ _

:i;

~~

~ .;:~?

:t

"-.~

~.

management

Information and analysis

BUSIness

\/

",suits

Figure 7-9 Award Criteria Framework

I. Planning, including design of processes, selection of measures. and deploy

ment of requirements.
2. Execution of plans.
3. Assessment of progress, taking into account internal and external results.
4, Revision of plans based upon assessment findings, learning, new inputs, and
new requirements.
,.
4. The criteria support goar-based diagnosis. The criteria and the scoring guidelines
make up a two-part diagnostic (assessment) system. The criteria are a set of 20
perfonnance-oriented requirements. The scoring guidelines indicate the assessment di
mensions-approach, deployment, and results--and the key factors used to assess
against each dimension. An assessment thus provides a profile of strengths and oppor
tunities for improvement relative to the 20 basic requirements. In this way, asse,sment
leads to actions, which contribute to the results composite previously described. This di
agnostic assessment is thus a useful management tool that goes beyond most penorm
ance reviews and is applicable to a wide range of strategies and management systems.

Criteria for Performance Excellence

-,.----------~-

Improvements in these seven areas contribute significantly to orgamzauon perlorm

ance, including financial penormance. The results also
the importance of sup
pliers, the needs of communities, and the needs of the nation.
2. The criteria are nonprescriptive and adaptable because
The focus is on results, not on procedures or tools. Organizations are encouraged
to develop and demonstrate creative, adaptive, and flexible approaches. for
meeting basic requirements. Nonprescriptive requirements are intended to fos
ter incremental and major "breakthrough" improvement.
Selection of tools, techniques. and systems usually depends upon factors such
as business type and size. the organization's stage of development, and em
ployee capabilities and responsibilities.
The focus is on common requirements within an organization rather than on spe
cific procedures, which fosters better understanding, communication, sharing
and alignment while supporting diversity and creativity in approaches.
3. The criteria support a systems approach to maintaining organization-wide goal
alignment. A systems approach to goal alignment, particularly when stIategy and goals
change over time, requires dynamic linkages among criteria items. In the criteria, action-

to

--..,..----------_._---

Table 7-1 shows the seven award categories and the 20 items with their point values. It
is important to note that almost half of the total score is based on results.
The Leadership category examines the company's leadership system and senior lead
ers' personal leadership. It examines how senior leaders and the leadership system ad
dress values. company directions, penormance expectations, a focus on customers and
other stakeholders. learning, and innovation. Also examined is how the company ad
dresses its societal responsibilities and provides support to key communities.
The Strategic Planning c2.tegory examines how the company sets strategic directions
and how it develops the critical strategies and activn plans to support the directions. Also
examined are how plans ate deployed and how penormance is tracked.
The Clistomer and Market Focus category examines how the company determines
requirements. expectations, and preferences of customers and markets. Also examined i5
how the company builds relationships with customers and determines their satisfaction.
TIle Information and Analysis category examines the selection. management, and
effectiveness of lise of information and data to support key company processes and ac
tion plans, and the company's penormance management system.
The Human Resource Focus category examines how the company enables employ
ees to develop and utilize their full potential, aligned with the company's objectives.
Also examined are the company's efforts to build and maintain a work environment and
work climate conducive to penonnance excellence. full partiCipation, and personal and
organizational growth.

PERFORMANCE MEASURES
The Process Management category examines the key aspects of process manage
ment, including customer-focused design, product and service delivery, support, and
supplier and partnering processes involving all work units. The category examines how
key processes are designed, implemented. managed, and improved to achieve better
performance.
The Business Results category examines the company's perfoImance and im
provement in key business areas: customer satisfaction, financial and marketplace per
formance, human resource results, supplier and partner performance. and operational
performance_ Also examined are performance levels relative to competitors.

TABLE 7-.1

Award Categories and Item Ustlng with Point Values

110

Leadership
............................80

1.1 Leadership System

1.2 Company Responsibility and Citizenship ....................30

Strategic Planning
2.1 Strategy Development Process .......................... .40

80

2.2 Company Strategy ...... .............. . ........... .40

3
Customer and Market focus
3.1 Customer and Market Knowledge. ............ . ....... .40

80

Scoring System
The system for scoring applicant responses is based on three evaluation dimensions: (I)
approach, (2) deployment, and (3) results.
Approach refers to how the applicant addresses the item requirements. The factors
used to evaluate approaches include:

3.2 Customer Satisfaction and Relationship Enhancement. . . . .. ., .40

4
Information and Analysis
4.1 Selection and Use of Information and Data .................2 5

80

4.2 Selection and Use of Comparative Information and Data ........15

Appropriateness of the methods to the requirements.

4.3 Analysis and Review of Company Performance ...............40

Effective use of the methods.

5
Human Resource Focus
5.1 Work Systems. . . . . . .

100

Degree to which the approach is systematic, integrated. and consistently applied;

embodies effective evaluationlimprovementlleaming cycles; and is based on reli
able information and data.

. ............................ .40

5.2 employee Education, Tra:ning, and Development

.....30

Evidence of innovative and/or significant and effectil'e adaptations of approaches

use(\ in other applications or types of businesses.

5.3 Empioyee Well-Being and Satisfaction .................... .30

Process Management
6.1 Management of Product and Service Processes ...... , .......60

6.2

Management of Support Processes .......................20

6.3

Managerrie"lt of Supplier and Partnering Processes. .., ......20

100

Deployment refers to the extent to which the applicant's approach is applied to all re
quirements of the item. The factors used to evaluate deployment include:
Use of the approach in addressing business and item requirements.

450

Use of the approach by all appropriate work units.

7 Business Results
7.1 Customer Satisfaction Results ..........................12 5

Results refer to outcomes in achieving the purposes given in the item. The factors
used to evaluate results include:

7.2 Financial and Market Results ........................... 125

7.3 Human Resource Results ...... . ......................50

7.4 Supplier and Partner Results

7.5 Company-Spedfic Results
TOTAL POINTS

Current performance.

. . .. . ...............25

. . . . . . . . . . . . . . . . . . . . . . 125

Performance relative to appropriate comparisons and/or benchmarks .

Rate, breadth, and importance of performance improvements.

1000

Demonstration of sustained improvement and/or sustained high-level performance.

Linkage of results measures to key performance measures identified in the busi
ness overview and in approach/deployment items.

1l

PERFORMANCE MEASURES

Use of the scoring system requires considerable training. Examiners receive more
than three days of training. with most of the time devoted to the scoring s~ stem. A
simpler system is given by the sample self-evaluation. which is shown in the next
section. This approach would be more appropriate for small and medium-sized
organizations.

_ _ The information in the plans is adequately disseminated within the

organization.
_ _ We evaluate and improve our planning process.
_ _ We constantly determine future requirements of the customer.

3. Cilstomer and Market Focus

~~rt1pl!_~el!:Evaluat~
The following self-evaluation consists of a series of statements. organized into the ma
jor categories of the Baldrige Award criteria. Use subjective judgment to grade how well
your organization conforms to the criteria by assigning a numerical rating from 1 to 10
to each statement, with 10 being high and 1 being low. Add the scores in each calegory
and normalize. For example. the five leadership items tOlallo 32; therefore the category
score is 32150 x 110 =70. Use the individual and total scores to monitor the status of the
quality management program. Use the first score as a baseline and subsequent scores to
monitor progress.
/. Leadership
___ Senior executives are actively and personally.involved in developing the
quality goals and standards for the organization. communicating these
goals. planning for quality, and supervising its implementation and
progress.
_ _ All levels of management demonstrate through their words and actions
that quality is the first priority within the organization.
_ _ There is a wiliingness to assist depanments and individual employees to
improve.
We operate in a matter consistent with a high sense ofethics, concern for
public health, and concern for the environment.
We have a system to evaluate the effectiveness of its leadership.
2. Slralegic Planning
._ _ We have an effective short-range (one to two years) plan for implement
ingTQM.
We have an effective long-range (three years or more) plan for leadership.
in quality and customer satisfaction.

Adapted. with permission. from O. W. Chase.lmple""'lIling TQM in a C(}1IStrucrion Company (Washington.

D.C.: Associated Gene,....:t Contractors of America. 1993).

24

__
__
__
__
__

__
__
__
__
__
__
__
__

We have developed strategies 10 maintain and build customer relationships.

We have effectivenieans to determine client expectations.
We effectively communicate those expectations to the work force.
We have established standards to meet those expectations.
We have developed effective means (surveys, visits, etc.) to measure
the level of client satisfaction with all processes that affect client
expectations.
We have a good client-contractor communications system.
We conduct a post-project follow-up with customers.
Employees visit customer sites. as appropriate. to help promote un
derstanding.
We operdy share information with the customer.
We are responsive to customer inquiries.
We are responsive to customer complaints.
Employees at all levels are alerted to report possible customer dissatis
.
faction with any product or service.
Those who interface with the customer receive special training (listening.
complaint resolution. negotiation. and the like).

4. Information and Analysis

_ _ We have an accurate and timely database that provides information on
customers, internal operations, organization performance, and COSts and
finances.
_ _ We benchmark ourselves against others who are leaders in panicular ar
eas and use the data to help improve quality.
_ _ Information collected is pertinent to the improvement effort.
_ _ Wormation collected is analyzed and actually used to support improvement.
_ _ We evaluate and improve our information and analysis system.

PERFORMANCE MEASURES

7. Business Results

5. Hllman Resource Focus

~\te can show quantifo:lble improvement in the follawing areas (rate each one):

_ _ Human resource and management plans support qUality.

_ _ We have an effective training program on quality awareness, teamwork.
and structured problem solving.
_ _ We provide effective training in technical skills.
_ _ The training program is open to all employees.
_ _ All employees are involved in process improvement and customer
satisfaction.
Employees are encouraged to make recommendations for improvement.
_ _. Employees, individually and .in teams, are recognized for improvement.
_ _ A positive work environment is maintained for the well-being, growth,
and positive attitude of employees.
_ _ Personnel practices such as recruitment. hiring. training. and other ser
vices are evaluated to determine their effectiveness.
_ _ Job site conditions are evaluated for possible improvement.

_ _ Safety/number of worker compensation claims.

_ _. Employee satisfactionfabsenteeisrnfturnover.

_ _ Grievances/strikes.

- - Satisfaction with our performance on the part of other parties with whom
we deal-i.e. architectsler.gineers. subcontractors. suppliers. government
agencies.
_ _ Punch (defect) lists.
_ _ On-time performance.
Reduced delivery times.

Financial return.

_ _ Rework.

_ _ Subcontractor quality.

___ Supplier quality.

_ _ Engineering processes.

6. Process Management

Logistical processes,

_ _ Administrative processes.

_ _ Construction processes.

_ _ Customer (client) satisfaction.

Repe at business.

Market share.

(These statements refer to all processes performed by the organization. including

adminrs/rative. logistical, engineering. alld cOils/rue/ion.)
___ All processes performed by this organization have been identified, and
the ownership of each process has been established.
_ _ All areas of duplication have been identified.
_ _ An effective system ex.ists within the organization to prioritize processes
for improvement.
_ _ Performance standards for processes have been established.
_ _ A system to measure process performance against the standards has been
developed.
_ _ We have an effective system for improving those processes identified as
needing improvement.
_ _ There is an effective means of disseminating "lessons leamed" informa
tion from project to project and throughout the organization.
_ _ Quality requirements are effectively communicated to subcontractors
and suppliers.
_ _ Subcontractor and supplier quality are effectively monitored and the re
sults are furnished to them.
_ _ We have an effective metho-d for evaluating the process improvement
system itself.

Comments
.-~-.---------"--

The MBNQA provides a plar, to keep improving all operations continuously and a sys
tem to measure these improvements accurately. Benchmarks are used to compare the or
ganization's performance with the world's best and to establish stretch goals. A close
partnership with suppliers and customers that feeds improvements back into the opera
tion is required. There is a long-lasting relationship with customers, so that their wants
are translated into products and services that go beyond delivery. Management from top
10 bottom is committed to improving quality, Preventing mistakes and looking for im
provement opportunities is built into the culture. There is a major investment in human
resources by means of training. motivation, and empowerment.
According to Dr. J. M. Juran, who studied the winners of the a'Nard, the gains have
been stunning. The gains can be accomplished by large and small U.S. organizations and
by U.S. workers. The gains include quality, productivity, and cycle time.

13