Dco Dissertation 6-24-04

ASSESSMENT OF CURRENT QUALITY MANAGEMENT PRACTICES
A Dissertation
Presented to
The School of Technology
Indiana State University
Terre Haute, Indiana
In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy
by
Darren C. Olson
May, 2004
© Darren C. Olson 2004

iii
ABSTRACT
The problem for this study was to assess the status and effectiveness of current
quality management practices, as stated by practitioners who are ASQ section officers.
A survey was conducted among practicing quality managers, who rated their employers’
implementation levels for categories contained in the ASQ body of knowledge for
certified quality managers (independent variables), and compliance levels with the ISO
9001:2000 requirements and the MBNQA criteria (two sets of dependent variables). All
ratings were based upon a 5-point Likert scale.
After applying a squared transformation to the data, ANCOVA was used to
determine if there were significant correlations between independent and dependent
variables. There were significant effects and /or two-way interactions correlated with 11
of the 12 dependent variables. These results were used to build a hierarchal model for
improving quality management efforts.
Participants also responded to demographics questions. Among the responses,
education level, status as a certified quality manager, the number of ASQ certifications
held, and company size all had significant correlations with some variables.
iv
ACKNOWLEDGEMENTS
I wish to express my deepest appreciation to my wife, Elizabeth, and children,

Daniel and Sarah, for their support and understanding. My work would not have been
possible without them.
Dr. John W. Sinn was my major advisor and mentor. I could not have developed
the knowledge, skills, abilities, and perspectives that I now possess without his teaching,
coaching, patience, and patronage. I will always be indebted to him.
I would also like to thank Dr. Todd Waggoner, Dr. Ming Zhou, Dr. Wallace
Carlson, Dr. William James, Dr. Ron Woolsey, Dr. Ernest Savage, Dr. Donna Trautman,
Dr. Bruce Dallman, Dr. Jeff Rybak, Dr. William Brauer, Kim Strickland, Sharon
Russell, and Linson Chamakkala for your support, guidance, and service.
Two organizations provided material support for this study. The School of
Graduate Studies at Indiana State University (ISU) provided monetary assistance under
its category II funding for proposed graduates student research. The Industrial
Technology (IT) Department at Bemidji State University (BSU) provided assistance with
office materials and postage, in addition to what was provided by ISU. The IT
Department at BSU also provided computer hardware and software that was used to
complete this study, and a graduate assistant, Linson Chamakkala, whose support
allowed me to dedicate more time to completing the study.
v
TABLE OF CONTENTS
ACKNOWLEDGEMENTS .............................................................................................. iv
LIST OF TABLES ............................................................................................................. x
LIST OF FIGURES ......................................................................................................... xv
CHAPTER 1. Introduction ................................................................................................ 1
The Context of the Problem ................................................................................... 1
Statement of the Problem ....................................................................................... 7
Significance of the Study ........................................................................................ 7
Research Objectives ............................................................................................ 10
Assumptions ........................................................................................................ 10
Limitations ........................................................................................................... 11
CHAPTER 2. Review of Literature ................................................................................ 12
Quality and Quality Management ........................................................................ 12
Defining Characteristics of Quality ......................................................... 12
Activities Used to Assure Quality ............................................................ 17
Organizing to Achieve Quality ................................................................. 24
Quality Managers and Professional Practice .......................................... 29
ASQ’s Vision for the Future .................................................................................. 38
Anticipating Needs ................................................................................... 39

vi
Models of Organizational Quality ........................................................... 43
Summary .................................................................................................. 54
CHAPTER 3. Method ..................................................................................................... 57
Restatement of the Problem ................................................................................. 57
Restatement of Objectives .................................................................................... 57
Research Design .................................................................................................. 58
Phase I: Review of Literature .................................................................. 59
Phase II: Develop the Study ..................................................................... 63
Questionnaire Refinement Using a Modified Delphi Process ...... 64
Time Testing the Revised Questionnaire....................................... 67
Phase III: Administer the Study and Analyze Results ............................... 68
Administer the Study ..................................................................... 68
Analyze the Results ...................................................................... 71
CHAPTER 4. Results ...................................................................................................... 72
Checking Normality Assumptions ......................................................................... 75
Histograms ............................................................................................... 79
Skewness and Kurtosis Z scores .............................................................. 89
Normality Tests ........................................................................................ 91
Data Transformation ........................................................................................... 98
Analyze the Data for Model Fit ............................................................ 121
Analysis of the Refined Model Using ANCOVA ................................... 146
Analysis of Demographics Data ........................................................................ 168
Is the Respondent a Certified Quality Manager? (CQM) ...................... 171

vii
Number of Years, QM Experience (EXP) .............................................. 173
Number of ASQ Certifications Held (NCERT) ...................................... 174
Responsibility Level (RESP) .................................................................. 176
Education Level (EDU) .......................................................................... 183
Number of Employees, Respondent’s Division or
Organization (EMPL) ............................................................................ 186
Number of Employees Under Respondent’s Direction (DIRECT) ......... 195
NAICS Code Categories of Respondents’ Employers (NAICS) .............. 196
ASQ Region (REGION)........................................................................... 202
Respondent Gender (GEN) .................................................................... 207
CHAPTER 5. Discussion .............................................................................................. 212
Conclusions ........................................................................................................ 212
Descriptive Statistics .............................................................................. 213
Normality Assumptions and Hypotheses ................................................ 218
Normality Assumptions .............................................................. 218
Hypotheses One and Two ........................................................... 219
ISO1T: Quality management system .............................. 220
ISO2T: Management responsibility ............................... 221
ISO3T: Resource management ...................................... 221
ISO4T: Product realization ............................................ 222
ISO5T: Measurement, analysis, and improvement ........ 222
MB1T: Leadership ......................................................... 223
MB2T: Strategic planning .............................................. 224

viii
MB3T: Customer and market focus ............................... 225
MB4T: Measurement, analysis, and
knowledge management ................................................. 225
MB5T: Human resource focus ....................................... 225
MB6T: Process management ......................................... 226
MB7T: Business results .................................................. 226
Hypotheses Three through Five: Correlations
With Demographics Items ...................................................................... 227
Is the Respondent a Certified Quality Manager? (CQM) .......... 227
Number of ASQ Certifications Held (NCERT) .......................... 227
Responsibility Level (RESP) ...................................................... 228
Education Level (EDU) ............................................................. 228
Number of Employees, Respondent’s Division or
Organization (EMPL) ................................................................ 229
NAICS Code Categories of Respondents’
Employers (NAICS) .................................................................... 230
EPILOGUE .................................................................................................................... 231
Interpretation of ANCOVA Results .................................................................... 231
Interpretation of Analyses Involving Demographics Items ............................... 235
Recommendations .............................................................................................. 236
REFERENCES .............................................................................................................. 240
APPENDIXES ............................................................................................................... 249

ix
APPENDIX A. Glossary of Selected Quality Management Tools
and Techniques .................................................................................................. 250
APPENDIX B. Table B1: A comparison of the ISO 9001:2000 Clauses
and the MBNQA Criteria categories with Part A of the ASQ BOK ................. 263
APPENDIX C. Human Subjects Review Board Approval ............................... 271
APPENDIX D. Transcript of Delphi Panel Proceedings .................................. 274
APPENDIX E. Survey Instrument, Solicitation Letter, and Informed
Consent Form ..................................................................................................... 288
APPENDIX F. Table F1: Responses and Variables Data ................................. 303
APPENDIX G. Table G1: Spearman’s Rho Correlation Data
Between Demographics Response Items and all Variables ............................... 345

x
LIST OF TABLES
Table 1 New Elements in the 2001 BOK ................................................................. 8
Table 2 Domains and major categories in Part A of the ASQ body of
knowledge for certified quality managers ................................................ 34
Table 3 Domains and major categories in Part B of the ASQ body of
knowledge for certified quality managers ................................................ 36
Table 4 BOK tools and techniques that could be used in
technology assessments ........................................................................... 42
Table 5 Overview of the ISO 9001:2000 clauses .................................................. 45
Table 6 Overview of the MBNQA Criteria categories .......................................... 47
Table 7 Phases of the study .................................................................................... 58
Table 8 Variable names and descriptions ............................................................. 62
Table 9 Mathematical definitions of independent and dependent variables ....... 73
Table 10 Demographics response items ................................................................. 74
Table 11 Descriptive statistics ................................................................................ 76
Table 12 One sample t-test, comparing mean variables responses to
the value 4.0 (Good) ................................................................................ 78
Table 13 Skewness and kurtosis Z-scores ............................................................... 90
Table 14 Normality tests on the dependent variables ............................................. 91

xi
Table 15 Skewness and kurtosis Z-scores for transformed variables ..................... 99
Table 16 Mathematical definitions of transformed data ....................................... 100
Table 17 Descriptive statistics for the transformed variables .............................. 101
Table 18 Normality tests on the transformed dependent variables ...................... 113
Table 19 Spearman’s Rho matrix for BOK vs. ISO transformed variables .......... 124
Table 20 Spearman’s Rho matrix for BOK vs. MB transformed variables .......... 125
Table 21 ANCOVA reports for independent variables vs. dependent variables
transformed data .................................................................................... 127
Table 22 Terms retained in refined models .......................................................... 145
Table 23 ANCOVA tests, parameter estimates, and lack of fit test,
dep. var. ISO1T ...................................................................................... 147
dep. var. ISO2T ...................................................................................... 149
dep. var. ISO3T ...................................................................................... 150
dep. var. ISO4T ...................................................................................... 152
dep. var. ISO5T ...................................................................................... 153
dep. var. MB1T ...................................................................................... 155
dep. var. MB2T ...................................................................................... 157

xii
dep. var. MB3T ...................................................................................... 159
dep. var. MB4T ...................................................................................... 161
dep. var. MB5T ...................................................................................... 162
dep. var. MB6T ...................................................................................... 164
dep. var. MB7T ...................................................................................... 165
Table 35 Final regression equations for each dependent variable ...................... 168
Table 36 Demographics response items, repeated ............................................... 170
Table 37 Significant correlations between item CQM and variables ................... 172
Table 38 Significant correlations between item NCERT and variables ............... 175
Table 39 Mean responses to BOK variables, sorted by responsibility level ......... 178
Table 40 Mean responses to ISO variables, sorted by responsibility level .......... 178
Table 41 Mean responses to MB variables, sorted by responsibility level ........... 178
Table 42 Tests for significant differences in variation of BOK variable
responses, grouped by responsibility level ............................................ 179
Table 43 Tests for significant differences in variation of ISO variable
Table 44 Tests for significant differences in variation of MB variable

xiii
Table 45 Significant correlations between item EDU and variables .................... 184
Table 46 Significant correlations between item EMPL and variables ................. 187
Table 47 Mean responses to BOK variables, sorted by NAICS code category .... 198
Table 48 Mean responses to ISO variables, sorted by NAICS code category ...... 198
Table 49 Mean responses to MB variables, sorted by NAICS code category ...... 199
responses, grouped by NAICS code category ........................................ 199
Table 51 Tests for significant differences in variation of ISO variable responses,
grouped by NAICS code category .......................................................... 200
responses, grouped by NAICS code category ........................................ 201
Table 53 Mean responses to BOK variables, sorted by ASQ region .................... 204
Table 54 Mean responses to ISO variables, sorted by ASQ region ...................... 204
Table 55 Mean responses to MB variables, sorted by ASQ region ...................... 205
responses, grouped by ASQ region ........................................................ 205
Table 59 Mean responses to BOK variables, sorted by gender ............................ 208
Table 60 Mean responses to ISO variables, sorted by gender .............................. 209
Table 61 Mean responses to MB variables, sorted by gender .............................. 209

xiv
responses, grouped by gender ................................................................ 209
Table 65 Final regression equations for each dependent variable ...................... 220
Table B1 A comparison of the ISO 9001:2000 Clauses and the MBNQA
Criteria categories with Part A of the ASQ BOK .................................. 263
Table F1 Responses and variables data ................................................................. 303
Table G1 Spearman’s Rho correlation data between demographics
response items and all variables ............................................................. 345

xv
LIST OF FIGURES
Figure 1 Time line for the study ............................................................................. 59
Figure 2 Histograms of all variables ...................................................................... 79
Figure 3 Q-Q normal probability plots of the dependent variables ........................ 92
Figure 4 Histograms of all transformed variables ................................................ 103
Figure 5 Q-Q normal probability plots of the transformed dependent variables . 114
Figure 6 Residual plot matrices for transformed dependent variables ................. 139
Figure 7 Residuals plot matrix, dependent variable ISO1T ................................. 148
Figure 12 Residuals plot matrix, dependent variable MB1T ................................. 156
Figure 15 Residuals plot matrix, dependent variable MB4T ................................ 162

xvi
Figure 19 Number and percent of respondents who are currently
certified quality managers ...................................................................... 171
Figure 20 Mean of BOK4: Customer Focused Organizations vs. item CQM ....... 172
Figure 21 Mean of MB3: Customer and Market Focus vs. item CQM .................. 173
Figure 22 Number of years QM experience held by respondents .......................... 174
Figure 23 Number of ASQ certifications held by respondents ............................. 175
Figure 24 Mean of MB3: Customer and Market Focus vs. item NCERT .............. 176
Figure 25 Responsibility level of respondents within their
employing organizations ........................................................................ 177
Figure 26 Variation in the mean of BOK4, categorized by respondent’s
responsibility level ................................................................................. 181
Figure 27 Variation in the mean of ISO1, categorized by respondent’s
Figure 30 Years of post-secondary education attained by respondents ................. 184
Figure 31 Mean of BOK1: Leadership vs. item EDU ............................................ 185
Figure 32 Mean of ISO4: Product Realization vs. item EDU ................................ 185
Figure 33 Number of employees at respondent’s primary unit of employment .... 186
Figure 34 Mean of BOK1: Leadership vs. item EMPL ........................................ 188
Figure 35 Mean of BOK3: Quality Management Tools vs. item EMPL ............... 189
xvii
Figure 36 Mean of BOK4: Customer-Focused Organizations vs. item EMPL ..... 189
Figure 37 Mean of BOK5: Supplier Performance vs. item EMPL ........................ 190
Figure 38 Mean of BOK6: Management vs. item EMPL ...................................... 190
Figure 39 Mean of BOK7: Training and Development vs. item EMPL ................ 191
Figure 40 Mean of ISO4: Product Realization vs. item EMPL .............................. 191
Figure 41 Mean of MB1: Leadership vs. item EMPL ............................................ 192
Figure 42 Mean of MB2: Strategic Planning vs. item EMPL ................................ 192
Figure 43 Mean of MB4: Measurement, Analysis, and Knowledge Management
vs. item EMPL ....................................................................................... 193
Figure 44 Mean of MB5: Human Resource Focus vs. item EMPL ....................... 193
Figure 45 Mean of MB6: Process Management vs. item EMPL ........................... 194
Figure 46 Mean of MB7: Business Results vs. item EMPL .................................. 194
Figure 47 Number of employees under direction of the respondent ...................... 195
Figure 48 NAICS Industrial classification prefix of respondent’s employer ......... 197
Figure 49 Variation in the mean of ISO1, categorized by NAICS code
prefix of respondent’s employer ............................................................ 202
Figure 50 Number of returned surveys by ASQ region ......................................... 203
Figure 51 Percentage of respondents, categorized by gender ................................ 208
Figure 52 Contour plot of the regression equation for ISO5T ............................... 223
Figure 53 A model for improving product realization ........................................... 234
Figure 54 A model for improving business results ................................................ 234

1
CHAPTER 1
Introduction
The Context of the Problem
In the mid-1990s, The American Society for Quality (ASQ) concluded a long-
term study aimed at providing guidance for ongoing development of the quality
profession. This study, called the Futures Project (Watson, 1998b), produced a vision
that led the organization to realign its strategy for the future. Watson summarized the
project outcomes. In discussing the decision to drop the word control from its name he
stated that the society is now “free from the restricting perception that the society cares
only about the technical aspects of control found in manufacturing applications (p.
110).” The society recognized the need to make the practice of quality a part of all work
processes, beyond the traditional application to manufacturing quality control.
Watson (1998b) emphasized that in the future all professionals will need to be
able to apply advanced quality tools, and that quality professionals will fill the role of
specialists, advising others. They will work in conjunction with personnel from various
functions, assisting in problem solving and in the application of advanced tools, as well
as building new competencies to assist in their roles as advisors. They will need more
knowledge about business and participate in aligning operations with strategic plans.
2
The Foresight 2020 Report was a continuation of the work started by the ASQ
Futures Team (ASQ Futures Team, n.d.). In this updated study, the team emphasized
that there will be fewer quality professionals in the future. In keeping with the sentiment
that quality professionals will work as advisors, so that others can apply appropriate
tools, the report emphasized that responsibility for quality will shift to business leaders.
Quality professionals will teach and coach others within the organization. They will also
provide leadership to the world at large, helping people from all areas of their
communities to understand the concepts of quality and apply them in all situations.
Watson (1998a) discussed how this vision of the future might unfold.
Technological advances have created a global economy in which a lot of information is
available to virtually anyone at any time. Combined with the expansion of business
partnerships and regional economic pacts, access to information will create a world in
which competitors can spring up anywhere, without much warning. The competitive
advantage afforded by being first to market will not last as long as in times past.
Traditional challenges, such as achieving design quality and controlling manufacturing
processes will continue to be vital, requiring more use of sophisticated methods.
Services rendered to the customer will become a more prominent way to gain
competitive advantage. Companies will need to use new, more advanced statistical
techniques, better management methods, and appropriate digital technologies. More of
the challenges are listed below:

3
1. Scanning the environment for potential sources of competition
2. Using sophisticated monitoring and analysis techniques to interface with
customers
3. Creating appropriate business strategies and sticking to them
4. Using technology to break down functional barriers
5. Setting and following strategies for information management
6. Dealing with the organizational consequences of access to information, such as
empowerment of the lower ranks and flattening of management structures
The list of challenges is formidable, and some of those outlined above are already
becoming a reality. In their review of the updated ISO 9001:2000 international quality
standard, Cianfrani, Tsiakals, and West (2001) outlined changes that the standard will
require or encourage companies to make. For example, a new introductory section to the
standard provides an overview of the principles that the International Organization for
Standardization (ISO) used as underlying themes in the 2000 revision. These principles
included the following items: (a) Customer focus, (b) leadership, (c) employee
involvement, (d) a process approach, (e) a systems approach to management, (f)
continuous improvement, (g) fact-based decision making, and (h) mutually beneficial
supplier relationships.
ISO 9001:2000 was restructured as compared to the 1994 standard. The 2000
revision is consolidated into eight clauses, some of which operationalize the updated
principles in the form of requirements. New requirements address monitoring customer
feedback and using data analysis techniques to determine satisfaction. The requirements
also address management of processes, treating them as systems with inputs and outputs.
4
The business world is rising to some of the challenges foreseen by the Futures
Team (Watson, 1998b). By virtue of requirements by governments and primary-
manufacturing organizations, ISO 9000 compliance has become a necessity for many
companies. ISO 9000 is growing and evolving. In the process it has adopted even more
principles that address the Futures Team’s findings. This is only one of the two
significant movements that have led a shift in management attitudes about quality.
Since 1987, when the Malcolm Baldrige National Quality Improvement Act was
passed in the United States, the Malcolm Baldrige National Quality Award (MBNQA)
has arisen and claimed a good deal of attention from managers in many economic
sectors. The award is administered by the National Institute of Standards and
Technology (NIST), a branch of the United States Department of Commerce (USDOC).
The MBNQA Criteria are organized into seven categories (U.S. Department of
Commerce, 2002) that address, even more closely than ISO 9000, most of the issues
brought forward by the ASQ Futures Team (Watson, 1998b). Each category focuses on
a management area, with the exception of one, which focuses on business results.
Rather than being a standard, as ISO 9000 is, the MBNQA system is a voluntary
set of criteria. Companies that want to be considered for the award must provide
evidence of adherence to its principles. An important difference from ISO 9000 is that
companies are freer to organize as appropriate for their own environments and internal
systems, as long as they can provide evidence that they meet the award criteria.
Many of the techniques prescribed in ISO 9000 are only suggestions in the
MBNQA Criteria.
5
Because the MBNQA is a voluntary set of criteria, rather than a standard, it may
not be readily apparent that the Baldrige system carries much influence in the business
world. However, there is important evidence that the Baldrige Criteria do exert a good
deal of influence. In addition to the few companies that actually apply for the award
annually, there is a large body of companies that use the criteria as an organizational
development guide and self-assessment tool. The U.S. Department of Commerce
estimated that two million copies of the criteria have been distributed since 1988. More
than one million of those copies were paper copies, the rest being distributed
electronically. They also estimated that a similar number of copies have been made by
recipients for further circulation. In the same report, the U.S. Department of Commerce
(2001) conservatively estimated that use of the criteria as an improvement guide and
self-assessment tool has saved businesses $2.17 billion, while costing them $119 million.
Finally, the U.S. Department of Commerce (personal communication, December 28,
2001) listed 44 states that, as of December, 2001, have instituted state quality awards
that are based upon the Baldrige criteria. The MBNQA has become a significant
influence upon business practices in the United States.
The growth of influence exhibited by ISO 9000 and the MBNQA are
encouraging signs that concepts about quality, in the United States at the least, are
becoming broader. Quality thinking is encompassing a wider variety of principles, just
as the ASQ Futures Team (Watson, 1998b) suggested it should.
However, no studies currently available in the literature have gauged the progress
of quality managers in implementing the recommendations that came out of the Futures
and the Foresight 2020 (ASQ, n.d.) studies. While the continued proliferation of ISO
6
9000 and the MBNQA are encouraging, there is reason to question to what degree
broadening concepts about quality have been accompanied by a parallel growth in the
application of the traditional tools and techniques of quality management. According to
Cianfrani et al. (2001) neither ISO 9001:2000 nor the MBNQA Criteria (U.S.
Department of Commerce, 2002) specifically addresses many of the fundamental tools
of quality management, such as statistical analysis, the seven quality control tools, and
the seven planning and management tools. These omissions may have been by design,
but they leave room for businesses to address quality without the use of specific quality
tools. Thus it is possible that the use of quality professionals as advisors and teachers in
the application of quality tools has not proliferated as has been envisioned by the ASQ
Futures Team (Watson, 1998b).
Research conducted since the ASQ Futures Team produced the Futures (Watson,
1998b) and Foresight 2020 (ASQ Futures Team, n.d.) studies did address the use of
quality management tools and techniques. For example, Grandzol and Gershon (1997)
conducted a study in which they attempted to identify common threads in total quality
management (TQM) practice, correlating the use of quality practices with a TQM
construct that they derived from a comprehensive literature review. Jayaram, Handfield,
and Ghosh (1997) correlated the use of quality tools with business strategies. Kannan,
Tan, Handfield, and Ghosh (1999) investigated the impact of quality tools and
techniques on business results. However, none of these studies was performed recently
enough to gauge the impact of changes to the quality management body of knowledge,
which reflect the findings of the ASQ Futures Team (Quality Management Division On-
7
line, 2001). Also, each of these studies used quality constructs created by their authors
rather than ones that are currently influential in actual practice.
Statement of the Problem
Significance of the Study
For the purposes of this research, quality management practitioners who are ASQ
section officers were surveyed. Respondents were asked to state estimations of their
employing organization’s level of progress in applying the principles contained in
categories of the certified quality manager body of knowledge (BOK) and effectiveness
in complying with principles contained in the categories of ISO 9001:2000 and the
MBNQA Criteria.
The inventory of quality management tools was entirely based upon the 2001
revision to the ASQ quality management body of knowledge (BOK). This is significant
because the ASQ Certification Committee revises the BOK on a five-year cycle (ASQ,
2001), meaning that the latest prior revision occurred when the Futures Study was just
being published and the Foresight 2020 study had not yet been conducted. The BOK is
modified by a process that involves a study done by an advisory committee that is then
reviewed and refined by a BOK committee and a panel of subject matter experts. Thus
the 2001 BOK is the first one to reflect the modified mission and vision of ASQ. The
new BOK contains many significant additions to its content (Quality Management
Division On-line, 2001). New elements are listed in Table 1.

8
Table 1
New elements in the 2001 BOK
Element Description
1A6 Constraint management
1A7 Negotiation techniques
1A8 Motivation techniques
1B2 Organization culture
1B3 Team building techniques
2A2 Market forces, industry trends, competitive analysis
2A3 Stakeholder groups
2A4 Technology trends and internal capabilities
2A5 Strength, weakness, opportunities, and threats analysis
2A7 Internal Capability Analysis
2B2 Competitive comparisons and benchmarks
2B3 Formulating quality policies
2C2 Deploying strategic goals and objectives into operational plans and
improvement projects
3A4 Tools for innovation and creativity
3B1 Process goals
3B4 Theory of constraints
3C5 Qualitative assessment
3C6 Analysis and use of survey results
3C7 Benchmarking: Internal and external

9
Table 1 (continued)
New elements in the 2001 BOK
Element Description
5A Supplier selection strategies and criteria
5B Techniques for communicating requirements to suppliers
5C Techniques for assessment and feedback of supplier performance
5D Supplier improvement strategies
5G Logistics and supply chain management
6A1 Principles of management
6B1 Communication techniques
6B2 Information systems
6B3 Knowledge management
6E3 Other industry and international standards
Correlations were analyzed between stated degrees of application of various
quality management tools and stated organizational success in meeting the requirements
of ISO 9001:2000 and MBNQA Criteria categories. This is significant because
ISO 9001:2000 and the MBNQA are widely recognized and applied, making them major
driving forces behind changes in quality practice. In other words, the study provided
both a check on the progress of quality management practices and a map that illustrates
correlations between professional practice and organizational effectiveness.

10
Research Objectives
The problem of the study was addressed through four objectives:
1. Assess the progress of quality management practitioners in applying the
principles contained in the categories of the Certified Quality Manager Body of
Knowledge (BOK), as stated by section officers in the American Society for
Quality (ASQ).
2. Gauge the effectiveness, as stated by ASQ section officers, of quality
management efforts in meeting the categorical requirements of the ISO
9001:2000 standard.
management efforts in meeting the categorical requirements of the Malcolm
Baldrige National Quality Award (MBNQA).
4. Investigate the relationships between the stated application levels from objective
one, treated as independent variables, and the stated effectiveness of quality
management efforts from objectives two and three, treated as dependent
variables, to determine if there are any possible main factor effects or
interactions.
Assumptions
The following assumptions were made for this study:
1. Respondents participated objectively and candidly.
2. Respondents possessed expertise adequate to address the questions at hand, and
they expressed their opinions and experiences satisfactorily.

11
3. The questionnaire appropriately represented the 2001 revision of the BOK,
ISO 9001:2000, and the 2002 MBNQA Criteria.
4. The responses of section officers were generalizable to the opinions and efforts
of other quality managers who are active in the profession through ASQ.
Limitations
The following limitations were inherent to the study:
1. The study was limited to a sample of quality managers who are serving as
officers in ASQ sections.
2. Potential respondents may have been prohibited from responding to a survey of
this nature due to issues of proprietary knowledge arising from their relationship
with their employers or clients.

12
CHAPTER 2
Review of Literature
The review of literature will focus on the current state of quality management,
the nature of the 2001 revision of the ASQ quality management body of knowledge
(BOK), and the nature of the 2002 Malcolm Baldrige National Quality Award Criteria
(MBNQA Criteria). The first section is concerned with quality and quality management.
It addresses how quality is defined, what activities lead to it, how to organize those
activities, and what constitutes professional quality management. The following section
contains an examination of ASQ’s vision for the future of quality, as represented by the
2001 BOK. The third section examines the prevailing forces in quality standards and
compares the leading standards with the BOK. The conclusion of this review
summarizes the previous sections.
Quality and Quality Management
Defining Characteristics of Quality
People interact with products and services continually. Whether at home, at
work, or at large, people use objects, receive services, and consume products that are not
entirely goods or services. As a result of such interactions, people are left with
impressions about how well their wants and needs were met, affecting perceived quality
levels. Product features and attributes, performance aspects, and other, less-tangible
13
product characteristics are all factors that can influence stakeholder impressions. The
degree to which such factors exist in a product can sometimes be measured objectively.
The degree to which other factors exist in a product can be subjective. Pirsig (1974)
suggested that quality is a fundamental, inherent property of an object, an absolute; our
interactions with an object provide us with both the objective and subjective measures of
that quality.
There are many definitions of quality in the literature. For example, Juran (1992)
stated that quality is fitness for use. Deming (1982/2000) defined quality according to
multiple perspectives, including that of production workers, management, consumers,
etc. Mizuno (1979/1988) defined total quality as a concept that entails adding value,
beyond merely satisfying consumers, as a consequence of production. Garvin (1987)
listed eight product dimensions by which quality can be judged. Each dimension
provided a different classification of observable product aspects that can be measured
and controlled, such as features, reliability, and performance, among others.
Both Shewhart (1931) and Ishikawa (1985) emphasized that quality is customer
defined. Companies must study the customer and take their needs into account when
they design and manufacture their products.
Feigenbaum (1983) taught that quality has many aspects, all of which must be
accounted for in a definition of quality. Ultimately it must be defined in terms of
customer satisfaction, which is not static.
Taguchi and Wu (1979) defined quality as the value lost by society when defects
occur in a product characteristic. Crosby’s (1979) definition was less ambiguous. He
taught that quality is conformance to specifications.

14
Hoyer and Hoyer (2001) classified quality definitions according to whether
they were primarily focused on product characteristics or on customer satisfaction.
Other themes running through the various definitions are that quality is abstract, people
perceive it in relative terms, it can be represented using substitute design specifications,
and it affects many stakeholders (e.g., production workers, management personnel,
consumers, and society at large).
Throughout the rest of this review and during the study, the term quality will be
used in the sense that it is inherently realized in a product and that indicators of quality
can be based on both objective specifications and subjective impressions, each of which
can be defined from the perspectives of multiple concerned parties. Attainment of
quality objectives can be determined by measuring conformance to specifications and
measuring and/or interpreting subjective reactions.
The term product will be used in reference to any combination of goods, services,
and consumables. Humphreys, Williams, and Meier (1997) lent credence to this
definition, suggesting that no good or service is completely one or the other. They
suggested that when we interact with something, there are both physical and
interpersonal aspects. For example, our satisfaction with a car is derived from objective
performance measures, subjective impressions (look, feel, etc.) and the service
associated with it (e.g., interactions with sales and service personnel). Satisfaction with
a hotel visit, which is mostly considered a service, is also affected by the physical
facilities. Thus, product will be used to denote the total package of goods and services
an organization delivers to the customer and/or user.

15
Once made or delivered, any product will have an actual level of quality; those
who interact with the product (i.e., employees, partners, customers, users, and the public)
will develop their perceptions of that quality. An organization can incorporate various
features and options into a design. Levels of fit, finish, durability, reliability,
serviceability, environmental friendliness, safety, and cost can be designed in.
Conformance to all such specifications can be monitored and controlled. Producers also
have to address subjective perceptions of quality (e.g., feel, taste, appearance, brand
prestige, and satisfaction with service). Producers of high-quality products are
successful at addressing all aspects of quality, to the appropriate levels.
Herrmann, Huber, and Braunstein (2000) discussed the importance of addressing
customer satisfaction, in addition to reaching goals for objective indicators of quality.
They stated that in the mind of the consumer a product’s quality is more than the sum of
its attributes, a concept borrowed from utility theory. Companies must view satisfaction
from marketing and behavioral science perspectives, meaning that customers will base
decisions about quality upon how well a product satisfies a complex set of expectations.
The starting point in achieving quality is to gauge customer desires and design products
accordingly. Once production has begun, satisfaction must be measured and products
must be modified as needed. Gustafsson and Johnson (1997) suggested an approach that
combines customer satisfaction modeling with quality function deployment (QFD), a
process by which abstract requirements are translated into specifications that can be met
by successive, downstream participants in the development, design, production, delivery,
and service of the product.

16
Implicit in this approach is the need to address quality over the entire life cycle
of a product. Life cycles begin with product development, which starts with the earliest
idea generation and continues through the design and pre-production phases. Concepts
are embodied in prototype designs that are tested and refined until almost ready for
production. Then production processes are developed. Once these are sufficiently
refined, the product goes into production and distribution. Continuous improvement
efforts make incremental changes to the design over the service life of the product.
These are based upon feedback from production, distribution, and service personnel,
from customers and end users, from sales personnel, etc. Eventually a product will
become obsolete and a company will develop something to replace it, ending the life
cycle.
Managing quality over the life cycle of the product requires organizations to
address requirements and constraints during all phases of the product life cycle. People
and organizations who interact with the product during its life cycle are stakeholders.
Each of them is an important source of information needed for defining quality. Raturi,
Houshmand, and Manek (1996) used the term total product development to describe the
process of accounting for stakeholder demands. They suggested making design
decisions based upon an analytic process that draws quality requirements from
customers, company strategies and policies, safety and environmental demands, and
constraints imposed by materials, manufacturing processes, etc.
Whether from external sources or internal ones, pressures can be brought to bear
either directly upon the product design or upon the performance of the organization. A
product may need to contain certain features, perform at certain levels, be defect free, or
17
comply with various laws, standards, and regulations. Additionally, an organization
may have to bring a product to market within a certain time frame and under certain
budget constraints, or it may have to achieve a certain level of profitability. All
pressures upon the product and the organization may end up being translated into
requirements. Performance of the product and the organization against these
requirements can be measured and used as indicators of quality.
Activities Used to Assure Quality
Achieving quality requires balancing what various stakeholders want and need,
translating this information into design requirements that are optimized to balance these
needs, and delivering products that conform to design specifications. The first step,
defining needs and choosing the right balance of requirements to meet those needs,
involves sorting out what needs are required from what ones are desirable, figuring out
what needs were not articulated, and producing a design that can satisfy the right balance
of needs.
Companies can use any of a number of techniques to optimize satisfaction.
These techniques are used to balance choices between design requirements and to take
account of downstream constraints upon the ability to conform to specifications.
Sometimes needs are ranked using cardinal numbers, based upon the opinions of
development team members. Other techniques use various optimization procedures. For
example, Derringer (1994), Dowlatshahi and Ashok (1997), Park and Kim (1998),
Vairaktarakis (1999), and Askin and Dawson (2000) presented various schemes for
generating and optimizing mathematical functions that represent satisfaction. There are
also techniques for taking into consideration constraints imposed by materials,

18
manufacturing processes, etc. For example, Otto and Ho (1996) and Shina and Saigal
(2000) presented methods for modeling process constraints as aggregates of the
constraints posed by individual production operations.
Design is the process of translating requirements into specifications and then
producing a document (on paper or electronic) that completely defines all aspects of the
product. As this process is carried out, designers should be working with personnel from
all relevant functional areas to gather input in the form of design constraints and
requirements. Thus the design process requires the integration of personnel from diverse
functions in specifying how the product is to be made.
Personnel from the various departments and functions need to receive
preliminary design information so that they can use it in planning their own processes
related to that product. Duties are carried out in an iterative give and take that is meant
to bring high quality products to market in minimal time and with minimal cost.
Concurrent engineering (CE) synchronizes the efforts of designers with those of
personnel from other functional areas. It is a practice that puts these personnel together
into cross-functional teams that are coordinated in the style of project management.
Activities are carried out in parallel as much as possible, communications are managed
along product lines, and data is shared between integrated computer systems.
Pawar, Haque, and Barson (1999) cited several factors that influence CE
outcomes, as indicated by team concordance (mutual agreement about project goals),
including the presence of a full-time project manager with strong leadership abilities,
commitment from participants who are not core team members, strong communication
links, and strong relationships between team members. Using case studies conducted in
19
large companies and involving complex development projects, Nihtila (1999) found
that integration between research/development and production is facilitated by the
following practices: (a) the use of standards, procedures, and plans; (b) setting
milestones; (c) conducting design reviews; (d) having a strong and experienced project
leader; and (e) using cross-functional teams.
Case studies conducted by Van Eijnatten and Simonse (1999) indicated that the
central management team should retain its identity throughout the life cycle of the
product, while other teams should be formed and dissolved as needed. These teams exist
in hierarchical levels under the central product management team, drawing employees
from all functional areas and all levels of the organization.
Concurrent engineering teams and their leaders have various technological tools
at their disposal, including computer aided design, design failure mode effects analysis,
simulation, finite element analysis, rapid prototyping, and others. Court, Culley, and
McMahon (1997) provided a good discussion about the types of tools available and the
need to retrieve, apply, and transfer this information among team members.
When a design is nearly complete, processes can be defined explicitly
(preliminary process planning having been started earlier in the development stages) and
operational quality plans can be formulated. Process and quality engineers work
together using various tools to create processes that optimize production and product
conformance, minimize costs, etc. They use historical data, tacit knowledge, and
implicit knowledge to guide their efforts. They do this in conjunction with the use of
quantitative tools like design of experiments, statistical process control (SPC), process
simulation, process failure mode effects analysis, time and motion studies, and Pareto
20
analysis. They also have powerful qualitative tools at their disposal such as process
flow charts, decision trees, brainstorming, mistake proofing, and root cause analysis. In
an iterative chain of fine-tuning, process and quality engineers test and refine until they
achieve capable processes. For an in-depth discussion of such activities, see chapters
three through six in Juran’s Quality Handbook (Juran & Godfrey, 1999).
Once processes are developed that are capable of achieving product
conformance, they must be implemented and controlled. Final test runs are performed
with production-ready designs and any minor adjustments to product and process designs
are made as needed. After fine-tuning, the processes are operationalized. According to
Cianfrani et al. (2001) ISO 9001:2000 requires the creation of a set of procedures and
work instructions to document those activities that are pertinent to the processes. Good
documentation will be accessible, easily interpreted, standard in layout, and cross-
referenced. Furthermore, if the procedures and instructions are not designed to work
within the company’s internal environment (e.g., not able to accommodate worker skill
levels, not compatible with company culture, or in conflict with policies or with other
procedures), then they will not be followed. Procedures and work instructions must
support process implementation and control, they must be feasible, and they must be
applied consistently.
Advanced quality planning takes place within the umbrella of concurrent
engineering (CE), in synchronization with requirements gathering, design, process
engineering, and quality engineering during the development stage. Dowlatshahi and
Ashok (1997) and Tsuda (1997) both discussed the use of quality function deployment
(QFD) during CE activities as a means of addressing quality during the development

21
stage. QFD is a process whereby quality planners can translate requirements into the
appropriate specifications for those who are involved with each phase of the product life
cycle, track implementation, and compare goals with performance. This is why quality
planning and concurrent engineering should be carried out in unison.
Operational quality plans are one output of advanced quality planning. They are
often referenced within process control plans, standard procedures, and work
instructions. Companies create them in order to control conformance. They specify how
and when to measure product characteristics and process parameters, how to provide
feedback, and how to respond to adverse stimuli (e.g., changing measurement
procedures, documenting solutions, isolating defective products, etc.) They also control
conformance by standardizing the use of mistake-proofing tools. One way to deploy
them is through the process control plan (PCP). This document controls the making and
delivery of the product, dividing production into operations and work steps. As
operational quality plans are made, they are deployed within the PCP. The Automotive
Industry Action Group (AIAG, 1994/1995) provided a guidebook for this process.
After production starts, there are still opportunities to affect product quality.
These include controlling conformance and making incremental quality improvements.
Juran (1992) referred to quality control during operations (production) as the quality
trilogy. The steady state of operations is to monitor measurable product and process
quality indicators, react to trouble signals and restore conditions to a state of control.
This is how conformance quality is achieved. Quality can be improved incrementally
through continuous improvement efforts. This involves using problem-solving tools to
address the problems causing the most negative effects within the company. Identifying
22
the worst operational problems and solving them will yield a steady state of operations
with a new, higher level of conformance quality. The term trilogy refers to the fact that
going from one steady state to a new one via improvements is a three-phased cycle (i.e.,
steady state, improvement, new steady state).
Continuous improvement efforts are aimed at making positive changes to product
designs and to processes. These are opportunities for changing overall quality levels by
making changes in design rather than degree of conformance.
Once the late stages of product development are reached, followed by production,
distribution, and service, it is difficult and expensive, but not impossible, to effect these
changes. Estimates vary, but annual costs caused by design corrections can run into
millions of dollars at mid-sized companies (Goldstein, 1990). This is because tooling
has already been produced, production facilities have already been set up, measurement
systems have already been implemented, and poor or outmoded work procedures have
been systematized, and consequently they have become bad habits of the organization
and its employees. Making changes to deep-seated practices requires going against the
momentum of the organization. Making needed changes is more difficult when products
are largely developed and when personnel are used to working in ways that are
incompatible with new demands.
The quality profession provides many tools that can be used to effect change,
even late in the development process. In order to make such changes, personnel must be
persistent, disciplined, and properly motivated. Management must be given motivation
to support such efforts. Ideas must be justified to them in their own language, the
bottom line (Juran, 1992). Management and employees must come to an agreement on
23
how to proceed. In this context, the resistance derived from organizational
momentum can be constructive (Waddell & Sohal, 1998). It may be derived from
ambivalence (not knowing how to proceed) or from legitimate concerns about how to
proceed. Piderit (2000) stated that these are circumstances where management and
personnel need to work together and make sure that the organization is moving ahead
properly. If favorable conditions can be established, then quality personnel can work to
solve problems and initiate change, even after production has started.
Distribution, sales and service personnel can also affect quality. They can
undertake their own development measures during the CE process, and they can exercise
their own quality control and improvement activities. They can also provide information
to personnel who are directly involved with early development activities by serving as
gatherers and interpreters of data from external customers.
Quality personnel can assist the distribution, sales, and service functions by
providing technical training and by co-developing downstream quality procedures. In
accordance with the ideas of Humphreys, Williams, and Meier (1997), there is no clear
distinction between goods and services. Quality is perceived from interactions with a
product’s physical aspects and with the personnel who are involved with the customer.
Thus distribution, sales, and service personnel can play key roles in determining the
inherent quality of current products, as well as in contributing to conformance,
incremental improvement, and the engineering of future products.
To this point, the discussion has been focused on how to achieve quality by
combining the efforts of personnel from all functional areas. The next section will focus
24
on how quality management tools and techniques are used to work within such
systems in achieving quality.
Organizing to Achieve Quality
There are many quality tools available for organizations to use. Picking which
ones to use and putting them together in a system that works requires skillful leadership
and management, careful planning, focused research, and a lot of hard work from all
parties. This section contains a discussion about how quality managers put the pieces
together and create a system that suits their needs.
Achieving quality requires efforts from personnel at all organization levels and
across all functions. From strategic planning, to the creation of quality systems (tactical
planning), product development (operational planning), production, distribution, sales,
and service, all activity related to the product can benefit from the use of quality tools
and techniques. As already discussed, these activities can affect both objective measures
of product and stakeholder perceptions.
Total quality management (TQM) is a management approach that integrates
quality assurance and continuous improvement by creating company-wide quality
systems, employing effective management methods, and using appropriate quality tools.
Company-wide quality systems are comprised of documented quality management
policies and the standard operating procedures and work instructions needed to translate
those policies into action.
TQM is widely understood in principle, but defining it precisely is difficult.
Consequently there are many approaches championed by consultants and by authors in

25
the popular press, and a different implementation scheme for every company that has
ever tried it (Wilkinson & Willmott, 1996).
It is still worthwhile to study TQM. No approach can claim to be the right one,
and no list of tools can claim to be all-inclusive. Quality management can be discussed
in terms of unifying principles and tools that have worked consistently over time and in
various contexts. The value of studying TQM lies in the ability to identify and
understand its core concepts so that organizations can adapt them to their own needs.
Grandzol and Gershon (1997) conducted a study in which they attempted to
identify the most effective TQM practices for achieving various quality outcomes. They
scanned the literature and selected what they felt was the best set of indicators for
measuring the effectiveness of TQM practices. A taxonomy created by Brown,
Hitchcock, and Willard (1994) closely paralleled the criteria set forth by the MBNQA.
Both sets of criteria focused on organizational and operational aspects of achieving
quality, with a heavy emphasis on organizational effectiveness. Because of the validity
gained by the close parallels, they used the criteria from Brown et al. as test indicators in
their study. The list contained seven categories, including (a) leadership, (b) continuous
improvement, (c) internal/external cooperation, (d) customer focus, (e) learning, (f)
employee fulfillment, and (g) process management. In their study, each category served
as a composite, independent variable representing quality outcomes.
Grandzol and Gershon (1997) conducted a survey aimed at correlating TQM
practices with quality performance indicators. They concluded that TQM effectiveness
and financial quality are functions of operational factors, and that operational quality is a
function of continuous improvement. They also found that product and service quality
26
are a function of customer focus and that customer satisfaction is a function of several
variables, including employee fulfillment, customer focus, and public responsibility. By
performing additional analysis, they also formulated grounded theories: leadership
affects financial quality and operational quality, and process management affects
operational quality.
One important aspect of Grandzol and Gershon’s (1997) results is that they found
TQM effectiveness to be largely dependent upon operational issues. This runs against
the opinions and findings of others, who believe that strategic issues are also critical to
achieving quality. For example, Ackoff (1999) argued that an organization’s strategic
alignment with its environment and its internal culture is vital to improvement efforts.
The ASQ futures Team (Watson, 1998b) found that strategic issues will be important to
the practice of quality in the future. In their survey about quality management tools,
Kannan et al. (1999) found that an organization’s orientation toward quality had a
significant impact upon performance. Recent studies by Curcovic, Melnyk, Calantone,
and Handfield (2000) and by Pannirselvam and Ferguson (2001) have validated the
internal consistency of MBNQA Criteria categories. These contain six items that gauge
organizational effectiveness and one that measures business results. These criteria are
strongly tied to strategic planning and alignment.
Despite conflicting information about quality management, understanding quality
and how to manage for its achievement are important precursors to taking appropriate
action. Understanding without application does not produce results. Put within the
context of Grandzol and Gershon’s (1997) findings, the challenge is to know what tools
27
can help an organization stabilize and improve operations, facilitate customer focus,
embrace public responsibility, and help employees gain fulfillment.
Quality management embodies many powerful tools and techniques that have
been created by quality practitioners or borrowed from other fields. A list of many of
these is provided in the glossary, found in Appendix A. The glossary is based upon the
tools and techniques contained in the BOK, with many inclusions that are widely used
and that fit within its rubric.
The tools and techniques listed in the glossary are used within management
systems like those previously described. Grandzol and Gershon (1997) emphasized the
need to have a customer focus, to pay attention to public responsibility and employee
fulfillment, to achieve operational stability, and to make operational improvements.
Meeting these needs is a matter of focusing on information gathering, setting priorities,
providing leadership, etc. Attending to these issues will help to enhance quality.
Controlling production operations leads to low levels of product variation. Making
operational improvements closes the gap between specifications and product outcomes
even farther. It can also lead to incremental improvements to product designs. These
activities require companies to select, and use, the proper tools for their circumstances.
A recent study contributed insights into how organizations can select the right
tools and techniques. Jayaram, Handfield, and Ghosh (1997) associated the use of
various quality tools with quality management strategies and indicators of quality
outcomes. This differs from the approach taken by Grandzol and Gershon (1997) who
were trying to link practices with performance indicators, but not with strategies.
Jayaram, Handfield, and Ghosh surveyed quality directors in manufacturing firms,

28
asking them to rank the level of use and impact of 38 quality tools. They also asked
respondents to rank their company’s emphasis on inspection, process control, process
improvement, and design quality strategies, and to rank the importance of the following
quality attributes according to their customers: (a) aesthetics, (b) features, (c) reliability,
(d) durability, (e) function, and (f) serviceability. In their analysis, they correlated the
use of tools with the various strategies and quality outcomes that were pursued. Their
study painted a useful picture, offering insight into how companies go about striving for
quality.
There are a large number and a wide variety of available tools and techniques.
Applying them requires working cross-functionally with personnel from all areas of the
organization, project management across functional departments, tracking and using data
and information from disparate sources, training, cultural changes, synchronization of
functions, and more. Every organization needs its own unique quality system. Knowing
which strategy to employ and how to put it into practice requires reflection and
experience. Flynn, Schroeder, and Sakakibara (1995) surveyed plant management
liaisons across three industries and in two cultures to gauge self-reported data on quality
management practices. They compared this data against quality performance indicators,
finding that both high and low achieving plants used similarly sophisticated quality
management systems. Plants with mediocre quality performance typically used more
simplistic approaches. The point is that while there is plenty of information available
about quality systems and quality tools and about what works and what does not in
various cases, a quality practitioner can master all of these things and still not be able to
implement an effective quality system.

29
What differentiates between successful and unsuccessful quality managers?
Some success factors are beyond the control of an individual manager. Other factors are
directly related to the individual. Exceptional managers may be able to succeed in
achieving quality goals, even in the presence of formidable factors that are not within his
or her direct sphere of control. The ensuing discussion will address what aspects of
practice define the professional quality manager.
Quality Managers and Professional Practice
How should an organization map out a quality system that fits its needs? How
should it go about planning for, implementing, and managing that system? How can it
become capable of using the tools that are needed and put them into practice at the right
times?
Answering these questions requires planning and focused research on the part of
quality managers. Organizations are complex, and they operate in ever-changing
environments. Creating and managing a TQM system requires the proper identification
of many research questions, selecting the tools to address these concerns, simultaneous
management, information gathering and problem solving, gathering and analyzing
volumes of data, deriving meaning from all the results, and managing the resultant
information so that it serves intended purposes. It requires planning in minute detail,
which in turn demands a sophisticated system for implementation and control. How do
quality managers go about the business of helping their organizations accomplish these
tasks?
Because the answer to this question is different in every case, quality managers
must engage in reflective practice. Schon (1983/1995) described this as thinking about
30
work as it unfolds (reflection in practice) and thinking about work from the outside,
when not actually engaged in the process (reflection on practice). He believed that
reflection on practice does not lend itself to critical self-examination. However, in a
study conducted among educators, Ferry and Ross-Gordon (1998) found that reflection
on practice does lead to changes in work practice. Both aspects of reflection will be
treated as practice that can help quality managers find answers to unique problems.
According to Schon (1983/1995) reflection in practice requires a practitioner to
frame problems by drawing parallels from his/her past experiences and from the existing
body of knowledge, formulate working theories, take appropriate actions, view the
consequences of those actions, then reformulate the problem, form new theories, and
revise actions. This process continues iteratively until acceptable solutions are
generated. Schon’s model is a good framework for discussion because, as Wales, Nardi,
& Stager (1993) stressed, it is a more contemporary treatment of professional practice
and reflection. Modern treatment of the topic can be traced back to Dewey (Willower,
1994).
Schon’s (1983/1995) model presupposes four constants that a practitioner brings
into any given situation. He used the term constant in reference to a pre-existing, rather
than unchanging, set of knowledge, skills, and personal interpretation schemes.
One constant is role frame. This means that a practitioner has a view of the role
he/she must fill in the course of work situations. Some of the roles that Schon
(1983/1995) examined are those of planners, designers, diagnosticians, teachers, and
managers.
31
Another aspect of role frames is that work can be pursued from either a
win/lose or a win/win perspective. A practitioner can attempt to control situations and
aim for predetermined outcomes (win/lose), or he/she can attempt to engage all
concerned parties in producing outcomes that are mutually agreeable (Schon,
1983/1995). This requires opening up assumptions for examination by others, and using
valid information to shape decisions. The practitioner chooses which of these two
approaches to take, based upon how he/she interprets his/her place in a given situation.
Quality managers find themselves, at various times, filling all of the roles just
described. Sometimes they are problem solvers. At other times they are systems
designers and trainers. They are always acting as managers. Sometimes they are
engaging in evolving work with others, sometimes they are dictating the use of actions
prescribed by standards, customer requirements, and company policies.
Two more of Schon’s (1983/1995) constants are strongly intertwined. One is the
appreciative system. This means the context in which the practitioner frames problems,
conducts inquiries, and reflects in/on practice. The other is the overarching theories used
to make sense of what the practitioner sees.
The prevailing appreciative systems in quality management are inseparable from
the major, overarching management theories. These include, but are not limited to,
scientific management (or Taylorism), systems theory, adaptive systems, and democratic
organization (Ackoff, 1999).
Scientific management is rooted in positivism, meaning that work can be
organized using reductionist, cause-effect relationships. Teixeira (1999) traced the roots
of modern quality management back to the era when scientific management was at its
32
strongest and Shewhart introduced the use of statistical tools for reducing variability
in manufacturing processes. During this era, SPC was created and manufacturers started
using design of experiments (DOE).
As the twentieth century progressed, it became clear to many that the principles
of scientific management were not adequate to deal with all of the quality challenges
presented by production.
Systems theory, as applied to management, asserts that organizations and
processes are complex arrangements of interrelated pieces. Phenomena do not always
occur in simple cause-effect relationships, as every action on one component or one
subsystem causes ripple effects throughout the organization. No problem can be solved
without understanding and accounting for systems issues. For a detailed discussion of
this topic, see Ackoff (1999).
Many quality management tools are designed to account for system complexity.
Root cause analysis tools help to look at problems from the bottom up. Fault-tree
analysis uses a top down approach. Systems diagrams and flow charts help users map
out and make sense of complex systems. Smith (1998) provided a good discussion about
systems diagnostics, drawing parallels from diagnostics research to quality management
practices. He discussed the use of both formal methods and many informal heuristics to
diagnose quality problems. He also cautioned that experts can fall into common fallacies
when framing problems, such as ignoring cues or considering irrelevant information,
locking into causative assumptions, selectively choosing information to support
erroneously made conclusions, and framing problems within the wrong context. Quality
managers must be cautious when using their expertise to solve complex problems.
33
In addition to accounting for complexity, quality management practices such
as cross-functional product development teams and quality circles favor adaptive
systems and democratic organization theories. Adaptive organizations will self-adjust
according to changing needs, evolving through a continuing process of monitoring the
environment, reorganizing accordingly, then obtaining and processing more feedback
(Rummler & Brache, 1995). Democratic organizations have flattened management
structures. The authority for making strategic and tactical decisions is often shifted to
lower levels of the hierarchy than in more traditional organizations. All of the above
appreciative systems have appropriate applications within the practice of quality
management. None of them is sufficient as a stand-alone perspective.
The final constant proposed by Schon (1983/1995) is the media, languages, and
repertoires that practitioners bring to their work. Media used by quality managers
include (a) electronic data gathering, (b) software-based reports and analysis, (c) written
communications and documentation, (d) paper- and software-based graphical
representations, (e) manual and automated measuring equipment, and (f) verbal dialogue.
The language of quality management includes vernacular that crosses over from business
management, science-based engineering disciplines, statistics, human resources, and
industrial engineering. The BOK for quality managers, which is overseen by ASQ
(2001) is a good surrogate for the repertoire of quality managers. This is because an
ASQ advisory committee develops and conducts job analysis surveys and uses cross-
industry, expert committees to formulate the BOK. The BOK and the accompanying
certification exam are updated frequently so as to reflect current practice, with the latest
revision occurring in 2001. Prospective certified quality managers must demonstrate

34
mastery of content as organized within seven domains, contained in Part A.
Candidates must also answer open-ended questions from Part B of the BOK, which
contains five domains. The domains and their major categories are shown in Table 2 and
Table 3.
If a quality manager is a master of the BOK (in all respects) and is effective at
professional practice as outlined above, then he/she is well equipped to help an
organization adapt its strategy and operations to evolving quality needs. This does not
mean that doing so would be easy or that success would be guaranteed. Because systems
are complex and people are involved with work processes, there is a large number of
factors that will influence a situation, any one or several of which could cause change
efforts to be less successful than planned for. The BOK contains techniques, tools, and
management philosophies that help quality managers deal with complexity.
This section has focused on how companies can organize to achieve quality and
what it is that quality mangers do. The following sections will focus on the ASQ vision
for the future of quality management. Following this, the ensuing section will examine
what key forces can be considered as models and driving forces influencing whether or
not industry moves toward this vision.
Table 2
Domains and major categories in Part A of the ASQ body of knowledge for certified
quality managers
I. Leadership
A. Organizational leadership
35
Table 2 (continued)
quality managers
B. Team Processes
II. Strategy development and deployment
A. Environmental analysis
B. Strategic planning and assessment
C. Deployment
III. Quality management tools
A. Problem solving tools
B. Process management approaches
C. Measurement: Analysis and metrics
IV. Customer-focused organizations
A. Customer identification and segmentation
B. Customer relationship management and commitment
V. Supplier performance
A. Supplier selection strategies and criteria
B. Techniques for communicating requirements to suppliers
C. Techniques for assessment and feedback of supplier performance
D. Supplier improvement strategies
E. Supplier certification programs
F. Partnerships and alliances with suppliers
G. Logistics and supply chain management

36
Table 2 (continued)
quality managers
VI. Management
A. Principles of management
B. Communications
C. Projects
D. Quality system
E. Quality models
VII. Training and development
A. Alignment with strategic planning and business needs
B. Training needs analysis
C. Materials and curriculum development
D. Methods of delivery
E. Evaluating effectiveness
Table 3
Domains and major categories in Part B of the ASQ body of knowledge for certified
quality managers
I. Strategic planning and deployment
A. Formulate organizational strategic plans
B. Formulate strategic quality plans
C. Focus on performance excellence
D. Formulate quality policies and procedures

37
Table 3 (continued)
quality managers
E. Interdepartmental collaboration for deployment of strategic plans
F. Performance improvement plans in keeping with company strategy
G. Resources needed to carry out performance improvement plans
H. Collaborate on the development and delivery of training programs
II. Customer focus
A. Account for customer expectations in design and delivery of products
B. Communication with customers as input for quality systems design
C. Evaluate customer feedback for continuous improvement feedback
D. Involve customers in product/service designs, process improvements
III. Departmental or organizational management
A. Define the quality mission in keeping with the organization’s mission
B. Establish quality goals
C. Manage budget and resource requirements
D. Quality staff selection, evaluation, and development
IV. Performance assessment
A. Evaluate effectiveness of quality systems
B. Assess organizational performance
C. Feedback loops to the organization for performance information
D. Use assessment information to establish continuous improvements

38
Table 3 (continued)
quality managers
V. Supplier performance management systems
A. Develop and implement a supplier management system
B. Increase value of supply chain using supplier performance information
C. Partnering with suppliers
ASQ’s Vision for the Future
Due to increasingly stringent expectations, organizations will have to work in
new ways to achieve quality. How should quality managers assist their employers in
adapting to these circumstances? Many organizations already have quality systems in
place. All of them are capable of meeting, to lesser or greater degrees, the quality
challenges presented by current markets. Whatever the level of quality system maturity
and effectiveness an organization has, it can benefit from increasing its ability to meet
current challenges, anticipate trends, and adapt to future needs. Such an approach to
quality management can help organizations meet the challenges of higher expectations.
Because the review up to this point has focused on how companies currently
organize to meet quality challenges, this section will discuss how companies can
anticipate and adapt to emerging needs. Organizations can find cues for future needs
within the 2001 BOK. These can come from internal sources and from the external
environment. Adaptation is a whole-organization effort. The ensuing material will
discuss issues related to anticipating future needs and adapting to them.

39
Anticipating Needs
Managing change is important for organizations that are pursuing quality
objectives. The environment within which an organization operates continuously
changes. New competitors arise, public sentiment and government oversight change,
technological developments render product lines and entire industries obsolete, enhanced
quality among products offered by the competition raises the bar for future product
offerings, etc. In a recent survey among ASQ certified quality mangers, Handfield,
Ghosh, and Fawcett (1998) found a degree of support for the notion that enhanced global
competition is driving the need for quality management to be involved with company-
wide change. This suggests that, at the very least, there is awareness within the quality
management profession itself that organization-transforming change is necessary to meet
emerging quality demands.
Inherent in the discussion up to this point is that quality improvements can come
in two major fashions. Incremental, evolutionary improvements can be made to existing
products and processes, and revolutionary changes can also be made.
Whether a quality manager is pursuing small, continuous improvements, or larger
systems-wide changes, the key is that change must occur. It was just mentioned that
there is a growing awareness of this need among quality managers. Literature within the
general business management profession also acknowledges this need. The ensuing
discussion, centered on work by Ackoff (1999) and Rummler and Brache (1995)
illustrated this point. Change must occur, and if an organization wants to guide where
that change will take it, then change must be managed.

40
There are a few basic steps inherent to managing change, as opposed to letting
change happen. The first is that an organization must understand itself and its
environment within the context of the challenges that it is likely to face. This in turn
requires being able to anticipate what challenges may come to bear within the business
environment. The final step is to plan for desired results, rather than removing undesired
results.
The first step, in the terms of Ackoff (1999), is understanding the organization
and its environment. This is called dissolving, or sorting out, the mess (a term referring
to the complex relationships between behaviors inherent to a system) that it is due to its
own nature and the state of its operating environment. Typical quality managers
expend a great deal of their efforts removing undesired results, either in the form of
“putting out fires” or making a lot of incremental changes to existing products and
processes. While such tasks are important, they do not contribute to sustained, managed
change.
Rummler and Brache (1995) stated that management is the key to an
organization’s ability to adapt, and that “adaptation is a process, not an event” (p. 13).
Successfully accomplishing a discrete task (e.g., “putting out a fire” or making an
incremental improvement) is an event. Doing so changes one piece of a complex
system. If such actions are not part of a larger strategy that accounts for the “mess” an
organization is in, then it is not part of an effective, sustained change program.
As mentioned above, one of the first challenges in sorting out an organization’s
“mess” entails understanding what types of challenges are likely to occur. Techniques
for anticipating needs do not produce guaranteed results. Some techniques are more
41
rigorous than others. All techniques are less accurate as the time horizon being
examined extends into the future. The purpose of trying to anticipate needs is not to
attempt absolute forecasts but to give organizations the ability to proactively plan for
possible developments in their environments. Ackoff (1999) stated that this is a critical
part of contingency planning, and it is key to being responsive and being able to adapt.
One method that organizations use to look ahead is technology assessment. The
Office of Technology Assessment (OTA) characterized technology assessment as a
“potentially important tool for understanding the future business environment” (OTA,
1977, p. 3), generally following the steps listed below:
1. Describe the technology (in the context of this study, the quality system)
2. Define the issue under study (in this context, changing quality expectations)
3. Determine the issue’s future course (How will the environment evolve?)
4. Identify policy actions (aspects of the quality system that need to be changed)
5. Suggest alternatives
6. Assess impacts (on the products, the organization, personnel, environment, etc.)
Technology assessment uses a combination of methods, depending upon the
situation. The methods used in any situation will depend upon the type information
under study, the nature of the information source, the needed degree of precision and
accuracy, etc. There are many research tools available for conducting such assessments.
The BOK, for example, embodies several, as shown in Table 4. The Office of
Technology Assessment (1977), published a report that contained a discussion about
factors that influence the value of results (including objectivity and clarity in defining the
research question, and flexibility in selecting research methods) e.g., forecasting

42
techniques and impact studies. If organizations exercise sufficient objectivity and
rigor, they can anticipate needs with enough accuracy that the results will be of value to
them in planning adaptations.
Table 4
BOK Tools and techniques that could be used in technology assessments
Affinity diagrams Network diagrams Root cause analysis
Balanced scorecards Organizational metrics Scatter diagrams
Brainstorming Pareto analysis Self-assessment
Cause and effect diagrams PDCA Statistical analysis
Customer management Process capability studies Strategic planning and
Decision support systems PDPC deployment
Design of experiments Process mapping Stratification
Flow charts Project management S.W.O.T.
FMEA Qualitative analysis Systems diagram
Force Field Analysis Quality costs Theory of constraints
Focus groups QFD Tree diagrams
Internal capability analysis Quality system audits Trend analysis
Interrelationship diagraphs Reliability engineering Value analysis
Knowledge management Resource allocation Work flow analysis
The ASQ updated its Futures Study (Watson, 1998b) with the Foresight 2020
Study (ASQ Futures Team, n.d.) because the future unfolded differently than anticipated.
While the results of the first study were generally good, it needed to be updated after
43
three years (ASQ news, 1999). For example, the organization updated its mission,
expanding its focus. Also, the speed of progress in the information revolution was faster
than anticipated. These differences from the assumptions about the near future, as
assumed in the first study, made it necessary to re-examine its assumptions about the
future. The accuracy of ASQ predictions can in part be justified by the recent
developments in the field of quality, as will be discussed later.
The above example is a case in point that anticipating future needs requires
caution. Guidelines like those published in the ASQ reports can be used as indicators of
issues to address, as long as it is realized that guidelines for proceeding into the future
are based on predictions. Organizations must generate the best information possible for
their own situations, and adapt accordingly.
Up to this point, the discussion has focused on how to define and achieve quality,
what quality management is, and how the profession can adapt to evolving needs.
The next section will contain a discussion about how companies can determine if the
actions they take are affecting the whole organization, as they should.
Models of Organizational Quality
Adaptation requires a disciplined process for designing and implementing
change. As the ASQ has envisioned, these efforts must be strategically oriented and
aimed at all aspects of work over the whole organization (ASQ Futures Team, n.d.,
Watson, 1998b). This is why it is so important for quality managers and their
organizations to engage in reflective practice. Working in teams that span the
organization, they must use a variety of perspectives to frame the situation, drawing from
current theory and from experience, and then iteratively reframe the problem as solutions
44
are conceptualized and refined. They must test, modify, and retest ideas until
acceptable solutions are achieved.
How can companies gauge the effectiveness of their quality improvement
efforts? Two quality documents stand out as candidates for being a standard in assessing
the effectiveness of quality efforts across the organization. The ISO 9001:2000 standard
and the MBNQA Criteria are widely understood and their principles are widely applied.
They are cross-industry quality system models that have proliferated sufficiently to be
used in such a manner. The systems have much in common both in terms of philosophy
and content. They also have significant differences.
Both ISO 9001:2000 and the MBNQA Criteria are organized according to
categories, divided along operational and strategic lines. ISO 9001: 2000 is organized
into eight clauses and an introduction. According to Cianfrani et al. (2001), each clause
addresses various aspects of the quality system. The MBNQA Criteria are organized
into criteria categories, of which there are seven. Six are focused on achieving quality;
the seventh is focused on results (U.S. Department of Commerce, 2002).
The ISO 9001: 2000 standard addresses much of what is in the BOK. It is
prescriptive in nature, requiring companies that wish to register to undergo an audit in
which quality systems are compared against the requirements of the standard. While it
does focus to an extent on the organization, its primary focus is on planning for, and
achieving product conformance. Its requirements focus mostly on operational issues at
the product development and production levels. The clauses and their main subsections
are summarized in Table 5.

45
Table 5
Overview of the ISO 9001:2000 clauses
Clause Description
0-3 Introduction, Scope, Normative Reference, and Terms and Definitions:
These clauses do not have any requirements
4 Quality management system
4.1 General requirements
4.2 Documentation requirements
5 Management responsibility
5.1 Management commitment
5.2 Customer focus
5.3 Quality policy
5.4 Planning
5.5 Responsibility, authority, and communication
5.6 Management Review
6 Resource Management
6.1 Provision of resources
6.2 Human resources
6.3 Infrastructure
6.4 Work environment
7 Product realization
7.1 Planning of product realization

46
Table 5 (continued)
Overview of the ISO 9001:2000 clauses
7.2 Customer-related process
7.3 Design and development
7.4 Purchasing
7.5 Production and service provision
7.6 Control of monitoring and measuring devices
8 Measurement, analysis, and improvement
8.1 General
8.2 Monitoring and measurement
8.3 Control of nonconforming product
8.4 Analysis of data
8.5 Improvement
The MBNQA Criteria are not a standard. They are part of the Malcolm Baldrige
National Quality Award, a voluntary system administered by the Baldrige national
Quality Program, within the National Institute of Standards and Technology (NIST), a
division of the U.S. Department of Commerce. The organizational quality model was
originally developed after passage of the Malcolm Baldrige national Quality Act in 1987,
and it has been revised continually since that time. Companies that wish to apply for the
award must submit extensive documentation as evidence that their business systems
merit recognition according to the criteria. Applications are scored by certified

47
examiners (U.S. Department of Commerce, 2002). The award criteria are summarized
by category and sub-category in Table 6.
Table 6
Overview of the MBNQA Criteria categories
Category Description
P Preface: Organizational profile: This section does not contain any
requirements, but evidence submitted for requirements is judged against
material provided by the organization in response to this section
1 Leadership
1.1 Organizational leadership
1.2 Public responsibility and citizenship
2 Strategic planning
2.1 Strategy development
2.2 Strategy deployment
3 Customer and market focus
3.1 Customer and market knowledge
3.2 Customer relationships and satisfaction
4 Measurement, analysis, and knowledge management
4.1 Measurement and analysis of organizational performance
4.2 Information management
5 Human resource focus
5.1 Work systems

48
Table 6 (continued)
Overview of the MBNQA Criteria categories
5.2 Employee education, training, and development
5.3 Employee well-being and satisfaction
6 Process management
6.1 Product and service processes
6.2 Business processes
6.3 Support processes
7 Business results
7.1 Customer-focused results
7.2 Financial and market results
7.3 Human resource results
7.4 Organizational effectiveness results
What makes the MBNQA Criteria a significant national model for organizational
quality? Since the inception of the award program relatively few companies have
applied for the award, and very few have won it. However, it is used by many
companies as a self-assessment tool. As mentioned previously, the U.S. Department of
Commerce (2001) estimated that over two million copies of the criteria have been
circulated since the program began, and the benefits to cost ratio for companies that
undergo such efforts has been estimated at $2.17 billion to $119 million. As of the time
of this review, 44 states have developed award systems based upon the MBNQA
Criteria.
49
Recent studies have lent credence, to a certain degree, to the validity of the
Baldrige model for organizational quality, an issue that will be addressed shortly. For
these reasons, the MBNQA Criteria are a viable contender to be considered for use as a
model of gauging the effectiveness of quality management efforts across the
organization.
Which of the two models is better suited as a tool for gauging the effectiveness of
organizational quality efforts? The ISO 9001: 2000 standard is a requirement to do
business in many industrial sectors, both in the United States and internationally. The
number of companies that have applied for the Baldrige Award does not compare with
the number of companies that have registered or attempted to register under the ISO
system. Additionally, the ISO standard does contain much of the same substance as the
BOK. One way to address the question is to compare ISO 9001: 2000 and the Baldrige
Criteria with the Part A of the BOK. Part B of the BOK is not included in this analysis
because its content is redundant with that of Part A, being organized separately in the
BOK as a means of creating two types of certification exam questions, rather than as a
means of introducing or organizing additional subject matter. The analysis matrix is
shown in Appendix Table B1.
The analysis matrix contains only those associations that can be specifically
drawn. In the case of both ISO 9001:2000 and the MBNQA Criteria, there are many
items that related in subject matter to areas in the BOK, even though there is no specific
mention of the BOK concept or tool in question. In such cases organizations following
either quality model often include, in their efforts, items that are contained in the BOK.
Yet when items are omitted in any type of standard or requirement, then it is possible to
50
comply without using such items. While omissions can be oversights, they can also
be by design. Purposeful omissions can indicate either a desire to allow flexibility or a
real difference in focus and philosophy. Whatever the reason for the various omissions,
no match was indicated if none was specified in the documents (except in a few special
circumstances, as noted in Appendix Table B1).
Table six shows that the BOK is more comprehensive than either ISO 9001: 2000
or the MBNQA Criteria. Both the Baldrige Criteria, and ISO 9000:2001, have content
areas not covered by the other, and both are lacking items from the BOK. Both sets of
criteria lack references to important traditional staples of the quality management tool
set, including statistical analysis, the seven quality control tools, the seven planning and
management tools, and quality function deployment. As pointed out by Tonk (2000), the
similarities between all three models are not an accident. The ASQ is involved with
development of both ISO standards and the Baldrige Criteria. Yet because ISO, and the
department of Commerce are separate bodies, the content of their quality models is
ultimately shaped by the objectives of their parent organizations.
ISO 9001:2000 is a significant model for organizational quality. As Appendix
Table B1 shows, it matches up with the BOK very closely. The ISO Technical
Committee 176 is responsible for the ISO 9001:2000 standard. According to TC 176
(ISO TC 176, n.d. a) the standard’s founding principles include (a) customer focus, (b)
leadership, (c) involvement of people, (d) a process approach, (e) a systems approach to
management, (f) continual improvement, (g) factual approach to decision making, and
(h) mutually beneficial supplier relationships. These principles are strongly aligned with
BOK principles. The significance of the standard goes beyond its similarity to the BOK.
51
Because it is an international standard, it has become a requirement for many
companies that wish to do business in most significant industry sectors. As of the end of
the year 2000, over 400,000 companies in 158 countries had successfully registered to
ISO 9000 (ISO TC 176, n.d. b).
The MBNQA Criteria are also a significant quality model. Even though they
also do not match up with the BOK item-for-item, they stand by themselves as a
significant model of organizational effectiveness. As previously mentioned, the criteria
have been widely, and successfully, applied as organizational improvement guidelines.
Recent studies have also suggested that the criteria might be a consistent set of
constructs for determining organizational effectiveness. Pannirselvam and Ferguson
(2001) found that the model for the MBQNA criteria is internally consistent, meaning
that the categories measure what they are intended to measure and that collectively they
are a fair representation of the efforts that are required to obtain results as identified in
the seventh criteria category. If these findings are valid, then quality goals can be set in
terms that will profit the spectrum of stakeholders within, and external to, the
organization.
In their study, Pannirselvam and Ferguson (2001) used examiners from the
Arizona Governor’s Quality Award, an award based on the MBQNA, to score the
information supplied by all companies that applied during 1993. The authors examined
the scores for the various categories, and for the performance data, as reported in the
application packets. After analysis, they found that quality management activities in the
various criteria categories explained the variability, either directly or indirectly, in
business results and customer satisfaction.

52
Other recent studies have examined the MBNQA Criteria. Curcovic et al.
(2000) surveyed quality managers in the automotive industry, at the plant level, to
determine their opinions about the content of the Baldrige Criteria and of TQM issues as
well. Constructs for each (Baldrige and TQM) were represented by questions in the
survey instrument, which was developed by, and pre-tested by, quality managers in the
automotive industry. Survey results were checked for content validity and the authors
used structural equation modeling to map MBNQA Criteria constructs to TQM
constructs. They found, as did Pannirselvam and Ferguson (2001), that the Baldrige
Criteria categories are internally consistent. They also found that the categories can be
successfully mapped to TQM constructs.
Wilson and Collier (2000) also studied the MBNQA model. In a manner similar
to Curcovic et al. (2000), they created a survey instrument, checked it for validity, and
administered it to managers in industry. They were able to identify causal models
leading to financial and business results in which leadership was the driving factor that
led to results, as mediated by various Baldrige elements.
These studies will not end debates about the validity of the MBNQA model or
about the profitability of quality management initiatives. While Pannirselvam and
Ferguson (2001) did use real evidentiary packages from award applicants, data was only
at the state level (from one state) and it was done using the 1993 Baldrige model.
Curcovic et al. (2000) used the 1997 Baldrige model, which is more recent. They
validated their instrument, but administered it at the plant level, which might have
missed potential input about organization-level factors. Wilson and Collier (2000) also
validated their instrument and administered it to respondents at the company level, but
53
their instrument was based upon the 1995 MBNQA model, which is a little more dated
than the model used by Curcovic et al. In all three cases, the studies used structural
equation modeling, and thus the results were dependent upon initial assumptions. The
path models produced by all three studies indicated model significance, but all three of
the path models were different. Based upon these studies, it cannot be concluded
definitively that the Baldrige model is internally consistent, or that it measures what it
claims to measure. However, the similarity in findings points out that quality and
business management goals might be made compatible, if managed properly.
Quality management systems fit integrally within business management systems,
operating in synchronization with them, drawing upon the same resources, and being
shaped in accordance with organizational design. Germain and Spears (1999) illustrated
this point in a recent study. They examined the effects, as perceived by senior managers,
of three classic organizational design factors on quality management. The first factor,
formalization, relates to the level of formal documentation in an organization.
Technocratic specialization, the second factor, is the division of non-production labor.
Strategic decentralization, the third factor, is the pushing of decision-making authority
lower in the management hierarchy. What Germain and Spears found is that these
factors completely mediate, either directly or indirectly, the shape of quality
management systems. Better quality management systems are generally associated with
higher degrees of technocratic specialization, strategic decentralization, and
formalization.
Adapting to meet evolving quality demands requires that organizations treat
quality management as a complex, technological process that is inseparable from

54
business management, a theme that runs through the BOK, ISO 9001:2000, and the
MBNQA criteria. Framing problems in this way will help managers to plan for, and
implement, quality management systems that are capable of changing in the face of
evolving demands.
Thus in addition to mastering the tools and techniques of their trade, successful
quality managers must understand the organization and its environment, anticipate and
adjust for future developments, and exercise prudent leadership and guidance. In their
study of the MBNQA criteria, Pannirselvam and Ferguson (2001) demonstrated that
business and quality goals do not have to be incompatible. This gives quality managers
the potential to influence change in terms that are desirable to all parties within the
organization.
Quality systems need to be continuously updated to meet changing demands. As
they frame problems, develop solutions, and measure results, quality managers take into
account the integration of quality and business management principles, deal with
complexity, develop an intimate understanding of the organization, and work with teams
of personnel from all functions who are involved with meeting quality objectives.
Summary
Adapting to changing needs requires quality managers to be dynamic. They must
be able to assume multiple roles. In addition to a thorough grounding in traditional
quality principles, they must have a complementary understanding of business
management and organizational theory, and a set of human relations skills. All of these
factors require successful quality managers to be exceptional individuals.

55
This review contained a discussion about quality, quality systems, professional
quality management, and the adaptation of quality management systems to evolving
expectations. The intent was to provide the context needed for understanding what
quality management will evolve into, if the ASQ Futures Team’s vision (Watson, 1998b,
ASQ Futures Team, n.d.) holds true.
The current BOK is a reflection of ASQ philosophy and of the input from subject
matter experts and practicing quality managers. It has close conceptual and content
parallels with both ISO 9001:2000 and the MBNQA Criteria.
Because ASQ has produced a vision of the near future for quality management
and has, in keeping with that vision, produced an up-to-date body of knowledge for
quality managers, it is important to look at actual practice and see if there is a reflection
of ASQ’s vision. Aside from ASQ, two prime forces that are helping to shape/reflect
quality management thought on wide scale are the ISO 9001:2000 standard and the
MBNQA Award.
This study is aimed at identifying relationships between the use of BOK
concepts, tools, and techniques by practicing quality managers and organizational
effectiveness in complying with ISO 9001:2000 and MBNQA Criteria requirements.
Other researchers have examined relationships between the use of quality tools and
associated outcomes or associated objectives. For example, studies by Grandzol and
Gershon (1997), Jayaram et al. (1997), and Kannan et al. (1999) have produced
important results using validated quality management and results models. This study
differs from those just discussed in that it is aimed at gauging the stated level of
implementation of quality management principles as they are spelled out in the latest
56
BOK. In this study, the stated levels were measured against stated levels of
organizational effectives as determined by ISO 9001:2000 and the MBNQA Criteria.
None of the models to be used in this study (The BOK, ISO 9001:2000, and the
MBNQA Criteria) are abstracted from widely applied, real-world documents.

57
CHAPTER 3
Method
The instrument was administered to quality management practitioners who are
ASQ section officers. It asked them to gauge the degree of use of the BOK concepts,
tools, and techniques at their employing organizations. It also asked them to rate the
degree of organizational effectiveness of their employers as judged by categories in ISO
9001:2000 and the Malcolm Baldrige National Quality Award (MBNQA) Criteria.
Results of these respondent ratings were analyzed in terms of the correlation between use
of items in the BOK with degree of organizational success in the various ISO 9001:2000
and MBNQA Criteria categories.
Restatement of the Problem
Restatement of Objectives
The problem of the study was addressed through four objectives:
1. Assess the progress of quality management practitioners in applying the
principles contained in the categories of the Certified Quality Manager Body of
Knowledge (BOK), as stated by section officers in the American Society for
Quality (ASQ).
58
management efforts in meeting the categorical requirements of the ISO
9001:2000 standard.
management efforts in meeting the categorical requirements of the Malcolm
Baldrige National Quality Award (MBNQA).
4. Investigate the relationships between the stated application levels from objective
one, treated as independent variables, and the stated effectiveness of quality
management efforts from objectives two and three, treated as dependent
variables, to determine if there are any possible main factor effects or
interactions.
Research Design
The study was conducted over three phases, as shown in Table 7. A timeline for
these activities is provided in Figure 1.
Table 7
Phases of the Study
Phase I. Review of Literature
i. Collect and review relevant literature
ii. Build initial draft of self-assessment tools
Phase II. Develop the Study
i. Select subject matter experts
ii. Refine self-assessment tools
Phase III. Administer Study and Analyze Results

59
Table 7 (continued)
Phases of the Study
i. Administer the instrument through a round of mailings
ii. Collect and Analyze data
iii. Generate a report of findings, including conclusions and
recommendations
Figure 1. Time line for the study
Phase I: Review of Literature
For the first phase of this study, literature was reviewed about quality
management in general and specifically in relation to the ASQ vision for the future.
Other items reviewed included materials relating to current forces (other than ASQ
itself) that are shaping thought about and practice of quality management. Sources of
60
information included journal articles, books, and electronic documents existing on the
Worldwide Web. Each resource was gathered using library resources, searches of
academic databases, and web searches.
The discussion generated by this review, and contained in this report, served as a
framework to support the need for and philosophy of this study. The outline of the
review was structured so as to build a case that supports the American Society for
Quality’s (ASQ) latest body of knowledge (BOK) for quality managers. This BOK has
been updated to reflect the findings of studies conducted by the society and other
contemporary views on quality management, such as ISO 9001: 2000.
As a consequence of the review, the BOK, ISO 9001:2000, and the MBNQA
Criteria were chosen as models for developing question sections in the survey
instrument. Each category of the BOK (Part A) will be treated as a separate construct,
and thus as an independent variable in analyzing the effectiveness of applying quality
management principles. These variables will be addressed by questions in sections of
the instrument. The requirements categories of ISO 9001:2000 and all of the MBNQA
categories will be treated as separate constructs. Each of these constructs will serve as a
dependent variable in the analysis just mentioned. The variables will be named
BOK1…BOK7, ISO1 … ISO5, and MB1…MB7. Successive variable numbers will be
tracked linearly down the lists of their associated categories, as depicted in Table 8.
These variables will be addressed by questions in sections two and three of the
instrument, respectively.
Statistical analyses were used to test hypotheses regarding the relationships
between independent and dependent variables, and between demographics items and all
61
variables. In the following hypotheses, independent variables were labeled BOKi,
where i = 1 … 7. Two-way interactions between BOK variables were labeled as
BOKxBOKy, where x = 1 … 7 and y = 1 … 7. The two sets of dependent variables were
labeled ISOj and MBk, where j = 1 … 5, and k = 1 … 7. Demographics items were
labeled DEMa, where a = 1 … 10.
ANCOVA tests were performed to determine whether the regression equations
between independent and dependent variables contained any non-zero regression
coefficients. The regression model contained all main effects and two-way interactions.
Hypotheses one and two tested regression models in which the regression coefficients
for main effects were labeled Cn, where n = 1 … 7. For two-way interactions, regression
coefficients were labeled Cxy, where x = 1 … 7 and y = 1 … 7. The regression equations
represent assumed relationships between independent and dependent variables. Listed
are the regression equations, the associated hypotheses, and the associated alternative
hypotheses that were tested:
7
ISOn  C1 BOK1  C 2 BOK 2  ...  C7 BOK 7  C
x , y 1
xy BOK x BOK y
H 10 : All C n and C xy  0 *
H 11 : C n  0 for at least one n or C xy  0 for at least one xy
7
MBn  C1 BOK1  C2 BOK 2  ...  C7 BOK 7  C
x , y 1
xy BOK x BOK y
H 2 0 : All C n and C xy  0*

62
*E.G., if all regression coefficients equaled zero, then there were no
significant relationships in the assumed regression equations.
The relationships between demographics items and all variables were tested
using Spearman’s Rho correlation test, and other methods as appropriate. The elements
in hypotheses three through five followed the labeling conventions mentioned above.
Listed are the null and its alternative for the demographics hypotheses that were tested:
H30: ISOn is not correlated with DEMa
H31: ISOn is correlated with DEMa
H40: MBn is not correlated with DEMa
H41: MBn is correlated with DEM a
H50: BOKn is not correlated with DEMa
H51: BOKn is correlated with DEMa
Table 8
Variable names and descriptions
Variable Type Description
BOK1 Independent Leadership

BOK2 Independent Strategy development and deployment
BOK3 Independent Quality management tools
BOK4 Independent Customer-focused organizations
BOK5 Independent Supplier performance
BOK6 Independent Management
BOK7 Independent Training and development
ISO1 Dependent Quality management system
63
Table 8 (continued)
Variable names and descriptions
ISO2 Dependent Management responsibility

ISO3 Dependent Resource management
ISO4 Dependent Product realization
ISO5 Dependent Measurement, analysis, and improvement
MB1 Dependent Leadership
MB2 Dependent Strategic planning
MB3 Dependent Customer and market focus
MB4 Dependent Measurement, Analysis, and Knowledge Management
MB5 Dependent Human resource focus
MB6 Dependent Process management
MB7 Dependent Business results
Phase II: Develop the Study
The survey was designed with a main section to collect data about the
independent and dependent variables, and another section to collect demographic data.
The demographics portion of the instrument was designed to collect information about
the background of respondents and the nature of their employers. The main section
asked respondents to assess the level of performance achieved by their employing
organization, using two categories of statements.
The first category of statements was drawn from the BOK and was used to
determine stated levels of application for quality management principles. Each sub-
category was represented by an item in the questionnaire. Average responses for each
64
sub-category item within a given category represented the stated level of application
for that category, or independent variable.
Questions in the main section asked respondents to rate their employers’ level of
effectiveness in complying with the ISO 9001:2000 and MBNQA Criteria categories
respectively. As with questions in the BOK section, each sub-category represented an
item on the questionnaire. The categorical scores were averages of the responses for all
sub-categorical items.
All questions were included in an initial draft of the survey. This document was
held ready for further development, pending approval of the study by the dissertation
committee and the Human Subjects Review Board (HSRB) at Bowling Green Sate
University (BGSU).
In addition to a draft of the survey document, the HSRB required a solicitation
letter and an informed consent form. These documents were developed, and the HSRB
granted approval to proceed (and subsequently continued that approval for another year).
A copy of the HSRB approval is included in Appendix C.
Questionnaire Refinement Using a Modified Delphi Process
After the dissertation committee approved the proposed study and the HSRB
granted approval to proceed, the instrument was refined by a panel of subject matter
experts. They were selected from a pool of candidates who were chosen according to
recommendations by officers in the Toledo Section of ASQ. Panel members were
selected according to the following criteria:
1. Must be a certified quality manager through ASQ
2. Must have at least ten years of experience at the quality manager level
65
3. Must be an exemplary manager who has demonstrated effective quality
management practices during his/her career, and who possesses a high degree of
subject matter expertise in quality management principles (particularly the
quality management BOK and the MBNQA Criteria).
Candidates were screened to verify their qualifications, to insure that they had
computer access, and verify computer literacy as follows:
1. The ability to send and receive email with attachments
2. The ability to read and edit Microsoft Word files
3. The ability to use web-based bulletin boards
4. The ability to use web-based chat
5. Access to a computer with internet access and a Java-enabled browser such as
Internet Explorer or Netscape
The panel was given a copy of the initial draft of the instrument and asked to
review it prior to participation. After a review period, panelists achieved majority
consensus on needed revisions, using a modified Delphi process. Each panelist had
copies of the instrument, the BOK, and the Baldrige Criteria throughout his/her
participation.
A modified Delphi process was conducted via the WorldWide Web. Panelists
reviewed items in the instrument for appropriateness and accuracy. The process they
followed is outlined below:
1. Panelists posted proposed statement modifications or replacement statements on
a web-based bulletin board.
2. They rated each other’s items on a five-point scale for perceived importance.
66
3. The average results of these scores were posted, and panelists were given a
chance to post responses to any item he/she felt were important, but that did not
score highly.
4. Panelists re-scored all items, once again using a five-point scale for importance,
based upon any responses posted in defense of items with low scores.
5. Final average scores were calculated, and all items scoring higher than 2.5 were
included in a subsequent round of scoring.
Rowe and Wright (1999) discussed a possible shortcoming associated with the
Delphi method that was addressed by allowing for discussion during rounds. The
concern was that participants might conform to group opinion rather than reach a
consensus. Allowing participants the opportunity to defend opinions and reflect on such
feedback from others helped to move the results closer to consensus opinion.
The Delphi process was continued for three rounds. This is in keeping with the
suggestions of Brockhoff (1975), who stated that the best results are usually obtained
after three rounds, and that further rounds may make the results worse. After the third
round descriptive statistics were generated for each item. Following the suggestion of
Scheibe, Skutsch, and Schofer (1975), achievement of consensus was judged using a
threshold value in which the inter-quartile range was equal to 1/5 of the total point scale.
In this study, the threshold will thus be 1.00. Consensus was deemed stable if the inter-
quartile range was less than, or equal to, 1.00 for an item after the third round. If this
criterion passed and the score for the item was greater than, or equal to, 2.5, then the
proposed change was made. A transcript of the Delphi panel discussions is included in
Appendix D.
67
Time Testing the Revised Questionnaire
The completed instrument was administered to a sample of 10 quality managers
in order to determine the average participation time. The HSRB required that this
information be included in the solicitation letter.
The sample size of 10 was determined by using an operating characteristic (OC)
curve for a two-tailed T-test and an alpha level of 0.05, using the method presented in
Montgomery (2001). The standard deviation for instrument completion time was an
unknown. But using the assumption that the difference between the slowest and fastest
participants was going to be 20 minutes, and that this time interval would represent +/- 3
standard deviations, two times the standard deviation was estimated at 6.67. The goal
was to estimate the mean completion time within +/- 10 minutes. The discrimination
ratio for the OC curve was 10/6.67, or 1.5. Looking this information up on an OC table,
the n-value was approximately 8, leading to a sample size of 5 (rounded up). In order to
add extra precision, 10 quality managers were sampled.
Of the original 10 test participants, 7 returned a completed survey. Their average
completion time was 9.4 minutes. The time quoted on the solicitation letter was 20
minutes or less, allowing for the 10 minute uncertainty surrounding the mean.
Once the anticipated completion time was added to the solicitation letter, the
survey packet was ready to be administered. A copy of the survey instrument and the
rest of the packet is included in Appendix E.

68
Phase III: Administer the Study and Analyze Results
Administer the Study
The questionnaire was administered to a sample of quality management
practitioners who were ASQ section officers, or who were recommended as substitutes
by their executive committees. The initial intent was to send solicitations to executive
committee chairpersons. However, ASQ does not share contact information for its
members, so solicitations to participate were sent to chairpersons when their contact
information could be found. If a chairperson could not be found, another committee
member was solicited, based upon who could be located.
The solicitation asked each committee member to choose a designated participant
from his/her section if he/she did not meet the guidelines, was unable to participate, and
could not recommend another committee member.
The guidelines for being eligible to participate included, in addition to those just
mentioned, that the respondent must be a quality management practitioner who is a
certified quality manager (CQM). The latter of these two requirements was changed to a
recommendation guideline in order to ensure a better response rate. This decision
reflected the difficulty I initially had finding qualified Delphi panelists to participate in
the questionnaire refinement process. In order to compensate for this change, the survey
instrument asked respondents whether or not they were currently a CQM. The intent
was to be able to check for correlations between a respondent’s status as a CQM and
his/her answers to variables response items.
The survey was directed at executive committee members because of the
influential role they play in the development of quality management practice. ASQ
69
sections engage in professional development, basic education, and informal
networking activities. Studies in the field of management suggest there may be a
positive relationship between external, professional activity and the diffusion of
knowledge within an employing organization.
For example, in a study conducted at a research facility, Tushman and Scanlan
(1981) found that boundary-spanning individuals are influential in the diffusion of
knowledge about practice within an organization. Boundary spanning refers to the
ability of an individual to maintain both external associations and internal networks. In a
study conducted among production planning and control personnel, Newell and Swan
(1995) found results similar to Tushman and Scanlan. They found that when individuals
with strong external and internal links engage in professional development, such activity
explains a portion of the variability in the implementation of technologies within an
organization.
The sample for this study was selected to access individuals who have strong
internal ties within their employing organizations and are influential participants in
external professional networks. Quality managers are networked within their own
organizations by virtue of their positions and the nature of their work. Section executive
committee members are influential members of external networks.
The ASQ encourages quality managers to develop cross-disciplinary skills and to
work with personnel from all functional areas to implement quality principles (Watson,
1998b). These are both characteristics of boundary spanning individuals. ASQ section
chairpersons, or officers who are designated as substitutes, will be treated as sampling
units who are influential in the practice of boundary-spanning quality managers.

70
The instrument was distributed via U.S. mail, when possible, and by email
when only email contact information was available. A second round of mailings was
sent to individuals who did not respond to round one. A third round was sent to
individuals who still had not responded after round two. Rounds two and three were also
distributed using a mixture of Us Mail and email delivery.
Because no contacts list was available, a contacts database was built. The
primary method for gathering contacts was to follow links from the ASQ web site to
sites maintained by individual sections. Other methods included conducting web
searches for section pages not linked to the ASQ site and finding an online copy of the
ASQ Organization Manual: July 2002-03 (2002), both of which added significant
numbers of contacts.
Building the contacts database was an ongoing process. At the time of the first
mailing (April 2003), executive committee members at approximately half of the
sections were still unaccounted for. The collection of contacts continued throughout the
administration of the questionnaire. Solicitations to each section contact were tracked
and continued throughout the spring and summer of 2003. Each section was approached
3 times prior to the end of the data collection period in October 2003. In the case of
negative and non-responses, new contacts were developed, when possible.
At the time of the study, ASQ maintained 252 sections. When the study was
proposed, the number was 247, but a new list with 5 additional sections was published.
Of the 252 sections, 244 were solicited. Committee members from 8 sections were not
solicited because contact information was never found.

71
Analyze the Results
The goals of the data analysis were to generate the following:
1. Descriptive statistics regarding the stated levels of implementation of concepts
contained in the BOK (organized by the categories of Part A). These statistics
will be grand averages of responses to each item in a particular category. All
responses will be provided on a five-point Likert scale
2. Descriptive statistics regarding the stated levels of effectiveness of quality
management efforts as judged by ISO 9001:2000 requirements categories.
3. Descriptive statistics regarding the stated levels of effectiveness of quality
management efforts as judged by the requirements in the MBNQA Criteria
4. Descriptions of the main effects, and interactions between effects, of quality
management practices contained in BOK categories (independent variables) upon
organizational effectiveness, as judged by ISO 9001:2000 and MBNQA Criteria
categories (two sets of dependent variables)
The analysis strategy and techniques used to achieve these objectives are discussed in
Chapter 4.
72
CHAPTER 4
Results
Data collection continued until October 10, 2003. The total number of completed
surveys was 88. This represented 34.9% of all North American sections, and 36.1% of
the sections solicited.
The data was analyzed using SPSS 11.5. Responses to variables questions and
demographic responses were coded and entered into the data table. For example, the
BOK category 1, Leadership, was composed of two response items. These were coded
as BOK1A and BOK1B. Each independent, and dependent, variable was defined in this
manner, as the mean of all scores for its sub-categorical items. Table 9 lists each
variable and the items it is composed of. A complete data table is located in Appendix F.
Demographics responses were also coded and entered into the data table. Table
10 lists the codes and coding schemes used for each demographics response item.
The data analysis was conducted using the following strategy, which served as an
outline for the ensuing discussion:
1. Generating descriptive statistics and checking normality assumptions
2. Data transformation
3. Analysis of the data for model fit
4. Analysis of refined model using ANCOVA

73
5. Analysis of demographics data
Table 9
Mathematical definitions of independent and dependent variables
Variable Definition
BOK1 MEAN(BOK1A,BOK1B)
BOK2 MEAN(BOK2A,BOK2B,BOK2C)
BOK3 MEAN(BOK3A,BOK3B,BOK3C)
BOK4 MEAN(BOK4A,BOK4B)
BOK5 MEAN(BOK5A,BOK5B,BOK5C,BOK5D,BOK5E,BOK5F,BOK5G)
BOK6 MEAN(BOK6A,BOK6B,BOK6C,BOK6D,BOK6E)
BOK7 MEAN(BOK7A,BOK7B,BOK7C,BOK7D,BOK7E)
ISO1 MEAN(ISO1A,ISO1B)
ISO2 MEAN(ISO2A,ISO2B,ISO2C,ISO2D,ISO2E,ISO2F)
ISO3 MEAN(ISO3A,ISO3B,ISO3C,ISO3D)
ISO4 MEAN(ISO4A,ISO4B,ISO4C,ISO4D,ISO4E,ISO4F)
ISO5 MEAN(ISO5A,ISO5B,ISO5C,ISO5D,ISO5E)
MB1 MEAN(MB1A,MB1B)
MB2 MEAN(MB2A,MB2B)
MB3 MEAN(MB3A,MB3B)
MB4 MEAN(MB4A,MB4B)
MB5 MEAN(MB5A,MB5B,MB5C)
MB6 MEAN(MB6A,MB6B)
MB7 MEAN(MB7A,MB7B,MB7C,MB7D,MB7E,MB7F)
74
Table 10
Demographics response items
Code Description
CQM 0 not a Certified quality manager
1 certified quality manager
EXP Number of years employed as a quality manager
NCERT Number of ASQ certifications held
RESP 1 No decision making authority
2 Single location – department or other unit

3 Single location – top management
4 Division or business unit
5 Organization management – single unit
6 Organization management – multiple units
7 Other (please specify)
EDU Number of years of education beyond high school
EMPL Number of employees at unit or organization:
1 1-4 7 500-999
2 5-9 8 1000-1499
3 10-19 9 1500-2499
4 20-49 10 2500-4999
5 50-99 11 5000-9999
6 100-499 12 10000+
DIRECT Number of employees under respondent’s direction

NAICS First two numbers of NAICS industrial code
REGION ASQ region that respondent’s section is located in (added after the fact by
extrapolation from respondent tracking data)
GENDER Added after the fact by extrapolation from respondent tracking data
75
Checking Normality Assumptions
Normality assumptions were checked using the approach suggested by Field
(2000). His first recommendation was to use a measurement scale with equal intervals
and make efforts to insure that data from different subjects is independent.
The data for this study was collected using a five-point Likert scale that was
designed to maintain equal intervals. One of the assumptions of this survey was that
respondents would answer questions objectively and candidly. If this assumption was
valid, then the combination of its validity and the Likert response scale provided a means
for establishing a scale with equal intervals.
Responses from each participant were independent from the responses of others
because each respondent was separated by geographic region from all other respondents.
Each person answered the questions in regards to a different place of employment, and
respondents had no interaction with each other.
Field (2000) also suggested that after the data is collected, descriptive statistics
and histograms should be examined and normality tests should be run. Descriptive
statistics and histograms were generated for each independent and dependent variable, as
shown in Table 11. A t-test was used to check for significant differences between the
mean responses to each variables item and the value 4.0 (which corresponded to the
response Good on the questionnaire). These results are shown in Table 12. Each
variable was tested for normality using the Kolmogorov-Smirnov and Shapiro-Wilk
tests.
76
Table 11
Descriptive statistics
BOK1 BOK2 BOK3 BOK4 BOK5

N Valid 88 88 88 88 88
Missing 0 0 0 0 0
Mean 3.915 3.542 3.902 4.074 3.332
Std. Error of Mean .0886 .0866 .0926 .0968 .0856
Median 4.000 3.667 4.000 4.250 3.429
Mode 3.5(a) 4.0 5.0 5.0 4.0
Std. Deviation .8312 .8123 .8683 .9083 .8030
Variance .6909 .6598 .7539 .8249 .6449
Skewness -.567 -.336 -.374 -1.002 -.152
Std. Error of Skewness .257 .257 .257 .257 .257
Kurtosis -.027 -.739 -.829 .559 -.375
Std. Error of Kurtosis .508 .508 .508 .508 .508
Range 3.5 3.3 3.0 3.5 3.7
Minimum 1.5 1.7 2.0 1.5 1.3
Maximum 5.0 5.0 5.0 5.0 5.0
a Multiple modes exist. The smallest value is shown
BOK6 BOK7 ISO1 ISO2 ISO3

N Valid 88 88 88 88 88
Missing 0 0 0 0 0
Mean 3.814 3.240 4.239 3.934 3.724
Std. Error of Mean .0846 .0965 .0808 .0872 .0829
Median 4.000 3.200 4.000 4.167 3.750
Mode 4.0 3.4 4.0(a) 5.0 4.0
Std. Deviation .7938 .9049 .7580 .8179 .7778
Variance .6301 .8188 .5746 .6689 .6049
Skewness -.420 -.345 -1.303 -.758 -.357
Kurtosis -.741 -.089 3.120 .187 -.499
Range 3.0 4.0 4.0 3.5 3.0
Minimum 2.0 1.0 1.0 1.5 2.0
Maximum 5.0 5.0 5.0 5.0 5.0
77
Table 11 (continued)
Descriptive statistics
ISO4 ISO5 MB1 MB2 MB3

N Valid 88 87 85 87 88
Missing 0 1 3 1 0
Mean 3.850 3.886 3.805 3.535 4.092
Std. Error of Mean .0725 .0859 .0972 .1003 .0840
Median 4.000 .000 4.000 3.500 4.000
Mode 4.0 4.0 4.0 4.0 5.0
Std. Deviation .6799 .8058 .9066 .9251 .7833
Variance .4622 .6494 .8218 .8559 .6135
Skewness -.561 -1.120 -.690 -.406 -.517
Kurtosis .049 2.178 .186 -.017 -.586
Range 3.2 4.0 3.5 4.0 3.0
Minimum 1.8 1.0 1.5 1.0 2.0
Maximum 5.0 5.0 5.0 5.0 5.0
MB4 MB5 MB6 MB7

N Valid 87 87 87 87
Missing 1 1 1 1
Mean 3.638 3.414 3.546 3.688
Std. Error of Mean .0993 .0934 .0941 .0800
Median 4.000 3.667 4.000 3.833
Mode 4.0 3.7(a) 4.0 4.0
Std. Deviation .9266 .8716 .8781 .7463
Variance .8587 .7596 .7711 .5569
Skewness -.640 -.306 -.816 -.557
Std. Error of Skewness .258 .258 .258 .258
Kurtosis .258 -.181 1.161 .466
Std. Error of Kurtosis .511 .511 .511 .511
Range 4.0 3.7 4.0 3.7
Minimum 1.0 1.3 1.0 1.3
Maximum 5.0 5.0 5.0 5.0
78
Table 12
One sample t-test, comparing mean variables responses to the value 4.0 (Good)
95% Confidence
Standard Sig. Mean Interval of the
N Mean Dev. t df 2-tailed Diff. Difference
Lower Upper
BOK1 88 3.915 .8312 -.962 87 .339 -.085 -.261 .091
BOK2 88 3.542 .8123 -5.293 87 .000 -.458 -.630 -.286
BOK3 88 3.902 .8683 -1.064 87 .290 -.098 -.282 .085
BOK4 88 4.074 .9083 .763 87 .448 .074 -.119 .266
BOK5 88 3.332 .8030 -7.803 87 .000 -.668 -.838 -.498
BOK6 88 3.814 .7938 -2.198 87 .031 -.186 -.354 -.018
BOK7 88 3.240 .9049 -7.875 87 .000 -.760 -.951 -.568
ISO1 88 4.239 .7580 2.953 87 .004 .239 .078 .399
ISO2 88 3.934 .8179 -.760 87 .449 -.066 -.240 .107
ISO3 88 3.724 .7778 -3.324 87 .001 -.276 -.440 -.111
ISO4 88 3.850 .6799 -2.064 87 .042 -.150 -.294 -.006
ISO5 88 3.886 .8058 -1.329 87 .187 -.114 -.285 .057
MB1 87 3.805 .9066 -2.010 86 .048 -.195 -.389 -.002
MB2 85 3.535 .9251 -4.631 84 .000 -.465 -.664 -.265
MB3 87 4.092 .7833 1.095 86 .277 .092 -.075 .259
MB4 87 3.638 .9266 -3.645 86 .000 -.362 -.560 -.165
MB5 87 3.414 .8716 -6.274 86 .000 -.586 -.772 -.400
MB6 87 3.546 .8781 -4.823 86 .000 -.454 -.641 -.267
MB7 87 3.688 .7463 -3.903 86 .000 -.312 -.471 -.153
Each variable was checked for normality both visually and analytically. The
following methods were used.
1. Histograms were plotted for each variable.
2. Skewness and kurtosis values for each variable, shown in Table 13, were
converted to Z-scores and checked for significance with a cutoff of 1.96.
3. The Kolmogorov-Smirnov and Shapiro-Wilk tests were used with significance
set at p < 0.05.

79
Histograms
Histograms were plotted for each variable, with normal curves superimposed
over the bars. Figure 2 shows each graph.
Variable BOK1: Leadership
20
Frequency
10
Std. Dev = .83

M ean = 3.91
0 N = 88.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK1
Figure 2. Histograms of all variables

80
Variable BOK2: Strategy Development and Deployment
30
20
Frequency
10
Std. Dev = .81

M ean = 3.54
0 N = 88.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK2
Variable BOK3: Quality Management Tools
30
20
Frequency
10
Std. Dev = .87

M ean = 3.90
0 N = 88.00
2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK3
Figure 2 (continued). Histograms of all variables

81
Variable BOK4: Customer-Focused Organizations
30
20
Frequency
10
Std. Dev = .91

M ean = 4.07
0 N = 88.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK4
Variable BOK5: Supplier Performance
30
20
Frequency
10
Std. Dev = .80

M ean = 3.33
0 N = 88.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK5

82
Variable BOK6: Management
30
20
Frequency
10
Std. Dev = .79

M ean = 3.81
0 N = 88.00
2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK6
Variable BOK7: Training and Development
30
20
Frequency
10
Std. Dev = .90

M ean = 3.24
0 N = 88.00
1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
BOK7

83
Variable ISO1: Quality Management System
50
40
Frequency
30
20
10 Std. Dev = .76

M ean = 4.2
0 N = 88.00
1.0 2.0 3.0 4.0 5.0
ISO1
Variable ISO2: Management Responsibility
30
20
Frequency
10
Std. Dev = .82

M ean = 3.93
0 N = 88.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
ISO2

84
Variable ISO3: Resource Management
30
20
Frequency
10
Std. Dev = .78

M ean = 3.72
0 N = 88.00
2.00 2.50 3.00 3.50 4.00 4.50 5.00
ISO3
Variable ISO4: Product Realization
20
Frequency
10
Std. Dev = .68

M ean = 3.85
0 N = 88.00
1.75 2.25 2.75 3.25 3.75 4.25 4.75
2.00 2.50 3.00 3.50 4.00 4.50 5.00
ISO4

85
Variable ISO5: Measurement, Analysis, and Improvement
40
Frequency 30
20
10
Std. Dev = .81
M ean = 3.89
0 N = 88.00
1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
ISO5
Variable MB1: Leadership
30
20
Frequency
10
Std. Dev = .91

M ean = 3.80
0 N = 87.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
MB1

86
Variable MB2: Strategic Planning
40
30
Frequency
20
10
Std. Dev = .93
M ean = 3.5
0 N = 85.00
1.0 2.0 3.0 4.0 5.0
MB2
Variable MB3: Customer and Market Focus
30
20
Frequency
10
Std. Dev = .78

M ean = 4.09
0 N = 87.00
2.00 2.50 3.00 3.50 4.00 4.50 5.00
MB3

87
Variable MB4: Measurement, Analysis, and Knowledge Management
50
40
Frequency
30
20
10 Std. Dev = .93

M ean = 3.6
0 N = 87.00
1.0 2.0 3.0 4.0 5.0
MB4
Variable MB5: Human Resource Focus
30
20
Frequency
10
Std. Dev = .87

M ean = 3.41
0 N = 87.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
MB5

88
Variable MB6: Process Management
40
Frequency 30
20
10
Std. Dev = .88
M ean = 3.55
0 N = 87.00
1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
MB6
Variable MB7: Business Results
40
30
Frequency
20
10
Std. Dev = .75
M ean = 3.69
0 N = 87.00
1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
MB7

89
Both the descriptive statistics and the histograms showed that the variables
data was not normally distributed. All of the variables were negatively skewed. At least
two reasons account for this: The employing organizations of respondents were actually
clustered at higher levels of performance, or the respondents tended to have a bias
toward rating their own organizations highly. Both the skewness values and the shapes
of the histograms demonstrated that the variables were universally skewed to the left.
The footnotes to Table 11 show that SPSS determined three variables, BOK1,
ISO1, and MB5, had multi-modal distributions. The histograms for variables ISO1 and
MB5 did not clearly illustrate that finding. The histogram for variable BOK1 showed a
second mode.
Skewness and Kurtosis Z Scores
The magnitude of skewness and kurtosis was not clear, based upon the raw
scores reported in the descriptive statistics. Field (2000) stated that this concern can be
addressed by generating skewness and kurtosis Z-scores using Equations 1-3. Skewness
and kurtosis results were converted using these equations. The resulting Z-scores are
reported in Table 13.
S 0 (Equation 1)
Z skewnss 
SE skewness
90
For positive values of kurtosis: (Equation 2)
K 0
Z kurtosis  
SE kurtosis
For negative values of kurtosis: (Equation 3)
K 0
Z kurtosis  
SE kurtosis
Table 13
Skewness and Kurtosis Z-scores
BOK1 BOK2 BOK3 BOK4 BOK5 BOK6 BOK7

Z-Skewness -2.21 -1.31 -1.46 -3.90 -0.59 -1.64 -1.34
Z-Kurtosis -0.23 -1.21 -1.28 1.05 -0.86 -1.21 -0.42
ISO1 ISO2 ISO3 ISO4 ISO5

Z-Skewness -5.07 -2.95 -1.39 -2.18 -4.36
Z-Kurtosis 2.48 0.61 -0.99 0.31 2.07
MB1 MB2 MB3 MB4 MB5 MB6 MB7

Z-Skewness -2.67 -1.55 -2.00 -2.48 -1.19 -3.16 -2.16
Z-Kurtosis 0.60 -0.18 -1.07 0.71 -0.59 1.51 0.96
91
Using a significance cutoff value of 1.96, the Z-scores showed that the
distributions for 11 of the variables were skewed significantly to the left, and the
distributions for 2 of the variables were significantly too sharp (their kurtosis values
were positive). These findings also showed that the variables data is not all normally
distributed.
Normality Tests
The dependent variables were tested for normality using the Kolmogorov-
Smirnov and Shapiro-Wilk tests with significance set at P < 0.05. Q-Q Normal
probability plots were generated at the same time. According to both normality tests,
every dependent variable scored significantly. This indicated the data for each
dependent variable was not normally distributed. The significance scores are shown in
Table 14. The normality plots, shown in Figure 3, confirmed these results.
Table 14
Normality tests on the dependent variables
Kolmogorov-Smirnov(a) Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
ISO1 .202 85 .000 .826 85 .000
ISO2 .124 85 .002 .939 85 .001
ISO3 .108 85 .016 .964 85 .018
ISO4 .159 85 .000 .958 85 .008
ISO5 .174 85 .000 .910 85 .000
MB1 .151 85 .000 .921 85 .000
MB2 .186 85 .000 .938 85 .000
MB3 .159 85 .000 .904 85 .000
MB4 .210 85 .000 .916 85 .000
MB5 .116 85 .006 .965 85 .022
MB6 .201 85 .000 .899 85 .000
MB7 .116 85 .007 .965 85 .022
a Lilliefors Significance Correction
92
Variable ISO1: Quality Management System
1.0
.5
0.0
Expected Normal
-.5
-1.0
-1.5
-2.0
-2.5
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value
Variable ISO2: Management Responsibility
1
Expected Normal
-1
-2
-3
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value
Figure 3. Q-Q Normal probability plots of the dependent variables

93
Variable ISO3: Resource Management
Expected Normal
0
-1
-2
-3
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value
Variable ISO4: Product Realization
2
Expected Normal
-1
-2
-3
2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value
Figure 3 (continued). Q-Q Normal probability plots of the dependent variables

94
Variable ISO5: Measurement, Analysis, and Improvement
Expected Normal
0
-1
-2
-3
0 1 2 3 4 5 6
Observed Value
Variable MB1: Leadership
1.5
1.0
.5
Expected Normal
0.0
-.5
-1.0
-1.5
-2.0
1 2 3 4 5 6
Observed Value

95
Variable MB2: Strategic Planning
Expected Normal
0
-1
-2
-3
0 1 2 3 4 5 6
Observed Value
Variable MB3: Customer and Market Focus
1.5
1.0
.5
Expected Normal
0.0
-.5
-1.0
-1.5
-2.0
-2.5
2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value

96
Variable MB4:Measurement, Analysis, and Knowledge Management
Expected Normal
0
-1
-2
-3
0 1 2 3 4 5 6
Observed Value
Variable MB5: Human Resource Focus
1
Expected Normal
-1
-2
-3
1 2 3 4 5 6
Observed Value

97
Variable MB6: Process Management
1
Expected Normal
-1
-2
-3
0 1 2 3 4 5 6
Observed Value
Variable MB7: Business Results
1
Expected Normal
-1
-2
-3
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Observed Value

98
Data Transformation
A data transformation strategy was determined using the ladder of powers.
According to Kirchner (2001), when data is left-skewed, raising the data to a power with
an exponent greater than one will help to normalize the spread. Because all variables
were left-skewed, 11 of them significantly so, squared and cubed transformations were
tested on the data.
Skewness and kurtosis Z-scores were calculated for both new sets of variables, as
shown in Table 15. The squared test variables were labeled with the suffix Ta, and the
cubed test variables were labeled with the suffix Tb. The table shows that the squared
transformation reduced the Z-scores to below 1.96 in all but two cases. BOK4Ta and
ISO1Ta had skewness Z-scores of –1.96 and –1.99 respectively, both of which were
close to being below the significance cutoff. In the case of the cubed transformations,
six of the skewness Z-scores scored significantly. Because of the results shown in Table
15, the squared transformation was used on the data.
The new variables were called BOK1T, BOK2T, BOK3T, etc. Table 16 names
and defines each transformed variable. These designations will be used throughout the
remainder of this report. Data tables for the transformed variables are located in
Appendix F.
Descriptive statistics and histograms were generated for the transformed
variables. The descriptive statistics are shown in Table 17, and the histograms are
shown in Figure 4.
99
Table 15
Skewness and kurtosis Z-scores for transformed variables
BOK1Ta BOK2Ta BOK3Ta BOK4Ta BOK5Ta BOK6Ta BOK7Ta

Z-Skewness -0.35 0.14 -0.25 -1.96 1.53 -0.31 1.33
Z-Kurtosis -1.27 -1.32 -1.54 -1.21 -0.47 -1.38 -0.82
BOK1Tb BOK2Tb BOK3Tb BOK4Tb BOK5Tb BOK6Tb BOK7Tb

Z-Skewness 0.84 1.38 0.56 -0.81 3.43 0.74 3.28
Z-Kurtosis -1.44 -1.21 -1.62 -1.55 1.20 -1.38 0.75
ISO1Ta ISO2Ta ISO3Ta ISO4Ta ISO5Ta

Z-Skewness -1.99 -1.12 0.27 -0.41 -1.23
Z-Kurtosis -0.56 -1.21 -1.23 -1.01 -0.66
ISO1Tb ISO2Tb ISO3Tb ISO4Tb ISO5Tb

Z-Skewness -0.55 0.01 1.57 0.93 0.44
Z-Kurtosis -1.47 -1.41 -1.17 -1.09 -1.20
MB1Ta MB2Ta MB3Ta MB4Ta MB5Ta MB6Ta MB7Ta

Z-Skewness -0.48 0.93 -0.83 0.18 1.28 0.36 0.22
Z-Kurtosis -1.18 -1.06 -1.46 -0.94 -0.69 0.47 -0.62
MB1Tb MB2Tb MB3Tb MB4Tb MB5Tb MB6Tb MB7Tb

Z-Skewness 0.94 2.65 0.02 2.15 3.23 2.92 1.87
Z-Kurtosis -1.35 -0.75 -1.61 -0.74 0.81 1.05 -0.39
100
Table 16
Mathematical definitions of transformed data
Variable Definition
BOK1T MEAN(BOK1A**2,BOK1B**2)
BOK2T MEAN(BOK2A**2,BOK2B**2,BOK2C**2)
BOK3T MEAN(BOK3A**2,BOK3B**2,BOK3C**2)
BOK4T MEAN(BOK4A**2,BOK4B**2)
BOK5T MEAN(BOK5A**2,BOK5B**2,BOK5C**2,BOK5D**2,BOK5E**2,
BOK5F**2,BOK5G**2)
BOK6T MEAN(BOK6A**2,BOK6B**2,BOK6C**2,BOK6D**2,BOK6E**2)
BOK7T MEAN(BOK7A**2,BOK7B**2,BOK7C**2,BOK7D**2,BOK7E**2)
ISO1T MEAN(ISO1A**2,ISO1B**2)
ISO2T MEAN(ISO2A**2,ISO2B**2,ISO2C**2,ISO2D**2,ISO2E**2,ISO2F**2)
ISO3T MEAN(ISO3A**2,ISO3B**2,ISO3C**2,ISO3D**2)
ISO4T MEAN(ISO4A**2,ISO4B**2,ISO4C**2,ISO4D**2,ISO4E**2,ISO4F**2)
ISO5T MEAN(ISO5A**2,ISO5B**2,ISO5C**2,ISO5D**2,ISO5E**2)
MB1T MEAN(MB1A**2,MB1B**2)
MB5T MEAN(MB5A**2,MB5B**2,MB5C**2)
MB7T MEAN(MB7A**2,MB7B**2,MB7C**2,MB7D**2,MB7E**2,MB7F**2)
101
Table 17
Descriptive statistics for the transformed variables
BOK1T BOK2T BOK3T BOK4T BOK5T

N Valid 88 88 88 88 88
Missing 0 0 0 0 0
Mean 16.1989 13.4508 16.1894 17.5170 12.1180
Std. Error of Mean .64996 .59244 .68777 .70606 .55242
Median 16.0000 13.6667 16.0000 18.2500 12.0000
Mode 12.50(a) 16.00 25.00 25.00 13.00(a)
Std. Deviation 6.09717 5.55756 6.45186 6.62347 5.18216
Variance 37.17552 30.88644 41.62655 43.87040 26.85482
Skewness -.089 .036 -.065 -.504 .393
Std. Error of .257 .257 .257 .257 .257
Kurtosis -.826 -.890 -1.206 -.742 -.111
Std. Error of .508 .508 .508 .508 .508
Range 22.50 22.00 21.00 22.50 22.86
Minimum 2.50 3.00 4.00 2.50 2.14
Maximum 25.00 25.00 25.00 25.00 25.00
BOK6T BOK7T ISO1T ISO2T ISO3T

N Valid 88 88 88 88 88
Missing 0 0 0 0 0
Mean 15.4545 11.5517 18.6023 16.4110 14.6335
Std. Error of Mean .61009 .60354 .61377 .62699 .59728
Median 16.0000 10.4000 16.0000 17.5000 14.2500
Mode 19.60 10.40 25.00 25.00 16.00
Std. Deviation 5.72315 5.66173 5.75763 5.88171 5.60300
Variance 32.75440 32.05514 33.15034 34.59448 31.39360
Skewness -.079 .341 -.510 -.289 .068
Std. Error of .257 .257 .257 .257 .257
Kurtosis -.967 -.339 -.161 -.750 -.770
Std. Error of .508 .508 .508 .508 .508
Range 20.20 24.00 24.00 22.17 21.00
Minimum 4.80 1.00 1.00 2.83 4.00
Maximum 25.00 25.00 25.00 25.00 25.00
102
Descriptive statistics for the transformed variables
ISO4T ISO5T MB1T MB2T MB3T

N Valid 88 88 87 85 87
Missing 0 0 1 3 1
Mean 15.6208 15.9472 15.4713 13.4176 17.4483
Std. Error of Mean .52626 .59861 .68372 .68317 .65429
Median 16.0000 16.0000 16.0000 12.5000 16.0000
Mode 16.00 16.00 16.00 16.00 25.00
Std. Deviation 4.93673 5.61547 6.37736 6.29855 6.10282
Variance 24.37135 31.53353 40.67068 39.67171 37.24439
Skewness -.106 -.317 -.123 .244 -.213
Std. Error of .257 .257 .258 .261 .258
Kurtosis -.517 -.221 -.714 -.576 -1.095
Std. Error of .508 .508 .511 .517 .511
Range 20.50 24.00 22.50 24.00 21.00
Minimum 4.50 1.00 2.50 1.00 4.00
Maximum
MB4T MB5T MB6T MB7T

N Valid 87 87 87 87
Missing 1 1 1 1
Mean 14.1782 12.5632 13.4080 14.4962
Std. Error of Mean .67058 .62033 .61172 .55289
Median 16.0000 13.6667 16.0000 15.1667
Mode 16.00 13.67 16.00 16.00
Std. Deviation 6.25475 5.78609 5.70573 5.15701
Variance 39.12196 33.47884 32.55540 26.59471
Skewness .045 .330 .094 .056
Std. Error of .258 .258 .258 .258
Kurtosis -.449 -.242 .111 -.196
Std. Error of .511 .511 .511 .511
Range 24.00 23.00 24.00 22.67
Minimum 1.00 2.00 1.00 2.33
Maximum 25.00 25.00 25.00 25.00
103
Variable BOK1T: Leadership, Transformed
40
30
Frequency
20
10
Std. Dev = 6.10
M ean = 16.2
0 N = 88.00
5.0 10.0 15.0 20.0 25.0
BOK1T
Variable BOK2T: Strategy Development and Deployment, Transformed
20
Frequency
10
Std. Dev = 5.56

M ean = 13.5
0 N = 88.00
2.5 7.5 12.5 17.5 22.5
5.0 10.0 15.0 20.0 25.0
BOK2T
Figure 4. Histograms of all transformed variables

104
Variable BOK3T: Quality Management Tools, Transformed
30
20
Frequency
10
Std. Dev = 6.45

M ean = 16.2
0 N = 88.00
5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0
BOK3T
Variable BOK4T: Customer-Focused Organizations, Transformed
30
20
Frequency
10
Std. Dev = 6.62

M ean = 17.5
0 N = 88.00
5.0 10.0 15.0 20.0 25.0
BOK4T
Figure 4 (continued). Histograms of all transformed variables

105
Variable BOK5T: Supplier Performance, Transformed
16
14
12
10
Frequency
4
Std. Dev = 5.18
2 M ean = 12.1
0 N = 88.00
2.0 6.0 10.0 14.0 18.0 22.0 26.0
4.0 8.0 12.0 16.0 20.0 24.0
BOK5T
Variable BOK6T: Management, Transformed
14
12
10
Frequency
4
Std. Dev = 5.72
2
M ean = 15.5
0 N = 88.00
4.0 8.0 12.0 16.0 20.0 24.0
6.0 10.0 14.0 18.0 22.0 26.0
BOK6T

106
Variable BOK7T: Training and Development, Transformed
16
14
12
10
Frequency
4
Std. Dev = 5.66
2 M ean = 11.6
0 N = 88.00
2.0 6.0 10.0 14.0 18.0 22.0 26.0
4.0 8.0 12.0 16.0 20.0 24.0
BOK7T
Variable ISO1T: Quality Management System, Transformed
50
40
30
20
Frequency
10 Std. Dev = 5.76

M ean = 18.6
0 N = 88.00
0.0 5.0 10.0 15.0 20.0 25.0
ISO1T

107
Variable ISO2T: Management Responsibility, Transformed
16
14
12
Frequency 10
4
Std. Dev = 5.88
2 M ean = 16.4
0 N = 88.00
2.0 6.0 10.0 14.0 18.0 22.0 26.0
4.0 8.0 12.0 16.0 20.0 24.0
ISO2T
Variable ISO3T: Resource Management, Transformed
30
20
Frequency
10
Std. Dev = 5.60

M ean = 14.6
0 N = 88.00
5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0
ISO3T

108
Variable ISO4T: Product Realization, Transformed
30
20
Frequency
10
Std. Dev = 4.94

M ean = 15.6
0 N = 88.00
4.0 8.0 12.0 16.0 20.0 24.0
6.0 10.0 14.0 18.0 22.0 26.0
ISO4T
Variable ISO5T: Measurement, Analysis, and Improvement, Transformed
30
20
Frequency
10
Std. Dev = 5.62

M ean = 15.9
0 N = 88.00
0.0 5.0 10.0 15.0 20.0 25.0
2.5 7.5 12.5 17.5 22.5
ISO5T

109
Variable MB1T: Leadership, Transformed
50
40
Frequency
30
20
10 Std. Dev = 6.38

M ean = 15.5
0 N = 87.00
5.0 10.0 15.0 20.0 25.0
MB1T
Variable MB2T: Strategic Planning, Transformed
40
30
Frequency
20
10
Std. Dev = 6.30
M ean = 13.4
0 N = 85.00
0.0 5.0 10.0 15.0 20.0 25.0
MB2T

110
Variable MB3T: Customer and Market Focus, Transformed
40
30
Frequency
20
10
Std. Dev = 6.10
M ean = 17.4
0 N = 87.00
5.0 10.0 15.0 20.0 25.0
MB3T
Variable MB4T: Measurement, Analysis, and Knowledge Management, Transformed
50
40
Frequency
30
20
10 Std. Dev = 6.25

M ean = 14.2
0 N = 87.00
0.0 5.0 10.0 15.0 20.0 25.0
MB4T

111
Variable MB5T: Human Resource Focus, Transformed
30
Frequency 20
10
Std. Dev = 5.79

M ean = 12.6
0 N = 87.00
2.5 7.5 12.5 17.5 22.5
5.0 10.0 15.0 20.0 25.0
MB5T
Variable MB6T: Process Management, Transformed
60
50
40
Frequency
30
20
10 Std. Dev = 5.71

M ean = 13.4
0 N = 87.00
0.0 5.0 10.0 15.0 20.0 25.0
MB6T

112
Variable MB7T: Business Results, Transformed
30
20
Frequency
10
Std. Dev = 5.16

M ean = 14.5
0 N = 87.00
2.0 6.0 10.0 14.0 18.0 22.0 26.0
4.0 8.0 12.0 16.0 20.0 24.0
MB7T
The descriptive statistics and histograms showed that, in general, the distributions
were made more symmetric by the data transformation. As discussed previously,
transforming the data brought all but two of the skewness and kurtosis Z-scores under
the significance limit. The remaining two variables scored significantly for skewness,
but only by small margins. The histograms appeared to show that squaring the data
widened the spread of the distributions and revealed some additional bimodal
distributions, but the SPSS output did not concur. The output labeled only two variables,
BOK1T and BOK5T, with bimodal distributions, as seen in Table 17. This reduced by
one the number of bimodal distributions, by one variable.
The transformed dependent variables were tested for normality, using the
Kolmogorov-Smirnov and Shapiro-Wilk tests with the significance level set at P < 0.05,
113
and their Q-Q normal probability plots were generated. The results of the normality
tests are shown in Table 18. The Q-Q normal probability plots are shown in Figure 5.
According to the normality tests, most of the dependent variables were still not
normally distributed after the transformation. The Kolmogorov-Smirnov test indicated
that 9 of the 12 distributions for dependent variables were still significantly not normally
distributed. The Shapiro-Wilk test indicated that 11 dependent variables were still not
normally distributed. The Q-Q normal probability plots appeared to show visible
straightening, relative to the corresponding plots for the un-transformed variables.
Table 18
Normality tests on the transformed dependent variables
Kolmogorov-Smirnov(a) Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
ISO1T .220 85 .000 .861 85 .000
ISO2T .082 85 .200(*) .959 85 .009
ISO3T .085 85 .191 .968 85 .035
ISO4T .102 85 .030 .976 85 .118
ISO5T .121 85 .004 .962 85 .013
MB1T .112 85 .010 .940 85 .001
MB2T .158 85 .000 .933 85 .000
MB3T .164 85 .000 .904 85 .000
MB4T .204 85 .000 .923 85 .000
MB5T .101 85 .031 .960 85 .009
MB6T .210 85 .000 .921 85 .000
MB7T .082 85 .200(*) .978 85 .163
* This is a lower bound of the true significance.
a Lilliefors Significance Correction
114
1.0
.5
Expected Normal 0.0
-.5
-1.0
-1.5
-2.0
-2.5
0 10 20 30
Observed Value
1
Expected Normal
-1
-2
-3
0 10 20 30
Observed Value
Figure 5. Q-Q normal probability plots of the transformed dependent variables

115
Expected Normal
0
-1
-2
-3
0 10 20 30
Observed Value

QQ
3
2
Expected Normal
-1
-2
-3
0 10 20 30
Observed Value
Figure 5 (continued). Q-Q normal probability plots of the transformed dependent
variables
116

QQ
2
1
Expected Normal
-1
-2
-3
0 10 20 30
Observed Value
1.5
1.0
.5
Expected Normal
0.0
-.5
-1.0
-1.5
-2.0
0 10 20 30
Observed Value
variables
117
1
Expected Normal
0
-1
-2
-3
-10 0 10 20 30
Observed Value
1.5
1.0
.5
Expected Normal
0.0
-.5
-1.0
-1.5
-2.0
-2.5
0 10 20 30
Observed Value
variables
118
Variable MB4T: Measurement, Analysis, and Knowledge Management,
Transformed
Expected Normal 1
-1
-2
-3
0 10 20 30
Observed Value
1
Expected Normal
-1
-2
-3
-10 0 10 20 30
Observed Value
variables
119
Expected Normal
0
-1
-2
-3
0 10 20 30
Observed Value
1
Expected Normal
-1
-2
-3
0 10 20 30
Observed Value
variables
120
The next stage of the research was to analyze the data for model fit. It was
contingent upon the data meeting normality assumptions or, if not, finding a suitable
transformation that would bring the data in line with normality assumptions.
Continuing the analysis depended upon whether or not the data was sufficiently
normal. After the transformation, most of the variables still tested as significantly non-
normal. However, 9 out of 11 transformed variables did not have significant skewness,
and none of them had significant kurtosis. This meant that the distributions of the
majority of transformed variables, while not normally distributed, did have symmetric
distributions.
According to Montgomery (2001), fixed effects ANOVA is robust to moderate
departures from normality. He stated that error distributions with tails that are thicker or
thinner than normal is more of a problem than skewed distributions, because the F-test is
only slightly affected by skewed variance. He also stated that the main effect of
departures from normality was a reduction in the true significance levels of results,
relative to the levels chosen in tests, and a reduction in the power of designs.
Harris (1994) made similar assertions. He stated that as long as two-tailed tests
are used, even when very non-normal populations are sampled and when the differences
in population variances are ten-fold, then the significance level is rarely more than 7%
when 5% is selected.
The SPSS 11.5 (2003) help file also supported these assertions. It contained
assertions that ANOVA “is robust to departures from normality, although the data should
be symmetric,” that tests should be run for homogeneity of variance, and residuals
should be examined.
121
The data transformation brought the distributions for variables within the
limits of departure from normality. It removed significant skewness from the
distributions of 9 of the 11 variables, and it removed significant kurtosis from all
variables.
Normality was tested by plotting residuals. Levene’s test for homogeneity of
variance would have been run if factorial analysis had been used. This was not possible
because all terms were entered as covariates.
Analyze the Data for Model Fit
There were two objectives in analyzing the data for model fit. The first was to
determine if the independent variables significantly described a portion of the variability
in the dependent variables. The second was to determine which main effects and two-
way interactions should be included in refined ANOVA models, based upon whether or
not they contributed significantly to the variation in dependent variables.
The initial models, derived from the hypotheses upon which the study was based,
were that the main effects of the independent variables and the two-way interactions
between them contributed to each dependent variable. The relationship is shown in
Equations 4 and 5, where the subscript n refers to the nth variable in a series.
After transformation, the equations for the initial models became as shown in
Equations 6 and 7. Once again, the subscript n refers to the nth variable in a series.
122
ISOn = Intercept + BOK1 + BOK2 + BOK3 + BOK4 + BOK5 + BOK6 + BOK7 +
BOK1xBOK2 + BOK1xBOK3 + BOK1xBOK4 + BOK1xBOK5 + BOK1xBOK6 +
BOK1xBOK7 + BOK2xBOK3 + BOK2xBOK4 + BOK2xBOK5 + BOK2xBOK6
+BOK2xBOK7 + BOK3xBOK4 + BOK3xBOK5 + BOK3xBOK6 + BOK3xBOK7 +
BOK6xBOK7 + Error (Equation 4)
MBn = Intercept + BOK1 + BOK2 + BOK3 + BOK4 + BOK5 + BOK6 + BOK7 +
BOK1xBOK7 + BOK2xBOK3 + BOK2xBOK4 + BOK2xBOK5 + BOK2xBOK6
+BOK2xBOK7 + BOK3xBOK4 + BOK3xBOK5 + BOK3xBOK6 + BOK3xBOK7 +
BOK6xBOK7 + Error (Equation 5)
ISOTn = Intercept + BOK1T + BOK2T + BOK3T + BOK4T + BOK5T + BOK6T +
BOK7T + BOK1TxBOK2T + BOK1TxBOK3T + BOK1TxBOK4T + BOK1TxBOK5T
+ BOK1TxBOK6T + BOK1TxBOK7T + BOK2TxBOK3T + BOK2TxBOK4T +
BOK2TxBOK5T + BOK2TxBOK6T +BOK2TxBOK7T + BOK3TxBOK4T +
BOK3TxBOK5T + BOK3TxBOK6T + BOK3TxBOK7T + BOK4TxBOK5T +
BOK6TxBOK7T + Error (Equation 6)

123
MBTn = Intercept + BOK1T + BOK2T + BOK3T + BOK4T + BOK5T + BOK6T +
BOK7T + BOK1TxBOK2T + BOK1TxBOK3T + BOK1TxBOK4T + BOK1TxBOK5T
+ BOK1TxBOK6T + BOK1TxBOK7T + BOK2TxBOK3T + BOK2TxBOK4T +
BOK2TxBOK5T + BOK2TxBOK6T +BOK2TxBOK7T + BOK3TxBOK4T +
BOK6TxBOK7T + Error (Equation 7)
The first step in analyzing the data for model fit was to test the independent vs.
dependent variables using the Spearman’s Rho correlation test with significance level set
at P < 0.05, in order to determine which variables varied significantly together. This test
was not helpful in screening for significant correlations because the correlation
coefficients between all variables tested significantly. The results of the tests are shown
in Tables 19 and 20.
Both model fit objectives were accomplished using analysis of covariance
(ANCOVA). It determined if the initial overall model tested significantly against the
dependent variables, and it determined which main effects and interactions tested
significantly in each model. In both cases, the f-test was used with the significance level
at P < 0.05.
The ANCOVA tests were run using type IV sums of squares. This excluded
cases where one of the variables in a comparison was missing data, and reduced the
degrees of freedom available for calculations involving the relevant variables. The
ANCOVA reports are shown in Table 21.

124
Residual plot matrices were generated during the ANCOVA tests. These
were used to check for linearity in the observed vs. predicted value plots and to identify
non-random patterns in the residuals. The residual plot matrices are shown in Figure 6.
Table 19
Spearman’s Rho matrix for BOK vs. ISO transformed variables
ISO1T ISO2T ISO3T ISO4T ISO5T

BOK1T Spearman’s Rho .635(**) .689(**) .520(**) .495(**) .560(**)
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
Sig. (2-tailed) .000 .000 .000 .000 .000
N 88 88 88 88 88
** Correlation is significant at the 0.01 level (2-tailed).

125
Table 20
Spearman’s Rho Coefficient matrix for BOK vs. MB transformed variables
MB1T MB2T MB3T MB4T

BOK1T Spearman’s Rho .671(**) .625(**) .519(**) .658(**)
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
Sig. (2-tailed) .000 .000 .000 .000
N 87 85 87 87
126
Spearman’s Rho Coefficient matrix for BOK vs. MB transformed variables
MB5T MB6T MB7T

BOK1T Spearman’s Rho .550(**) .602(**) .629(**)
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
Sig. (2-tailed) .000 .000 .000
N 87 87 87
127
Table 21
ANCOVA reports for independent variables vs. dependent variables – transformed data
Tests of Between-Subjects Effects: Independents vs. ISO1T

Type IV Partial
Sum of Mean Eta
Source Squares df Square F Sig. Squared
Corrected Model 1894.393(a) 28 67.657 4.033 .000 .657
Intercept 29.875 1 29.875 1.781 .187 .029
BOK1T 39.124 1 39.124 2.332 .132 .038
BOK2T 15.731 1 15.731 .938 .337 .016
BOK3T .261 1 .261 .016 .901 .000
BOK4T 13.308 1 13.308 .793 .377 .013
BOK5T .000 1 .000 .000 .997 .000
BOK6T 82.696 1 82.696 4.930 .030 .077
BOK7T 6.904 1 6.904 .412 .524 .007
BOK1T * BOK2T 2.221 1 2.221 .132 .717 .002
BOK1T * BOK3T 4.109 1 4.109 .245 .622 .004
BOK1T * BOK4T 4.923 1 4.923 .293 .590 .005
BOK1T * BOK5T 2.319 1 2.319 .138 .711 .002
BOK1T * BOK6T 2.672 1 2.672 .159 .691 .003
BOK1T * BOK7T .612 1 .612 .036 .849 .001
BOK2T * BOK3T .497 1 .497 .030 .864 .001
BOK2T * BOK4T 12.587 1 12.587 .750 .390 .013
BOK2T * BOK5T 57.012 1 57.012 3.399 .070 .054
BOK2T * BOK6T 19.464 1 19.464 1.160 .286 .019
BOK2T * BOK7T 2.209 1 2.209 .132 .718 .002
BOK3T * BOK4T 2.957 1 2.957 .176 .676 .003
BOK3T * BOK5T 45.925 1 45.925 2.738 .103 .044
BOK3T * BOK6T .100 1 .100 .006 .939 .000
BOK3T * BOK7T 8.161 1 8.161 .487 .488 .008
BOK4T * BOK5T .014 1 .014 .001 .977 .000
BOK4T * BOK6T 45.482 1 45.482 2.711 .105 .044
BOK4T * BOK7T 1.142 1 1.142 .068 .795 .001
BOK5T * BOK6T 8.703 1 8.703 .519 .474 .009
BOK5T * BOK7T 34.690 1 34.690 2.068 .156 .034
BOK6T * BOK7T 2.710 1 2.710 .162 .689 .003
Error 989.686 59 16.774
Total 33336.000 88
Corrected Total 2884.080 87
a R Squared = .657 (Adjusted R Squared = .494)
128

Partial
Type IV Sum Mean Eta
Source of Squares df Square F Sig. Squared
Intercept .037 1 .037 .003 .954 .000
BOK1T 1.659 1 1.659 .155 .695 .003
BOK2T 61.530 1 61.530 5.755 .020 .089
BOK3T 4.351 1 4.351 .407 .526 .007
BOK4T 40.070 1 40.070 3.748 .058 .060
BOK5T 4.601 1 4.601 .430 .514 .007
BOK6T 27.642 1 27.642 2.585 .113 .042
BOK7T 6.297 1 6.297 .589 .446 .010
BOK1T * BOK2T 1.815 1 1.815 .170 .682 .003
BOK1T * BOK3T .343 1 .343 .032 .858 .001
BOK1T * BOK4T 10.388 1 10.388 .972 .328 .016
BOK1T * BOK5T .089 1 .089 .008 .928 .000
BOK1T * BOK6T 20.079 1 20.079 1.878 .176 .031
BOK1T * BOK7T 13.673 1 13.673 1.279 .263 .021
BOK2T * BOK3T .312 1 .312 .029 .865 .000
BOK2T * BOK4T 30.914 1 30.914 2.891 .094 .047
BOK2T * BOK5T 3.999 1 3.999 .374 .543 .006
BOK2T * BOK6T .020 1 .020 .002 .966 .000
BOK2T * BOK7T .628 1 .628 .059 .809 .001
BOK3T * BOK4T .068 1 .068 .006 .937 .000
BOK3T * BOK5T 6.773 1 6.773 .633 .429 .011
BOK3T * BOK6T 15.664 1 15.664 1.465 .231 .024
BOK3T * BOK7T 7.424 1 7.424 .694 .408 .012
BOK4T * BOK5T 6.507 1 6.507 .609 .438 .010
BOK4T * BOK6T 10.376 1 10.376 .970 .329 .016
BOK4T * BOK7T 44.454 1 44.454 4.158 .046 .066
BOK5T * BOK6T .383 1 .383 .036 .850 .001
BOK5T * BOK7T 13.125 1 13.125 1.228 .272 .020
BOK6T * BOK7T .791 1 .791 .074 .787 .001
Error 630.824 59 10.692
Total 26709.917 88
129

Partial
Intercept 82.168 1 82.168 4.873 .031 .076
BOK1T 8.318 1 8.318 .493 .485 .008
BOK2T 67.327 1 67.327 3.993 .050 .063
BOK3T 4.968 1 4.968 .295 .589 .005
BOK4T 24.344 1 24.344 1.444 .234 .024
BOK5T 4.365 1 4.365 .259 .613 .004
BOK6T .243 1 .243 .014 .905 .000
BOK7T 24.706 1 24.706 1.465 .231 .024
BOK1T * BOK2T 18.908 1 18.908 1.121 .294 .019
BOK1T * BOK3T 3.231 1 3.231 .192 .663 .003
BOK1T * BOK4T .858 1 .858 .051 .822 .001
BOK1T * BOK5T 16.951 1 16.951 1.005 .320 .017
BOK1T * BOK6T 3.509 1 3.509 .208 .650 .004
BOK1T * BOK7T 5.023 1 5.023 .298 .587 .005
BOK2T * BOK3T 5.380 1 5.380 .319 .574 .005
BOK2T * BOK4T 20.268 1 20.268 1.202 .277 .020
BOK2T * BOK5T 6.804 1 6.804 .404 .528 .007
BOK2T * BOK6T 5.710 1 5.710 .339 .563 .006
BOK2T * BOK7T .024 1 .024 .001 .970 .000
BOK3T * BOK4T 3.622 1 3.622 .215 .645 .004
BOK3T * BOK5T 1.344 1 1.344 .080 .779 .001
BOK3T * BOK6T 7.966 1 7.966 .472 .495 .008
BOK3T * BOK7T 9.869 1 9.869 .585 .447 .010
BOK4T * BOK5T 2.898 1 2.898 .172 .680 .003
BOK4T * BOK6T .984 1 .984 .058 .810 .001
BOK4T * BOK7T 48.888 1 48.888 2.899 .094 .047
BOK5T * BOK6T 1.816 1 1.816 .108 .744 .002
BOK5T * BOK7T .050 1 .050 .003 .957 .000
BOK6T * BOK7T 2.153 1 2.153 .128 .722 .002
Error 994.872 59 16.862
Total 21575.563 88
130

Type IV Partial
Sum of Mean Eta
Intercept 38.492 1 38.492 3.952 .051 .063
BOK1T 6.176 1 6.176 .634 .429 .011
BOK2T 77.151 1 77.151 7.920 .007 .118
BOK3T 18.424 1 18.424 1.891 .174 .031
BOK4T 44.750 1 44.750 4.594 .036 .072
BOK5T 7.744 1 7.744 .795 .376 .013
BOK6T 52.841 1 52.841 5.425 .023 .084
BOK7T .419 1 .419 .043 .836 .001
BOK1T * BOK2T .178 1 .178 .018 .893 .000
BOK1T * BOK3T 20.061 1 20.061 2.060 .157 .034
BOK1T * BOK4T 1.201 1 1.201 .123 .727 .002
BOK1T * BOK5T 2.286 1 2.286 .235 .630 .004
BOK1T * BOK6T 5.543 1 5.543 .569 .454 .010
BOK1T * BOK7T .878 1 .878 .090 .765 .002
BOK2T * BOK3T 5.501 1 5.501 .565 .455 .009
BOK2T * BOK4T 8.818 1 8.818 .905 .345 .015
BOK2T * BOK5T 25.003 1 25.003 2.567 .114 .042
BOK2T * BOK6T .490 1 .490 .050 .823 .001
BOK2T * BOK7T 3.496 1 3.496 .359 .551 .006
BOK3T * BOK4T .036 1 .036 .004 .952 .000
BOK3T * BOK5T 1.476 1 1.476 .152 .698 .003
BOK3T * BOK6T 3.525 1 3.525 .362 .550 .006
BOK3T * BOK7T .175 1 .175 .018 .894 .000
BOK4T * BOK5T 6.483 1 6.483 .666 .418 .011
BOK4T * BOK6T 11.041 1 11.041 1.133 .291 .019
BOK4T * BOK7T 18.662 1 18.662 1.916 .172 .031
BOK5T * BOK6T 13.249 1 13.249 1.360 .248 .023
BOK5T * BOK7T .258 1 .258 .026 .871 .000
BOK6T * BOK7T .576 1 .576 .059 .809 .001
Error 574.706 59 9.741
Total 23593.226 88
131

Type IV
Sum of Mean Partial Eta
Intercept 1.539 1 1.539 .155 .696 .003
BOK1T .241 1 .241 .024 .877 .000
BOK2T 163.187 1 163.187 16.396 .000 .217
BOK3T .429 1 .429 .043 .836 .001
BOK4T 41.395 1 41.395 4.159 .046 .066
BOK5T .138 1 .138 .014 .907 .000
BOK6T 55.375 1 55.375 5.564 .022 .086
BOK7T 13.433 1 13.433 1.350 .250 .022
BOK1T * BOK2T 15.417 1 15.417 1.549 .218 .026
BOK1T * BOK3T 3.912 1 3.912 .393 .533 .007
BOK1T * BOK4T 28.777 1 28.777 2.891 .094 .047
BOK1T * BOK5T .634 1 .634 .064 .802 .001
BOK1T * BOK6T 10.486 1 10.486 1.054 .309 .018
BOK1T * BOK7T 39.302 1 39.302 3.949 .052 .063
BOK2T * BOK3T 1.709 1 1.709 .172 .680 .003
BOK2T * BOK4T 17.975 1 17.975 1.806 .184 .030
BOK2T * BOK5T 55.359 1 55.359 5.562 .022 .086
BOK2T * BOK6T 15.151 1 15.151 1.522 .222 .025
BOK2T * BOK7T 8.199 1 8.199 .824 .368 .014
BOK3T * BOK4T 10.227 1 10.227 1.027 .315 .017
BOK3T * BOK5T 18.486 1 18.486 1.857 .178 .031
BOK3T * BOK6T 38.733 1 38.733 3.892 .053 .062
BOK3T * BOK7T .502 1 .502 .050 .823 .001
BOK4T * BOK5T 6.755 1 6.755 .679 .413 .011
BOK4T * BOK6T 56.954 1 56.954 5.722 .020 .088
BOK4T * BOK7T 11.616 1 11.616 1.167 .284 .019
BOK5T * BOK6T 5.951 1 5.951 .598 .442 .010
BOK5T * BOK7T .787 1 .787 .079 .780 .001
BOK6T * BOK7T 13.272 1 13.272 1.333 .253 .022
Error 587.225 59 9.953
Total 25122.863 88
132
Tests of Between-Subjects Effects: Independents vs. MB1T

Type IV Partial
Sum of Mean Eta
Intercept .877 1 .877 .047 .829 .001
BOK1T 120.859 1 120.859 6.505 .013 .101
BOK2T 2.192 1 2.192 .118 .733 .002
BOK3T 2.135 1 2.135 .115 .736 .002
BOK4T .109 1 .109 .006 .939 .000
BOK5T 25.383 1 25.383 1.366 .247 .023
BOK6T 20.359 1 20.359 1.096 .300 .019
BOK7T 7.488 1 7.488 .403 .528 .007
BOK1T * BOK2T 8.802 1 8.802 .474 .494 .008
BOK1T * BOK3T 8.229 1 8.229 .443 .508 .008
BOK1T * BOK4T 5.632 1 5.632 .303 .584 .005
BOK1T * BOK5T 4.719 1 4.719 .254 .616 .004
BOK1T * BOK6T .838 1 .838 .045 .833 .001
BOK1T * BOK7T 1.750 1 1.750 .094 .760 .002
BOK2T * BOK3T 5.233 1 5.233 .282 .598 .005
BOK2T * BOK4T 3.254 1 3.254 .175 .677 .003
BOK2T * BOK5T 11.854 1 11.854 .638 .428 .011
BOK2T * BOK6T .664 1 .664 .036 .851 .001
BOK2T * BOK7T 9.786 1 9.786 .527 .471 .009
BOK3T * BOK4T 80.272 1 80.272 4.320 .042 .069
BOK3T * BOK5T .809 1 .809 .044 .835 .001
BOK3T * BOK6T 5.460 1 5.460 .294 .590 .005
BOK3T * BOK7T 6.203 1 6.203 .334 .566 .006
BOK4T * BOK5T 52.272 1 52.272 2.813 .099 .046
BOK4T * BOK6T 10.172 1 10.172 .547 .462 .009
BOK4T * BOK7T 11.875 1 11.875 .639 .427 .011
BOK5T * BOK6T 41.235 1 41.235 2.219 .142 .037
BOK5T * BOK7T 3.287 1 3.287 .177 .676 .003
BOK6T * BOK7T 1.056 1 1.056 .057 .812 .001
Error 1077.638 58 18.580
Total 24322.000 87
133

Type IV Sum Mean Partial Eta
Intercept 29.814 1 29.814 2.788 .101 .047
BOK1T 48.071 1 48.071 4.496 .038 .074
BOK2T 9.162 1 9.162 .857 .359 .015
BOK3T 8.600 1 8.600 .804 .374 .014
BOK4T 34.491 1 34.491 3.226 .078 .054
BOK5T 12.156 1 12.156 1.137 .291 .020
BOK6T 23.474 1 23.474 2.195 .144 .038
BOK7T .017 1 .017 .002 .968 .000
BOK1T * BOK2T 57.754 1 57.754 5.402 .024 .088
BOK1T * BOK3T .635 1 .635 .059 .808 .001
BOK1T * BOK4T 84.314 1 84.314 7.886 .007 .123
BOK1T * BOK5T 11.585 1 11.585 1.083 .302 .019
BOK1T * BOK6T 3.158 1 3.158 .295 .589 .005
BOK1T * BOK7T .266 1 .266 .025 .875 .000
BOK2T * BOK3T 6.956 1 6.956 .651 .423 .011
BOK2T * BOK4T 76.485 1 76.485 7.153 .010 .113
BOK2T * BOK5T 1.028 1 1.028 .096 .758 .002
BOK2T * BOK6T 40.794 1 40.794 3.815 .056 .064
BOK2T * BOK7T 4.659 1 4.659 .436 .512 .008
BOK3T * BOK4T 3.420 1 3.420 .320 .574 .006
BOK3T * BOK5T 1.911 1 1.911 .179 .674 .003
BOK3T * BOK6T 8.216 1 8.216 .768 .384 .014
BOK3T * BOK7T 4.208 1 4.208 .394 .533 .007
BOK4T * BOK5T 24.992 1 24.992 2.337 .132 .040
BOK4T * BOK6T 8.981 1 8.981 .840 .363 .015
BOK4T * BOK7T 31.026 1 31.026 2.902 .094 .049
BOK5T * BOK6T 51.836 1 51.836 4.848 .032 .080
BOK5T * BOK7T 11.401 1 11.401 1.066 .306 .019
BOK6T * BOK7T 59.084 1 59.084 5.526 .022 .090
Error 598.753 56 10.692
Total 18635.250 85
134

Type IV
Mean Partial Eta
Source Sum of df F Sig.
Square Squared
Squares
Intercept 75.713 1 75.713 4.612 .036 .074
BOK1T 1.782 1 1.782 .109 .743 .002
BOK2T 3.007 1 3.007 .183 .670 .003
BOK3T 2.344 1 2.344 .143 .707 .002
BOK4T 145.140 1 145.140 8.840 .004 .132
BOK5T 42.496 1 42.496 2.588 .113 .043
BOK6T 12.522 1 12.522 .763 .386 .013
BOK7T 21.942 1 21.942 1.336 .252 .023
BOK1T * BOK2T 65.375 1 65.375 3.982 .051 .064
BOK1T * BOK3T 12.687 1 12.687 .773 .383 .013
BOK1T * BOK4T 59.764 1 59.764 3.640 .061 .059
BOK1T * BOK5T 6.779 1 6.779 .413 .523 .007
BOK1T * BOK6T 72.742 1 72.742 4.431 .040 .071
BOK1T * BOK7T 42.058 1 42.058 2.562 .115 .042
BOK2T * BOK3T 13.754 1 13.754 .838 .364 .014
BOK2T * BOK4T 36.658 1 36.658 2.233 .141 .037
BOK2T * BOK5T 9.814 1 9.814 .598 .443 .010
BOK2T * BOK6T 48.873 1 48.873 2.977 .090 .049
BOK2T * BOK7T 4.808 1 4.808 .293 .590 .005
BOK3T * BOK4T 31.164 1 31.164 1.898 .174 .032
BOK3T * BOK5T 12.979 1 12.979 .791 .378 .013
BOK3T * BOK6T .745 1 .745 .045 .832 .001
BOK3T * BOK7T 15.223 1 15.223 .927 .340 .016
BOK4T * BOK5T 15.687 1 15.687 .955 .332 .016
BOK4T * BOK6T 7.916 1 7.916 .482 .490 .008
BOK4T * BOK7T .953 1 .953 .058 .810 .001
BOK5T * BOK6T 2.983 1 2.983 .182 .671 .003
BOK5T * BOK7T 14.136 1 14.136 .861 .357 .015
BOK6T * BOK7T 39.649 1 39.649 2.415 .126 .040
Error 952.257 58 16.418
Total 29689.500 87
135

Type IV Partial
Sum of Mean Eta
Intercept .294 1 .294 .020 .889 .000
BOK1T 18.433 1 18.433 1.245 .269 .021
BOK2T 25.588 1 25.588 1.728 .194 .029
BOK3T 3.431 1 3.431 .232 .632 .004
BOK4T .751 1 .751 .051 .823 .001
BOK5T 6.153 1 6.153 .416 .522 .007
BOK6T 3.666 1 3.666 .248 .621 .004
BOK7T 89.446 1 89.446 6.042 .017 .094
BOK1T * BOK2T .236 1 .236 .016 .900 .000
BOK1T * BOK3T 9.164 1 9.164 .619 .435 .011
BOK1T * BOK4T 11.684 1 11.684 .789 .378 .013
BOK1T * BOK5T 5.546 1 5.546 .375 .543 .006
BOK1T * BOK6T .959 1 .959 .065 .800 .001
BOK1T * BOK7T 1.941 1 1.941 .131 .719 .002
BOK2T * BOK3T .933 1 .933 .063 .803 .001
BOK2T * BOK4T 3.860 1 3.860 .261 .612 .004
BOK2T * BOK5T .645 1 .645 .044 .835 .001
BOK2T * BOK6T 3.006 1 3.006 .203 .654 .003
BOK2T * BOK7T 37.600 1 37.600 2.540 .116 .042
BOK3T * BOK4T .762 1 .762 .052 .821 .001
BOK3T * BOK5T .035 1 .035 .002 .962 .000
BOK3T * BOK6T .151 1 .151 .010 .920 .000
BOK3T * BOK7T 9.871 1 9.871 .667 .418 .011
BOK4T * BOK5T 4.382 1 4.382 .296 .588 .005
BOK4T * BOK6T 1.245 1 1.245 .084 .773 .001
BOK4T * BOK7T 3.880 1 3.880 .262 .611 .004
BOK5T * BOK6T 23.904 1 23.904 1.615 .209 .027
BOK5T * BOK7T 36.661 1 36.661 2.476 .121 .041
BOK6T * BOK7T 65.225 1 65.225 4.406 .040 .071
Error 858.668 58 14.805
Total 20853.250 87
136

Partial
Intercept .445 1 .445 .027 .869 .000
BOK1T 20.469 1 20.469 1.257 .267 .021
BOK2T .675 1 .675 .041 .839 .001
BOK3T 21.030 1 21.030 1.291 .261 .022
BOK4T 13.544 1 13.544 .831 .366 .014
BOK5T 1.293 1 1.293 .079 .779 .001
BOK6T 52.152 1 52.152 3.202 .079 .052
BOK7T 71.292 1 71.292 4.377 .041 .070
BOK1T * BOK2T .312 1 .312 .019 .890 .000
BOK1T * BOK3T 61.147 1 61.147 3.754 .058 .061
BOK1T * BOK4T 15.501 1 15.501 .952 .333 .016
BOK1T * BOK5T 16.732 1 16.732 1.027 .315 .017
BOK1T * BOK6T 11.876 1 11.876 .729 .397 .012
BOK1T * BOK7T 13.480 1 13.480 .828 .367 .014
BOK2T * BOK3T 2.416 1 2.416 .148 .702 .003
BOK2T * BOK4T 2.521 1 2.521 .155 .695 .003
BOK2T * BOK5T 2.519 1 2.519 .155 .696 .003
BOK2T * BOK6T 11.934 1 11.934 .733 .396 .012
BOK2T * BOK7T 9.057 1 9.057 .556 .459 .009
BOK3T * BOK4T 7.564 1 7.564 .464 .498 .008
BOK3T * BOK5T 37.609 1 37.609 2.309 .134 .038
BOK3T * BOK6T 45.727 1 45.727 2.807 .099 .046
BOK3T * BOK7T 18.473 1 18.473 1.134 .291 .019
BOK4T * BOK5T 117.583 1 117.583 7.218 .009 .111
BOK4T * BOK6T 2.100 1 2.100 .129 .721 .002
BOK4T * BOK7T 20.029 1 20.029 1.230 .272 .021
BOK5T * BOK6T 71.677 1 71.677 4.400 .040 .071
BOK5T * BOK7T 11.271 1 11.271 .692 .409 .012
BOK6T * BOK7T 1.897 1 1.897 .116 .734 .002
Error 944.785 58 16.289
Total 16610.778 87
137

Type IV Partial
Sum of Mean Eta
Intercept 19.570 1 19.570 1.795 .186 .030
BOK1T 80.152 1 80.152 7.350 .009 .112
BOK2T 7.766 1 7.766 .712 .402 .012
BOK3T .135 1 .135 .012 .912 .000
BOK4T 1.769 1 1.769 .162 .689 .003
BOK5T 10.838 1 10.838 .994 .323 .017
BOK6T 38.164 1 38.164 3.500 .066 .057
BOK7T 129.501 1 129.501 11.876 .001 .170
BOK1T * BOK2T .323 1 .323 .030 .864 .001
BOK1T * BOK3T 9.080 1 9.080 .833 .365 .014
BOK1T * BOK4T 16.523 1 16.523 1.515 .223 .025
BOK1T * BOK5T 2.911 1 2.911 .267 .607 .005
BOK1T * BOK6T 4.673 1 4.673 .429 .515 .007
BOK1T * BOK7T .318 1 .318 .029 .865 .001
BOK2T * BOK3T 12.847 1 12.847 1.178 .282 .020
BOK2T * BOK4T 22.194 1 22.194 2.035 .159 .034
BOK2T * BOK5T 9.777 1 9.777 .897 .348 .015
BOK2T * BOK6T 1.524 1 1.524 .140 .710 .002
BOK2T * BOK7T 5.273 1 5.273 .484 .490 .008
BOK3T * BOK4T 76.649 1 76.649 7.029 .010 .108
BOK3T * BOK5T .153 1 .153 .014 .906 .000
BOK3T * BOK6T 5.096 1 5.096 .467 .497 .008
BOK3T * BOK7T 1.059 1 1.059 .097 .756 .002
BOK4T * BOK5T 24.313 1 24.313 2.230 .141 .037
BOK4T * BOK6T .868 1 .868 .080 .779 .001
BOK4T * BOK7T 14.600 1 14.600 1.339 .252 .023
BOK5T * BOK6T 66.894 1 66.894 6.135 .016 .096
BOK5T * BOK7T 44.898 1 44.898 4.117 .047 .066
BOK6T * BOK7T .022 1 .022 .002 .965 .000
Error 632.460 58 10.904
Total 18440.250 87
138

Type IV Sum Mean Partial Eta
Intercept 13.108 1 13.108 1.285 .262 .022
BOK1T 6.629 1 6.629 .650 .423 .011
BOK2T .018 1 .018 .002 .966 .000
BOK3T 5.001 1 5.001 .490 .487 .008
BOK4T 21.185 1 21.185 2.077 .155 .035
BOK5T .195 1 .195 .019 .891 .000
BOK6T 7.016 1 7.016 .688 .410 .012
BOK7T 26.330 1 26.330 2.581 .114 .043
BOK1T * BOK2T 2.798 1 2.798 .274 .602 .005
BOK1T * BOK3T 7.257 1 7.257 .711 .402 .012
BOK1T * BOK4T 10.403 1 10.403 1.020 .317 .017
BOK1T * BOK5T 2.211 1 2.211 .217 .643 .004
BOK1T * BOK6T .007 1 .007 .001 .980 .000
BOK1T * BOK7T .199 1 .199 .020 .889 .000
BOK2T * BOK3T 1.763 1 1.763 .173 .679 .003
BOK2T * BOK4T 1.229 1 1.229 .120 .730 .002
BOK2T * BOK5T 16.023 1 16.023 1.571 .215 .026
BOK2T * BOK6T .083 1 .083 .008 .929 .000
BOK2T * BOK7T .028 1 .028 .003 .958 .000
BOK3T * BOK4T 38.494 1 38.494 3.774 .057 .061
BOK3T * BOK5T 49.536 1 49.536 4.856 .032 .077
BOK3T * BOK6T .071 1 .071 .007 .934 .000
BOK3T * BOK7T 1.196 1 1.196 .117 .733 .002
BOK4T * BOK5T 5.924 1 5.924 .581 .449 .010
BOK4T * BOK6T .088 1 .088 .009 .926 .000
BOK4T * BOK7T 15.776 1 15.776 1.546 .219 .026
BOK5T * BOK6T 14.852 1 14.852 1.456 .232 .024
BOK5T * BOK7T 6.297 1 6.297 .617 .435 .011
BOK6T * BOK7T 2.115 1 2.115 .207 .651 .004
Error 591.656 58 10.201
Total 20569.230 87
139
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual
Figure 6. Residual plot matrices for transformed dependent variables

140
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual
Figure 6 (continued). Residual plot matrices for transformed dependent variables

141
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual

142
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual

143
Variable MB4T: Measurement, Analysis, and Knowledge Management,
Transformed
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual

144
Observed
Predicted
Std. Residual
Observed
Predicted
Std. Residual
The residual plot matrices seemed to indicate heteroscedasticity in most
variables. The trends between observed and predicted values were all positive, and most
145
of them were fairly linear. The relationships between residuals and predicted values
seemed to be random in all cases. However, the plots of most variables indicated weak
positive correlations between residuals and observed values. In the case of ISO1T, the
relationship seemed not only positive, but exponential as well.
The results of the ANCOVA tests were subjected to a significance screening
level of P < 0.05. All tests were considered to be two-tailed because the hypotheses did
not specify the direction of relationships between independent and dependent variables.
Subjecting the data in Table 21 to these criteria showed that all models were significant.
In each model, the significant main effects and interactions were retained for
further analysis. Table 22 shows the terms retained for each refined model.
Terms that did not test significantly in the ANCOVA were not retained in the
refined models. This was to insure that ensuing analyses would have better estimates of
error and thus be more powerful (i.e., be more able to avoid type II error).
Table 22
Terms retained in refined models
Variable Terms Included in Refined Model
ISO1T BOK6T
ISO2T BOK2T, BOK4TxBOK7T
ISO3T BOK2T
ISO4T BOK2T, BOK4T, BOK6T
ISO5T BOK2T, BOK4T, BOK6T, BOK2TxBOK5T, BOK4TxBOK6T
MB1T BOK1T, BOK3TxBOK4T
BOK1T, BOK1TxBOK2T, BOK1TxBOK4T, BOK2TxBOK4T,
MB2T
BOK5TxBOK6T, BOK6TxBOK7T
146
Terms retained in refined models
Variable Terms Included in Refined Model
MB5T BOK7T, BOK4TxBOK5T, BOK5TxBOK6T

MB6T BOK1T, BOK7T, BOK3TxBOK4T, BOK5TxBOK6T,
MB7T BOK3TxBOK5T
Analysis of the Refined Model Using ANCOVA
The main effects and two-way interactions, listed in Table 22, were analyzed
using SPSS 11.5. Each dependent variable was entered into the univariate ANOVA tool,
and independent variables were entered as covariates. Independent variables were not
treated as fixed or random factors.
It was necessary to use covariates in the analysis because the independent
variables (BOK1T … BOK7T) were not controlled factors, and because the data
consisted of survey responses. It would not have been possible to elicit responses in
controlled factor level combinations, and therefore it would not have been possible to
create and run a factorially designed experiment. Random factor analysis could not be
used because the variables represented pre-selected phenomena on a finite scale.
Had it been appropriate to use factorial analysis, the number of responses
required to test the main effects and two-way interactions would have been prohibitive.
A full factorial analysis would have required a*b*c…*n responses, where a, b, c, etc.,
are the number of levels for each factor and n is the number of replicates. With 4
midpoints on the un-transformed variables and 24 on the transformed variables, it would

147
have required thousands of responses to analyze either the un-transformed or the
transformed data in a full factorial experiment. A partial factorial analysis would have
required 1 degree of freedom for each main factor level encountered and 1 for each
factor level combination encountered in the interactions. There were not enough
responses to analyze just three main effects and their two-way interactions on any of the
response variables, given the number of factor levels encountered in the data.
Because of the reasons just discussed, it was necessary to conduct the analyses on
the refined models using ANCOVA. All terms were entered into the univariate ANOVA
tool as covariates. The models were built to test all main effects and two-way
interactions presented in Table 22. Type IV sums of squares were used, and the
significance level was set at 0.05. The results of these analyses are shown in Tables 23-
34, which show the significance test results, estimates of parameters, and lack of fit tests.
Figures 7-18 show the residuals plot matrices.
Table 23
ANCOVA tests, parameter estimates, and lack of fit test, dep. var. ISO1T
Type IV Sum Mean Partial Eta Observed

Source of Squares df Square F Sig. Squared Power(a)
Corrected
1182.677(b) 1 1182.677 59.780 .000 .410 1.000
Model
Intercept 785.406 1 785.406 39.700 .000 .316 1.000
BOK6T 1182.677 1 1182.677 59.780 .000 .410 1.000
Error 1701.403 86 19.784
Total 33336.000 88
Corrected
2884.080 87
Total
a Computed using alpha = .05
b R Squared = .410 (Adjusted R Squared = .403)
148
Partial
Std. 95% Confidence Eta Observed
Parameter B Error t Sig. Interval Squared Power(a)
Lower Upper
Bound Bound
Intercept 1.37
8.646 6.301 .000 5.918 11.374 .316 1.000
2
BOK6T .644 .083 7.732 .000 .479 .810 .410 1.000
Sum of Mean Partial Eta Observed

Source Squares df Square F Sig. Squared Power(a)
Lack of Fit 637.041 27 23.594 1.308 .193 .374 .856
Pure Error 1064.362 59 18.040
Observed
Predicted
Std. Residual
Model: Intercept + BOK6T
Figure 7. Residuals plot matrix, dependent variable ISO1T

149
Table 24
Type IV
Corrected
1471.040(b) 2 735.520 40.632 .000 .489 1.000
Model
Intercept 807.852 1 807.852 44.628 .000 .344 1.000
BOK2T 185.338 1 185.338 10.238 .002 .108 .886
BOK4T *
317.732 1 317.732 17.552 .000 .171 .985
BOK7T
Error 1538.679 85 18.102
Total 26709.917 88
Corrected
3009.719 87
Total
Partial
Lower Upper
Bound Bound
Intercept 7.999 1.197 6.680 .000 5.618 10.379 .344 1.000
BOK2T .351 .110 3.200 .002 .133 .569 .108 .886
BOK4T *
.017 .004 4.190 .000 .009 .024 .171 .985
BOK7T

Lack of
1518.429 81 18.746 3.703 .104 .987 .492
Fit
Pure Error 20.250 4 5.063
150
Observed
Predicted
Std. Residual
Model: Intercept + BOK2T + BOK4T*BOK7T
Table 25
Type IV Partial
Sum of Mean Eta Observed
Corrected
820.153(b) 1 820.153 36.907 .000 .300 1.000
Model
Intercept 659.211 1 659.211 29.665 .000 .256 1.000
BOK2T 820.153 1 820.153 36.907 .000 .300 1.000
Error 1911.090 86 22.222
Total 21575.563 88
Corrected
2731.244 87
Total
151
Partial
Lower Upper
Bound Bound
Intercept 1.325.44
7.202 .000 4.574 9.831 .256 1.000
2 7
BOK2T 6.07
.552 .091 .000 .372 .733 .300 1.000
5

Lack of
473.690 18 26.316 1.245 .253 .248 .753
Fit
Pure Error 1437.401 68 21.138
Observed
Predicted
Std. Residual
Model: Intercept + BOK2T

152
Table 26
Type IV Partial
Corrected
1139.607(b) 3 379.869 32.537 .000 .537 1.000
Model
Intercept 229.520 1 229.520 19.659 .000 .190 .992
BOK2T 4.221 1 4.221 .362 .549 .004 .091
BOK4T 294.610 1 294.610 25.234 .000 .231 .999
BOK6T 117.439 1 117.439 10.059 .002 .107 .880
Error 980.700 84 11.675
Total 23593.226 88
Corrected
2120.307 87
Total
Partial
Lower Upper
Bound Bound
Intercept 5.150 1.161 4.434 .000 2.840 7.459 .190 .992
BOK2T -.055 .092 -.601 .549 -.238 .127 .004 .091
BOK4T .367 .073 5.023 .000 .222 .512 .231 .999
BOK6T .310 .098 3.172 .002 .116 .504 .107 .880

Lack of Fit 889.367 77 11.550 .885 .646 .907 .224
Pure Error 91.333 7 13.048
153
Observed
Predicted
Std. Residual
Model: Intercept + BOK2T + BOK4T + BOK6T
Table 27
Type IV Partial
Corrected
1694.097(b) 5 338.819 26.477 .000 .618 1.000
Model
Intercept 14.599 1 14.599 1.141 .289 .014 .184
BOK2T 17.284 1 17.284 1.351 .249 .016 .210
BOK4T 182.013 1 182.013 14.224 .000 .148 .961
BOK6T 345.944 1 345.944 27.034 .000 .248 .999
BOK2T *
31.152 1 31.152 2.434 .123 .029 .338
BOK5T
BOK4T *
121.399 1 121.399 9.487 .003 .104 .861
BOK6T
Error 1049.320 82 12.797
Total 25122.863 88
Corrected
2743.417 87
Total
154
Partial
Lower Upper
Bound Bound
Intercept -2.855 2.673 -1.068 .289 -8.172 2.462 .014 .184
BOK2T -.165 .142 -1.162 .249 -.449 .118 .016 .210
BOK4T .629 .167 3.771 .000 .297 .960 .148 .961
BOK6T 1.203 .231 5.199 .000 .743 1.663 .248 .999
BOK2T *
.010 .006 1.560 .123 -.003 .023 .029 .338
BOK5T
BOK4T *
-.035 .011 -3.080 .003 -.058 -.012 .104 .861
BOK6T
Partial
Lack of Fit 1049.320 82 12.797 . . 1.000 .
Pure Error .000 0 .
155
Observed
Predicted
Std. Residual
Model: Intercept + BOK2T + BOK4T + BOK6T
+ BOK2T*BOK5T + BOK4T*BOK6T
Table 28
ANCOVA tests, parameter estimates, and lack of fit test, dep. var. MB1T
Type IV Partial
Corrected
1789.636(b) 2 894.818 44.006 .000 .512 1.000
Model
Intercept 186.277 1 186.277 9.161 .003 .098 .849
BOK1T 483.213 1 483.213 23.764 .000 .221 .998
BOK3T *
214.261 1 214.261 10.537 .002 .111 .894
BOK4T
Error 1708.042 84 20.334
Total 24322.000 87
Corrected
3497.678 86
Total
156
Std. 95% Confidence Partial Eta Observed

Lower Upper
Bound Bound
Intercept 4.176 1.380 3.027 .003 1.432 6.920 .098 .849
BOK1T .500 .102 4.875 .000 .296 .703 .221 .998
BOK3T *
.011 .003 3.246 .002 .004 .017 .111 .894
BOK4T

Lack of Fit 1240.563 64 19.384 .829 .721 .726 .463
Pure Error 467.479 20 23.374
Observed
Predicted
Std. Residual
Figure 12. Residuals plot matrix, dependent variable MB1T

157
Table 29
Partial
Type IV Sum Mean Eta Observed
Corrected
2258.567(b) 6 376.428 27.342 .000 .678 1.000
Model
Intercept .253 1 .253 .018 .893 .000 .052
BOK1T 84.740 1 84.740 6.155 .015 .073 .688
BOK1T *
5.899 1 5.899 .429 .515 .005 .099
BOK2T
BOK1T *
117.042 1 117.042 8.501 .005 .098 .821
BOK4T
BOK2T *
194.085 1 194.085 14.097 .000 .153 .960
BOK4T
BOK5T *
.240 1 .240 .017 .895 .000 .052
BOK6T
BOK6T *
41.687 1 41.687 3.028 .086 .037 .405
BOK7T
Partial
Error 1073.856 78 13.767
Total 18635.250 85
Corrected
3332.424 84
Total

Lower Upper
Bound Bound
Intercept .309 2.281 .135 .893 -4.233 4.851 .000 .052
BOK1T .708 .286 2.481 .015 .140 1.277 .073 .688
BOK1T *
-.007 .011 -.655 .515 -.028 .014 .005 .099
BOK2T
158

Lower Upper
Bound Bound
BOK1T *
-.026 .009 -2.916 .005 -.043 -.008 .098 .821
BOK4T
BOK2T *
.037 .010 3.755 .000 .018 .057 .153 .960
BOK4T
BOK5T *
-.001 .006 -.132 .895 -.013 .011 .000 .052
BOK6T
BOK6T *
.009 .005 1.740 .086 -.001 .019 .037 .405
BOK7T

Lack of Fit 1073.856 78 13.767 . . 1.000 .
Pure Error .000 0 .
159
Observed
Predicted
Std. Residual
Model: Intercept + BOK1T + BOK1T*BOK2T + BOK1T*BOK4T
+ BOK2T*BOK4T + BOK5T*BOK6T + BOK6T*BOK7T
Table 30
Type IV Partial
Corrected 50.51
1748.909(b) 2 874.454 .000 .546 1.000
Model 5
Intercept 20.95
362.741 1 362.741 .000 .200 .995
5
BOK4T 45.77
792.364 1 792.364 .000 .353 1.000
3
BOK1T *
75.945 1 75.945 4.387 .039 .050 .544
BOK6T
Error 1454.109 84 17.311
Total 29689.500 87
Corrected
3203.017 86
Total
160

Lower Upper
Bound Bound
Intercept 5.773 1.261 4.578 .000 3.265 8.280 .200 .995
BOK4T .561 .083 6.766 .000 .396 .727 .353 1.000
BOK1T *
.007 .003 2.095 .039 .000 .013 .050 .544
BOK6T

Lack of Fit 1369.442 77 17.785 1.470 .312 .942 .371
Pure Error 84.667 7 12.095
Observed
Predicted
Std. Residual

161
Table 31
Partial
Corrected
1685.421(b) 2 842.711 42.159 .000 .501 1.000
Model
Intercept 378.462 1 378.462 18.934 .000 .184 .990
BOK7T 63.070 1 63.070 3.155 .079 .036 .419
BOK6T *
23.232 1 23.232 1.162 .284 .014 .187
BOK7T
Error 1679.067 84 19.989
Total 20853.250 87
Corrected
3364.489 86
Total

Lower Upper
Bound Bound
Intercept 6.303 1.449 4.351 .000 3.422 9.184 .184 .990
BOK7T .490 .276 1.776 .079 -.059 1.039 .036 .419
BOK6T *
.011 .010 1.078 .284 -.009 .031 .014 .187
BOK7T

Lack of Fit 1494.776 74 20.200 1.096 .473 .890 .375
Pure Error 184.292 10 18.429
162
Observed
Predicted
Std. Residual
Table 32
Partial
Corrected
1232.708(b) 3 410.903 20.714 .000 .428 1.000
Model
Intercept 461.123 1 461.123 23.246 .000 .219 .997
BOK7T 146.696 1 146.696 7.395 .008 .082 .767
BOK4T *
49.954 1 49.954 2.518 .116 .029 .348
BOK5T
BOK5T *
1.684 1 1.684 .085 .772 .001 .060
BOK6T
Error 1646.472 83 19.837
Total 16610.778 87
Corrected
2879.180 86
Total
163

Lower Upper
Bound Bound
Intercept 5.339 1.107 4.821 .000 3.136 7.541 .219 .997
BOK7T .350 .129 2.719 .008 .094 .606 .082 .767
BOK4T *
.012 .007 1.587 .116 -.003 .026 .029 .348
BOK5T
BOK5T *
.003 .009 .291 .772 -.016 .021 .001 .060
BOK6T

Lack of Fit 1646.472 83 19.837 . . 1.000 .
Pure Error .000 0 .
Observed
Predicted
Std. Residual
Model: Intercept + BOK7T + BOK4T*BOK5T + BOK6T*BOK5T

164
Table 33
Partial
Corrected
1658.948(b) 5 331.790 23.558 .000 .593 1.000
Model
Intercept 87.285 1 87.285 6.197 .015 .071 .691
BOK1T 22.076 1 22.076 1.567 .214 .019 .236
BOK7T 79.493 1 79.493 5.644 .020 .065 .651
BOK3T *
2.769 1 2.769 .197 .659 .002 .072
BOK4T
BOK5T *
120.380 1 120.380 8.547 .004 .095 .823
BOK6T
BOK5T *
32.365 1 32.365 2.298 .133 .028 .322
BOK7T
Error 1140.816 81 14.084
Total 18440.250 87
Corrected
2799.764 86
Total

Lower Upper
Bound Bound
Intercept 3.481 1.398 2.489 .015 .699 6.264 .071 .691
BOK1T .119 .095 1.252 .214 -.070 .308 .019 .236
BOK7T .423 .178 2.376 .020 .069 .777 .065 .651
BOK3T *
.002 .004 .443 .659 -.005 .009 .002 .072
BOK4T
BOK5T *
.029 .010 2.924 .004 .009 .049 .095 .823
BOK6T
BOK5T *
-.021 .014 -1.516 .133 -.048 .007 .028 .322
BOK7T
165

Lack of Fit 1140.816 81 14.084 . . 1.000 .
Pure Error .000 0 .
Observed
Predicted
Std. Residual
Model: Intercept + BOK1T + BOK7T + BOK3T*BOK4T
+ BOK5T*BOK6T + BOK5T*BOK7T
Table 34
Type IV
Corrected
813.172(b) 1 813.172 46.893 .000 .356 1.000
Model
Intercept 2927.314 1 2927.314 168.810 .000 .665 1.000
166
Type IV
BOK3T *
813.172 1 813.172 46.893 .000 .356 1.000
BOK5T
Error 1473.973 85 17.341
Total 20569.230 87
Corrected
2287.145 86
Total
95%
Std. Confidence Partial Eta Observed
Lower Upper
Bound Bound
Intercept 10.123 .779 12.993 .000 8.574 11.673 .665 1.000
BOK3T *
.020 .003 6.848 .000 .014 .026 .356 1.000
BOK5T

Lack of Fit 1351.221 76 17.779 1.304 .353 .917 .411
Pure Error 122.752 9 13.639
167
Observed
Predicted
Std. Residual
Model: Intercept + BOK3T*BOK5T
The lack of fit tests did not score significantly for any of the models. However,
in the cases of variables ISO5T, MB2T, MB5T, and MB6T, there were not enough
degrees of freedom to carry out the lack of fit test.
In a few cases, terms included in the refined models did not test significantly
during the second ANCOVA. These terms were not retained in the final models. Table
35 shows the regression equation for each dependent variable with only the significant
terms retained.
168
Table 35
Final regression equations for each dependent variable
Dep. Variable Final Regression Equation

ISO1T = 8.646 + .644*BOK6T
ISO2T = 7.999 + .351*BOK2T + .017*BOK4TxBOK7T
ISO 3T = 7.202 + .552*BOK2T
ISO4T = 5.150 + .367*BOK4T + .310*BOK6T
ISO5T = .629*BOK4T + 1.203*BOK6T -.035*BOK4TxBOK6T
MB1T = 4.176 + .500*BOK1T + .011*BOK3TxBOK4T
MB2T = .708*BOK1T -.026*BOK1TxBOK4T + .037*BOK2TxBOK4T
MB4T = No significant terms
MB5T = 5.339 + .350*BOK7T
MB6T = 3.481 + .423*BOK7T +.029*BOK5TxBOK6T
MB7T = 10.123 + .020*BOK3TxBOK5T
Analysis of Demographics Data
Each demographics response item was analyzed according to its nature. The
frequencies of responses for each item were illustrated using a chart appropriate for the
situation.
When applicable, items were analyzed to see if there was significant correlation
between responses and mean variable levels (for both independent and dependent
variables). Correlation checks were conducted using the Spearman Correlation

169
Coefficient because the data did not meet normality assumptions. Any correlation
that tested significantly at P < 0.05 was reported and charted or graphed appropriately.
The complete set of correlation data from these analyses is in Appendix G.
All analyses conducted between demographics response items and variables were
done using the untransformed variables. The purpose was to provide a better illustration
of the relationships between demographics items and mean rankings for variables data.
There were two precautions taken so that the untransformed variables could be
used. The first, mentioned above, was that correlations were checked using Spearman’s
Rho to accommodate the non-normality of the data.
The second precaution was that a more conservative probability level was used
when ANOVA techniques were employed. The purpose of the demographics analyses
was not to build models of independent variable-dependent variable relationships, as was
done between the BOK variables and the ISO and MB variables. ANOVA was used
with categorical demographics response items to compare grouped means of variables
data. For example, the mean responses to variables items were compared using NAICS
code prefixes and ASQ regions as grouping categories. Using ANOVA in these
circumstances allowed for a comparison of within- and between-group differences in
variation. When these techniques were used, a more conservative probability level (P <
.01) was used to compensate for the non-normality of the data.
The demographics response items, originally listed in Table 10, are repeated in
Table 36 for reference purposes. The presentation of results follows the order
represented in this table.

170
Table 36
Demographics response items, repeated
Code Description
CQM 0 not a Certified quality manager
1 certified quality manager
EXP Number of years employed as a quality manager
NCERT Number of ASQ certifications held
RESP 1 No decision making authority

7 Other (please specify)
EDU Number of years of education beyond high school
EMPL Number of employees at unit or organization:
1 1-4 7 500-999
2 5-9 8 1000-1499
3 10-19 9 1500-2499
4 20-49 10 2500-4999
5 50-99 11 5000-9999
6 100-499 12 10000+
DIRECT Number of employees under respondent’s direction

NAICS First two numbers of NAICS industrial code
REGION ASQ region number that respondent’s section is in (added after the fact
by extrapolation from respondent tracking data)
GENDER Added after the fact by extrapolation from respondent tracking data
171
Is the Respondent a Certified Quality Manager? (CQM)
The data was checked for significant correlations between respondent status as a
certified quality manager and the mean response levels for all variables. This was
checked using a bivariate Spearman’s Rho test, with the significance level set at P <
0.05. The number of respondents who were currently certified quality managers at the
time they took the survey is charted in Figure 19. All of the Spearman’s coefficients are
reported in Appendix G. Significant correlations are shown in Table 37.
No Response (2)
2.3%
Yes (37)
42.0%
No (49)
55.7%
Figure 19. Number and percent of respondents who are currently certified quality
managers
172
Table 37
Significant correlations between item CQM and variables
Hold CQM?
BOK4 Spearman’s Rho -.218
Sig. (2-tailed) .044
N 86
MB3 Spearman’s Rho -.238
N 85
Respondent status as a CQM tested significantly against the means of BOK4,
Customer-Focused Organizations, and MB3, Customer and Market Focus. Graphs of the
variation in means vs. CQM status are shown in Figures 20 and 21.
4.5
Mean BOK4: Customer Focused Organization
4.0
3.5
3.0
2.5
Missing No Yes
Is the Respondent a CQM?
Figure 20. Mean of BOK4: Customer Focused Organizations vs. item CQM
173
4.4
Mean MB3: Customer and Market Focus

4.2
4.0
3.8
3.6
3.4
Missing No Yes
Is the Respondent a CQM?
Figure 21. Mean of MB3: Customer and Market Focus vs. item CQM
Number of Years, QM Experience (EXP)
The data was checked for significant correlations between the number of years of
quality management experience reported by respondents and the mean response levels
for all variables. This was checked using a bivariate Spearman’s Rho test, with the
significance level set at P < 0.05. Figure 22 shows the distribution of respondent
experience levels. All of the Spearman’s coefficients are reported in Appendix G. The
test did not show any significant correlations.

174
30
20
10
Std. Dev = 7.54

Mean = 14.6
0 N = 88.00
5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0
Years of QM Experience
Figure 22. Number of years QM experience held by respondents
Number of ASQ Certifications Held (NCERT)
The data was checked for significant correlations between the number of ASQ
certifications held by respondents and the mean response levels for all variables. The
analysis was performed using a bivariate Spearman’s Rho test, with the significance
level set at P < 0.05. Figure 23 depicts the number of certifications reported by
respondents. All of the Spearman’s coefficients are reported in Appendix G. Significant
correlations are shown in Table 38.

175
30
Number of Respondents Holding
20
10
0
0 1 2 3 4 5 6 7 8
Number of ASQ Certifications Held
Figure 23. Number of ASQ certifications held by respondents
Table 38
Significant correlations between item NCERT and variables
Number of ASQ Certifications

MB3 Spearman’s Rho -.258
N 87
The number of ASQ certifications held by respondents tested significantly
against the mean of MB3, Customer and Market Focus. The chart in Figure 24 depicts
the MB3 vs. NCERT trend.

176
5.5
Mean MB3: Customer and Market Focus

5.0
4.5
4.0
3.5
3.0
2.5
2.0
0 1 2 3 4 5 6 7 8
Number of ASQ Certifications Held
Figure 24. Mean of MB3: Customer and Market Focus vs. item NCERT
Responsibility Level (RESP)
The mean response levels of all variables were determined using stated
responsibility levels as categories. The variation in response levels was tested for
significant differences in means when compared according to responsibility level.
ANOVA was used, with a significance level of P < 0.01. The conservative probability
level was used to compensate for the non-normality of the data. Figure 25 shows the
breakdown of reported responsibility levels. Categorized response level means are
shown in Tables 39-41. Results of the significance tests are reported for each group of
variables individually in Tables 42-44

177
30
Number of Responses
20
10
0
0
999
2.
Responsibility Level
Legend:
1 No decision making authority
7 Other
999 Don’t know
0 No response
Figure 25. Responsibility level of respondents within their employing organizations
Table 39
Mean responses to BOK variables, sorted by responsibility level
Responsibility BOK BOK BOK BOK BOK BOK BOK

Level 1 2 3 4 5 6 7
0 3.0 2.7 2.0 2.5 2.3 2.0 1.0
1 3.5 3.7 3.0 4.0 3.1 3.4 3.4
2 3.8 3.4 3.7 3.7 3.1 3.6 3.0
3 4.0 3.6 4.1 4.2 3.5 3.9 3.1
4 3.7 3.5 4.2 4.2 3.7 3.8 3.6
178
Mean responses to BOK variables, sorted by responsibility level
Responsibility BOK BOK BOK BOK BOK BOK BOK

Level 1 2 3 4 5 6 7
5 4.2 3.6 4.2 4.5 3.5 4.1 3.5
6 3.9 3.6 3.7 4.2 3.3 3.8 3.3
7 5.0 4.7 5.0 4.0 4.0 4.8 5.0
Table 40
Mean responses to ISO variables, sorted by responsibility level
Responsibility
Level ISO1 ISO2 ISO3 ISO4 ISO5
0 1.0 1.5 2.0 2.5 1.0
1 3.5 3.7 3.3 3.3 3.2
2 4.2 3.6 3.5 3.6 3.6
3 4.3 4.1 3.8 4.0 4.0
4 4.4 4.0 3.7 3.9 4.2
5 4.4 4.2 4.0 4.2 4.2
6 4.3 4.0 3.9 3.8 3.9
7 5.0 5.0 4.0 4.0 5.0
Table 41
Mean responses to MB variables, sorted by responsibility level
Resp. Level MB1 MB2 MB3 MB4 MB5 MB6 MB7

0 4.0 2.5 2.5 1.0 1.3 2.5 2.7
1 4.0 --- 3.5 4.5 3.7 3.0 4.0
2 3.5 3.3 3.9 3.4 3.2 3.3 3.4
3 4.0 3.7 4.2 3.8 3.6 3.7 3.9
4 3.5 3.5 3.9 3.4 3.3 3.7 3.7
5 3.9 3.6 4.3 3.9 3.6 3.7 3.9
6 4.0 3.8 4.1 3.7 3.5 3.7 3.7
7 5.0 5.0 5.0 5.0 4.0 4.0 4.0
179
Table 42
Tests for significant differences in variation of BOK variable responses, grouped by
responsibility level
Sum of Mean
Squares df Square F Sig.
BOK1 * Resp. Level Between Groups 5.307 8 .663 .956 .476
Within Groups 54.804 79 .694
Total 60.111 87
BOK2 * Resp. Level Between Groups 4.042 8 .505 .748 .649
Total 57.403 87
BOK3 * Resp. Level Between Groups 10.600 8 1.325 1.903 .071
Total 65.591 87
Total 71.770 87
BOK5 * Resp. Level Between Groups 6.058 8 .757 1.195 .313
Total 56.105 87
Total 54.819 87
Total 71.239 87
Table 43
Tests for significant differences in variation of ISO variable responses, grouped by
Sum of Mean
ISO1 * Resp. Level Between Groups 18.030 8 2.254 5.571 .000
Total 49.989 87
180
Sum of Mean
Total 58.197 87
ISO3 * Resp. Level Between Groups 6.955 8 .869 1.504 .169
Total 52.630 87
Total 40.216 87
Total 56.495 87
Table 44
Tests for significant differences in variation of MB variable responses, grouped by
Sum of Mean
MB1 * Resp. Level Between Groups 6.146 8 .768 .929 .498
Total 70.678 86
MB2 * Resp. Level Between Groups 7.656 8 .957 1.132 .352
Total 71.894 84
Total 52.764 86
MB4 * Resp. Level Between Groups 13.059 8 1.632 2.095 .046
Total 73.845 86
MB5 * Resp. Level Between Groups 8.096 8 1.012 1.379 .219
Total 65.326 86
MB6 * Resp. Level Between Groups 3.776 8 .472 .589 .784
181
Sum of Mean
Total 66.316 86
Total 47.894 86
The means of several items, including BOK4, ISO1, ISO2, and ISO5, varied
significantly with responsibility level. Figures 26-29 depict these relationships.
5.0
Mean BOK4: Customer-Focused Org.
4.5
4.0
3.5
3.0
2.5
2.0
1.5
0 1 2 3 4 5 6 7 999
Respondent's Responsibility Level
Figure 26. Variation in the mean of BOK4, categorized by respondent’s responsibility
level
182
Mean ISO1: Quality Management System

5
0
0 1 2 3 4 5 6 7 999
Figure 27. Variation in the mean of ISO1, categorized by respondent’s responsibility
level
6
Mean ISO2: Management Responsibility
1
0 1 2 3 4 5 6 7 999
level
183
Mean ISO5: Measurement/Analysis/Improvement

6
0
0 1 2 3 4 5 6 7 999
level
Education Level (EDU)
The data was checked for significant correlations between the number of years of
post-secondary education reported by respondents and the mean response levels for all
variables. The analysis was performed using a bivariate Spearman’s Rho test, with the
significance level set at P < 0.05. Figure 30 shows the education levels reported by
respondents. All of the Spearman’s coefficients are reported in Appendix G. Significant
correlations are shown in Table 45.

184
Number of Respondents who Attained Each Level

30
20
10
0
0 2 3 4 5 6 7 8 10
Years of Post-Secondary Education
Figure 30. Years of post-secondary education attained by respondents
Table 45
Significant correlations between item EDU and variables
Years of Post-Secondary
Education
BOK1 Spearman’s Rho -.219
N 88
ISO4 Spearman’s Rho -.243
N 88
Significant correlations were found between education level and the variables
BOK1, Leadership, and ISO4, Product Realization. These relationships are shown in
Figures 31-32.
185
5.0
4.8
Mean BOK1: Leadership 4.6
4.4
4.2
4.0
3.8
3.6
0 2 3 4 5 6 7 8 10
Figure 31. Mean of BOK1: Leadership vs. item EDU
4.4
Mean ISO4: Product Realization
4.2
4.0
3.8
3.6
3.4
0 2 3 4 5 6 7 8 10
Figure 32. Mean of ISO4: Product Realization vs. item EDU

186
Number of Employees, Respondent’s Division or Organization (EMPL)
The data was checked for significant correlations between the number of
employees at the respondents’ division or organization and the mean response levels for
all variables. The analysis was performed using a bivariate Spearman’s Rho test, with
the significance level set at P < 0.05. Figure 33 shows the distribution of
organization/division sizes that were reported by respondents. All of the Spearman’s
coefficients are reported in Appendix G. Significant correlations are shown in Table 46.
40
Number of Respondents in Category
35
30
20
10 11
10
9
7
3 4 3 3
0
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Number of Employees at Division or Organization
Figure 33. Number of employees at respondent’s primary unit of employment

187
Table 46
Significant correlations between item EMPL and variables
# Employees
BOK1 Spearman’s Rho .245
N 88
N 88
N 88
N 88
N 88
N 88
ISO4 Spearman’s Rho .257
N 88
MB1 Spearman’s Rho .309
N 87
N 85
N 87
N 87
N 87
N 87
188
As listed in Table 46, there were several significant correlations between
variables and the number of employees at a respondent’s division or organizations.
These relationships are depicted by the charts in Figure 34 through Figure 46.
5.5
5.0
Mean BOK1: Leadership
4.5
4.0
3.5
3.0
2.5
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
1-4
5-9
10000+
No Response
Figure 34. Mean of BOK1: Leadership vs. item EMPL

189
5.0
Mean BOK3: Quality Mgmt. Tools

4.5
4.0
3.5
3.0
2.5
2.0
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
1-4
5-9
10000+
No Response
Figure 35. Mean of BOK3: Quality Management Tools vs. item EMPL
5.5
Mean BOK4: Customer-Focused Org.
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Figure 36. Mean of BOK4: Customer-Focused Organizations vs. item EMPL

190
4.5
Mean BOK5: Supplier Performance

4.0
3.5
3.0
2.5
2.0
1.5
1000-1499
1500-2499
2500-4999
100-499
500-999
10-19
20-49
50-99
1-4
5-9
10000+
No Response
Figure 37. Mean of BOK5: Supplier Performance vs. item EMPL
5.5
5.0
Mean BOK6: Management
4.5
4.0
3.5
3.0
2.5
2.0
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Figure 38. Mean of BOK6: Management vs. item EMPL

191
5.0
Mean BOK7: Training/Development

4.5
4.0
3.5
3.0
2.5
2.0
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Figure 39. Mean of BOK7: Training and Development vs. item EMPL
4.5
Mean ISO4: Product Realization
4.0
3.5
3.0
2.5
1000-1499
1500-2499
2500-4999
100-499
500-999
1-4
5-9
10-19
20-49
50-99
10000+
No Response
Figure 40. Mean of ISO4: Product Realization vs. item EMPL

192
5.0
4.5
Mean MB1: Leadership

4.0
3.5
3.0
2.5
2.0
1.5
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Figure 41. Mean of MB1: Leadership vs. item EMPL
5.0
Mean MB2: Strategic Planning
4.5
4.0
3.5
3.0
2.5
2.0
1.5
100-499
500-999
1000-1499
1500-2499
2500-4999
1-4
5-9
10-19
20-49
50-99
10000+
No Response
Figure 42. Mean of MB2: Strategic Planning vs. item EMPL

193
5.5
Mean MB4: Information and Analysis

5.0
4.5
4.0
3.5
3.0
2.5
100-499
500-999
1000-1499
1500-2499
2500-4999
1-4
5-9
10-19
20-49
50-99
10000+
No Response
Figure 43. Mean of MB4: Measurement, Analysis, and Knowledge Management vs.
item EMPL
5.5
Mean MB5: Human Resource Focus
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1000-1499
1500-2499
2500-4999
100-499
500-999
10-19
20-49
50-99
1-4
5-9
10000+
No Response
Figure 44. Mean of MB5: Human Resource Focus vs. item EMPL
194
4.5
Mean MB6: Process Management

4.0
3.5
3.0
2.5
2.0
1-4
5-9
10-19
20-49
50-99
100-499
500-999
1000-1499
1500-2499
2500-4999
10000+
No Response
Figure 45. Mean of MB6: Process Management vs. item EMPL
4.5
Mean MB7: Business Results
4.0
3.5
3.0
2.5
2.0
100-499
500-999
1000-1499
1500-2499
2500-4999
1-4
5-9
10-19
20-49
50-99
10000+
No Response
Figure 46. Mean of MB7: Business Results vs. item EMPL

195
Number of Employees Under Respondent’s Direction (DIRECT)
The data was checked for significant correlations between the number of
employees under the direction of respondents and the mean response levels for all
variables. This was checked using a bivariate Spearman’s Rho test, with the significance
level set at P < 0.05. Figure 47 shows the distribution of respondent experience levels.
All of the Spearman’s coefficients are reported in Appendix G. The test did not show
any significant correlations.
16
14
Number of Respondents Per Level
12
10
0
0
1
2
3
4
5
6
7
10
11
12
13
14
15
16
17
19
20
22
25
30
35
40
45
50
70
300
425
Employees Under Direction of Respondent
Figure 47. Number of employees under direction of the respondent

196
NAICS Code Categories of Respondents’ Employers (NAICS)
The mean response levels of all variables were determined using reported NAICS
code prefixes as categories. The variation in response levels was tested for significant
differences in means when compared against NAICS code categories. ANOVA was
used, with a significance level of P < 0.01. The conservative probability level was used
to compensate for the non-normality of the data and the fact that most NAICS code
categories were represented by very few respondents. Figure 48 depicts the number of
respondents who reported various industrial code categories. Categorized response level
means are shown in Tables 47-49. Results of the significance tests are reported for each
group of variables individually in Tables 50-52

197
60
Number of Respondents from Category

50
40
30
20
10
0
22 31 32 33 44 48 49 54 61 81 92
NAICS Code Category
Legend
22 Utilities
31 Manufacturing: Food, beverage, tobacco, textiles, apparel
32 Manufacturing: Wood, paper, petro-chemical, plastics, and printing
33 Manufacturing: Metal, electrical and electronics, transportation, furniture
44 Retail trade
48 Transportation (except postal and couriers)
49 Transportation: Postal and couriers
54 Professional, scientific, and technical services
61 Educational services
81 Other services (except public administration)
92 Public administration
Figure 48. NAICS Industrial classification prefix of respondent’s employer

198
Table 47
Mean responses to BOK variables, sorted by NAICS code category
NAICS
Code
Category BOK1 BOK2 BOK3 BOK4 BOK5 BOK6 BOK7
22 5.0 4.7 5.0 4.0 4.0 4.8 5.0
31 3.5 3.0 3.3 3.0 2.7 3.2 2.6
32 3.9 3.5 3.7 4.2 3.2 3.8 3.2
33 3.9 3.6 4.0 4.1 3.4 3.9 3.3
44 3.5 4.3 4.0 5.0 2.6 4.0 2.4
48 4.5 4.3 4.7 5.0 4.3 5.0 4.2
49 4.0 3.3 3.3 3.5 3.1 3.6 3.4
54 3.9 3.3 3.6 4.1 3.3 3.7 3.3
61 3.0 2.5 2.3 2.3 2.3 2.2 1.9
81 3.8 3.4 3.9 4.0 2.6 3.8 3.1
92 4.5 4.0 5.0 5.0 3.7 4.0 3.8
Table 48
Mean responses to ISO variables, sorted by NAICS code category
NAICS
Code
Categor
y ISO1 ISO2 ISO3 ISO4 ISO5
22 5.0 5.0 4.0 4.0 5.0
31 4.5 3.3 3.3 2.5 2.8
32 4.4 4.2 3.8 3.9 3.9
33 4.3 3.9 3.7 3.9 3.9
44 4.0 5.0 4.5 3.7 4.2
48 5.0 4.7 4.8 4.8 5.0
49 4.0 3.7 3.5 4.0 4.0
54 4.1 3.7 3.7 3.5 3.7
61 1.5 2.1 2.6 2.7 1.9
81 4.3 3.6 3.5 3.6 3.9
92 5.0 5.0 5.0 4.0 4.2
199
Table 49
Mean responses to MB variables, sorted by NAICS code category
NAICS
Code
Category MB1 MB2 MB3 MB4 MB5 MB6 MB7
22 5.0 5.0 5.0 5.0 4.0 4.0 4.0
31 4.0 3.0 3.5 2.0 3.3 3.5 3.2
32 4.2 3.4 4.1 3.8 3.8 3.5 3.7
33 3.7 3.6 4.0 3.6 3.3 3.5 3.7
44 4.0 5.0 5.0 4.0 3.7 4.0 4.2
48 4.5 5.0 5.0 5.0 4.3 4.5 5.0
49 3.5 4.0 4.0 4.0 4.0 4.0 4.0
54 3.8 3.4 4.4 3.6 3.6 3.6 3.6
61 3.8 2.3 2.8 2.3 2.2 2.8 2.5
81 3.7 3.2 4.7 3.5 3.7 4.0 3.9
92 4.5 3.5 5.0 5.0 4.0 4.0 4.0
Table 50
NAICS code category
Sum of Mean
BOK1 * NAICS Code Cat. Between Groups 3.936 10 .394 .540 .857
Total 60.111 87
BOK2 * NAICS Code Cat. Between Groups 5.889 10 .589 .880 .555
Total 57.403 87
BOK3 * NAICS Code Cat. Between Groups 10.349 10 1.035 1.443 .178
Total 65.591 87
BOK4 * NAICS Code Cat. Between Groups 11.070 10 1.107 1.404 .194
Total 71.770 87
BOK5 * NAICS Code Cat. Between Groups 7.329 10 .733 1.157 .333
Total 56.105 87
200
NAICS code category
Sum of Mean
Total 54.819 87
Total 71.239 87
Table 51
NAICS code category
Sum of Mean
ISO1 * NAICS Code Cat. Between Groups 17.677 10 1.768 4.213 .000
Total 49.989 87
Total 58.197 87
ISO3 * NAICS Code Cat. Between Groups 6.323 10 .632 1.051 .410
Total 52.630 87
ISO4 * NAICS Code Cat. Between Groups 7.489 10 .749 1.762 .082
Total 40.216 87
Total 56.495 87
201
Table 52
NAICS code category
Sum of Mean
MB1 * NAICS Code Cat. Between Groups 4.990 10 .499 .577 .827
Total 70.678 86
MB2 * NAICS Code Cat. Between Groups 11.101 10 1.110 1.351 .220
Total 71.894 84
MB3 * NAICS Code Cat. Between Groups 9.415 10 .942 1.651 .109
Total 52.764 86
MB4 * NAICS Code Cat. Between Groups 12.872 10 1.287 1.604 .121
Total 73.845 86
MB5 * NAICS Code Cat. Between Groups 8.178 10 .818 1.088 .382
Total 65.326 86
Total 66.316 86
Total 47.894 86
The analyses showed that the mean of one item varied significantly vs. NAICS
code category. With the significance level set at P < 0.01, the ANOVA showed
significant results for ISO1, Quality Management System. The chart in Figure 49 depicts
this relationship.
202
Mean ISO1: Quality Management System 5
1
22 31 32 33 44 48 49 54 61 81 92
NAICS Code Prefix of Respondent's Employer
Figure 49. Variation in the mean of ISO1, categorized by NAICS code prefix of
respondent’s employer
ASQ Region (REGION)
The mean response levels of all variables were determined using as categories the
ASQ region in which respondents’ sections were located as categories. The variation in
response levels was tested for significant differences in means when compared against
ASQ regions. ANOVA was used, with a significance level of P < 0.01. The
conservative probability level was used to compensate for the non-normality of the data.
Figure 50 depicts the number of respondents who responded from, the number of
sections in, and the response rate for each region. Categorized response level means are
203
shown in Tables 53-55. Results of the significance tests are reported for each group
of variables individually in Tables 56-58. No significant relationships were found.
# Sections Responding to Survey 12
10
R10 (15)
R11 (29)
R12 (16)
R13 (15)
R14 (27)
R15 (28)
R1 (12)
R2 (11)
R3 (10)
R4 (15)
R5 (11)
R6 (21)
R7 (13)
R8 (12)
R9 (17)
ASQ Region*
*Number of sections per region shown in parentheses
Percentages represent return rates for each region
Figure 50. Number of returned surveys by ASQ region

204
Table 53
Mean responses to BOK variables, sorted by ASQ region
ASQ
Region BOK1 BOK2 BOK3 BOK4 BOK5 BOK6 BOK7
1 3.3 3.0 4.0 4.0 3.2 3.4 3.5
2 4.0 3.4 3.9 3.7 2.7 3.8 3.3
3 4.1 3.9 4.0 4.1 3.5 4.0 3.6
4 4.1 3.7 4.1 4.7 3.5 4.2 3.6
5 4.0 3.8 4.3 4.1 3.7 4.2 3.4
6 3.5 3.3 3.6 3.7 3.4 3.4 3.1
7 3.3 3.2 3.7 4.3 2.8 3.3 2.8
8 4.3 3.9 4.1 4.3 3.1 4.1 2.7
9 3.9 3.5 3.9 4.2 3.2 3.9 3.2
10 3.7 3.1 3.8 4.3 3.3 2.9 3.0
11 4.0 3.6 4.1 4.0 3.8 4.1 3.3
12 3.9 3.4 3.7 4.0 3.0 3.8 3.2
13 4.1 3.1 3.8 3.4 3.3 3.9 3.2
14 4.3 3.9 4.0 4.5 3.5 4.1 3.9
15 4.2 3.8 4.0 4.0 3.6 3.8 3.2
Table 54
Mean responses to ISO variables, sorted by ASQ region
ASQ Region ISO1 ISO2 ISO3 ISO4 ISO5

1 4.5 4.0 3.9 4.2 4.2
2 3.8 3.6 3.7 3.6 3.8
3 4.8 4.4 4.0 3.9 4.1
4 4.5 4.5 4.0 4.0 4.3
5 4.5 4.3 4.1 4.4 4.4
6 3.9 3.4 3.4 3.7 3.7
7 4.1 3.6 3.5 3.8 3.8
8 4.3 4.1 3.9 4.1 4.2
9 4.5 4.1 3.8 4.1 3.9
10 4.3 4.1 3.8 4.1 3.6
11 4.3 4.0 3.5 3.8 3.9
12 4.0 3.6 3.7 3.5 3.5
13 4.2 3.7 3.5 3.6 3.9
14 4.3 4.1 3.8 4.2 4.2
15 4.2 4.0 3.8 3.6 3.6
205
Table 55
Mean responses to MB variables, sorted by ASQ region
ASQ
Region MB1 MB2 MB3 MB4 MB5 MB6 MB7
1 3.5 3.5 4.0 4.3 3.2 3.8 3.6
2 3.5 2.5 3.3 3.0 3.0 3.0 2.8
3 4.4 4.0 4.2 4.3 3.8 3.9 4.0
4 4.0 4.4 4.8 4.0 3.9 3.9 4.0
5 3.8 3.1 3.5 3.8 3.3 3.1 3.4
6 3.2 3.2 3.8 3.4 2.9 3.2 3.3
7 3.3 3.2 4.3 3.3 3.6 3.4 3.7
8 4.1 3.8 4.3 3.8 3.3 3.5 3.8
9 4.0 3.4 4.1 3.9 3.6 3.6 4.0
10 3.2 3.0 4.0 3.3 3.3 3.7 3.6
11 3.9 3.8 4.0 3.4 3.2 3.7 3.6
12 4.1 3.4 4.0 3.2 3.3 3.6 3.7
13 3.9 3.4 3.8 3.6 3.2 3.7 3.4
14 4.3 4.2 4.3 4.1 3.8 3.9 4.1
15 3.6 3.4 4.3 3.7 3.6 3.3 3.6
Table 56
Tests for significant differences in variation of BOK variable responses, grouped by ASQ
region
Sum of Mean
BOK1 * ASQ Region Between Groups 8.245 14 .589 .829 .636
Total 60.111 87
Total 57.403 87
Total 65.591 87
Total 71.770 87
206
Tests for significant differences in variation of BOK variable responses, grouped by ASQ
region
Sum of Mean
Total 56.105 87
Total 54.819 87
Total 71.239 87
Table 57
Tests for significant differences in variation of ISO variable responses, grouped by ASQ
region
Sum of Mean
ISO1 * ASQ Region Between Groups 5.596 14 .400 .657 .807
Total 49.989 87
ISO2 * ASQ Region Between Groups 10.100 14 .721 1.095 .377
Total 58.197 87
Total 52.630 87
Total 40.216 87
Total 56.495 87
207
Table 58
Tests for significant differences in variation of MB variable responses, grouped by ASQ
region
Sum of Mean
MB1 * ASQ Region Between Groups 11.617 14 .830 1.012 .452
Total 70.678 86
MB2 * ASQ Region Between Groups 15.206 14 1.086 1.341 .206
Total 71.894 84
MB3 * ASQ Region Between Groups 8.668 14 .619 1.011 .452
Total 52.764 86
MB4 * ASQ Region Between Groups 10.229 14 .731 .827 .638
Total 73.845 86
Total 65.326 86
Total 66.316 86
Total 47.894 86
Respondent Gender (GEN)
The mean response levels of all variables were determined using respondent
gender as categories. The variation in response levels was tested for significant
differences in means when compared against gender. ANOVA was used, with a
significance level of P < 0.01. The conservative probability level was used to
compensate for the non-normality of the data. Figure 51 depicts the breakdown of
208
respondents by gender. Categorized response level means are shown in Tables 59-
61. Results of the significance tests are reported for each group of variables individually
in Tables 62-64. No significant relationships were found.
Female
13.6%
Male
86.4%
Figure 51. Percentage of respondents, categorized by gender
Table 59
Mean responses to BOK variables, sorted by gender
Gender BOK1 BOK2 BOK3 BOK4 BOK5 BOK6 BOK7

Male 3.9 3.6 3.9 4.1 3.4 3.8 3.3
Female 4.0 3.2 3.8 4.2 3.1 3.9 2.9
209
Table 60
Mean responses to ISO variables, sorted by gender
Gender ISO1 ISO2 ISO3 ISO4 ISO5

Male 4.2 3.9 3.7 3.8 3.9
Female 4.3 3.9 3.8 4.0 4.1
Table 61
Mean responses to MB variables, sorted by gender
Gender MB1 MB2 MB3 MB4 MB5 MB6 MB7

Male 3.8 3.6 4.1 3.7 3.4 3.6 3.7
Female 3.9 3.3 4.0 3.4 3.5 3.4 3.7
Table 62
gender
Sum of Mean
BOK1 * Gender Between Groups .026 1 .026 .038 .846
Total 60.111 87
BOK2 * Gender Between Groups 1.675 1 1.675 2.585 .112
Total 57.403 87
Total 65.591 87
Total 71.770 87
Total 56.105 87
210
gender
Sum of Mean
Total 54.819 87
BOK7 * Gender Between Groups 1.310 1 1.310 1.611 .208
Total 71.239 87
Table 63
gender
Sum of Mean
ISO1 * Gender Between Groups .125 1 .125 .215 .644
Total 49.989 87
Total 58.197 87
Total 52.630 87
Total 40.216 87
Total 56.495 87
211
Table 64
gender
Sum of Mean
MB1 * Gender Between Groups .175 1 .175 .211 .647
Total 70.678 86
Total 71.894 84
Total 52.764 86
Total 73.845 86
Total 65.326 86
Total 66.316 86
Total 47.894 86
212
CHAPTER 5
Discussion
Conclusions
The analyses pointed to several key relationships between independent and
dependent variables. Respondents rated the performance of their employers as good, or
better, for 7 of the 19 variables. Mean respondent ratings were significantly higher than
‘good’ for only one variable but in spite of this finding, there were several important,
positive correlations between independent variables, their two-way interactions, and
dependent variables.
There were also some important relationships between demographics response
items and both independent and dependent variables. Contemplating these results may
help organizations accurately measure, and fine-tune, their quality management efforts.
The following subsections contain discussions about the conclusions from this
study. Descriptive statistics related to mean response levels are discussed first, followed
by an evaluation of the study hypotheses and conclusions regarding demographics item
responses and their implications.

213
Descriptive Statistics
The first three research objectives were focused on assessing the stated
effectiveness of the respondents’ employing organizations in implementing the BOK,
ISO 9001:2000, and MBNQA category requirements. The mean response for each
variable was tested using a two-tailed t-test, with the significance level at 0.05, to
determine if there was a difference from the value 4.0 (corresponding to good).
The only variable to score significantly above 4.0 was ISO1 (quality management
system). This indicated that for all other categories, the average respondent felt his/her
organization’s performance could not be rated as better than good.
The response to several variables was not significantly different from 4.0,
meaning that the average respondent felt his/her organization’s level of implementation
could be rated as good. These variables are listed in below:
1. BOK1: Leadership
2. BOK3: Quality management tools
3. BOK4: Customer-focused organizations
4. ISO2: Management responsibility
5. ISO5: Measurement, analysis, and improvement
6. MB3: Customer and market focus
Average responses to the remainder of variables were all significantly less than
4.0, but all were greater than 3.0. This means that respondents rated their organizations’
implementation levels between somewhat and good. Variables in this category are listed
below:
1. BOK2: Strategy development and deployment

214
2. BOK5: Supplier performance
3. BOK6: Management
4. BOK7: Training and development
5. ISO3: Resource management
6. ISO4: Product realization
7. MB1: Leadership
8. MB2: Strategic planning
9. MB4: Measurement, analysis, and knowledge management
10. MB5: Human resource focus
11. MB6: Process management
12. MB7: Business results
There were some apparent discrepancies in results. The first was in regards to
leadership. BOK1 (leadership) did not score significantly different from 4.0, but MB1
(leadership) scored significantly less than 4.0. The actual average scores were only
slightly different (3.915 vs. 3.805), yet this difference was enough to make one, but not
the other, significantly less than 4.0. Each variable was composed of two items; both
had organizational leadership as a sub-category, while the second sub-category for
BOK1 was team processes, and for MB1 was social responsibility. It was the identical
element in each variable that scored differently. Organizational leadership scored a
mean of 4.11 for BOK1 and 3.92 for BOK1. The sub-categories that were unique to
their variables, BOK1B (team processes) and MB1B (social responsibility) scored
virtually the same (3.70 and 3.69 respectively). Both had nearly identical standard
deviations and confidence intervals. It is unclear what led to such a discrepancy.

215
The second apparent discrepancy was related to management. The variable
ISO1 (quality management system) was the only variable to score significantly higher
than 4.0. But the variable BOK6 (management) scored significantly less than 4.0. ISO1
focuses specifically on ISO-mandated quality management system requirements. BOK6
focuses on other aspects of management, in addition to the quality management system.
Comparison of the means of each categorical sub-item indicated that it was the other
items in BOK6 that caused the apparent discrepancy. BOK6D (quality system) scored
4.11 (which was not significantly higher than 4.0) and all other aspects of BOK6 scored
less than 4.0.
A third apparent discrepancy was between ISO5 (measurement, analysis, and
improvement), which scored as ‘good’, and MB4 (measurement, analysis, and
knowledge management), which scored significantly less than ‘good’. However, the two
items are more similar in name than they are in actuality. ISO5 focuses primarily on
measuring and analyzing data about products, processes, customer satisfaction, and
quality systems. MB4 starts with measurement of operations, but also focuses on the
whole organization, and on strategic decision-making, innovation, and knowledge
management. Given the different nature of the two variables, it was not surprising that
the more complicated requirement, MB4, scored less than ISO5.
Respondents rated their organizations’ implementation levels as ‘good’ or better
than ‘good’ in several key areas. Respondents rated ISO1 (quality management system)
as better than ‘good’. Two key elements of quality management systems, BOK3 (quality
management tools), and ISO5 (measurement, analysis, and improvement) scored as
‘good’. Respondents felt that their organizations were doing a good job of taking
216
management responsibility (ISO2) and possibly with exercising leadership
(discussed above). Customer focus also scored as ‘good’ for both BOK4 (customer-
focused organizations), and MB3 (customer and market focus).
Despite these strong points, respondents rated their organizations as being less
than strong in key outcomes-oriented variables. The scores for ISO4 (product
realization) and MB7 (business results) were both less than 4.0 (3.850 and 3.688
respectively).
Considering these responses in the absence of the remainder of results might
have lead to an uncomfortable conclusion. With key quality system variables being
rated higher than outcome-oriented variables, it might appear that there is a
disconnection between quality efforts and resulting quality. However, there were some
key quality system variables that respondents rated as less than good.
It is also possible that respondents might have over-rated their quality
management systems and use of quality tools. However, a study by Michalisin and
White (2001) indicated that managers tend to make accurate statements about their
organizations’ commitment to quality.
Given the complex nature of organizations and the fact that several variables
scored less than ‘good’, it is probable that some of the lower-scoring variables were
contributing factors to the ratings for outcome-oriented variables.
For example, both measures of strategic planning, BOK2 (strategy development
and deployment) and MB2 (strategic planning) scored significantly less than ‘good’.
Strategic planning is an activity that focuses an organization’s direction on strengths,

217
weaknesses, opportunities, and threats. Deficiencies in strategic planning could very
well contribute to deficiencies in outcomes-oriented quality measures.
Another pair of related variables that scored significantly less than ‘good’ was
BOK7 (training and development) and MB5 (human resource focus). If organizations
are struggling with some aspects of how they relate with, employ, and develop their
human resources, then it is possible that even well-designed quality systems may not
function up to their potentials.
Other variables scored significantly less than ‘good’, including BOK5 (supplier
performance), ISO3 (resource management), and MB6 (process management). All three
of these variables represent critical areas of operations that can contribute to, or detract
from, outcomes-centered measures of quality.
The mean responses to all variables presented some interesting results with
potentially important implications for organizations, quality professionals, and quality
practitioners. The purpose of the analysis portion of this study was to identify, and
discuss, the possible effects of stated performance for independent variables items (BOK
categorical requirements), and interactions between independent variables items, on the
performance for dependent variables items (ISO and MB categorical requirements).
Both of the outcomes-focused quality measures discussed to this point, ISO4 (product
realization) and MB7 (business results), were included in the dependent variables sets.
The ensuing discussion is focused on testing the normality assumptions and on
the ANCOVA results, which were aimed at evaluating the hypotheses proposed earlier.
218
Normality Assumptions and Hypotheses
Normality Assumptions
Sufficient evidence suggests that the means and variances are not normally
distributed. All variables data were significantly non-normal according to all indicators
and tests. The normal probability plots of independent variables indicates curvature in
most instances. Performing a squared transformation of the data reshaped nearly all the
distributions, making them sufficiently symmetric to proceed with the ANCOVA, but
this did not bring most variables within the limits of normality.
It is possible that the structure of the Likert response scale contributed to the non-
normality of the means and variances. The distributions of all variables are skewed to
the left. Providing a scale with more differentiation between choices might have helped
to center the distributions by allowing a broader range of response values. The centering
that resulted from the squared transformation supports this assertion. A seven- or ten-
point scale may have resulted in normally distributed responses.
Variation in the independent variables did significantly predict a portion of the
variation in the dependent variables, in all cases except for one. The variable MB4T
(measurement, analysis, and knowledge management) did not have any significant terms
after the refined models were tested using ANCOVA. All other dependent variables had
at least one significant term, and the lack of fit test did not score significantly for any
model. In some cases, there were not enough degrees of freedom to calculate an F-score
for the lack of fit test, but in each case the partial Eta score was 1.0.
219
Hypotheses One and Two
7
ISOn  C1 BOK1  C 2 BOK 2  ...  C7 BOK 7  C
x , y 1
xy BOK x BOK y
H 10 : All C n and C xy  0
7
MBn  C1 BOK1  C2 BOK 2  ...  C7 BOK 7  C
x , y 1
xy BOK x BOK y
H 2 0 : All C n and C xy  0
There was sufficient evidence to suggest that there were significant main effects
between independent variables and dependent variables except in the case of MB4T
(measurement, analysis, and knowledge management). There are many similar aspects
to the categorical items of the BOK, the ISO criteria, and the MBNQA, so the existence
of significant main effects was not unexpected.
There is also sufficient evidence to support the existence of significant two-way
interactions between independent variables on the dependent variables ISO2T, ISO5T,
MB1T, MB2T, MB3T, MB6T, and MB7T. These results were not unexpected either,
given the complex nature of organizations.
The following subsections contain discussions about the analysis for each
dependent variable. The discussions are focused on significant main effects and two-
way interactions. Table 35 is repeated in this section as Table 65 for quick reference.
220
Table 65
Final regression equations for each dependent variable
Dep. Variable Final Regression Equation

ISO1T = 8.646 + .644*BOK6T
ISO2T = 7.999 + .351*BOK2T + .017*BOK4TxBOK7T
ISO 3T = 7.202 + .552*BOK2T
ISO4T = 5.150 + .367*BOK4T + .310*BOK6T
ISO5T = .629*BOK4T + 1.203*BOK6T -.035*BOK4TxBOK6T
MB2T = .708*BOK1T -.026*BOK1TxBOK4T + .037*BOK2TxBOK4T
MB4T = No significant terms
MB5T = 5.339 + .350*BOK7T
MB6T = 3.481 + .423*BOK7T +.029*BOK5TxBOK6T
MB7T = 10.123 + .020*BOK3TxBOK5T
ISO1T: Quality management system
The mean response to ISO1T (quality management system) was positively
correlated with BOK6T (management). It was not surprising that there is a positive
correlation between the two variables; as discussed previously, BOK6 is composed of a
broader set of management issues. As ratings for BOK6 rose, ratings for ISO1 should
rise as well, since all of the elements of ISO1 are also covered in BOK6.
221
ISO2T: Management responsibility
There was an error in transcribing ISO2 and its sub-categorical items onto the
questionnaire. Sub-category one, management commitment, was mistakenly used as the
categorical label instead of being included as a sub-category. Two aspects of
management review, sub-category six, were each listed as separate sub-categories.
Therefore the number of sub-categories was correct, but one item was not included and
two items represented another. This error carried over from the first draft and
throughout all reviews and development activities for the study. The error was not
detected until the last stages of data analysis, as Chapter Five was being drafted. It is not
known if this error biased the results, and, if there was any bias, whether or not it skewed
the mean response up or down.
The mean response to ISO2T (management responsibility), as it appeared in the
questionnaire, was positively correlated with BOK2T (strategy development and
deployment) and with a two-way interaction between BOK4T (customer-focused
organizations) and BOK7T (training and development). The factors with significant,
positive correlations to management responsibility were all outside the realm of
manufacturing operations. Responses indicated that in organizations with a focus on
strategy and a combined focus on customers and employees, management took
responsibility for quality. Management and quality management tools did not factor in.
ISO3T: Resource management
The mean response to ISO3T (resource management) was positively correlated
with BOK2T (strategy development and deployment). This indicated that improvements
in providing resources, managing human resources, creating adequate infrastructure, and

222
creating a proper work environment were all tied with making improvements in
developing and deploying strategies. It seems reasonable that organizations doing a poor
job developing and implementing strategies would be inherently unable to manage
resources because they would be spending resources in the wrong pursuits and/or
pursuing mismatched goals that do not necessarily compliment each other.
ISO4T: Product realization
The mean response to ISO4T (product realization) was positively correlated with
the main effects of two variables: BOK4T (customer-focused organizations) and BOK6T
(management). This indicates that developing and producing quality products is tied to
having effective management and to being customer-focused.
There was no correlation with quality management tools (BOK3T) or with
supplier performance (BOK5T). Both of these items are related conceptually to product
quality and incremental improvement, but there is no correlation with product
realization. It would be interesting to know if the results would be the same if the use of
quality management tools or the management of supplier performance were inadequate.
ISO5T: Measurement, analysis, and improvement
The mean response to ISO5T (measurement, analysis, and improvement) is
positively correlated with two main effects, BOK4T (customer-focused organizations)
and BOK6T (management), and negatively correlated with a two-way interaction
between the same two variables.
It is possible that this result indicated a non-linear relationship between the
dependent and independent variables, one in which increasing performance for BOK4T
and BOK6T would present diminishing returns. The mathematical justification for this
223
is that as the mean response values rise, the product of BOK4T and BOK6T (a
subtractive term in the equation) would become increasingly significant. Figure 52
shows curvature in the response surface, suggesting that this proposition might be
accurate.
Figure 52. Contour plot of the regression equation for ISO5T
The X-axis represents BOK4T, the Y-axis represents BOK6T, and the Z-axis represents
ISO5T. The shape of the surface does show some curvature.
MB1T: Leadership
The mean response to MB1T (leadership) is positively correlated with BOK1T
(leadership) and with the interaction between BOK3T (quality management tools) and
BOK4T (customer-focused organizations). The correlation between MB1T and BOK1T
was to be expected. The positive interaction between BOK3T and BOK4T is interesting
224
in that using quality management tools might not seem, on the surface, to be related
to leadership. However, these results indicated that when an organization is customer-
focused, then successful use of quality management tools could help to improve
leadership. Perhaps quality management tools provide the operational means to
implement customer focus, indicating that management is exercising leadership in
achieving quality.
MB2T: Strategic planning
The mean response to MB2T (strategic planning) is significantly correlated with
several terms. It is positively associated with BOK1T (leadership) and the interaction
between BOK2T (strategy development and deployment) and BOK4T (customer-
focused organizations). It is negatively associated with the interaction between BOK1T
(leadership) and BOK4T (customer-focused organizations).
The tie-in with leadership is understandable. Organizations cannot develop and
deploy strategies if management is not exercising leadership. Scattered efforts cannot
produce or carry out effective strategies.
Instead of having a significant, direct correlation upon strategic planning
(MB2T), the effect of strategy development and deployment (BOK2T) effect was
moderated by having a customer-focused organization (BOK4T). This suggests that
without customer focus, it would be possible to develop and deploy ineffective and/or
inappropriate strategies.
As was the case with ISO5T, the negative interaction term is hard to interpret
because the terms involved also have positive effects upon strategic planning. It is
225
possible that this relationship also exhibits diminishing returns from improvement
efforts.
MB3T: Customer and market focus
The mean response to MB3T (customer and market focus) is positively correlated
with BOK4T (customer focused organizations) and the interaction between BOK1T
(leadership) and BOK6T (management). The direct tie between similar categories was
predicable. The interaction between leadership and management suggests that efforts to
achieve customer and market focus need to be driven from the top down. The
motivation needs to come from management through leadership, not just by decree.
MB4T: Measurement, analysis, and knowledge management
No main effects or interactions are significantly associated with MB4T
(measurement, analysis, and knowledge management). The lack of a significant
correlation with BOK3T (quality management tools) is somewhat surprising. It is
possible that the MBNQA focus on organization-level analysis and knowledge
management is sufficiently different from BOK3C (measurement; assessment and
metrics) in the minds of respondents to elicit uncorrelated responses. It is also possible
that because BOK3C is only one item among three (within BOK3), it did not sufficiently
alter the mean response to cause a significant correlation between BOK3T and MB4T.
MB5T: Human resource focus
The mean response to MB5T (human resource focus) is positively correlated with
BOK7T (business results). This suggests that organizations with better business results
make human resource issues a higher priority than ones that are less successful.
If the converse relationship were also true (that business results are positively
226
correlated with a human resource focus), then this result would be ironic. It would
indicate that organizations with less impressive business results should increase their
focus on human resources.
MB6T: Process management
The mean response to MB6T (process management) is positively correlated with
BOK7T (training and development) and with the interaction between BOK5T (supplier
performance) and BOK6T (management). This result makes sense intuitively. Process
management requires a good deal of technical expertise, which would be enhanced by a
focus on training and development. It also requires commitment from management. The
interaction between management and supplier performance focuses suggests that a focus,
driven by management, across the supply chain is required.
MB7T: Business results
The mean response to MB7T (business results) is positively correlated with the
interaction between BOK3T (quality management tools) and BOK5T (supplier
performance). This suggests that a focus on quality management tools, across the supply
chain, is required to drive good business results.
While this finding can hardly settle the debate about the effect of quality efforts
on the bottom line, it is interesting that according to respondent ratings, effective use of
quality management tools by all involved parties was the only factor effecting business
results. Some might respond that this reflects bias of practitioners in the quality
profession. However it is unlikely that 88 respondents who had no connection with each
other could have jointly, and subconsciously, manipulated answers to such a broad set of
questions, covering complex material, so that the results would turn out as they did.
227
Hypotheses Three through Five: Correlations With Demographics Items
H30: ISOn is not correlated with DEMa

H31: ISOn is correlated with DEMa
H40: MBn is not correlated with DEMa

H41: MBn is correlated with DEM a
H50: BOKn is not correlated with DEMa

H51: BOKn is correlated with DEMa
The responses to some of the demographics questions yielded correlations with
the mean responses of some variables. This section contains discussions about those
demographic question-variable correlations that tested significantly using Spearman’s
Rho with the significance level set at 0.05, as presented in Chapter 4.
Is the Respondent a Certified Quality Manager? (CQM)
Respondent status as a certified quality manager is negatively correlated with
BOK4 (customer-focused organizations) and MB3 (customer and market focus). This
might suggest that certified quality mangers had a heightened perception of what it
means to be customer- and market-focused and/or were more sensitive to the their
organizations’ relationship with customers and the market. Perhaps the knowledge and
professional activity requirements associated with being a CQM made these respondents
more aware of the reality of their organizations’ external relationships.
Number of ASQ Certifications Held (NCERT)
The number of ASQ certifications held by respondents is negatively correlated
with MB3 (customer and market focus). As with respondent status as a CQM, this result
may indicate that respondents who were more professionally active were either better-
228
qualified to make such judgments or were more in tune with their organizations’
external relationships.
Responsibility Level (RESP)
There is significant variation in the mean responses to ISO1 (quality management
system), ISO2 (management responsibility), and ISO5 (measurement, analysis, and
improvement) as differentiated by respondent responsibility level. The profile of
respondents (shown in Figure 25) indicates that most respondents were employed at
levels 2, 3, 5 and 6. These responses correspond to departmental or top management at a
single location (2 and 3) and to organization-level management of either single or
multiple units (5 and 6). Figures 27-29 show that respondents who stated their
responsibility level at 3, 5, and 6 rated these variables almost identically. The most
significant difference, among these most well represented groups, was the mean
responses of participants at responsibility level 2. This may indicate that respondents
who were managers at the department level had a different perception of management,
the quality management system, and measurement, analysis, and improvement than those
at higher levels of management. Perhaps the view of those who work “in the trenches”
was slightly less optimistic because of their closer involvement with daily operations.
Education Level (EDU)
The amount of post-secondary education obtained by respondents is negatively
correlated with BOK1 (leadership) and with ISO4 (product realization). The majority of
respondents indicated that they had attained 4-6 years of post-secondary education,
placing them at the bachelor’s and master’s degree levels, or somewhere in between. In
the case of both variables, there is a sharp drop in mean response levels after the two-
229
year post-secondary education level. As with respondent status as a CQM and the
number of ASQ certifications held by respondents, it may be that the heightened level of
education provided these respondents with a better awareness of actual conditions. It is
interesting that in each case, perceptions of performance decreased as education and
qualification levels increased.
Number of Employees, Respondent’s Division or Organization (EMPL)
There is positive correlations between the number of employees at the
respondents’ organization and the mean responses to the following variables:
1. BOK1: Leadership
2. BOK3: Quality management tools
3. BOK4: Customer-focused organization
4. BOK5: Supplier performance
5. BOK6: Management
6. BOK7: Training and development
7. ISO4: Product realization
8. MB1: Leadership
9. MB2: Strategic planning
10. MB4: Information and analysis
11. MB5: Human resource focus
12. MB6: Process management
13. MB7: Business results
The trend for each one of the variables listed above is upward. The fact that the
trend is always upward and that it applies to most variables suggests that the general
230
level of quality-related activities is better at larger organizations. This makes sense
intuitively because larger organizations generally have a wider talent pool and a larger
cash flow, allowing them to engage in more quality activities than smaller organizations.
NAICS Code Categories of Respondents’ Employers (NAICS)
There is a significant difference in the mean response to ISO1 (quality
management system) as differentiated by the NAICS code of the respondents’
organization. The most pronounced differences in mean responses occurred in cases
where there were only one or two respondents from the respective code categories. Even
with a significance score of 0.00, it is difficult to lend too much credence to these results
because of the low number of responses in many code categories.
The one exception is with category 54 (professional, scientific, and technical
services), which is represented by 9 respondents. There is a pronounced drop in mean
response from participants employed in code 54 organizations as compared to those
employed by code 32 and 33 organizations (both manufacturing sector codes), which
constituted a majority of the respondent pool. Most of the code 54 respondents were
from quality consulting firms (the two exceptions were employees of testing labs). It is
plausible that the quality management systems at code 54 organizations are significantly
different from manufacturing companies, although the fact that they were rated as less
effective is interesting.
231
EPILOGUE
Interpretation of ANCOVA Results
It is interesting, and possibly significant, that respondents rated their
organizations’ quality management systems as being better than good, but did not rate
anything else better than good. Can a good or better quality management system exist in
an organization in which all other aspects of quality are less than good? Can an
organization have a good or better quality management system without elevating other
aspects of quality into the same range? Flynn et al. (1995), found that high-quality and
low-quality plants in their study had similarly complex quality systems. The difference
between good and poor performance wasn’t the systems in place, it was the
organizations’ ability to use the system.
In light of those observations, it is possible there were mitigating factors that
contributed to the lag between having good quality management systems and achieving
good results in other areas. From a logical standpoint, results must follow efforts, not
precede them. The fact that respondents rated their organizations’ quality management
systems so highly could be a leading indicator of other improvements to follow.
How can organizations improve the other elements of their quality systems?
Many organizations will not pursue improvement projects based solely upon the desire
to achieve higher levels of quality. The answer to how quality systems can be improved
must be built around a strategy that will produce tangible results in terms of the bottom
line.
The two dependent variables in this study that most closely represented end
results are ISO4 (product realization) and MB7 (business results). All of the other
232
quality variables in this study, both independent and dependent, measured aspects of
strategic and operational performance. Implementing all of the BOK elements and
complying with the ISO 9001:2000 and MBNQA requirements are critical to achieving
quality, but in the end organizations must consistently excel with product realization and
business results. If they fail to achieve success in these two areas, it is unlikely that they
will thrive over time.
Organizations can start building a plan for sustained success, based upon the
ANCOVA results of this study, by first aiming to improve product realization (ISO4)
and business results (MB7). The cornerstone, or first level, for this plan would be to
improve product realization by focusing the organization on customers (BOK4) and
strengthening management (BOK6), and to improve business results by effectively using
quality management tools (BOK3), both internally and within supplier organizations
(BOK5).
The next step in building an action plan would be to add a second level by
looking into the activities that help to continuously improve the first-level activities. The
most direct counterparts to BOK4 (customer-focused organizations) and BOK6
(management) within the dependent variable sets are MB3 (customer and market focus)
and ISO2 (management responsibility) respectively. The most direct counterparts to
BOK3 (quality management tools) are ISO5 (measurement, analysis, and improvement),
MB4 (measurement, analysis, and knowledge management), and MB6 (process
management). The most direct counterpart to BOK5 (supplier performance) is ISO4
(product realization).
233
The action plan can be extended to a second level by adding the BOK
elements that correlate positively with MB3, ISO2, ISO5, MB4, and MB6 (selected in
the previous paragraph). This requires adding the remainder of the BOK elements.
MB3 (customer and market focus) is correlated in part with BOK1 (leadership). ISO2
(management responsibility) is correlated in part with BOK2 (strategy development and
deployment) and BOK7 (training and development).
In addition to adding the elements just discussed, organizations must continue to
focus on first-level activities. All of the BOK elements from the first level are positively
correlated with some of the dependent variables, listed in the previous paragraph, that
drive the selection of second-level activities.
The two-level plan can be represented with models for improving product
realization and business results. Figures 53 and 54 illustrate these models, which clarify
the points made in the preceding discussion.

234
Figure 53. A model for improving product realization
Figure 54. A model for improving business results

235
BOK4 (customer-focused organizations) and BOK6 (management) were
recurring themes in these models. They were involved in significant correlations with
many of the ISO and MBNQA variables. This suggested that good management and
customer focus were key elements in most quality-related activities and supported their
continued use as primary themes in the tenets of the quality profession.
Interpretation of Analyses Involving Demographics Items
The correlation analyses conducted between mean responses to variables and the
responses to demographics questions produced some interesting results that suggested
other recommendations in addition to the models just discussed. Education level,
obtaining ASQ certifications, and adjusting methods according to organization size all
emerged as potentially significant factors in achieving quality objectives.
The amount of post-secondary education, obtaining CQM status, and the number
of ASQ certifications held all seem to provide managers with a less optimistic view of
their organizations’ quality performance in some key areas. It is possible that the more
optimistic ratings from managers with less education and fewer certifications are more
accurate, but logic and intuition suggest that the first proposition is true.
Another interesting finding was the predominance of respondents (40%) who
indicated their primary unit of employment had 100-499 employees. Results of this
study indicated that there might be an economy of scale that improves the chances of
successfully implementing quality activities as organization size increases. If the profile
of organization sizes in this study is close to the norm across North America, then it
would be advantageous to develop knowledge about how smaller-sized companies can
improve their chances of success in implementing quality activities.

236
The third limitation listed in the study proposal was that the sensitive nature
of topics in the questionnaire might cause some employers to prohibit surveyed
individuals from responding. It is difficult to gauge whether or not this limitation had a
significant effect on the survey response rate. Only two of the individuals who
responded to the invitation to participate indicated that they could not participate because
of restrictions on revealing proprietary information about their employers. It is unknown
how many non-responses were due to this reason. Managers sampled from 93 sections
did not respond to any of the invitations. It is not known how many of these non-
responses might have been due to prohibitions from employers. Judging only by the
number of replies that gave employer restrictions as a reason for declining the invitation
to participate, this limitation does not seem to warrant any further action in future
studies. However, in view of the large number of non-replies, researchers who attempt
to replicate this study may find it helpful to find additional measures to allay employer
concerns about sharing proprietary information.
Recommendations
The sample for this study consisted of one quality manager who was a member of
the executive committee, or who was recommended by an executive committee member
from each North American section of ASQ. Responses were received from 88 of the
252 sections (35%). The theory behind this sampling plan was that active leaders in the
profession would be best able to summarize the performance of their employers as asked
for in the questionnaire, and that their responses would be leading indicators of where
the state of quality management practice was heading.

237
In light of these facts, it should be noted that the data from this study might
not be generalizable to quality management practitioners among North American section
leaders. The number of returned questionnaires was sufficient to yield a power level of
1.00 for all corrected models in the ANCOVAs, but the observed power for individual
model elements ranged across the entire spectrum from very low up to 1.00. Assuming
that the results are generalizable to the specified population is not strongly supported.
Generalizing the data to represent quality management practice among the
employers of all North American quality managers is not justied. Even if the observed
power were very high in all cases, the nature of the sample would prohibit such a broad
generalization. Individuals were not randomly selected. They were hand picked because
of their leadership positions. This biased the results toward the opinions of more
experienced and more professionally active quality managers.
Whether or not the results of this study can be generalized beyond the sample, it
would still be advisable for the broader population of quality managers to pay heed to the
key findings in this study because of the conservative alpha values used throughout the
analyses. Using conservative alpha values made it likely that there were not any falsely
significant results. Any errors in identifying significant correlations would most likely
be due to beta risk or type II error. If the models contained in the results are erroneous, it
is highly probable that they actually should contain more components than stated. It
would be better to pay attention to conservative results where the significant results are
very likely to be accurate than to ignore all results because some potentially significant
relationships might have been missed. The following recommendations were made
according to this philosophy.

238
Quality managers should continue (or start) to pursue additional education
and ASQ certifications (the CQM in particular and others as they are able). They should
pay attention to the significant correlations found between the variables in this study and
use this knowledge to help their employers build plans for improving quality systems,
like the models presented in Figures 53 and 54. They should customize these plans to
accommodate their unique situations and remember to create plans that start with the
fundamentals and then build upon initial successes.
The mean responses to many of the variables in this study are positively
correlated with organization size, so research should be conducted to see if these results
can be confirmed. If the results are confirmed, then further studies should be conducted
to determine how smaller companies can increase their chances of successfully
improving their quality efforts.
Because the sample for this study was narrowly focused on quality management
leaders and broadly focused in other respects, it should be replicated among other
populations. The questionnaire should be administered to quality managers regardless of
ASQ activity, and samples should be focused on organizations in particular industrial
sectors and of particular sizes. If the study is replicated, researchers should look into the
advisability of increasing the range of the Likert response scale to 7 or 10 points.
Combined with a wider Likert scale, a larger sample might also yield responses that are
more normally distributed.
There should be studies that further explore the relationships between education
level, professional certification, and perceptions of organization performance. If the
findings of this study are confirmed and expanded upon, then ASQ could use this
239
information to support continued emphasis on certification and education and to fine-
tune offerings to the needs of industry. Increasing the ability of practitioners to
accurately assess performance can be of great benefit to their employers.

240
REFERENCES
Ackoff, R. L. (1999). Re-creating the corporation: A design of organizations for the 21st
century. New York: Oxford University Press.
AIAG (1994/1995). Advanced product quality planning and control plan: APQP (2nd
printing). Southfield, MI: AIAG.
Askin, R. G., & Dawson, D. W. (2000). Maximizing customer satisfaction by optimal
specification of engineering characteristics. IIE Transactions, 32, 9-20.
ASQ. (2001). Certified quality manager (revised 7/01) [Brochure]. Milwaukee, WI:
Author.
ASQ. (2002). ASQ Organization Manual: July 2002-3. Retrieved April 3, 2003, from
http://www.asq.org/region3/info/ASQorgmanual2002-03.pdf
ASQ Futures Team. (n.d.). Foresight 2020: Implications. Retrieved June 22, 2001,
from http://www.asqnet.org/member/futures/implications.shtml.
ASQ news. (1999, September). Quality Progress, 32(9). Retrieved from
http://www.asqnet.org/members/news/qualityprogress/1999/0999/18news_sept99
.html
241
Brockhoff, K. (1975). The performance of forecasting groups in computer dialogue
and face-to-face discussion. In H. A. Linstone & M. Turoff (Eds). The Delphi
method: Techniques and applications (pp. 291-321). Reading, MA: Addison-
Wesley.
Brown, M., Hitchcock, D., & Willard, M. (1994). Why TQM fails and what to do about
it. New York: Irwin Professional.
Cianfrani, C. A., Tsiakals, J. J., & West, J. E. (2001). ISO 9001:2000 explained (2nd
ed.). Milwaukee, WI: Quality Press.
Court, A. W., Culley, S. J., & McMahon, C. A. (1997). The influence of information
technology in new product development: Observations of an empirical study of
the access of engineering design information. International Journal of
Information Management, 17(5), 359-375.
Crosby, P. B. (1979). Quality is free. New York: McGraw Hill.
Curcovic, S., Melnyk, S., Calantone, R., & Handfield, R. (2000). Validating the
Malcolm Baldrige National Quality Award framework through structural
equation modeling. International Journal of Production Research, 38(4), 765-
791.
Deming, W. E. (1982/2000). Out of the crisis (1st MIT Press edition). Cambridge, MA:
The MIT Press.
Derringer, G. C. (1994). A balancing act: Optimizing a product’s properties. Quality
Progress, 27(6), 51-58.

242
Dowlatshahi, S., & Ashok, M. S. (1997). Design optimization in concurrent
engineering: A team approach. Concurrent Engineering: Research and
Applications, 5(2), 145-154.
Feigenbaum, A. V. (1983). Total quality control (3rd ed.). New York: McGraw Hill.
Ferry, N. M., & Ross-Gordon, J. M. (1998). An inquiry into Schon’s epistemology of
practice: Exploring links between experience and reflective practice. Adult
Education Quarterly, 48(2), 98-112.
Field, A. (2000). Discovering statistics using SPSS for Windows. London: Sage
Publications
Flynn, B. B., Schroeder, R., & Sakakibara, S. (1995). Determinants of quality
performance in high- and low-quality plants. Quality Management Journal, 2(2),
8-25.
Garvin, D. A. (1987). Competing on the eight dimensions of quality. Harvard Business
Review, 65(6), 101-109.
Germain, R., & Spears, N. (1999). Quality management and its relationship with
organizational context and design. International Journal of Quality and
Reliability Management, 16(4), 371-391.
Goldstein, R. (1990). The cost of engineering design corrections. 44th Annual Quality
Congress Transactions, ASQC, San Francisco, 549-554.
Grandzol, J. R., & Gershon, M. (1997). Which TQM practices really matter: An
empirical investigation. Quality Management Journal, 4(4), 43-59.
Gustafsson, A., & Johnson, M. D. (1997). Bridging the quality-satisfaction gap.
Quality Management Journal, 4(3), 27-43.

243
Handfield, R., Ghosh, S., & Fawcett, S. (1998). Quality-driven change and its
effects on financial performance. Quality Management Journal, 5(3), 13-30.
Harris, R. J. (1994). ANOVA: Analysis of variance primer. Itasca, IL: Peacock
Publishers, Inc.
Herrmann, A., Huber, F., & Braunstein, C. (2000). Market-driven product and service
design: Bridging the gap between customer needs, quality management, and
customer satisfaction. International Journal of Production Economics, 66, 77-
96.
Humphreys, M. A., Williams, M. R. & Meier, R. L. (1997). Leveraging the total market
offering in the agile enterprise. Quality Management Journal, 5(1), 60-74.
Hoyer, R. W., & Hoyer, B. B. Y. (2001). What is quality?: Learn how each of eight
well-known gurus answers this question. Quality Progress, 34(7), 53-62.
Ishikawa, K. (1985). What is total quality control? The Japanese way. Englewood
Cliffs, NJ: Prentice Hall.
ISO TC 176 (n.d.). Quality management principles. Retrieved February 8, 2002, from
http://isotc176sc2.elysium-ltd.net/QMP.html.
ISO TC 176 (n.d.). The ISO survey of ISO 9000 and ISO 14000 certificates, tenth cycle:
Up to and including 31 December, 2000. Retrieved February 8, 2002, from
http://isotc176sc2.elysium-ltd.net/survey10thcycle.pdf.
Jayaram, J., Handfield, R., & Ghosh, S. (1997). The application of quality tools in
achieving quality attributes and strategies. Quality Management Journal, 5(1),
75-100.
244
Juran, J. M. (1992). Juran on quality by design: The new steps for planning quality
into goods and services. New York: The Free Press.
Juran, J. M., & Godfrey, A. B. (1999). Juran’s Quality Handbook (5th ed.). New York:
McGraw-Hill.
Kannan, V. R., Tan, K. C., Handfield, R. B. & Ghosh, S. (1999). Tools and techniques of
quality management: An empirical investigation or their impact on performance.
Quality Management Journal, 6(3), 34-49.
Kirchner, J. (2001). Data analysis toolkit #3: Tools for transforming data. Retrieved
October 17, 2003, from Berkely University, EPS 120 – Analysis of
Environmental Data Web site:
http://ist-socrates.berkeley.edu/~epsc120/Toolkits/Toolkit_03.pdf
Michalisin, M. D., & White, G.P. (2001). An empirical study of the posturing-
implementation gap in quality management. Quality Management Journal, 8(1),
34-54.
Mizuno, S. (Ed.). (1979/1988). Management for quality improvement: the 7 new QC
tools (Productivity Press, trans.). Cambridge, MA: Productivity Press. (Original
work published 1979).
Montgomery, D. C. (2001). Design and analysis of experiments (5th ed.). New York:
John Wiley and Sons, Inc.
Newell, S., & Swan, J. (1995). Professional associations as important mediators of the
innovation process. Science Communication, 16(4), 371-387.

245
Nihtila, J. (1999). R&D-Production integration in the early phases of new product
development projects. Journal of Engineering and Technology Management, 16,
55-81.
Office of Technology Assessment. (1977). Technology assessment in business and
government. Washington D. C.: Office of Technology Assessment. (NTIS No.
PB-273164)
Otto, K. N., & Ho, C. M. (1996). Modeling manufacturing quality constraints for
product development. Concurrent Engineering: Research and Applications,
4(4), 333-346.
Pannirselvam, G. P., & Ferguson, L. A. (2001). A study of the relationships between
the Baldrige categories. International Journal of Quality and Reliability
Management, 18(1), 14-34.
Park, T., & Kim, K. J. (1998). Determination of an optimal set of design requirements
using house of quality. Journal of Operations Management, 16, 569-581.
Pawar, K. S., Haque, B., & Barson, R. J. (1999). Establishing concordance within
concurrent engineering teams. Concurrent Engineering: Research and
Applications, 7(3), 215-229.
Piderit, S. K. (2000). Rethinking resistance and recognizing ambivalence: A
multidimensional view of attitudes toward an organizational change. Academy of
Management Review, 25(4), 783-794.
Pirsig, R. M. (1974). Zen and the art of motorcycle maintenance: A inquiry into values.
New York: Morrow.

246
Quality Management Division On-line (2001). The 2001 American Society for
Quality certified quality manager body of knowledge (BOK): How the 2001 BOK
compares to the 1995 BOK. Retrieved February 8, 2002, from http://www.asq-
qmd.org/cqm_bok95.html.
Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and
analysis. International Journal of forecasting, 15, 353-375.
Raturi, A. S., Houshmand, A. A., & Manek, H. S. (1996). A decision support
framework for product development. Quality Management Journal, 4(1), 57-71.
Rummler, G. A., & Brache, A. P. (1995). Improving performance: How to manage the
white space on the organization chart. San Francisco: Jossey-Bass.
Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology.
In H. A. Linstone & M. Turoff (Eds.). The Delphi method: Techniques and
applications (pp. 262-287). Reading, MA: Addison-Wesley.
Schon, D. A. (1983/1995). The reflective practitioner: How professionals think in
action (2nd paperback ed.). Aldershot, England: Arena.
Shewhart, W. A. (1931). Economic control of quality manufactured product. New
York: D. Van Nostrand Co.
Shina, S. G., & Saigal, A. (2000). Using Cpk as a design tool for new system
development. Quality Engineering, 12(4), 551-560.
Smith, G. F. (1998). Determining the cause of quality problems: Lessons learned from
diagnostic disciplines. Quality Management Journal, 5(2), 24-41.
Taguchi, G., & Wu, Y. (1979). Introduction to offline quality control. Negaya, Japan:
Central Japan Quality Control Association.

247
Teixeira, A. F. (1999). How to navigate in the sea of quality management literature.
Strategic Change, 8, 143-151.
Tonk, H. S. (2000). Integrating ISO 9001:2000 and the Baldrige Criteria. Quality
Progress, 33(8), 51-55.
Tsuda, Y. (1997). Models after concurrent engineering product development processes.
Quality Engineering, 9(4), 641-651.
Tushman, M. L., & Scanlan, T. J. (1981). Boundary spanning individuals: Their role in
information transfer and their antecedents. Academy of Management Journal,
24(2), 289-305.
U.S. Department of Commerce. (2001). Economic evaluation of the Baldrige national
Quality Program (Planning Report 01-3). Gaithersburg, MD: National Institute
of Standards and Technology, Technology Administration.
U.S. Department of Commerce. (2002). 2003 Criteria for performance excellence
(Report Number T1114). Gaithersburg, MD: Baldrige National Quality Program,
National Institute of Standards and Technology, Technology Administration.
Vairaktarakis, G. L. (1999). Optimization tools for design and marketing of
new/improved products using the house of quality. Journal of Operations
Management, 17, 645-663.
Van Eijnatten F. M. & Simonse, L. W. L. (1999). Organizing for creativity, quality and
speed in product creation processes. Quality and Reliability Engineering
International, 15, 411-416.
Waddell, D., & Sohal, A. S. (1998). Resistance: A constructive tool for change
management. Management Decision, 36(8), 543-548.

248
Wales, C. E., Nardi, A. H., & Stager, R. A. (1993). Emphasizing critical thinking
and problem solving. In Lynn Curry & Jon F. Wergin (Eds.), Educating
professional: Responding to new expectations for competence and accountability
(pp. 178-211). San Francisco: Jossey-Bass.
Watson, G. H. (1998a). Digital hammers and electronic nails: Tools of the next
generation. Quality Progress, 31(7), 21-26.
Watson, G. H. (1998b). The emancipation of quality: Building bridges and closing
gaps. Quality Progress, 31(8), 110-114.
Wilkinson, A., & Willmott, H. (1996). Quality management, problems, and pitfalls: A
critical perspective. International Journal of Quality and Reliability
Management, 13(2), 55-65.
Willower, D. J. (1994). Dewey’s theory of inquiry and reflective administration.
Journal of Educational Administration, 32(1), 5-22.
Wilson, D. D., & Collier, D. A. (2000). An empirical investigation of the Malcolm
Baldrige National Quality Award causal model. Decision Sciences, 31(2), 361-
383.
249
APPENDIXES
250
APPENDIX A
Glossary of Quality Management Tools, Techniques, and Concepts

251
Affinity diagrams - Classifying issues by categorical columns
Balanced scorecards - A method of tracking organizational performance along both
financial, and non-financial lines using weighted percentages for each indicator
Benchmarking - Comparison of performance in various areas to that of competitors or
other target organizations
Brainstorming – Meetings conducted according to certain rules that facilitate the free
generation and sharing of ideas
Cause and effect (Ishikawa) diagrams - Graphically sorts root causes by man,
machine, materials, and methods
Catch-ball - Give and take method for finalizing plans and allocating resources between
layers of management
Check sheets – Sheets, formatted in various ways, which are used to reduce the
collection of yes/no data to an act of placing a check in the appropriate spot.
Communications techniques – Effective use of written, verbal, and non-verbal
techniques to communicate vertically and horizontally within/without the organization,
receive feedback, assure understanding, assure compliance, etc.
Conflict Resolution – Using brainstorming, bargaining, voting, and other tools to
resolve conflicts among stakeholders
Constraint management – Removing constraints and bottlenecks from various work
processes
Continuous improvement (Kaizen) - A systematic approach to incremental
improvement that follows a plan-do-check-act (or a derivative thereof) cycle

252
Customer management – Accounting for all stakeholders in determining quality
requirements, monitoring success, evaluating feedback ,etc., differentiating between
customer types and prioritizing their requirements, attending to customer service for all
stakeholders, making goals and plans for customer retention, anticipating and planning to
meet future requirements
Cycle time reduction – A major objective of process management techniques that can
use a variety of tools to achieve shortened cycle times by identifying unneeded activities,
more efficient work methods, better machine operations, etc.
Decision support systems - Systems, usually computerized, for aiding in the decision
making process by providing information, analyzing data, etc., often tied into a
knowledge management system
Design of experiments - Statistical experimentation; using Taguchi or factorial methods
to identify relationships between control factors and dependent variables
Empowerment – Giving personnel the authority to act in matters for which they are
given responsibility
Evolutionary operation - Generation of response surface maps for process parameter
optimization through a continuing cycle of incremental improvements that are generated
by factorial experiments
Failure mode effects analysis (FMEA) - Analysis of the seriousness of potential failure
modes by listing all conceivable mechanisms for failure and using multiplicative ratings
to judge severity; multiplication factors include seriousness of the problem, likelihood of
occurrence, and likelihood of detection.

253
Flow charts – Charts that depict the flow of control (using arrows) in a process,
showing various actions (such as start/stop, decisions, and inputs) symbolically as nodes
Focus groups - Use of panels made up of customers, experts, etc., in order to generate
ideas about an issue
Force field analysis - Analysis of factors favoring and fighting a planned action by
listing them graphically on either side of the desired action and rating them on an ordinal
scale
Graphical representation of statistics - Any of a number of graphs used to depict
distribution, variance, correlation, etc., including histograms, curves, box plots, stem-leaf
diagrams, scatter plots, regression curves, etc.
Histogram – A statistical graph that plots numbers of occurrences for measured outputs,
with ranges of output values represented on the horizontal axis, number of occurrences
on the vertical axis, and a smoothed trend line connecting the top (number of
occurrences) of each range so as to provide a visual estimation of the shape of the data’s
distribution
Human resources management – Applying the principles of staffing, professional
development, needs analysis and training, and alignment of personnel with company
strategy.
Information systems – Use information systems and employ appropriate technologies
as needed to facilitate communications and knowledge management
Internal capability analysis – Measuring resources, skills and other capabilities to
identify gaps between what is needed and what is currently available

254
Internal customers – Treating employees and partners as customers and suppliers to
each other throughout the range of production operations, a method that is used to
improve various points of the production process with the goal of overall product
improvement: it also helps personnel who do not deal directly with external stakeholders
to appreciate their own part in achieving quality
Interrelationship diagraphs (or relationship diagrams) – A method of depicting
relationships between components of a system or a complex problem by representing
various components with circles and interconnecting them with arrows that show causal
relationships
Kanbans – The use of small-lot order tags in a pull production system; tags follow the
work in progress until completion. When used in conjunction with careful analysis,
Kanban systems can help producers to adjust lot sizes so as to minimize the amount of
inventory and work in progress.
Knowledge management – Cataloguing data, information, and knowledge, then
creating a system to facilitate access by the right people at the right times; the system
should also help the organization to generate new linkages between pieces of
information, and add value by making tacit knowledge explicit.
Load leveling – Optimization of workflow (for people and/or machines) by identifying
bottlenecks and synchronizing work rates for all operations so as to eliminate stoppages
and slow downs
Loss functions - Tracking financial variations as a function of customer response to
product variation
255
Management principles – Applying management techniques such as leadership,
planning, controlling, organizing, staffing, etc.
Management techniques – Applying various approaches to management (such as
theory X/Y and personality profiling) when called for by the situation at hand and
according to the needs of the organization due to its size, industry, etc.
Matrix diagrams – Categorization of data and information in matrix form to track
and/or illuminate interrelationships between items
Metrology – The science of measurement, encompasses a large variety of methods for
gauging mechanical phenomena; in quality practice the most usual applications are for
dimensional inspection and in-process measurements
Mistake proofing – Elimination of errors by implementing safeguards that prevent the
occurrence of potential failure mechanisms
Motion studies – Analysis of human operator motions in minute detail so as to identify
and eliminate wasteful and/or potentially harmful motions
Motivation – Using various methods to sustain employee enthusiasm
Negotiation – Identifying common goals among various stakeholders and facilitating
cooperation in achieving them
Network diagram (or Arrow diagram) – A process flow charting technique that
utilizes such tools as GANTT charts and associated node diagrams (activities are shown
in their sequences and feedback loops) for the purpose of planning and of identifying
what tasks may hang up a process or may be eliminated altogether

256
Organizational metrics – Measuring all aspects of company strategy deployment by
recognizing and tracking the effects that strategic plans made at the top of the
organization have on all components of the system
Pareto analysis – Ranking of problems according to severity criteria, for example the
number of defects: the purpose is to identify the few problems that cause the bulk of
undesired consequences
PDCA (Plan-do-check-act) – A popular cycle used in improvement projects that
amounts to using iterative feedback loops
Prioritization matrix (or matrix diagram) – A tool in which various priorities are
plotted against each other in the rows and columns of a matrix, and each cell contains the
strength of the relationship between two (or more, if the matrix is multidimensional)
priorities: the values in rows and columns for each priority can then be summed to show
the net effect of a priority based upon how it relates to all other priorities
Process capability studies – Determination of the ability of a process to consistently
produce a certain result, within tolerances that are determined by the probability of
achieving them, once the process is in statistical control
Process control plans – Documents that follow parts throughout production; these plans
are broken down by individual operation and provide instructions for process settings,
quality control activities, etc.
Process decision program chart (PDPC) – Used to examine possible outcomes of an
action by linearly tracing various possibility paths from the action forward in time along
various possible cause and effect chains

257
Process design – Determination of optimal processes using statistical and graphical
techniques
Process management – Application of a project management paradigm to individual
processes; once the process is set, its capability is tracked, goals and plans for
incremental improvement are made, implemented, and followed up on.
Process mapping – The use of flow charts to represent a process from the first operation
to the last, depicting feedback, parallel and serial operations, checkpoints, etc., along the
way
Project management – Prioritizing projects using cost-benefit analyses, planning
activities, estimating costs, monitoring feedback, managing risk, setting milestones,
checking progress, revising plans, maintaining documentation and communications
Qualitative analysis – Accounting for subjective quality measures by translating them
into appropriate objective surrogates, then monitoring and analyzing related data for use
in improvements, planning, etc.
Quality circles – The use of voluntarily formed continuous improvement teams in
focused projects
Quality costs – The process of determining measures that indicate how much poor
quality is costing the company (e.g., due to warranty service, scrap, field failures, list
customers, etc.), tracking performance, and initiating cost reduction projects accordingly
Quality function deployment – The process of building quality into a product by
gathering requirements, then translating them in successive steps into design
requirements, design specifications, process requirements, process specifications, etc.;
each translation is facilitated by team use of a complex matrix

258
Quality policies – Setting quality policies in alignment with company plans, goals,
and policies, then being able to deploy policies through goals, plans, procedures,
instructions, and metrics at the quality function level
Quality system audits – Comparison of quality systems to external quality system
requirements (usually a standard like ISO 9000) comparison of actual quality practices,
to those specified by the system under scrutiny
Registration to quality standards – A process of implementing a quality system and
successfully passing an audit against a standard (like ISO 9000)
Reliability and validity – Insuring that research designs, data collection, and analysis
are performed such that the results of research adequately represent the construct under
study, that various phenomena in the study belong to the construct, and that items
measure what they are supposed to measure
Reliability engineering – Using analysis techniques that represent a system and its
failure modes and applies probability analysis to determine data such as predicted life
cycles and mean time between failures based upon various design options
Resource allocation – Planning for the provision of resources to needed functions, then
monitoring needs, usage, etc., and correctly prioritizing resource deployment
Root cause analysis – Any of a number of tools that are designed to help problem
solvers get to the fundamental cause of a problem; includes such tools as Ishikawa
diagrams, 8D, asking why five times, etc.
Sampling – An acceptance inspection method that operates on the principle that a
sample drawn from a batch of objects will exhibit the same distribution of defects or
defectives as that of the entire batch

259
Scatter diagrams – XY plots that depict each occurrence of a phenomenon as a
single point, with the control variable plotted on the X axis and the dependent variable
on the Y axis. The plot shows if there is a possible trend line to the data
Self assessment – Setting metrics for the performance of the organization at all levels,
and particularly for the quality management system, that can adequately measure outputs
and be compared to goals, checking performance against goals and making correction as
needed, using qualitative techniques to accomplish the same objectives, performing and
using internal audits, etc.
Standard operating procedures – Creation of an optimum procedure for a task or set of
tasks, and the use of documentation to reinforce uniform use of the procedure
Standardization – Creation or selection of a set of standards with which to comply; can
apply to quality systems, engineering practices, business practices, part design
parameters, etc.
Statistical analysis – using the array of statistical tools to monitor and interpret data
from stakeholders, operations, etc., then revising plans and actions accordingly
Statistical process control – Generation of graphs and charts that represent the
statistical properties of a measurement in a process stream (such as a part dimension, a
machine setting, or even a product attribute) so as to gain an early warning when non-
random variation occurs, to see what the level of random variation is, etc.; used in
conjunction with closed loop problem solving in order to eliminate non-random causes
of variation and to assist in the process of decreasing the amount of random variation
Strategic planning and deployment – assessing strategic position, aligning strategy
with the business environment and with stakeholder needs (internal and external),
260
translating strategic plans into goals for management, deploying goals vertically and
horizontally, translating goals into action plans, measuring compliance, etc.
Stratification – the categorization of data along different dimensions, such as tracking
scrap rates by shift, operator, machine, etc.
Supplier management – Setting criteria for the selection of suppliers, communicating
requirements to suppliers, monitoring their performance, developing partnerships with
them, helping them to improve performance, managing supply chain logistics and
optimizing relationships to control costs and quality, etc.
Surveys – Using survey techniques among customers and other stakeholders to obtain
needed systems feedback, then appropriately analyzing the data and deploying it through
revised strategic plans, goals, and action plans as needed
S.W.O.T. – A strategic planning tool in which participants plot strengths, weaknesses,
opportunities, and threats on a four-sectored chart
Systems diagram – a flow chart that is used to depict the functions of a system, rather
than a process stream
Systems thinking – realizing that the principles of complexity apply to business systems
as well, that organizations are complex systems consisting of various functions, partners
and suppliers, resources, personnel, etc.
Team Techniques – being aware of the dynamics of team maturation and using that
knowledge to help build effective teams and facilitate their operations, including the use
of goals and agendas, aligning team missions with company goals, coaching,
empowerment, performance measurement, etc.

261
Theory of constraints – realizing that resources and capabilities are limited, and
compensating accordingly so as to still achieve desired ends, chiefly by realizing that a
few key variables usually govern the bulk of variability in desired outcomes, identifying
those key variables, and controlling them
Time studies – Like motion studies, the idea is to eliminate tasks, operations, etc., that
are deemed to be too wasteful, except that time is studied rather than motion
TQM – a philosophy of managing all aspects of quality with a whole-organization
approach to quality management and improvement. Quality managers are required to
understand the philosophical and practical contributions to this broad topic and apply
various techniques in appropriate situations
Tree diagrams – Sometimes called systematic diagrams, they are used in the
deployment of strategic plans by showing the cascading effect of plans, with the top
action being expanded into subsequent required actions in the various functions as the
level of planning moves vertically down the organization.
Trend analysis – A forecasting tool that attempts to illustrate what may happen in the
future based upon what has been happening (e.g., using past rates of increase in sales to
extrapolate into the future). In the BOK it is to be used in analyzing internal
performance measures, market forces, industry trends, technology, the competition, etc.
Value analysis – A process whereby the attributes valued by customers are determined
and then ranked in order of importance, followed by comparing an organizational self-
analysis of performance on those items to customer rankings of the same

262
Work flow analysis – The use of layout diagrams, flow charts, time and motion
studies, etc., to examine how work gets done from physical layout and process flow
perspectives; the idea is to eliminate bottlenecks, increase efficiency, make better use of
space, etc.
Work instructions – A part of standard operating procedures, these instructions serve
two main purposes: to standardize work methods and to serve as a teaching guide and
reference
263
APPENDIX B
Table B1
A comparison of the ISO 9001:2000 Clauses and the MBNQA Criteria categories with
Part A of the ASQ BOK

264
Table B1
Quality Management Body of Knowledge, ASQ ISO 9001: 2002

2000 MBNQA
Criteria
___ ___
I. Leadership
___ ___
A. Organizational Leadership
a
1. Organizational development Not covered P.1
2. Organizational culture Not covered 5.1
b
3. ASQ code of ethics Not covered 1.2
4. Techniques for facilitating or managing 8.2, 8.4, 8.5 5.2
organizational change
a
5. Organizational roadblocks Not covered P.2
6. Constraint management Not covered Not covered
7. Negotiation techniques Not covered Not covered
8. Motivation techniques Not covered 5.1, 5.3
9. Conflict resolution techniques Not covered Not covered
10. Employee empowerment 5.5 1.1, 2.2
___ ___
B. Team Processes
1. Types of teams Not covered 5.1
2. Team formation and evolution Not covered 5.1
3. Team-building techniques Not covered 5.1
4. Team facilitation techniques Not covered Not covered
5. Team leadership techniques Not covered 5.1
6. Team performance evaluation 7.3 5.1
7. Team reward and recognition Not covered 2.2, 5.1
265
Table B1 (continued)

2000 MBNQA
Criteria
___ ___
II. Strategy Development and Deployment
___ ___
A. Environmental Analysis
1. Legal and regulatory factors 7.2 1.2, 2.1
2. Market forces, industry trends, competitive Not covered 1.1
analysis
c
3. Stakeholder groups 7.2 1.1
4. Technology trends and internal capabilities Not covered 2.1
5. Strengths, weaknesses, opportunities, and Not covered 1.1, 2.1
threats analysis
6. Customer/employee surveys and feedback 7.2 3.1
7. Internal capability analysis 8.2 1.1
___ ___
B. Strategic Planning and Assessment
1. Strategic planning techniques and models Not covered 2.1
2. Competitive comparisons and benchmarks Not covered 2.1
3. Formulating quality policies 5.3 2.1
___ ___
C. Deployment
d e
1. Assure integration between strategic techniques (5.3, 5.4, 2.2
and models 5.5)
2. Deploy strategic goals and objectives into 5.3, 5.4, 7.1 2.2
operational plans and improvement projects
3. Resource allocation planning activities 6.1, 6.3 2.2
4. Metrics and goals that drive organizational 5.3, 5.4 2.2, 4.1
performance
266

2000 MBNQA
Criteria
___ ___
III. Quality Management Tools
___ ___
A. Problem Solving Tools
1. The seven quality control tools Not covered Not covered
2. The seven planning and management tools Not covered Not covered
3. Root cause analysis, plan-do-check-act and 8.5 4.1, 6.1
other, like models
4. Tools for innovation and creativity Not covered 1.1, 5.1, 6.2
5. Cost of quality Not covered 6.1, 6.2, 6.3
___ ___
B. Process Management Approaches
1. Process goals 7.1 6.1, 6.2, 6.3
2. Cycle time reduction Not covered 6.1
3. Process analysis and documentation 7.1, 7.3, 7.5, 6.1, 6.2, 6.3
8.2
4. Theory of constraints Not covered 2.1
f
5. Theory of variation (7.1, 7.5) Not covered
___ ___
C. Measurement: Assessment and Metrics
1. Statistical analysis Not covered Not covered
2. Trend analysis 8.4 4.1, 7.1, 7.2,
7.3, 7.4
3. Process capability Not covered 6.1
4. Reliability and validity 7.3 3.1, 4.2
5. Qualitative assessment Not covered Not covered
6. Analysis and use of survey results Not covered 1.1, 3.2, 5.3,
7.1
g
7. Benchmarking: Internal and external 8.2 2.2, 3.2, 4.1,
6.1, 6.3
267

2000 MBNQA
Criteria
___ ___
IV. Customer-Focused Organizations
___ ___
A. Customer Identification and Segmentation
1. Internal customers Not covered 6.3
2. External customers 7.2, 7.3 3.1, 3.2, 7.1
___ ___
B. Customer Relationship Management and
Commitment
1. Determining and assuring customer 5.2, 6.1, 7.2, 3.1, 3.2, 7.1
satisfaction 7.3, 8.2, 8.4
2. Customer service principles 7.5 Not covered
3. Multiple-customer management Not covered 6.1
4. Customer retention/loyalty Not covered 3.1, 7.1
5. Anticipate customer expectations, priorities, 7.2 3.1, 6.1, 7.1
needs
6. Deploy the voice of the customer through Not covered Not covered
QFD
___ ___
V. Supplier Performance
A. Supplier Selection Strategies and Criteria 7.4 6.2
Techniques for Communicating Requirements to Suppliers 7.4 1.1, 4.2, 6.1
Techniques for Assessment and Feedback of Supplier 7.4, 8.4 6.1
Performance
D. Supplier Improvement Strategies 7.4 Not covered
E. Supplier Certification Programs 7.4 Not covered
F. Partnerships and Alliances with Suppliers Not covered 2.2, 4.2, 6.1,
6.2
G. Logistics and Supply Chain Management Not covered 6.2, 7.4
268

2000 MBNQA
Criteria
___ ___
VI. Management
___ ___
A. Principles of Management
1. Principles of management 5.4, 5.5, 5.6, 1.1, 2.1, 2.2,
6.2, 8.2 4.1, 4.2, 5.1,
6.1, 6.2, 6.3
h
2. Total quality management (4.1, 8.5) i(1.1, 2.1, 2.2,
4.1, 4.2, 5.1,
6.1, 6.2, 6.3)
3. Management styles Not covered 5.1, 5.2, 5.3
4. Organizational structures Not covered 2.1
j
5. Business systems and interdependence of (6.4, 7.3) 5.1
functions
6. Staffing 5.5, 6.2 5.1, 5.2
___ ___
B. Communications
1. Communication techniques Not covered Not covered
2. Information systems 6.3 4.2
3. Knowledge management Not covered 4.2
___ ___
C. Projects
1. Project justification and prioritization Not covered 7.2
techniques
k l
2. Project planning and estimation (7.1, 7.3) (1.2, 2.1, 6.2)
m
3. Monitor and measure project activity 7.3 6.1, 6.2, 6.3
n
4. Project documentation and related procedures 7.3 6.1, 6.2, 6.3,
7.4
269

2000 MBNQA
Criteria
___ ___
D. The Quality System
o
1. The quality function mission 5.3 P.1
2. Quality plan deployment in the organization 8.5 2.2, 6.1, 6.2,
6.3
3. Review of the effectiveness of the quality 5.6, 7.2, 8.2, 6.1, 6.2, 6.3,
system 8.4 7.1, 7.2, 7.3,
7.4
___ ___
E. Quality Models
1. MBQNA Criteria for Performance Not covered Not applicable
Excellence
2. ISO 9000 Not applicable 7.4
3. Major industry and other international 7.2 7.4
standards
___ ___
VII. Training and Development
Alignment With Strategic Planning and Business Needs 6.2 5.1, 5.2, 7.3
B. Training Needs Analysis 6.2 5.1, 5.2
C. Training Materials and Curriculum Development Not covered Not covered
D. Methods of training delivery 6.2 5.2
E. Techniques for Evaluating Training Effectiveness 6.2 5.2, 7.3
a
This section is not scored by examiners, but it does affect the judgment of answers to
scored questions that relate back to this item
b
Covers ethics in general without mentioning the ASQ Code of Ethics
c
Provides a blanket statement about additional requirements
d
These sections relate but do not specifically mention covered topics
270
the Part A of the ASQ BOK
e
This section does not mention vertical and horizontal deployment but otherwise matches
up with the BOK item
f
These sections touch upon monitoring output without ever mentioning variation
g
This section loosely implies external benchmarking
h
These sections cover aspects of TQM without mentioning it or its proponents
i
The MBNQA Criteria are fundamentally about whole-organization management for
quality, however it does not mention TQM or any of its proponents by name
j
These sections do not mention business systems specifically but their content is related
k
These sections touch upon project planning without mentioning any methods and
without discussing estimation
l
Estimation is not covered
m
This section covers the topic without mentioning any specific methods and without
mentioning risk assessment
n
Mentions the use of information from previous products, does not mention repeatability
or any type of planning cycle
o
The MBNQA Criteria system does not discriminate between the quality function and
the rest of the organization. Section P.1 covers organizational mission. While not
graded, it affects the replies to sections that are graded.
271
APPENDIX C
Human Subjects Review Board Approval

272
273
274
APPENDIX D
Transcript of Delphi Panel Proceedings

275
The following text is a complete transcript of the dialogue between Delphi panelists.
Four panelists were selected. Three members participated in round one, and two
continued past that point to the end of round three. One panelist dropped out before ever
participating. The numerical information preceding each comment is the scoring for that
item by the participants, which was originally threaded onto the discussion as attached
replies. In the round three transcript, the mean scores are preceded by an M, and the
interquartile ranges are preceded by and I.

276
Round One Transcript
BOK
3, 5 Change business unit to (dept, division, etc)
2, 5 Phrase "for which you have decision-making authority" implies this will be sent
only to people which have decision making authority. If person does not have
decision making authority, what do they do?
1, 5 Each line item needs further clarification (notes, phrases in parenthesis). People
do not have hours to go back and forth between CQM, Baldrige and ISO 9001
references to refresh their memory on what each item means. If in doubt, I would guess
they will circle "don't know" rather than take the time to clearly
understand what the question is.
5 This applies to front end of survey, not BOK. There needs to be a short introduction
explaining the purpose of the survey and what will be done with the replies.
There needs to be a "sales pitch" to persuaded the person to take the time to reply. I do
not think many people would take the 30 minutes or so needed to think
through replies.
ISO 9001
1, 5 Each line item needs further clarification (notes, phrases in parenthesis). People do
not have hours to go back and forth between CQM, Baldrige and ISO 9001
they will circle "don't know" rather than take the time to clearly understand what the
question is.
277
MBNQA
not have hours to go back and forth between CQM, Baldrige and ISO 9001 references to
refresh their memory on what each item means. If in doubt, I would guess they will
circle "don't know" rather than take the time to clearly understand what the question is.
Demographics
4, 4 Clarify whether intent is years holding title of "Quality Manager" or manager of a
quality function (supervisor, manager, director,VP, etc)
5, 4 Question 4 - Suggest you add phrase (select one) or (select all that apply)
5, 4 Question 5 - Replace "post-secondary" with "beyond high school"
1, 4 Question 6 - replace "organization" with "company"
1, 4 At end of survey, instruct where to send if survey is hardcopy.
General Comment
4, 4 FORMAT: Title each Rating Column on each page: "Not at All" to "Strong" --
Will be easier for respondents and ensure understanding/consistent responses from them
Use Current Versions
4, 5 Use current versions of the three documents. Maintain updates.
Corrections needed in CQM 2001: Wording of Module 7, sections D and E.
MBNQA now called "Baldrige National Quality Program-- 2003 Criteria for
Performance Excellence" - Multiple changes; I noticed esp. 1B, Title of Category 4 and
4B, 5B, and all of categories 6 and 7. I am less familiar with ISO, but should confirm
wordings in version 2000. Include dates in titles, e.g., CQM should state (version 2001)
278
Demographics
5, 3 (Need respondents to identify certifications more fully. There are many.)
1. Do you currently hold a quality manager
certification? From which professional organizations
(ASQ, NHQP, etc.)? ___ Y ___ N Please specify:
Question 3
5, 4 Needs the word "list". Should read "Please list .........."
Responses 3,4,5- BOK, ISO, MBNQA Documents
PANELIST 2 SAID: Each line item needs further clarification (notes, phrases in
parenthesis). People do not have hours to go back and forth between CQM,
Baldrige and ISO 9001 references to refresh their memory on what each item means. If
in doubt, I would guess they will circle "don't know" rather than take the time
to clearly understand what the question is.
RESPONSE: I assume that respondents will rate their organizations based on their
reading/understanding of the categories and sub-categories' titles only, in the 3
master documents. I believe this is adequate, and the survey is LESS threatening as it
stands. In other words, they simply receive 6 page survey we received.
--- Will the survey contain source documents as panelist 2 recommended? That would
be too overwhelming!
It would aid consistency of responses to survey, but few managers would read them,
anyway. --- It would be a generous and educational gesture if your cover letter or
an appendix offered these 3 documents on a website "for further reading, if you are
interested" -- I appreciated having them in one location myself. You also could
279
provide links to the appropriate organization websites. There, respondents could
"learn more about these organizations and assessment systems if you wish..."
RESPONSE #3: INTRO MATERIAL
SCORE= 5
#3 SAID: This applies to front end of survey, not BOK. There needs to be a short
introduction explaining the purpose of the survey and what will be done with the
replies. There needs to be a "sales pitch" to persuaded the person to take the time to
reply. I do not think many people would take the 30 minutes or so needed to
think through replies.
RESPONSE: DARREN'S COVER LETTER SHOULD ACCOMPLISH THIS.
Round Two Transcript
BOK
2, 5 Phrase "for which you have decision-making authority" implies this will be sent
only to people which have decision making authority. If person does not have
decision making authority, what do they do?
1, 5 Each line item needs further clarification (notes, phrases in parenthesis). People
do not have hours to go back and forth between CQM, Baldrige and ISO 9001
they will circle "don't know" rather than take the time to clearly understand what the
question is.
5 This applies to front end of survey, not BOK. There needs to be a short introduction
explaining the purpose of the survey and what will be done with the replies. There needs
280
to be a "sales pitch" to persuaded the person to take the time to reply. I do not think
many people would take the 30 minutes or so needed to think through replies.
ISO 9001
MBNQA
Demographics
4, 4 Clarify whether intent is years holding title of "Quality Manager" or manager of a
quality function (supervisor, manager, director, VP, etc)
5, 4 Question 4 - Suggest you add phrase (select one) or (select all that apply)
5, 4 Question 5 - Replace "post-secondary" with "beyond high school"
1, 4 Question 6 - replace "organization" with "company"
1, 4 At end of survey, instruct where to send if survey is hardcopy.

281
General comment
4, 4 FORMAT: Title each Rating Column on each page: "Not at All" to "Strong" --
Will be easier for respondents and ensure understanding/consistent responses from them
4, 5 Use current versions of the three documents. Maintain updates.
wordings in version 2000. Include dates in titles, e.g., CQM should state (version 2001)
Demographics
5, 3 (Need respondents to identify certifications more fully. There are many.)
1. Do you currently hold a quality manager certification? From which professional
organizations (ASQ, NHQP, etc.)? ___ Y ___ N Please specify:
Question 3
5, 4 Needs the word "list". Should read "Please list .........."
Responses 3,4,5- BOK, ISO, MBNQA DOCUMENTS
PANELIST 2 SAID: Each line item needs further clarification (notes, phrases in
parenthesis). People do not have hours to go back and forth between CQM, Baldrige and
ISO 9001 references to refresh their memory on what each item means. If in doubt, I
would guess they will circle "don't know" rather than take the time to clearly understand
what the question is.

282
RESPONSE: I assume that respondents will rate their organizations based on their
reading/understanding of the categories and sub-categories' titles only, in the 3 master
documents. I believe this is adequate, and the survey is LESS threatening as it stands.
In other words, they simply receive 6 page survey we received. --- Will the survey
contain source documents as panelist 2 recommended? That would be too
overwhelming! It would aid consistency of responses to survey, but few managers
would read them, anyway. --- It would be a generous and educational gesture if your
cover letter or an appendix offered these 3 documents on a website "for further reading,
if you are interested" -- I appreciated having them in one location myself. You also
could provide links to the appropriate organization websites. There, respondents could
Score = 5
#3 SAID: This applies to front end of survey, not BOK. There needs to be a short
introduction explaining the purpose of the survey and what will be done with the
reply. I do not think many people would take the 30 minutes or so needed to think
through replies.
Round Three Transcript
BOK
(For Managers) M=3.5 IR = 3 Phrase "for which you have decision-making

283
authority" implies this will be sent only to people which have decision making
authority.
If person does not have decision making authority, what do they do?
(For general population) M=5 IR=0 Phrase "for which you have decision-making
authority" implies this will be sent only to people which have decision making authority.
If person does not have decision making authority, what do they do?
M=3 IR=4 Each line item needs further clarification (notes, phrases in
M=4.5 IR=1 This applies to front end of survey, not BOK. There needs to be a
short introduction explaining the purpose of the survey and what will be done with the
replies.
There needs to be a "sales pitch" to persuade the person to take the time to reply.
I do not think many people would take the 30 minutes or so needed to think through
replies.
ISO 9001

284
M=4.5 IR=1 (New addition for the 3rd round, evolved as a reply) I think the
instrument should be used with only the 3 documents, no further elaborations and no
accompanying documents. In the cover letter, people may be referred to appropriate
websites or books, however, if they wish to learn more. This would be an educational
opportunity in quality management.
MBNQA
Demographics
M=3.5 IR=1 Clarify whether intent is years holding title of "Quality Manager"
or manager of a quality function (supervisor, manager, director, VP, etc)
M=4.5 IR=1 Question 4 - Suggest you add phrase (select one) or (select all that
apply)
M=3.5 IR=1 Question 5 - Replace "post-secondary" with "beyond high school"
M=3 IR=5 Question 6 - replace "organization" with "company"

285
(New to 3rd round)
M=4.5 IR=1 On second thought, I agree organization is better than
company
M=2.5 IR=3 At end of survey, instruct where to send if survey is hardcopy.
M=4.5, IR = 1 (New to 3rd round) Information on submitting completed surveys
might appear on the cover; very visible there. also in cover letter.
General Comment
M=4 IR=0 FORMAT: Title each Rating Column on each page: "Not at All" to
"Strong" -- Will be easier for respondents and ensure understanding/consistent responses
from them
M=4.5 IR=1 Use current versions of the three documents. Maintain updates.
wordings in version 2000.
Include dates in titles, e.g., CQM should state (version 2001)
Demographics
M=4 IR=2 (Need respondents to identify certifications more fully. There are
many.)
286
1. Do you currently hold a quality manager
certification? From which professional organizations
(ASQ, NHQP, etc.)? ___ Y ___ N Please specify:
Question 3
M=4.5 IR=1 Needs the word "list". Should read "Please list .........."
Responses 3,4,5- BOK, ISO, MBNQA DOCUMENTS
M=4.5 IR=1 PANELIST 2 SAID: Each line item needs further clarification
(notes, phrases in parenthesis). People do not have hours to go back and forth between
CQM, Baldrige and ISO 9001 references to refresh their memory on what each item
means. If in doubt, I would guess they will circle "don't know" rather than take the time
to clearly understand what the question is.
RESPONSE: I assume that respondents will rate their organizations based on
their reading/understanding of the categories and sub-categories' titles only, in the 3
master documents. I believe this is adequate, and the survey is LESS threatening as it
stands. In other words, they simply receive 6 page survey we received. --- Will the
survey contain source documents as panelist 2 recommended? That would be too
overwhelming! It would aid consistency of responses to survey, but few managers
would read them, anyway. --- It would be a generous and educational gesture if your
cover letter or an appendix offered these 3 documents on a website "for further reading,
if you are interested" -- I appreciated having them in one location myself. You also
could provide links to the appropriate organization websites. There, respondents could

287
Withdrawn by author, no other votes tendered
#3 SAID: This applies to front end of survey, not BOK. There needs to be a
short introduction explaining the purpose of the survey and what will be done with the
reply. I do not think many people would take the 30 minutes or so needed to think
through replies.

288
APPENDIX E
Survey Instrument, Solicitation Letter, and Informed Consent Form

289
Note: The appearance of the survey instrument has been altered to fit the
format of this dissertation. In its original form, the instrument used all 12-point fonts
and margins were smaller. Font sizes in the tables had to be reduced significantly to
keep the page format and order intact. These alterations made the response tables harder
to read and changed the aspect ratio of the pages. Also, the page numbers were altered
to fit the numbering of the dissertation document.

290
Department of Technology Systems
Bowling Green State University College of Technology
Bowling Green, OH 43403-0302
(419) 372-2439
Assessment of Current Quality Management Practices
All results will be held confidential. Neither your name nor your information
about employer identity will be associated with any data.
© 2003 Darren Olson
A self-addressed, stamped envelope is provided for your convenience. Please send completed survey to:
Darren Olson
9541 Chad Dr. NW
Bemidji, MN 56601
291
Principles from the 2001 ASQ Certified Quality Manager Body of Knowledge
Please review the principles listed below (lettered sub-categories) and circle the response
that best describes the level of implementation of that sub-category in your organization.
Answer the question in reference to the department or division for which you have
decision-making authority, or in which you are primarily employed.
Very Little
Not At All
Somewhat
Body of Knowledge Categories and Sub-Categories
Strong
Good
Leadership
A. Organizational leadership 1 2 3 4 5 Don’t know
B. Team processes 1 2 3 4 5 Don’t know
Strategy Development and Deployment
A. Environmental analysis 1 2 3 4 5 Don’t know
B. Strategic planning and assessment 1 2 3 4 5 Don’t know
C. Deployment 1 2 3 4 5 Don’t know
Quality Management Tools
A. Problem solving tools 1 2 3 4 5 Don’t know
B. Process management approaches 1 2 3 4 5 Don’t know
C. Measurement: Assessment and metrics 1 2 3 4 5 Don’t know
Customer-Focused Organizations
A. Customer identification and segmentation 1 2 3 4 5 Don’t know
B. Customer relationship management and commitment 1 2 3 4 5 Don’t know
Supplier Performance
A. Supplier selection strategies and criteria 1 2 3 4 5 Don’t know
B. Techniques for communicating requirements to suppliers 1 2 3 4 5 Don’t know
C. Techniques for assessment and feedback of supplier performance 1 2 3 4 5 Don’t know
D. Supplier improvement strategies 1 2 3 4 5 Don’t know
E. Supplier certification programs 1 2 3 4 5 Don’t know
F. Partnerships and alliances with suppliers 1 2 3 4 5 Don’t know
G. Logistics and supply chain management 1 2 3 4 5 Don’t know
Management
A. Principles of management 1 2 3 4 5 Don’t know
B. Communications 1 2 3 4 5 Don’t know
C. Projects 1 2 3 4 5 Don’t know
D. Quality system 1 2 3 4 5 Don’t know
E. Quality models 1 2 3 4 5 Don’t know
Training and Development
A. Alignment with strategic planning and business needs 1 2 3 4 5 Don’t know
B. Training needs analysis 1 2 3 4 5 Don’t know
C. Training materials and curriculum development 1 2 3 4 5 Don’t know
D. Methods of delivery 1 2 3 4 5 Don’t know
E. Evaluating effectiveness 1 2 3 4 5 Don’t know
Please continue to the next page

292
Principles from ISO 9001:2000
Please review the principles listed below (lettered sub-clauses) and circle the response
that best describes the level of implementation of that sub-clause in your organization.
Answer the question in reference to the department or division for which you have
decision-making authority, or in which you are primarily employed.
Very Little
Not At All
Somewhat
ISO 9001:2000 Requirement Clauses and Sub-Clauses
Strong
Good
Quality Management System
A. General requirements 1 2 3 4 5 Don’t know
B. Documentation requirements 1 2 3 4 5 Don’t know
Management Commitment
A. Customer focus 1 2 3 4 5 Don’t know

B. Quality policy 1 2 3 4 5 Don’t know
C. Planning 1 2 3 4 5 Don’t know
D. Responsibility, authority, and communication 1 2 3 4 5 Don’t know
E. Management review 1 2 3 4 5 Don’t know
F. Review output 1 2 3 4 5 Don’t know
Resource Management
A. Provision of resources 1 2 3 4 5 Don’t know
B. Human resources 1 2 3 4 5 Don’t know
C. Infrastructure 1 2 3 4 5 Don’t know
D. Work environment 1 2 3 4 5 Don’t know
Product Realization
A. Planning of product realization 1 2 3 4 5 Don’t know
B. Customer-related process 1 2 3 4 5 Don’t know
C. Design and development 1 2 3 4 5 Don’t know
D. Purchasing 1 2 3 4 5 Don’t know
E. Production and service provision 1 2 3 4 5 Don’t know
F. Control of monitoring and measuring devices 1 2 3 4 5 Don’t know
Measurement, Analysis, and Improvement

A. General 1 2 3 4 5 Don’t know
B. Monitoring and measurement 1 2 3 4 5 Don’t know
C. Control of nonconforming product 1 2 3 4 5 Don’t know

D. Analysis of data 1 2 3 4 5 Don’t know
E. Improvement 1 2 3 4 5 Don’t know

293
Principles from the Baldrige National Quality Program: Criteria for Performance
Excellence
Please review the principles listed below (lettered sub-categories) and circle the response that
best describes the level of implementation of that sub-category in your organization. Answer the
question in reference to the department or division for which you have decision-making
authority, or in which you are employed. Response levels are: (1) Not at all, (2) Very Little, (3)
Somewhat, (4) Good, and (5) Strong.
Very Little
Somewhat
Not at all
Strong
Baldrige Categories and Sub-Categories
Good
Leadership
A. Organizational leadership 1 2 3 4 5 Don’t know
B. Social responsibility 1 2 3 4 5 Don’t know
Strategic Planning
A. Strategy development 1 2 3 4 5 Don’t know
B. Strategy deployment 1 2 3 4 5 Don’t know
Customer and Market Focus

A. Customer and market knowledge 1 2 3 4 5 Don’t know
B. Customer relationships and satisfaction 1 2 3 4 5 Don’t know
Measurement, Analysis, and Knowledge

A. Measurement & analysis of organizational performance 1 2 3 4 5 Don’t know
B. Information and knowledge management 1 2 3 4 5 Don’t know
Human Resource Focus

A. Work systems 1 2 3 4 5 Don’t know
B. Employee learning and motivation 1 2 3 4 5 Don’t know
C. Employee well-being and satisfaction 1 2 3 4 5 Don’t know
Process Management
A. Value creation processes 1 2 3 4 5 Don’t know
B. Support processes 1 2 3 4 5 Don’t know
Business Results
A. Customer-focused results 1 2 3 4 5 Don’t know
B. Product and service results 1 2 3 4 5 Don’t know
C. Financial and market results
D. Human resource results 1 2 3 4 5 Don’t know
E. Organizational effectiveness results 1 2 3 4 5 Don’t know
F. Governance and social responsibility results 1 2 3 4 5 Don’t know

294
Respondent background information
Please answer the following questions using your knowledge about your employing organization
and your understanding of the relevant documents.
1. Do you currently hold an ASQ quality manager certification (please circle one)?
No Yes
2. How many years of experience do you have working in a quality management role (include
all employers)?
_________
3. Please list all ASQ certifications that you currently hold.
4. Please indicate the level of responsibility for which you have the primary decision making
authority in matters related to quality. Check all that apply.
I do not have decision making authority _____
Single location – department or other unit _____
Single location – top management _____
Division or business unit _____
Organization management – single unit _____
Organization management – multiple units _____
Other (please specify) _____
5. Please specify to the nearest number of years the amount of education you have attained
beyond high school.
_________

295
6. In your present position, how many people are employed at your current organization
(please select one)?
____ Not applicable ____ 50-99 ____ 2500-4999
____ 1-4 ____ 100-499 ____ 5000-9999
____ 5-9 ____ 500-999 ____ 10000
____ 10-19 ____ 1000-1499
____ 20-49 ____ 1500-2499
7. How many employees are under your direction as a quality manager (if question is not
applicable, please answer none)?
__________

296
8. What are the first two digits in the NAICS code for your organization (circle
the appropriate choice)?
List of NAICS codes
11 Agriculture, forestry, fishing, and hunting
21 Mining
22 Utilities
23 Construction
31 Manufacturing (food, beverage and tobacco products)
31 Manufacturing (textiles, apparel, leather and allied products)
32 Manufacturing (wood products and paper)
32 Printing and related support activities
32 Manufacturing (petroleum and coal products)
32 Manufacturing (chemical, plastics and rubber products, non-metallic minerals)
33 Manufacturing (primary metal, fabricated metal products, machinery)
33 Manufacturing (computer and electronic products)
33 Manufacturing (electrical equipment, appliances, and components)
33 Manufacturing (transportation equipment)
33 Manufacturing (furniture and related products)
33 Manufacturing (miscellaneous)
42 Wholesale trade
44 Retail trade (motor vehicle and parts dealers)
44 Retail trade (furniture, home furnishings, electronics, appliances)
44 Retail trade (building materials, garden equipment and supplies)
44 Retail trade (food and beverages, health and personal care)
44 Retail trade (gasoline station)
44 Retail trade (clothing and clothing accessories)
45 Retail trade (sporting goods, hobbies, books, music)
45 Retail trade (general merchandise, miscellaneous stores, non-store)
48 Transportation (except postal service, couriers, and messengers)
49 Transportation (postal service, couriers, and messengers)
49 Warehousing
51 Information
52 Finance and insurance
53 Real estate and rental/leasing
54 Professional, scientific, and technical services
55 Management of companies and enterprises
56 Administrative support
56 Waste management and remediation services
61 Educational services
62 Health care
62 Social assistance
71 Arts, recreation, and entertainment
81 Other services (except public administration)
92 Public administration
! Important: Please read the informed-consent form that accompanies this
survey, and sign below if you agree to its terms. This survey cannot be
processed without said consent.
Thank you for your participation

297
I have read the informed consent form supplied with this survey and I agree to its
terms, signed:
Signature _______________________________________________
Print Name _______________________________________________
If you would like to receive a summary of the results, please indicate an address or email
address where you would like the document sent:
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
Please use the remainder of this page to provide any additional comments you would
like to make.
Thank you for your participation

298

Bowling Green, OH 43403-0302
(419) 372-2439
This study is being conducted by Mr. Darren Olson, and is chaired by Dr. John W. Sinn,
Department of Technology Systems, Bowling Green State University. Inquiries may be
addressed to:
Office of Research Compliance

Mr. Rich Rowlands, Compliance Officer
201 South Hall, BGSU
Bowling Green, OH 43403
(419) 372-7716
A self-addressed, stamped envelope is provided for your convenience. Please send

completed survey to:
Darren Olson
9541 Chad Dr. NW
Bemidji, MN 56601
Faculties of: Aviation Studies, Construction Management and Technology, Electronics and Computer Technology, and Manufacturing Technology
299
Bowling Green State University Department of Technology Systems
College of Technology
Bowling Green, Ohio 43403-0302
(419) 372-2439
9541 Chad Dr. NW

Bemidji, MN 56601
XXX, 2003
Dear XXX:
I am an assistant professor at Bemidji State University and am working on my doctoral

degree through Bowling Green State University. My dissertation is in the form of a
survey addressed to practicing quality managers across North America. In particular, I
will be sampling one person who serves in a leadership position within each ASQ
section.
The survey asks respondents to rate the performance of their employing organization
according to the categories of ISO 9001:2000, The 2003 Baldrige Criteria, and the items
within ASQ’s body of knowledge for certified quality managers. The form lists each
category (or clause) and its sub-items. It asks respondents to rate performance according
to each sub-category (or sub-clause), using a five-point scale. Being intimately familiar
with each standard is not a pre-requisite. All responses will be kept confidential, and no
results will be associated with the identity of any respondent. Judging by test-
respondents, I anticipate that most participants will complete the form in less than 20
minutes.
I am contacting you because of your section leadership activities in ASQ. If you are not
able to take the survey, could you pass this package along to someone in your section
who is a practicing quality manager and whom you regard as an exemplary member of
the section?
I feel that this study will help members of the profession to see the state of quality
management practice across the continent, as seen by leaders in the field, and to gauge
its effectiveness in achieving measures of organizational success. I intend to submit the
results for publication in Quality Progress and in the Quality Management Journal. I
will also offer respondents the chance to receive a copy of the results.
I would appreciate your help with completing my research. Thank you for your
consideration.
Sincerely,
Darren Olson
218-755-2948 Work
218-444-5891 Home
dolson@bemidjistate.edu
300
(419) 372-2439
INFORMED CONSENT FORM FOR SURVEY RESPONDENTS – PLEASE REVIEW AND
SIGN
Study Title: Assessment of Current Quality Management Practices

Principal Investigator: Darren C. Olson
Contact Information: 9541 Chad Dr. NW
Bemidji, MN 56601
dolson@bemidjistate.edu
Human Subjects Review Board: If you have any questions or concerns, you may also
contact the Chair of the Human Subjects Review Board for Bowling Green State University at
(419) 372-7716.
Sponsoring Institution: Bowling Green State University, Bowling Green, OH 43403
Why You Have Been Contacted: You were invited to participate in this study because of your
standing as a section officer for the American Society for Quality.
Purpose and Nature of the Study: This study will pursue the following objectives:
1. Assess the progress of quality management practitioners in applying the principles contained
in the categories of the Certified Quality Manager Body of Knowledge (BOK), as stated by
section officers in the American Society for Quality (ASQ).
2. Gauge the effectiveness, as stated by ASQ section officers, of quality management efforts in
meeting the categorical requirements of the ISO 9001:2000 standard.
3. Gauge the effectiveness, as stated by ASQ section officers, of quality management efforts in
meeting the categorical requirements of the Malcolm Baldrige National Quality Award
(MBNQA).
4. Investigate the relationships between the stated application levels from objective one, treated
as independent variables, and the stated effectiveness of quality management efforts from
objectives two and three, treated as dependent variables, to determine if there are any
possible main factor effects or interactions.
301
(419) 372-2439
Potential Benefits of the Study: This project can be of benefit to quality professionals and
practitioners. It has the potential to determine if quality managers across North America feel that
there are gaps between ASQ-defined quality management practices and the requirements of some
internationally respected definitions of organizational quality. Particularly it will compare self-
rated effectiveness in applying quality management principles from ASQ’s body of knowledge to
principles from ISO 9001:2000 and the Baldrige Award criteria.
What is Expected of Participants? Participants will self-rate the performance of their
employing organization using evaluation items that are taken directly from the American Society
for Quality certified quality manager body of knowledge, ISO 9001:2000, and the Malcolm
Baldrige National Quality Award Criteria. They will also provide basic background information
about the size of the business unit they work in, the nature of the industry their employer
operates in (using NAICS codes), years of quality management experience, other quality
certifications held, level of decision making authority within the company, currency of quality
management certification, and number of years of post-secondary education.
Participation is Voluntary: Participation in this study is voluntary. If you do not want or are
unable to take part, no information regarding your association with the researcher will be shared
with any parties.
Confidentiality: Sometimes organizations will explicitly or systematically guard the privacy of
information that may be considered trade secrets, offering competitive advantage. In such cases,
employers may require employees to forego participation in studies like this one, or they may
require certain precautions to be taken that will insure protection of trade secrets.
In light of this, this informed consent statement is being provided to potential

participants, who must acknowledge understanding of its principles and must attest to
having appropriate clearance from authorities within their organizations.
302
(419) 372-2439
Confidentiality measures will include the following precautions:
1. Names of participants and their employing organizations will be collected and

retained, but not reported.
2. There will be no interaction between participants.
Eligibility to Participate
If you wish to participate, please sign on the lines below and return this page with your
survey. Alternately you may sign on the appropriate lines located on page seven of the
survey instrument. Your responses will be discarded if you do not sign in one of these
two places. By signing the lines below or in the survey, you acknowledge your
agreement with the following statement:
I certify that I have read the material contained above, under the heading Informed
Consent Form for Survey Respondents, and I am authorized, by all appropriate
personnel at my employing organization, to answer the background questions as
described in the accompanying consent form. I am likewise authorized to answer
questions about my own assessment of my employing organization’s level of
implementation of principles contained in the certified quality manager body of
knowledge and in the Malcolm Baldrige National Quality Award Criteria. I have been
informed that no information about me or about my employer will be reported
throughout the course of this study.
Please sign your name here: ______________________________

Please Print your Name Here: ______________________________
Date signed: ______________________________
303
APPENDIX F
Table F1
Responses and Variables Data

304
CODE BOK1A BOK1B BOK2A BOK2B BOK2C

1 5 4 4 4 4
2 4 5 3 5 2
3 4 3 2 3 3
4 5 4 999 4 4
5 3 3 2 3 2
6 5 5 4 5 5
7 5 5 4 4 5
8 3 3 4 3 3
9 4 3 2 3 1
10 5 4 3 4 4
11 4 3 4 4 0
12 5 5 5 5 4
13 5 5 3 5 4
14 4 4 4 3 3
15 5 4 2 2 2
16 5 5 4 5 5
17 4 4 4 5 4
18 5 5 5 5 5
19 4 4 3 4 4
20 4 3 999 3 3
21 5 5 4 4 4
22 3 4 4 4 3
23 4 0 3 5 4
24 5 5 5 5 5
25 5 5 5 5 4
26 5 5 4 3 3
27 5 5 4 5 4
28 2 2 2 3 3
29 3 3 5 4 4
30 4 4 2 4 4
31 5 0 2 3 4
32 4 2 3 3 3
33 2 2 2 2 2
34 3 4 2 3 2
35 3 3 3 2 2
36 4 3 3 4 2
37 4 2 3 3 2
38 4 5 5 4 3
39 5 4 4 5 4
40 4 4 1 3 3
41 3 2 2 2 2
305

42 4 4 4 4 4
43 5 4 3 4 4
44 4 4 4 4 4
45 5 4 3 4 4
46 5 4 3 3 3
47 5 4 4 5 5
48 3 3 3 3 3
49 2 3 1 2 2
50 5 4 3 3 3
51 5 5 4 5 5
52 3 4 2 3 4
53 5 2 3 4 4
54 4 4 2 4 4
55 4 4 4 4 4
56 5 5 4 4 4
57 5 4 4 5 4
58 2 1 2 3 3
59 4 3 5 5 4
60 5 5 4 5 4
61 4 3 3 3 3
62 4 3 3 3 3
63 4 4 3 4 4
64 3 1 2 3 2
65 4 4 4 5 4
66 4 3 4 4 3
67 4 4 4 4 5
68 3 4 4 2 2
69 5 5 4 5 5
70 5 4 4 4 4
71 4 4 3 4 3
72 4 4 2 4 4
73 4 2 2 2 2
74 4 4 4 4 4
75 5 4 3 4 4
76 3 3 3 2 3
77 5 5 4 4 4
78 3 2 5 3 3
79 4 4 3 4 4
80 4 3 4 5 4
81 4 5 4 5 4
82 3 3 3 3 2
83 4 3 3 5 5
306

84 4 3 4 3 3
85 4 4 3 3 3
86 4 3 3 3 3
87 5 3 4 4 4
88 4 3 4 4 4
307
CODE BOK3A BOK3B BOK3C BOK4A BOK4B

1 4 4 3 5 5
2 3 4 5 5 5
3 4 4 4 4 4
4 3 3 3 999 4
5 3 3 2 2 2
6 4 4 5 4 5
7 5 4 5 5 5
8 4 3 4 4 4
9 3 3 3 4 5
10 4 3 4 4 4
11 3 4 4 4 3
12 5 5 5 4 4
13 4 3 4 5 5
14 3 2 5 4 3
15 5 4 5 4 5
16 5 4 5 5 4
17 5 5 5 5 5
18 5 5 5 5 5
19 5 5 5 3 3
20 4 4 2 999 3
21 5 5 4 5 5
22 3 3 3 4 4
23 5 5 5 5 5
24 5 5 5 5 5
25 5 5 5 4 4
26 5 4 4 5 4
27 5 5 5 4 4
28 2 3 2 5 4
29 5 4 3 4 3
30 4 5 5 5 4
31 4 3 4 5 5
32 4 3 4 4 4
33 2 2 4 1 2
34 3 4 4 4 5
35 3 3 3 3 4
36 4 4 2 3 3
37 1 2 3 3 2
38 5 5 5 5 5
39 4 4 3 5 4
40 4 4 3 2 1
41 3 3 3 3 3
42 4 4 4 4 4
43 3 4 4 5 5
44 5 5 4 4 5
45 5 4 5 3 3
308
CODE BOK3A BOK3B BOK3C BOK4A BOK4B

46 4 4 4 4 4
47 4 5 5 5 5
48 3 2 2 4 4
49 3 3 3 2 4
50 3 3 4 4 4
51 5 5 5 4 5
52 2 3 3 4 3
53 5 5 4 5 5
54 4 4 4 5 5
55 4 4 5 5 5
56 5 5 4 5 5
57 4 5 5 5 5
58 2 3 3 3 2
59 4 3 3 4 4
60 4 4 4 4 5
61 3 2 3 3 3
62 3 3 3 4 5
63 4 4 3 5 4
64 4 4 3 4 5
65 5 5 5 5 5
66 3 4 4 4 3
67 5 5 5 5 5
68 3 1 2 4 3
69 5 5 5 4 5
70 5 4 5 4 5
71 3 3 3 3 3
72 5 4 4 4 4
73 2 4 3 2 1
74 4 4 2 3 3
75 5 4 4 2 3
76 3 3 4 4 3
77 5 5 5 5 5
78 2 2 2 4 4
79 5 5 5 5 4
80 4 4 4 4 4
81 5 4 4 5 5
82 2 3 2 3 3
83 3 4 5 5 5
84 5 4 3 5 5
85 5 4 5 4 4
86 5 5 5 3 4
87 4 3 4 5 5
88 5 5 5 5 5
309
CODE BOK5A BOK5B BOK5C BOK5D BOK5E

1 4 4 5 4 3
2 3 3 4 2 3
3 4 4 4 3 3
4 4 4 4 4 4
5 3 2 2 2 999
6 4 4 3 4 0
7 5 4 4 4 5
8 2 2 2 2 2
9 3 2 2 1 2
10 5 4 3 3 3
11 4 3 4 4 4
12 4 4 4 4 4
13 4 4 3 3 3
14 4 4 3 2 2
15 4 3 3 3 2
16 4 5 4 4 3
17 5 5 5 5 5
18 4 4 5 5 5
19 2 2 2 2 2
20 1 3 1 1 1
21 5 5 4 5 4
22 3 3 4 3 3
23 3 4 4 3 4
24 5 5 5 5 5
25 4 4 4 4 4
26 4 4 3 3 2
27 4 4 3 3 3
28 4 4 3 2 1
29 4 3 3 4 4
30 3 3 3 2 2
31 3 4 3 2 1
32 3 3 3 3 3
33 3 2 1 2 4
34 3 3 2 2 1
35 4 4 3 3 3
36 3 3 3 3 2
37 3 3 3 2 1
38 4 4 4 3 4
39 4 4 4 3 3
40 4 3 4 4 3
41 3 2 3 2 1
42 4 4 4 4 4
43 4 4 4 3 3
44 4 4 4 4 4
310
CODE BOK5A BOK5B BOK5C BOK5D BOK5E

45 4 4 4 3 3
46 5 4 3 4 4
47 4 5 4 4 4
48 4 4 2 2 2
49 3 3 2 4 2
50 2 4 4 3 3
51 4 5 5 4 4
52 4 4 2 2 2
53 3 3 3 2 1
54 4 4 4 4 4
55 4 4 5 5 4
56 5 5 5 5 5
57 4 4 4 4 4
58 2 3 2 2 2
59 4 4 3 3 1
60 4 4 3 3 3
61 2 4 3 3 2
62 3 4 3 3 3
63 4 4 3 4 4
64 3 5 4 3 1
65 4 4 4 3 3
66 4 4 4 3 3
67 4 4 4 4 4
68 2 2 2 1 1
69 4 4 3 3 1
70 4 4 5 4 4
71 3 3 3 3 3
72 4 4 4 3 3
73 3 3 3 2 2
74 3 4 5 4 4
75 3 3 3 4 3
76 4 4 3 2 1
77 5 5 5 5 4
78 2 2 2 2 2
79 5 4 4 4 3
80 3 4 2 2 2
81 3 3 2 4 3
82 3 4 4 4 2
83 4 4 3 2 1
84 4 3 4 3 3
85 4 4 4 4 4
86 3 4 2 3 2
87 4 4 4 555 555
88 4 4 4 3 4
311
CODE BOK5F BOK5G BOK6A BOK6B BOK6C

1 4 4 4 4 4
2 4 4 3 2 2
3 2 3 3 4 4
4 4 4 3 3 3
5 999 999 3 3 2
6 2 4 5 5 4
7 5 4 5 4 4
8 2 2 4 3 3
9 3 2 4 5 3
10 4 4 5 4 0
11 3 3 4 3 3
12 4 4 4 4 4
13 3 3 4 5 4
14 3 4 4 4 4
15 2 2 4 4 4
16 3 4 5 5 5
17 5 5 4 4 5
18 4 4 5 5 5
19 2 2 3 4 3
20 1 1 4 3 3
21 5 4 5 5 5
22 3 3 4 4 3
23 3 4 4 4 5
24 5 5 5 5 5
25 4 4 5 5 4
26 4 3 4 3 4
27 5 3 4 4 3
28 1 1 2 1 999
29 5 4 4 3 3
30 2 2 4 4 5
31 2 2 5 4 4
32 4 2 3 3 5
33 4 2 2 2 2
34 1 2 2 2 3
35 3 3 4 3 3
36 2 3 4 3 3
37 3 1 3 3 2
38 3 4 3 3 4
39 4 3 4 4 4
40 3 3 3 4 3
41 1 2 2 2 2
42 4 4 4 4 4
43 2 2 5 4 4
44 4 4 5 5 5
45 4 4 4 4 4
312
CODE BOK5F BOK5G BOK6A BOK6B BOK6C

46 4 4 4 5 4
47 5 3 4 5 5
48 2 2 3 2 2
49 2 3 2 3 3
50 2 3 4 4 4
51 4 4 5 4 4
52 3 3 4 3 5
53 4 4 4 4 5
54 3 3 5 5 4
55 5 5 4 4 4
56 5 5 4 4 4
57 5 5 5 5 5
58 3 3 2 2 2
59 4 4 4 4 5
60 4 4 4 4 5
61 3 3 3 2 2
62 3 3 4 4 4
63 5 4 4 4 4
64 4 4 4 3 3
65 4 4 4 4 4
66 3 3 3 3 4
67 4 4 5 5 5
68 2 2 3 3 3
69 2 3 5 4 3
70 4 4 5 5 4
71 3 4 3 4 4
72 4 3 4 4 4
73 3 2 3 2 2
74 4 4 4 4 3
75 3 3 4 4 4
76 1 2 3 3 0
77 4 5 5 5 5
78 2 1 3 3 2
79 4 5 4 4 4
80 3 3 4 3 4
81 999 3 4 5 5
82 3 4 3 2 3
83 2 2 4 4 4
84 4 4 4 3 4
85 3 4 4 4 4
86 3 3 4 5 4
87 555 3 4 4 4
88 4 4 5 5 5
313
CODE BOK6D BOK6E BOK7A BOK7B BOK7C BOK7D

1 4 4 4 3 3 3
2 3 2 3 2 2 3
3 4 4 3 4 3 3
4 4 3 2 2 3 3
5 2 2 2 3 3 3
6 5 4 4 4 4 4
7 5 4 4 4 4 4
8 3 3 3 2 3 3
9 3 2 3 2 4 5
10 5 5 4 3 3 3
11 4 4 3 4 3 2
12 5 5 4 4 4 4
13 5 4 5 5 5 4
14 3 3 4 2 3 4
15 5 4 2 1 2 3
16 5 5 5 4 4 5
17 5 4 4 3 3 4
18 5 5 4 5 5 4
19 4 3 4 2 2 2
20 4 3 4 2 2 2
21 5 4 4 4 4 4
22 3 3 3 3 4 4
23 4 3 3 3 3 4
24 5 5 5 5 5 5
25 5 5 5 5 5 5
26 4 4 4 3 4 4
27 5 4 3 3 4 4
28 4 999 4 2 2 2
29 3 3 3 3 3 3
30 5 4 4 3 3 3
31 5 5 3 4 3 4
32 5 4 2 2 2 2
33 3 2 1 1 2 1
34 3 3 2 3 3 3
35 3 4 2 3 4 4
36 4 2 3 3 2 2
37 1 1 1 1 1 1
38 5 5 4 3 4 5
39 3 3 5 3 999 4
40 4 2 1 1 2 1
41 3 3 2 2 3 2
42 4 4 4 4 4 4
43 4 3 4 3 2 2
44 5 5 5 5 5 5
45 4 4 4 3 4 3
314
CODE BOK6D BOK6E BOK7A BOK7B BOK7C BOK7D

46 5 4 3 3 3 3
47 5 5 4 4 4 5
48 3 3 2 3 2 3
49 3 2 2 3 3 3
50 4 3 4 4 4 4
51 5 5 5 4 5 4
52 3 2 3 3 3 3
53 5 2 4 3 3 3
54 5 5 5 5 3 4
55 4 4 4 4 5 5
56 5 4 4 4 4 4
57 5 5 5 4 4 4
58 4 3 2 2 2 2
59 5 4 4 5 4 4
60 5 4 4 3 3 4
61 4 3 3 2 2 3
62 4 4 3 4 3 3
63 4 4 3 3 3 3
64 3 2 2 1 2 2
65 5 4 4 4 4 5
66 3 2 2 2 3 3
67 4 4 4 4 4 4
68 3 3 1 1 1 1
69 5 5 2 2 3 2
70 4 4 4 4 4 4
71 3 3 4 3 3 3
72 4 4 4 3 3 3
73 4 1 2 2 2 2
74 3 3 3 3 2 2
75 5 5 4 4 4 3
76 4 2 3 3 2 3
77 5 5 5 4 4 4
78 3 3 3 3 2 3
79 5 4 4 3 3 3
80 4 4 3 3 3 4
81 5 999 5 5 5 4
82 3 2 2 3 3 3
83 4 4 3 3 2 3
84 4 4 5 3 3 4
85 5 4 3 4 3 3
86 5 3 4 3 4 4
87 5 4 4 4 4 4
88 999 5 5 4 4 4
315
CODE BOK7E ISO1A ISO1B ISO2A ISO2B

1 4 4 4 4 5
2 1 5 5 5 5
3 3 4 4 3 5
4 3 5 5 5 5
5 3 2 2 2 2
6 3 5 4 4 5
7 4 5 5 5 5
8 3 3 5 4 3
9 3 4 4 5 4
10 3 5 5 4 3
11 3 5 5 4 5
12 4 5 5 4 5
13 4 5 5 5 5
14 4 4 4 3 4
15 2 5 5 5 5
16 5 5 5 4 5
17 3 5 5 5 5
18 4 5 5 5 5
19 2 4 4 3 4
20 3 4 4 4 3
21 4 5 5 5 5
22 3 3 4 4 4
23 3 5 5 5 5
24 5 5 5 5 5
25 5 5 5 5 5
26 3 4 5 4 5
27 3 5 5 4 5
28 1 4 4 5 3
29 3 4 3 4 4
30 3 5 5 4 3
31 3 5 5 5 5
32 2 5 5 3 5
33 1 2 3 1 3
34 3 3 4 5 4
35 4 3 3 4 4
36 3 4 5 4 4
37 1 1 1 2 1
38 3 5 5 5 5
39 3 4 4 5 4
40 1 4 4 2 3
41 2 3 4 3 3
42 4 4 4 4 4
43 3 5 5 4 5
44 5 4 4 4 4
45 3 4 5 4 5
316
CODE BOK7E ISO1A ISO1B ISO2A ISO2B

46 3 4 5 5 5
47 4 5 5 5 5
48 3 3 3 4 4
49 3 4 4 5 3
50 4 4 4 4 4
51 4 5 5 5 5
52 2 4 3 3 4
53 3 5 5 5 5
54 4 3 4 4 4
55 5 4 5 5 4
56 4 4 4 4 5
57 4 5 5 5 5
58 2 4 4 2 3
59 4 4 4 4 5
60 4 5 5 5 5
61 2 4 4 4 4
62 4 4 4 5 4
63 3 5 5 5 5
64 1 4 4 5 4
65 4 5 5 5 4
66 4 4 4 3 2
67 4 4 4 5 4
68 1 3 3 4 4
69 1 5 5 5 5
70 4 4 4 5 4
71 3 3 3 3 3
72 3 4 4 4 4
73 2 4 3 2 3
74 3 4 3 3 3
75 4 4 5 4 4
76 3 4 5 3 5
77 4 5 5 5 5
78 2 4 4 3 4
79 3 4 4 4 5
80 3 4 4 4 4
81 3 4 4 5 4
82 2 3 4 2 3
83 1 4 4 5 5
84 3 4 4 4 5
85 2 4 4 3 4
86 4 4 5 4 4
87 3 4 5 5 4
88 4 5 5 5 5
317
CODE ISO2C ISO2D ISO2E ISO2F ISO3A ISO3B

1 4 4 5 5 4 4
2 5 5 5 4 4 5
3 4 4 5 4 3 3
4 5 5 4 4 4 3
5 3 3 3 3 3 3
6 4 5 4 3 3 4
7 4 5 5 4 4 5
8 2 3 2 3 2 2
9 4 4 4 3 3 3
10 4 4 5 4 3 3
11 5 4 4 4 4 4
12 4 4 4 4 3 3
13 5 5 5 5 5 4
14 4 4 3 4 4 3
15 5 5 5 5 4 4
16 5 5 5 5 5 5
17 4 5 4 4 4 4
18 5 5 5 5 5 5
19 3 3 3 2 2 3
20 3 3 2 3 4 3
21 4 5 5 5 4 4
22 3 3 4 4 3 3
23 4 5 4 4 5 5
24 5 5 5 5 5 5
25 5 5 5 5 4 4
26 5 4 4 3 3 4
27 4 4 4 4 4 3
28 4 4 2 2 4 5
29 3 3 3 3 3 2
30 4 4 4 4 4 4
31 4 4 4 4 5 4
32 4 4 5 5 4 4
33 2 2 1 1 1 3
34 3 3 2 2 4 4
35 3 3 4 4 3 3
36 3 3 3 3 3 3
37 1 3 1 1 3 2
38 5 5 5 5 5 5
39 4 4 4 4 3 3
40 3 3 2 1 3 3
41 3 3 3 3 3 4
42 4 4 4 4 4 4
43 3 4 5 5 4 3
44 4 4 4 4 4 4
45 4 4 5 4 3 4
318
CODE ISO2C ISO2D ISO2E ISO2F ISO3A ISO3B

46 3 4 5 4 4 4
47 5 5 5 5 4 4
48 3 2 2 2 3 2
49 4 4 3 3 4 4
50 4 4 4 4 4 4
51 5 5 5 5 4 4
52 3 3 3 3 4 4
53 4 4 5 5 4 4
54 3 3 4 4 3 3
55 4 4 5 5 3 3
56 4 5 5 4 4 4
57 5 5 4 4 5 4
58 3 2 3 2 2 2
59 4 5 4 3 4 4
60 5 5 4 4 4 4
61 3 3 4 4 3 3
62 4 4 3 3 3 3
63 4 4 5 5 4 4
64 3 3 2 2 3 2
65 4 5 4 4 3 4
66 3 3 1 1 4 4
67 4 4 4 4 5 4
68 3 3 3 2 3 3
69 5 5 5 5 4 4
70 4 4 4 4 4 4
71 3 3 3 3 3 3
72 4 4 4 4 4 4
73 2 3 3 3 3 3
74 3 3 3 3 3 3
75 4 4 5 5 4 3
76 4 4 5 5 3 3
77 5 4 5 5 5 5
78 3 3 3 3 2 3
79 4 4 4 4 4 4
80 3 3 4 4 4 4
81 4 4 4 3 3 3
82 3 2 2 3 3 2
83 5 5 5 5 4 4
84 3 3 3 4 3 4
85 3 4 4 4 3 3
86 3 4 4 3 3 3
87 4 4 3 3 5 4
88 5 5 5 5 5 5
319
CODE ISO3C ISO3D ISO4A ISO4B ISO4C ISO4D

1 5 5 4 4 4 4
2 4 5 4 5 5 5
3 4 4 4 4 4 4
4 5 5 4 3 3 5
5 3 4 3 3 3 999
6 4 4 4 4 3 4
7 4 5 5 5 3 4
8 2 3 2 3 4 3
9 3 4 3 4 4 3
10 4 4 4 4 3 4
11 4 4 4 4 5 4
12 4 4 4 4 4 4
13 4 4 4 5 1 4
14 4 3 4 4 4 4
15 3 4 5 5 5 5
16 5 5 5 5 4 5
17 5 5 5 5 5 5
18 5 5 4 4 5 5
19 2 2 3 2 2 2
20 4 4 3 4 3 3
21 5 5 4 5 5 5
22 3 4 3 3 3 4
23 5 5 3 4 4 4
24 5 5 5 5 5 5
25 4 4 4 4 4 4
26 4 3 5 5 999 4
27 4 4 4 4 555 3
28 5 5 4 5 5 5
29 2 2 3 3 2 4
30 4 4 4 4 555 4
31 5 5 5 5 4 4
32 5 5 5 5 5 3
33 2 2 1 1 1 2
34 3 3 3 3 4 2
35 3 3 3 4 3 4
36 3 4 3 3 3 2
37 2 1 2 3 3 4
38 5 5 4 4 4 4
39 4 4 4 4 4 3
40 2 2 2 2 2 3
41 4 4 4 4 1 3
42 4 4 4 4 4 4
43 4 4 4 4 4 4
44 4 4 4 4 4 4
45 4 4 4 4 4 4
320
CODE ISO3C ISO3D ISO4A ISO4B ISO4C ISO4D

46 4 4 5 5 4 4
47 5 4 4 5 5 5
48 3 2 2 2 3 3
49 4 4 4 4 4 4
50 4 4 4 3 3 3
51 5 5 5 5 4 4
52 4 3 4 4 4 4
53 4 5 3 4 4 5
54 3 3 3 4 4 4
55 4 3 4 4 4 4
56 4 4 5 5 4 4
57 5 5 5 5 5 5
58 2 2 4 3 4 3
59 4 5 4 4 4 4
60 4 5 5 5 5 4
61 3 3 2 3 3 3
62 3 4 4 4 3 3
63 4 4 4 5 4 4
64 3 3 3 4 4 5
65 4 4 4 5 4 4
66 4 5 4 3 2 3
67 4 5 4 4 4 4
68 2 3 4 3 3 3
69 5 5 5 5 1 5
70 4 4 4 4 4 4
71 3 3 4 3 3 3
72 3 3 4 4 4 4
73 3 3 3 2 3 3
74 2 3 3 4 2 3
75 3 3 4 4 3 3
76 3 3 3 3 2 4
77 4 5 4 4 4 4
78 3 4 3 3 3 3
79 4 4 4 4 4 4
80 4 5 4 3 4 4
81 4 4 4 5 4 4
82 3 2 3 3 3 4
83 5 5 5 4 4 3
84 4 4 5 5 4 4
85 3 4 5 4 4 4
86 3 3 999 3 3 3
87 4 4 5 5 4 4
88 5 5 5 5 5 5
321
CODE ISO4E ISO4F ISO5A ISO5B ISO5C ISO5D

1 4 4 4 5 4 4
2 5 5 4 5 4 3
3 4 4 4 4 4 3
4 4 5 3 3 5 4
5 3 2 3 3 999 3
6 4 3 4 3 4 4
7 4 5 5 4 5 4
8 3 3 3 3 3 2
9 3 5 5 4 5 4
10 3 5 4 5 4 4
11 4 4 4 4 4 3
12 4 4 4 4 4 4
13 4 1 5 5 3 4
14 4 4 4 4 4 4
15 5 5 4 3 5 4
16 5 5 5 5 5 5
17 5 4 5 5 5 5
18 5 5 5 5 4 5
19 2 3 3 3 3 3
20 4 4 4 4 4 4
21 5 1 5 5 5 4
22 4 3 3 3 4 3
23 0 5 4 4 5 4
24 5 5 5 5 5 5
25 4 4 5 5 5 5
26 4 4 5 4 5 4
27 4 4 4 4 4 3
28 5 5 3 4 5 2
29 4 4 4 4 4 4
30 4 5 5 5 5 5
31 4 3 4 5 5 3
32 5 5 5 5 5 5
33 2 4 1 1 1 1
34 3 2 3 4 4 4
35 4 4 4 4 4 4
36 2 2 3 3 4 2
37 2 1 1 1 1 1
38 4 4 5 5 4 3
39 4 3 4 4 999 4
40 3 3 2 2 4 3
41 3 4 3 3 3 3
42 4 4 4 4 4 4
43 4 5 4 5 5 4
44 4 4 4 4 4 4
45 4 4 4 4 4 4
322
CODE ISO4E ISO4F ISO5A ISO5B ISO5C ISO5D

46 4 4 4 4 5 4
47 4 5 5 5 5 4
48 3 2 3 3 2 3
49 4 4 4 4 5 3
50 3 3 4 4 4 3
51 5 5 5 5 4 5
52 4 4 3 4 4 3
53 4 5 5 5 5 5
54 4 4 3 4 4 4
55 4 5 4 4 4 4
56 4 5 4 4 4 4
57 5 4 5 5 5 5
58 3 4 4 3 3 3
59 4 3 4 4 5 4
60 4 5 4 5 5 4
61 4 4 4 4 4 4
62 3 4 3 3 3 3
63 4 5 4 5 5 5
64 4 5 4 4 5 3
65 4 5 4 5 5 4
66 4 3 3 3 3 3
67 4 4 4 4 4 4
68 3 4 3 3 4 2
69 5 5 5 5 5 5
70 4 4 4 4 3 4
71 3 3 3 3 3 3
72 4 5 4 4 4 4
73 3 4 4 4 3 3
74 3 3 3 3 3 3
75 4 5 4 5 5 5
76 4 4 3 3 5 3
77 4 4 4 4 5 4
78 3 4 3 2 3 1
79 4 4 4 4 5 4
80 4 4 4 4 4 4
81 999 4 4 4 4 4
82 3 2 3 3 3 2
83 5 1 5 4 4 4
84 4 4 3 3 4 3
85 4 5 4 4 5 5
86 4 5 5 5 5 5
87 5 5 5 5 5 5
88 5 5 5 5 5 4
323
CODE ISO5E MB1A MB1B MB2A MB2B MB3A

1 4 4 4 4 4 5
2 3 4 3 4 4 5
3 3 4 2 3 3 4
4 4 4 999 999 999 5
5 2 3 4 2 2 3
6 4 999 999 999 999 999
7 5 4 5 4 4 5
8 3 3 2 3 3 5
9 5 4 3 3 2 5
10 3 5 5 4 4 4
11 4 4 5 4 4 4
12 4 4 4 4 4 4
13 4 5 4 5 4 5
14 4 3 4 4 4 4
15 4 5 3 2 2 5
16 5 5 5 5 5 5
17 5 4 3 4 4 4
18 5 5 5 5 5 5
19 4 3 2 3 2 3
20 4 3 4 3 2 5
21 5 5 5 4 4 5
22 3 4 4 0 0 4
23 5 5 4 4 4 5
24 5 5 5 5 5 5
25 5 5 5 5 5 5
26 4 5 5 4 3 4
27 3 4 4 5 4 4
28 3 2 2 3 3 5
29 4 3 3 4 3 4
30 4 5 2 3 3 3
31 4 4 4 3 3 4
32 4 2 1 3 2 2
33 1 2 1 1 1 3
34 3 3 4 2 2 4
35 4 4 4 2 2 4
36 2 4 4 3 3 3
37 1 3 5 3 2 3
38 4 4 5 4 3 5
39 4 5 4 5 5 5
40 3 4 4 3 3 4
41 3 3 2 2 2 3
42 4 5 5 3 3 4
43 4 5 5 4 4 4
44 4 5 5 5 5 4
45 4 4 3 4 4 3
324
CODE ISO5E MB1A MB1B MB2A MB2B MB3A

46 4 4 4 4 4 4
47 4 5 4 5 4 5
48 3 3 2 3 3 4
49 4 3 3 3 3 3
50 3 4 4 3 3 3
51 5 5 5 4 4 5
52 3 3 3 3 4 4
53 4 5 5 4 3 4
54 3 3 4 4 4 4
55 4 3 4 3 3 4
56 4 4 3 3 3 4
57 5 5 4 5 5 5
58 2 2 1 1 1 3
59 4 4 4 3 4 4
60 4 5 4 5 4 5
61 3 4 4 4 3 3
62 3 4 5 4 4 4
63 4 4 3 4 4 5
64 3 3 4 3 3 5
65 4 4 4 4 4 5
66 3 3 3 4 3 4
67 4 4 5 4 4 4
68 2 3 3 3 3 4
69 5 4 4 5 3 5
70 4 5 3 4 4 5
71 3 3 3 4 3 4
72 4 4 3 3 3 3
73 2 2 1 2 2 3
74 3 4 4 4 4 4
75 5 4 4 4 4 3
76 3 3 3 2 3 3
77 4 5 4 5 5 5
78 2 3 2 3 3 5
79 5 5 5 4 4 5
80 4 4 4 4 4 4
81 4 5 5 5 4 3
82 2 3 2 3 2 3
83 4 4 4 5 5 5
84 4 4 3 5 5 4
85 3 3 4 3 3 4
86 5 3 3 3 3 4
87 4 5 4 4 4 5
88 4 5 5 4 4 5
325
CODE MB3B MB4A MB4B MB5A MB5B MB5C

1 4 4 4 4 3 3
2 5 4 3 4 4 4
3 3 4 4 4 3 4
4 5 4 4 5 5 4
5 3 3 4 3 3 3
6 999 999 999 999 999 999
7 5 5 5 4 5 5
8 4 3 3 3 3 3
9 5 4 3 4 4 4
10 5 4 4 3 3 3
11 4 5 5 4 4 4
12 4 4 4 3 3 3
13 5 5 5 5 5 5
14 4 4 4 4 4 4
15 4 4 4 4 3 4
16 4 5 5 5 5 5
17 5 4 5 5 3 3
18 5 5 5 5 4 4
19 2 4 2 3 2 2
20 4 3 4 3 3 3
21 5 4 4 5 5 5
22 3 5 4 4 3 4
23 5 4 4 4 4 4
24 5 5 5 5 5 5
25 5 5 5 4 4 4
26 4 4 4 5 4 3
27 4 4 4 3 3 3
28 5 1 3 3 2 1
29 3 3 3 4 3 3
30 4 3 2 3 3 3
31 5 4 4 4 3 3
32 2 2 2 2 2 2
33 3 1 1 1 2 2
34 4 4 3 3 4 4
35 3 3 3 3 4 3
36 4 2 2 3 3 4
37 2 1 1 1 1 2
38 5 5 5 4 4 4
39 5 4 4 4 3 3
40 4 3 3 2 3 2
41 3 3 2 3 2 3
42 4 4 4 4 4 4
43 5 4 4 4 4 4
44 4 4 4 4 4 3
45 3 4 4 4 3 3
326
CODE MB3B MB4A MB4B MB5A MB5B MB5C

46 4 4 4 4 4 5
47 5 5 4 4 3 4
48 4 2 2 2 2 2
49 4 3 4 3 2 3
50 3 4 3 3 3 3
51 5 5 5 5 4 4
52 3 4 3 2 2 2
53 5 4 4 5 5 5
54 4 2 2 2 2 4
55 4 4 4 3 4 3
56 5 4 4 3 3 3
57 5 5 5 4 4 5
58 2 3 2 2 1 1
59 4 4 4 999 3 3
60 5 4 4 4 3 3
61 3 3 2 2 2 2
62 4 3 3 4 4 4
63 4 4 4 5 4 4
64 4 3 3 2 2 2
65 4 5 5 4 4 3
66 3 4 3 3 4 4
67 4 4 4 4 4 4
68 3 2 2 2 1 2
69 5 4 4 3 3 2
70 5 4 4 4 4 4
71 4 3 3 3 4 4
72 3 4 4 3 3 3
73 2 3 2 2 2 3
74 4 3 3 3 4 3
75 4 4 3 3 3 3
76 3 2 3 3 2 3
77 5 4 4 5 4 4
78 4 2 3 3 2 3
79 5 4 4 4 4 4
80 3 4 4 4 3 4
81 4 5 4 3 4 4
82 3 3 2 3 4 3
83 5 4 4 4 3 4
84 5 3 4 3 3 3
85 4 4 3 4 3 4
86 3 3 4 4 4 3
87 5 4 4 4 5 3
88 5 5 5 5 5 5
327
CODE MB6A MB6B MB7A MB7B MB7C MB7D

1 3 3 4 4 4 4
2 4 4 5 4 5 4
3 3 4 3 4 4 3
4 4 4 4 4 4 4
5 3 3 2 3 3 2
6 999 999 999 999 999 999
7 4 5 5 5 4 5
8 4 3 3 3 3 3
9 3 3 4 4 4 3
10 4 4 4 3 5 3
11 3 3 4 4 5 3
12 4 4 4 4 4 3
13 4 4 5 4 4 3
14 4 4 4 4 4 4
15 2 3 5 5 5 3
16 5 5 4 5 5 5
17 5 5 4 4 3 3
18 5 5 5 5 5 4
19 1 1 2 2 4 3
20 4 3 4 4 5 3
21 5 5 5 5 5 5
22 3 3 4 0 4 4
23 4 4 5 5 5 5
24 5 5 5 5 5 5
25 4 4 4 4 4 4
26 4 4 5 5 2 3
27 4 4 5 5 4 4
28 2 3 5 5 5 3
29 3 3 3 4 4 3
30 3 3 4 3 4 4
31 3 4 5 4 4 4
32 1 1 3 3 4 3
33 1 1 1 1 3 1
34 3 4 4 4 0 4
35 3 3 3 4 3 3
36 4 3 3 3 4 3
37 2 3 2 3 3 2
38 4 4 5 4 0 3
39 4 4 5 5 5 3
40 3 3 1 2 3 3
41 3 3 3 4 0 3
42 4 4 3 3 3 3
43 4 4 4 5 5 4
44 4 4 4 4 0 4
45 4 3 4 4 3 3
328
CODE MB6A MB6B MB7A MB7B MB7C MB7D

46 4 4 4 4 5 4
47 5 5 5 5 4 4
48 2 2 4 3 0 3
49 3 4 4 4 4 3
50 4 4 3 4 4 4
51 4 4 5 4 4 4
52 3 4 4 4 0 3
53 4 4 5 4 0 4
54 3 2 3 4 5 3
55 4 4 3 4 0 4
56 3 3 5 4 4 3
57 4 5 5 5 5 5
58 3 2 2 3 0 2
59 4 4 4 4 0 4
60 4 4 5 4 0 3
61 4 3 3 3 3 3
62 4 4 5 4 4 4
63 4 4 5 4 3 4
64 2 2 4 3 0 2
65 4 4 4 4 4 3
66 4 3 2 4 5 2
67 4 4 4 4 0 4
68 2 1 2 2 4 2
69 3 3 5 5 4 3
70 3 4 4 4 0 4
71 4 4 4 3 4 4
72 3 4 4 4 0 3
73 2 3 2 3 2 3
74 4 4 4 4 4 4
75 4 4 4 4 0 3
76 3 3 3 4 3 2
77 5 5 5 5 5 4
78 3 2 2 4 4 3
79 4 4 4 4 0 4
80 4 3 3 3 4 4
81 4 4 4 4 5 3
82 3 3 4 3 4 3
83 4 4 4 5 5 3
84 4 3 5 4 999 3
85 999 4 3 4 5 3
86 3 3 3 3 0 3
87 4 4 5 5 999 4
88 5 5 5 5 5 5
329
CODE MB7E MB7F CQM EXP NCERT RESP

1 4 4 0 6.0 0 2
2 4 3 0 4.0 0 2
3 3 2 0 4.0 2 3
4 4 4 1 7.0 3 3
5 2 2 999 10.0 2 999
6 999 999 0 6.0 1 2
7 5 5 1 16.0 1 5
8 3 2 0 19.0 0 2
9 3 2 1 9.0 3 5
10 4 5 1 4.0 1 2
11 4 5 0 24.0 1 6
12 3 4 1 20.0 3 2
13 4 3 0 13.0 1 2
14 4 4 999 15.0 1 2
15 4 3 1 9.0 3 5
16 5 5 1 27.0 2 5
17 3 3 1 25.0 3 5
18 5 5 0 26.0 1 3
19 2 1 1 21.0 8 2
20 3 4 0 14.0 1 2
21 5 5 0 4.0 0 6
22 4 4 0 30.0 1 1
23 5 5 0 5.0 1 3
24 5 5 1 25.0 2 5
25 4 4 1 14.0 3 7
26 4 5 0 12.0 0 3
27 4 4 0 18.0 1 5
28 2 3 0 20.0 6 2
29 3 4 1 9.0 5 4
30 3 2 0 24.0 1 4
31 4 3 0 11.0 1 5
32 3 2 1 8.0 3 5
33 1 1 0 12.0 1 2
34 3 3 0 15.0 1 6
35 3 3 1 15.0 1 5
36 3 3 0 11.0 1 2
37 2 4 1 19.0 1 0
38 4 4 0 7.0 1 5
39 3 3 1 15.0 3 2
40 2 1 1 16.0 2 2
41 3 2 1 12.0 3 2
42 3 3 0 18.0 0 6
43 4 4 0 12.0 0 5
44 4 4 1 22.0 3 6
45 3 4 1 13.0 3 2
330
CODE MB7E MB7F CQM EXP NCERT RESP

46 4 4 1 15.0 7 2
47 4 5 0 4.0 1 6
48 2 2 0 4.0 2 5
49 3 2 0 5.0 0 4
50 4 2 0 19.0 0 6
51 4 4 0 10.0 0 6
52 3 3 0 40.0 0 5
53 4 5 1 15.0 4 5
54 3 3 1 5.0 5 5
55 3 4 1 12.0 4 4
56 4 3 0 7.0 0 5
57 5 5 1 28.0 4 6
58 2 1 1 8.0 3 2
59 4 5 0 12.0 1 2
60 4 3 0 11.0 1 5
61 3 4 0 15.0 1 6
62 4 4 0 17.0 2 3
63 4 4 0 10.0 0 6
64 3 3 0 8.0 0 3
65 4 4 0 30.0 0 5
66 2 2 1 27.0 1 6
67 4 5 1 29.0 5 2
68 2 2 1 5.0 3 6
69 4 3 0 20.0 1 3
70 4 4 0 14.0 2 5
71 3 4 1 13.0 2 5
72 4 3 1 20.0 7 3
73 3 1 0 13.0 2 2
74 4 4 0 21.0 0 2
75 4 4 1 25.0 4 3
76 2 2 1 13.0 3 6
77 4 4 0 8.0 0 4
78 3 2 0 24.0 2 6
79 4 4 0 5.0 0 3
80 4 3 1 18.0 4 3
81 4 3 0 15.0 3 2
82 3 2 1 10.0 3 3
83 4 4 0 6.0 0 6
84 4 4 0 14.0 0 6
85 4 4 0 15.0 0 5
86 3 3 1 20.0 2 2
87 4 5 0 20.0 2 2
88 5 5 1 10.0 1 5
331
CODE EDU EMPL DIRECT NAICS REGION

1 4 7 1 32 15
2 8 9 0 33 10
3 5 5 2 33 13
4 6 6 5 32 15
5 6 999 4 61 2
6 2 7 6 33 2
7 6 7 12 33 15
8 5 4 5 81 7
9 5 5 1 54 4
10 4 7 13 33 11
11 5 10 50 32 3
12 7 7 3 33 3
13 8 2 1 54 3
14 5 7 6 49 14
15 4 7 13 33 9
16 6 7 4 33 9
17 4 8 6 33 11
18 5 9 17 33 8
19 6 4 19 33 15
20 6 7 7 81 12
21 6 7 425 81 12
22 4 7 0 32 9
23 6 9 40 33 9
24 4 9 0 33 14
25 7 11 4 22 4
26 4 13 1 32 14
27 4 7 7 33 12
28 4 5 5 33 7
29 7 5 3 33 7
30 4 7 3 33 2
31 0 7 3 32 12
32 6 7 11 33 5
33 4 5 0 54 15
34 7 9 30 54 7
35 8 7 0 33 12
36 4 7 20 31 12
37 7 7 0 61 12
38 6 8 14 92 15
39 5 6 0 33 6
40 4 13 20 33 13
41 5 5 3 33 10
42 4 7 2 32 5
43 3 6 1 33 8
44 8 11 300 33 12
45 8 7 5 33 11
332
CODE EDU EMPL DIRECT NAICS REGION

46 6 8 20 33 7
47 2 11 0 33 15
48 6 6 10 33 14
49 8 6 0 33 1
50 6 8 45 33 13
51 2 7 0 54 5
52 3 8 0 54 6
53 7 5 5 32 7
54 10 8 17 33 11
55 7 13 16 33 10
56 2 5 1 33 6
57 4 10 0 48 14
58 4 7 2 33 9
59 10 5 1 33 6
60 4 7 2 33 9
61 4 7 5 33 3
62 6 7 10 32 9
63 4 7 22 33 4
64 6 7 4 33 6
65 4 9 16 33 1
66 6 7 2 33 6
67 0 10 70 33 12
68 6 7 2 32 8
69 4 5 5 33 8
70 2 7 40 33 15
71 7 9 4 33 15
72 0 7 7 33 5
73 6 4 2 33 6
74 4 3 5 54 11
75 10 9 25 33 13
76 6 6 0 32 11
77 5 8 1 54 4
78 7 8 35 33 12
79 4 8 15 33 11
80 6 7 3 33 8
81 5 8 0 33 14
82 4 7 20 33 6
83 8 7 2 44 4
84 4 10 0 33 4
85 7 9 40 33 8
86 6 6 4 32 6
87 5 5 2 54 13
88 6 7 5 32 3
333
CODE BOK1 BOK2 BOK3 BOK4 BOK5 BOK6

1 4.5 4.0 3.7 5.0 4.0 4.0
2 4.5 3.3 4.0 5.0 3.3 2.4
3 3.5 2.7 4.0 4.0 3.3 3.8
4 4.5 4.0 3.0 4.0 4.0 3.2
5 3.0 2.3 2.7 2.0 2.3 2.4
6 5.0 4.7 4.3 4.5 3.5 4.6
7 5.0 4.3 4.7 5.0 4.4 4.4
8 3.0 3.3 3.7 4.0 2.0 3.2
9 3.5 2.0 3.0 4.5 2.1 3.4
10 4.5 3.7 3.7 4.0 3.7 4.8
11 3.5 4.0 3.7 3.5 3.6 3.6
12 5.0 4.7 5.0 4.0 4.0 4.4
13 5.0 4.0 3.7 5.0 3.3 4.4
14 4.0 3.3 3.3 3.5 3.1 3.6
15 4.5 2.0 4.7 4.5 2.7 4.2
16 5.0 4.7 4.7 4.5 3.9 5.0
17 4.0 4.3 5.0 5.0 5.0 4.4
18 5.0 5.0 5.0 5.0 4.4 5.0
19 4.0 3.7 5.0 3.0 2.0 3.4
20 3.5 3.0 3.3 3.0 1.3 3.4
21 5.0 4.0 4.7 5.0 4.6 4.8
22 3.5 3.7 3.0 4.0 3.1 3.4
23 4.0 4.0 5.0 5.0 3.6 4.0
24 5.0 5.0 5.0 5.0 5.0 5.0
25 5.0 4.7 5.0 4.0 4.0 4.8
26 5.0 3.3 4.3 4.5 3.3 3.8
27 5.0 4.3 5.0 4.0 3.6 4.0
28 2.0 2.7 2.3 4.5 2.3 2.3
29 3.0 4.3 4.0 3.5 3.9 3.2
30 4.0 3.3 4.7 4.5 2.4 4.4
31 5.0 3.0 3.7 5.0 2.4 4.6
32 3.0 3.0 3.7 4.0 3.0 4.0
33 2.0 2.0 2.7 1.5 2.6 2.2
34 3.5 2.3 3.7 4.5 2.0 2.6
35 3.0 2.3 3.0 3.5 3.3 3.4
36 3.5 3.0 3.3 3.0 2.7 3.2
37 3.0 2.7 2.0 2.5 2.3 2.0
38 4.5 4.0 5.0 5.0 3.7 4.0
39 4.5 4.3 3.7 4.5 3.6 3.6
40 4.0 2.3 3.7 1.5 3.4 3.2
41 2.5 2.0 3.0 3.0 2.0 2.4
42 4.0 4.0 4.0 4.0 4.0 4.0
43 4.5 3.7 3.7 5.0 3.1 4.0
44 4.0 4.0 4.7 4.5 4.0 5.0
45 4.5 3.7 4.7 3.0 3.7 4.0
334
CODE BOK1 BOK2 BOK3 BOK4 BOK5 BOK6

46 4.5 3.0 4.0 4.0 4.0 4.4
47 4.5 4.7 4.7 5.0 4.1 4.8
48 3.0 3.0 2.3 4.0 2.6 2.6
49 2.5 1.7 3.0 3.0 2.7 2.6
50 4.5 3.0 3.3 4.0 3.0 3.8
51 5.0 4.7 5.0 4.5 4.3 4.6
52 3.5 3.0 2.7 3.5 2.9 3.4
53 3.5 3.7 4.7 5.0 2.9 4.0
54 4.0 3.3 4.0 5.0 3.7 4.8
55 4.0 4.0 4.3 5.0 4.6 4.0
56 5.0 4.0 4.7 5.0 5.0 4.2
57 4.5 4.3 4.7 5.0 4.3 5.0
58 1.5 2.7 2.7 2.5 2.4 2.6
59 3.5 4.7 3.3 4.0 3.3 4.4
60 5.0 4.3 4.0 4.5 3.6 4.4
61 3.5 3.0 2.7 3.0 2.9 2.8
62 3.5 3.0 3.0 4.5 3.1 4.0
63 4.0 3.7 3.7 4.5 4.0 4.0
64 2.0 2.3 3.7 4.5 3.4 3.0
65 4.0 4.3 5.0 5.0 3.7 4.2
66 3.5 3.7 3.7 3.5 3.4 3.0
67 4.0 4.3 5.0 5.0 4.0 4.6
68 3.5 2.7 2.0 3.5 1.7 3.0
69 5.0 4.7 5.0 4.5 2.9 4.4
70 4.5 4.0 4.7 4.5 4.1 4.4
71 4.0 3.3 3.0 3.0 3.1 3.4
72 4.0 3.3 4.3 4.0 3.6 4.0
73 3.0 2.0 3.0 1.5 2.6 2.4
74 4.0 4.0 3.3 3.0 4.0 3.4
75 4.5 3.7 4.3 2.5 3.1 4.4
76 3.0 2.7 3.3 3.5 2.4 3.0
77 5.0 4.0 5.0 5.0 4.7 5.0
78 2.5 3.7 2.0 4.0 1.9 2.8
79 4.0 3.7 5.0 4.5 4.1 4.2
80 3.5 4.3 4.0 4.0 2.7 3.8
81 4.5 4.3 4.3 5.0 3.0 4.8
82 3.0 2.7 2.3 3.0 3.4 2.6
83 3.5 4.3 4.0 5.0 2.6 4.0
84 3.5 3.3 4.0 5.0 3.6 3.8
85 4.0 3.0 4.7 4.0 3.9 4.2
86 3.5 3.0 5.0 3.5 2.9 4.2
87 4.0 4.0 3.7 5.0 3.8 4.2
88 3.5 4.0 5.0 5.0 3.9 5.0
335
CODE BOK7 ISO1 ISO2 ISO3 ISO4 ISO5

1 3.4 4.0 4.5 4.5 4.0 4.2
2 2.2 5.0 4.8 4.5 4.8 3.8
3 3.2 4.0 4.2 3.5 4.0 3.6
4 2.6 5.0 4.7 4.3 4.0 3.8
5 2.8 2.0 2.7 3.3 2.8 2.8
6 3.8 4.5 4.2 3.8 3.7 3.8
7 4.0 5.0 4.7 4.5 4.3 4.6
8 2.8 4.0 2.8 2.3 3.0 2.8
9 3.4 4.0 4.0 3.3 3.7 4.6
10 3.2 5.0 4.0 3.5 3.8 4.0
11 3.0 5.0 4.3 4.0 4.2 3.8
12 4.0 5.0 4.2 3.5 4.0 4.0
13 4.6 5.0 5.0 4.3 3.2 4.2
14 3.4 4.0 3.7 3.5 4.0 4.0
15 2.0 5.0 5.0 3.8 5.0 4.0
16 4.6 5.0 4.8 5.0 4.8 5.0
17 3.4 5.0 4.5 4.5 4.8 5.0
18 4.4 5.0 5.0 5.0 4.7 4.8
19 2.4 4.0 3.0 2.3 2.3 3.2
20 2.6 4.0 3.0 3.8 3.5 4.0
21 4.0 5.0 4.8 4.5 4.2 4.8
22 3.4 3.5 3.7 3.3 3.3 3.2
23 3.2 5.0 4.5 5.0 4.0 4.4
24 5.0 5.0 5.0 5.0 5.0 5.0
25 5.0 5.0 5.0 4.0 4.0 5.0
26 3.6 4.5 4.2 3.5 4.4 4.4
27 3.4 5.0 4.2 3.8 3.8 3.6
28 2.2 4.0 3.3 4.8 4.8 3.4
29 3.0 3.5 3.3 2.3 3.3 4.0
30 3.2 5.0 3.8 4.0 4.2 4.8
31 3.4 5.0 4.3 4.8 4.2 4.2
32 2.0 5.0 4.3 4.5 4.7 4.8
33 1.2 2.5 1.7 2.0 1.8 1.0
34 2.8 3.5 3.2 3.5 2.8 3.6
35 3.4 3.0 3.7 3.0 3.7 4.0
36 2.6 4.5 3.3 3.3 2.5 2.8
37 1.0 1.0 1.5 2.0 2.5 1.0
38 3.8 5.0 5.0 5.0 4.0 4.2
39 3.8 4.0 4.2 3.5 3.7 4.0
40 1.2 4.0 2.3 2.5 2.5 2.8
41 2.2 3.5 3.0 3.8 3.2 3.0
42 4.0 4.0 4.0 4.0 4.0 4.0
43 2.8 5.0 4.3 3.8 4.2 4.4
44 5.0 4.0 4.0 4.0 4.0 4.0
45 3.4 4.5 4.3 3.8 4.0 4.0
336
CODE BOK7 ISO1 ISO2 ISO3 ISO4 ISO5

46 3.0 4.5 4.3 4.0 4.3 4.2
47 4.2 5.0 5.0 4.3 4.7 4.6
48 2.6 3.0 2.8 2.5 2.5 2.8
49 2.8 4.0 3.7 4.0 4.0 4.0
50 4.0 4.0 4.0 4.0 3.2 3.6
51 4.4 5.0 5.0 4.5 4.7 4.8
52 2.8 3.5 3.2 3.8 4.0 3.4
53 3.2 5.0 4.7 4.3 4.2 4.8
54 4.2 3.5 3.7 3.0 3.8 3.6
55 4.6 4.5 4.5 3.3 4.2 4.0
56 4.0 4.0 4.5 4.0 4.5 4.0
57 4.2 5.0 4.7 4.8 4.8 5.0
58 2.0 4.0 2.5 2.0 3.5 3.0
59 4.2 4.0 4.2 4.3 3.8 4.2
60 3.6 5.0 4.7 4.3 4.7 4.4
61 2.4 4.0 3.7 3.0 3.2 3.8
62 3.4 4.0 3.8 3.3 3.5 3.0
63 3.0 5.0 4.7 4.0 4.3 4.6
64 1.6 4.0 3.2 2.8 4.2 3.8
65 4.2 5.0 4.3 3.8 4.3 4.4
66 2.8 4.0 2.2 4.3 3.2 3.0
67 4.0 4.0 4.2 4.5 4.0 4.0
68 1.0 3.0 3.2 2.8 3.3 2.8
69 2.0 5.0 5.0 4.5 4.3 5.0
70 4.0 4.0 4.2 4.0 4.0 3.8
71 3.2 3.0 3.0 3.0 3.2 3.0
72 3.2 4.0 4.0 3.5 4.2 4.0
73 2.0 3.5 2.7 3.0 3.0 3.2
74 2.6 3.5 3.0 2.8 3.0 3.0
75 3.8 4.5 4.3 3.3 3.8 4.8
76 2.8 4.5 4.3 3.0 3.3 3.4
77 4.2 5.0 4.8 4.8 4.0 4.2
78 2.6 4.0 3.2 3.0 3.2 2.2
79 3.2 4.0 4.2 4.0 4.0 4.4
80 3.2 4.0 3.7 4.3 3.8 4.0
81 4.4 4.0 4.0 3.5 4.2 4.0
82 2.6 3.5 2.5 2.5 3.0 2.6
83 2.4 4.0 5.0 4.5 3.7 4.2
84 3.6 4.0 3.7 3.8 4.3 3.4
85 3.0 4.0 3.7 3.3 4.3 4.2
86 3.8 4.5 3.7 3.0 3.6 5.0
87 3.8 4.5 3.8 4.3 4.7 4.8
88 4.2 5.0 5.0 5.0 5.0 4.6
337
CODE MB1 MB2 MB3 MB4 MB5 MB6 MB7

1 4.0 4.0 4.5 4.0 3.3 3.0 4.0
2 3.5 4.0 5.0 3.5 4.0 4.0 4.2
3 3.0 3.0 3.5 4.0 3.7 3.5 3.2
4 4.0 0.0 5.0 4.0 4.7 4.0 4.0
5 3.5 2.0 3.0 3.5 3.0 3.0 2.3
6 0.0 0.0 0.0 0.0 0.0 0.0 0.0
7 4.5 4.0 5.0 5.0 4.7 4.5 4.8
8 2.5 3.0 4.5 3.0 3.0 3.5 2.8
9 3.5 2.5 5.0 3.5 4.0 3.0 3.3
10 5.0 4.0 4.5 4.0 3.0 4.0 4.0
11 4.5 4.0 4.0 5.0 4.0 3.0 4.2
12 4.0 4.0 4.0 4.0 3.0 4.0 3.7
13 4.5 4.5 5.0 5.0 5.0 4.0 3.8
14 3.5 4.0 4.0 4.0 4.0 4.0 4.0
15 4.0 2.0 4.5 4.0 3.7 2.5 4.2
16 5.0 5.0 4.5 5.0 5.0 5.0 4.8
17 3.5 4.0 4.5 4.5 3.7 5.0 3.3
18 5.0 5.0 5.0 5.0 4.3 5.0 4.8
19 2.5 2.5 2.5 3.0 2.3 1.0 2.3
20 3.5 2.5 4.5 3.5 3.0 3.5 3.8
21 5.0 4.0 5.0 4.0 5.0 5.0 5.0
22 4.0 0.0 3.5 4.5 3.7 3.0 4.0
23 4.5 4.0 5.0 4.0 4.0 4.0 5.0
24 5.0 5.0 5.0 5.0 5.0 5.0 5.0
25 5.0 5.0 5.0 5.0 4.0 4.0 4.0
26 5.0 3.5 4.0 4.0 4.0 4.0 4.0
27 4.0 4.5 4.0 4.0 3.0 4.0 4.3
28 2.0 3.0 5.0 2.0 2.0 2.5 3.8
29 3.0 3.5 3.5 3.0 3.3 3.0 3.5
30 3.5 3.0 3.5 2.5 3.0 3.0 3.3
31 4.0 3.0 4.5 4.0 3.3 3.5 4.0
32 1.5 2.5 2.0 2.0 2.0 1.0 3.0
33 1.5 1.0 3.0 1.0 1.7 1.0 1.3
34 3.5 2.0 4.0 3.5 3.7 3.5 3.6
35 4.0 2.0 3.5 3.0 3.3 3.0 3.2
36 4.0 3.0 3.5 2.0 3.3 3.5 3.2
37 4.0 2.5 2.5 1.0 1.3 2.5 2.7
38 4.5 3.5 5.0 5.0 4.0 4.0 4.0
39 4.5 5.0 5.0 4.0 3.3 4.0 4.0
40 4.0 3.0 4.0 3.0 2.3 3.0 2.0
41 2.5 2.0 3.0 2.5 2.7 3.0 3.0
42 5.0 3.0 4.0 4.0 4.0 4.0 3.0
43 5.0 4.0 4.5 4.0 4.0 4.0 4.3
44 5.0 5.0 4.0 4.0 3.7 4.0 4.0
45 3.5 4.0 3.0 4.0 3.3 3.5 3.5
338
CODE MB1 MB2 MB3 MB4 MB5 MB6 MB7

46 4.0 4.0 4.0 4.0 4.3 4.0 4.2
47 4.5 4.5 5.0 4.5 3.7 5.0 4.5
48 2.5 3.0 4.0 2.0 2.0 2.0 2.8
49 3.0 3.0 3.5 3.5 2.7 3.5 3.3
50 4.0 3.0 3.0 3.5 3.0 4.0 3.5
51 5.0 4.0 5.0 5.0 4.3 4.0 4.2
52 3.0 3.5 3.5 3.5 2.0 3.5 3.4
53 5.0 3.5 4.5 4.0 5.0 4.0 4.4
54 3.5 4.0 4.0 2.0 2.7 2.5 3.5
55 3.5 3.0 4.0 4.0 3.3 4.0 3.6
56 3.5 3.0 4.5 4.0 3.0 3.0 3.8
57 4.5 5.0 5.0 5.0 4.3 4.5 5.0
58 1.5 1.0 2.5 2.5 1.3 2.5 2.0
59 4.0 3.5 4.0 4.0 3.0 4.0 4.2
60 4.5 4.5 5.0 4.0 3.3 4.0 3.8
61 4.0 3.5 3.0 2.5 2.0 3.5 3.2
62 4.5 4.0 4.0 3.0 4.0 4.0 4.2
63 3.5 4.0 4.5 4.0 4.3 4.0 4.0
64 3.5 3.0 4.5 3.0 2.0 2.0 3.0
65 4.0 4.0 4.5 5.0 3.7 4.0 3.8
66 3.0 3.5 3.5 3.5 3.7 3.5 2.8
67 4.5 4.0 4.0 4.0 4.0 4.0 4.2
68 3.0 3.0 3.5 2.0 1.7 1.5 2.3
69 4.0 4.0 5.0 4.0 2.7 3.0 4.0
70 4.0 4.0 5.0 4.0 4.0 3.5 4.0
71 3.0 3.5 4.0 3.0 3.7 4.0 3.7
72 3.5 3.0 3.0 4.0 3.0 3.5 3.6
73 1.5 2.0 2.5 2.5 2.3 2.5 2.3
74 4.0 4.0 4.0 3.0 3.3 4.0 4.0
75 4.0 4.0 3.5 3.5 3.0 4.0 3.8
76 3.0 2.5 3.0 2.5 2.7 3.0 2.7
77 4.5 5.0 5.0 4.0 4.3 5.0 4.5
78 2.5 3.0 4.5 2.5 2.7 2.5 3.0
79 5.0 4.0 5.0 4.0 4.0 4.0 4.0
80 4.0 4.0 3.5 4.0 3.7 3.5 3.5
81 5.0 4.5 3.5 4.5 3.7 4.0 3.8
82 2.5 2.5 3.0 2.5 3.3 3.0 3.2
83 4.0 5.0 5.0 4.0 3.7 4.0 4.2
84 3.5 5.0 4.5 3.5 3.0 3.5 4.0
85 3.5 3.0 4.0 3.5 3.7 4.0 3.8
86 3.0 3.0 3.5 3.5 3.7 3.0 3.0
87 4.5 4.0 5.0 4.0 4.0 4.0 4.6
88 5.0 4.0 5.0 5.0 5.0 5.0 5.0
339
CODE BOK1T BOK2T BOK3T BOK4T BOK5T BOK6T

1 20.50 16.00 13.67 25.00 16.29 16.00
2 20.50 12.67 16.67 25.00 11.29 6.00
3 12.50 7.33 16.00 16.00 11.29 14.60
4 20.50 16.00 9.00 16.00 16.00 10.40
5 9.00 5.67 7.33 4.00 5.25 6.00
6 25.00 22.00 19.00 20.50 12.83 21.40
7 25.00 19.00 22.00 25.00 19.86 19.60
8 9.00 11.33 13.67 16.00 4.00 10.40
9 12.50 4.67 9.00 20.50 5.00 12.60
10 20.50 13.67 13.67 16.00 14.29 22.75
11 12.50 16.00 13.67 12.50 13.00 13.20
12 25.00 22.00 25.00 16.00 16.00 19.60
13 25.00 16.67 13.67 25.00 11.00 19.60
14 16.00 11.33 12.67 12.50 10.57 13.20
15 20.50 4.00 22.00 20.50 7.86 17.80
16 25.00 22.00 22.00 20.50 15.29 25.00
17 16.00 19.00 25.00 25.00 25.00 19.60
18 25.00 25.00 25.00 25.00 19.86 25.00
19 16.00 13.67 25.00 9.00 4.00 11.80
20 12.50 9.00 12.00 9.00 2.14 11.80
21 25.00 16.00 22.00 25.00 21.14 23.20
22 12.50 13.67 9.00 16.00 10.00 11.80
23 16.00 16.67 25.00 25.00 13.00 16.40
24 25.00 25.00 25.00 25.00 25.00 25.00
25 25.00 22.00 25.00 16.00 16.00 23.20
26 25.00 11.33 19.00 20.50 11.29 14.60
27 25.00 19.00 25.00 16.00 13.29 16.40
28 4.00 7.33 5.67 20.50 6.86 7.00
29 9.00 19.00 16.67 12.50 15.29 10.40
30 16.00 12.00 22.00 20.50 6.14 19.60
31 25.00 9.67 13.67 25.00 6.71 21.40
32 10.00 9.00 13.67 16.00 9.29 16.80
33 4.00 4.00 8.00 2.50 7.71 5.00
34 12.50 5.67 13.67 20.50 4.57 7.00
35 9.00 5.67 9.00 12.50 11.00 11.80
36 12.50 9.67 12.00 9.00 7.57 10.80
37 10.00 7.33 4.67 6.50 6.00 4.80
38 20.50 16.67 25.00 25.00 14.00 16.80
39 20.50 19.00 13.67 20.50 13.00 13.20
40 16.00 6.33 13.67 2.50 12.00 10.80
41 6.50 4.00 9.00 9.00 4.57 6.00
42 16.00 16.00 16.00 16.00 16.00 16.00
43 20.50 13.67 13.67 25.00 10.57 16.40
44 16.00 16.00 22.00 20.50 16.00 25.00
45 20.50 13.67 22.00 9.00 14.00 16.00
340
CODE BOK1T BOK2T BOK3T BOK4T BOK5T BOK6T

46 20.50 9.00 16.00 16.00 16.29 19.60
47 20.50 22.00 22.00 25.00 17.57 23.20
48 9.00 9.00 5.67 16.00 7.43 7.00
49 6.50 3.00 9.00 10.00 7.86 7.00
50 20.50 9.00 11.33 16.00 9.57 14.60
51 25.00 22.00 25.00 20.50 18.57 21.40
52 12.50 9.67 7.33 12.50 8.86 12.60
53 14.50 13.67 22.00 25.00 9.14 17.20
54 16.00 12.00 16.00 25.00 14.00 23.20
55 16.00 16.00 19.00 25.00 21.14 16.00
56 25.00 16.00 22.00 25.00 25.00 17.80
57 20.50 19.00 22.00 25.00 18.57 25.00
58 2.50 7.33 7.33 6.50 6.14 7.40
59 12.50 22.00 11.33 16.00 11.86 19.60
60 25.00 19.00 16.00 20.50 13.00 19.60
61 12.50 9.00 7.33 9.00 8.57 8.40
62 12.50 9.00 9.00 20.50 10.00 16.00
63 16.00 13.67 13.67 20.50 16.29 16.00
64 5.00 5.67 13.67 20.50 13.14 9.40
65 16.00 19.00 25.00 25.00 14.00 17.80
66 12.50 13.67 13.67 12.50 12.00 9.40
67 16.00 19.00 25.00 25.00 16.00 21.40
68 12.50 8.00 4.67 12.50 3.14 9.00
69 25.00 22.00 25.00 20.50 9.14 20.00
70 20.50 16.00 22.00 20.50 17.29 19.60
71 16.00 11.33 9.00 9.00 10.00 11.80
72 16.00 12.00 19.00 16.00 13.00 16.00
73 10.00 4.00 9.67 2.50 6.86 6.80
74 16.00 16.00 12.00 9.00 16.29 11.80
75 20.50 13.67 19.00 6.50 10.00 19.60
76 9.00 7.33 11.33 12.50 7.29 9.50
77 25.00 16.00 25.00 25.00 22.43 25.00
78 6.50 14.33 4.00 16.00 3.57 8.00
79 16.00 13.67 25.00 20.50 17.57 17.80
80 12.50 19.00 16.00 16.00 7.86 14.60
81 20.50 19.00 19.00 25.00 9.33 22.75
82 9.00 7.33 5.67 9.00 12.29 7.00
83 12.50 19.67 16.67 25.00 7.71 16.00
84 12.50 11.33 16.67 25.00 13.00 14.60
85 16.00 9.00 22.00 16.00 15.00 17.80
86 12.50 9.00 25.00 12.50 8.57 18.20
87 17.00 16.00 13.67 25.00 14.25 17.80
88 12.50 16.00 25.00 25.00 15.00 25.00
341
CODE BOK7T ISO1T ISO2T ISO3T ISO4T ISO5T MB1T

1 11.80 16.00 20.50 20.50 16.00 17.80 16.00
2 5.40 25.00 23.50 20.50 23.50 15.00 12.50
3 10.40 16.00 17.83 12.50 16.00 13.20 10.00
4 7.00 25.00 22.00 18.75 16.67 15.00 16.00
5 8.00 4.00 7.33 10.75 8.00 7.75 12.50
6 14.60 20.50 17.83 14.25 13.67 14.60 0.00
7 16.00 25.00 22.00 20.50 19.33 21.40 20.50
8 8.00 17.00 8.50 5.25 9.33 8.00 6.50
9 12.60 16.00 16.33 10.75 14.00 21.40 12.50
10 10.40 25.00 16.33 12.50 15.17 16.40 25.00
11 9.40 25.00 19.00 16.00 17.50 14.60 20.50
12 16.00 25.00 17.50 12.50 16.00 16.00 16.00
13 21.40 25.00 25.00 18.25 12.50 18.20 20.50
14 12.20 16.00 13.67 12.50 16.00 16.00 12.50
15 4.40 25.00 25.00 14.25 25.00 16.40 17.00
16 21.40 25.00 23.50 25.00 23.50 25.00 25.00
17 11.80 25.00 20.50 20.50 23.50 25.00 12.50
18 19.60 25.00 25.00 25.00 22.00 23.20 25.00
19 6.40 16.00 9.33 5.25 5.67 10.40 6.50
20 7.40 16.00 9.33 14.25 12.50 16.00 12.50
21 16.00 25.00 23.50 20.50 19.50 23.20 25.00
22 11.80 12.50 13.67 10.75 11.33 10.40 16.00
23 10.40 25.00 20.50 25.00 16.40 19.60 20.50
24 25.00 25.00 25.00 25.00 25.00 25.00 25.00
25 25.00 25.00 25.00 16.00 16.00 25.00 25.00
26 13.20 20.50 17.83 12.50 19.60 19.60 25.00
27 11.80 25.00 17.50 14.25 14.60 13.20 16.00
28 5.80 16.00 12.33 22.75 23.50 12.60 4.00
29 9.00 12.50 11.33 5.25 11.67 16.00 9.00
30 10.40 25.00 14.83 16.00 17.80 23.20 14.50
31 11.80 25.00 19.00 22.75 17.83 18.20 16.00
32 4.00 25.00 19.33 20.50 22.33 23.20 2.50
33 1.60 6.50 3.33 4.50 4.50 1.00 2.50
34 8.00 12.50 11.17 12.50 8.50 13.20 12.50
35 12.20 9.00 13.67 9.00 13.67 16.00 16.00
36 7.00 20.50 11.33 10.75 6.50 8.40 16.00
37 1.00 1.00 2.83 4.50 7.17 1.00 17.00
38 15.00 25.00 25.00 25.00 16.00 18.20 20.50
39 14.75 16.00 17.50 12.50 13.67 16.00 20.50
40 1.60 16.00 6.00 6.50 6.50 8.40 16.00
41 5.00 12.50 9.00 14.25 11.17 9.00 6.50
42 16.00 16.00 16.00 16.00 16.00 16.00 25.00
43 8.40 25.00 19.33 14.25 17.50 19.60 25.00
44 25.00 16.00 16.00 16.00 16.00 16.00 25.00
45 11.80 20.50 19.00 14.25 16.00 16.00 12.50
342
CODE BOK7T ISO1T ISO2T ISO3T ISO4T ISO5T MB1T

46 9.00 20.50 19.33 16.00 19.00 17.80 16.00
47 17.80 25.00 25.00 18.25 22.00 21.40 20.50
48 7.00 9.00 8.83 6.50 6.50 8.00 6.50
49 8.00 16.00 14.00 16.00 16.00 16.40 9.00
50 16.00 16.00 16.00 16.00 10.17 13.20 16.00
51 19.60 25.00 25.00 20.50 22.00 23.20 25.00
52 8.00 12.50 10.17 14.25 16.00 11.80 9.00
53 10.40 25.00 22.00 18.25 17.83 23.20 25.00
54 18.20 12.50 13.67 9.00 14.83 13.20 12.50
55 21.40 20.50 20.50 10.75 17.50 16.00 12.50
56 16.00 16.00 20.50 16.00 20.50 16.00 12.50
57 17.80 25.00 22.00 22.75 23.50 25.00 20.50
58 4.00 16.00 6.50 4.00 12.50 9.40 2.50
59 17.80 16.00 17.83 18.25 14.83 17.80 16.00
60 13.20 25.00 22.00 18.25 22.00 19.60 20.50
61 6.00 16.00 13.67 9.00 10.50 14.60 16.00
62 11.80 16.00 15.17 10.75 12.50 9.00 20.50
63 9.00 25.00 22.00 16.00 19.00 21.40 12.50
64 2.80 16.00 11.17 7.75 17.83 15.00 12.50
65 17.80 25.00 19.00 14.25 19.00 19.60 16.00
66 8.40 16.00 5.50 18.25 10.50 9.00 9.00
67 16.00 16.00 17.50 20.50 16.00 16.00 20.50
68 1.00 9.00 10.50 7.75 11.33 8.40 9.00
69 4.40 25.00 25.00 20.50 21.00 25.00 16.00
70 16.00 16.00 17.50 16.00 16.00 14.60 17.00
71 10.40 9.00 9.00 9.00 10.17 9.00 9.00
72 10.40 16.00 16.00 12.50 17.50 16.00 12.50
73 4.00 12.50 7.33 9.00 9.33 10.80 2.50
74 7.00 12.50 9.00 7.75 9.33 9.00 16.00
75 14.60 20.50 19.00 10.75 15.17 23.20 16.00
76 8.00 20.50 19.33 9.00 11.67 12.20 9.00
77 17.80 25.00 23.50 22.75 16.00 17.80 20.50
78 7.00 16.00 10.17 9.50 10.17 5.40 6.50
79 10.40 16.00 17.50 16.00 16.00 19.60 25.00
80 10.40 16.00 13.67 18.25 14.83 16.00 16.00
81 20.00 16.00 16.33 12.50 17.80 16.00 25.00
82 7.00 12.50 6.50 6.50 9.33 7.00 6.50
83 6.40 16.00 25.00 20.50 15.33 17.80 16.00
84 13.60 16.00 14.00 14.25 19.00 11.80 12.50
85 9.40 16.00 13.67 10.75 19.00 18.20 12.50
86 14.60 20.50 13.67 9.00 13.60 25.00 9.00
87 14.60 20.50 15.17 18.25 22.00 23.20 20.50
88 17.80 25.00 25.00 25.00 25.00 21.40 25.00
343
CODE MB2T MB3T MB4T MB5T MB6T MB7T GEN

1 16.00 20.50 16.00 11.33 9.00 16.00 0
2 16.00 25.00 12.50 16.00 16.00 17.83 0
3 9.00 12.50 16.00 13.67 12.50 10.50 1
4 0.00 25.00 16.00 22.00 16.00 16.00 0
5 4.00 9.00 12.50 9.00 9.00 5.67 0
6 0.00 0.00 0.00 0.00 0.00 0.00 0
7 16.00 25.00 25.00 22.00 20.50 23.50 0
8 9.00 20.50 9.00 9.00 12.50 8.17 0
9 6.50 25.00 12.50 16.00 9.00 11.67 1
10 16.00 20.50 16.00 9.00 16.00 16.67 0
11 16.00 16.00 25.00 16.00 9.00 17.83 0
12 16.00 16.00 16.00 9.00 16.00 13.67 0
13 20.50 25.00 25.00 25.00 16.00 15.17 0
14 16.00 16.00 16.00 16.00 16.00 16.00 0
15 4.00 20.50 16.00 13.67 6.50 18.17 0
16 25.00 20.50 25.00 25.00 25.00 23.50 0
17 16.00 20.50 20.50 14.33 25.00 11.33 0
18 25.00 25.00 25.00 19.00 25.00 23.50 0
19 6.50 6.50 10.00 5.67 1.00 6.33 0
20 6.50 20.50 12.50 9.00 12.50 15.17 0
21 16.00 25.00 16.00 25.00 25.00 25.00 0
22 0.00 12.50 20.50 13.67 9.00 16.00 0
23 16.00 25.00 16.00 16.00 16.00 25.00 0
24 25.00 25.00 25.00 25.00 25.00 25.00 1
25 25.00 25.00 25.00 16.00 16.00 16.00 0
26 12.50 16.00 16.00 16.67 16.00 17.33 0
27 20.50 16.00 16.00 9.00 16.00 19.00 0
28 9.00 25.00 5.00 4.67 6.50 16.17 0
29 12.50 12.50 9.00 11.33 9.00 12.50 0
30 9.00 12.50 6.50 9.00 9.00 11.67 1
31 9.00 20.50 16.00 11.33 12.50 16.33 1
32 6.50 4.00 4.00 4.00 1.00 9.33 1
33 1.00 9.00 1.00 3.00 1.00 2.33 0
34 4.00 16.00 12.50 13.67 12.50 13.20 0
35 4.00 12.50 9.00 11.33 9.00 10.17 0
36 9.00 12.50 4.00 11.33 12.50 10.17 0
37 6.50 6.50 1.00 2.00 6.50 7.67 1
38 12.50 25.00 25.00 16.00 16.00 16.40 0
39 25.00 25.00 16.00 11.33 16.00 17.00 0
40 9.00 16.00 9.00 5.67 9.00 4.67 1
41 4.00 9.00 6.50 7.33 9.00 9.40 0
42 9.00 16.00 16.00 16.00 16.00 9.00 0
43 16.00 20.50 16.00 16.00 16.00 19.00 1
44 25.00 16.00 16.00 13.67 16.00 16.00 0
45 16.00 9.00 16.00 11.33 12.50 12.50 0
344
CODE MB2T MB3T MB4T MB5T MB6T MB7T GEN

46 16.00 16.00 16.00 19.00 16.00 17.50 1
47 20.50 25.00 20.50 13.67 25.00 20.50 0
48 9.00 16.00 4.00 4.00 4.00 8.40 0
49 9.00 12.50 12.50 7.33 12.50 11.67 0
50 9.00 9.00 12.50 9.00 16.00 12.83 0
51 16.00 25.00 25.00 19.00 16.00 17.50 0
52 12.50 12.50 12.50 4.00 12.50 11.80 0
53 12.50 20.50 16.00 25.00 16.00 19.60 1
54 16.00 16.00 4.00 8.00 6.50 12.83 0
55 9.00 16.00 16.00 11.33 16.00 13.20 0
56 9.00 20.50 16.00 9.00 9.00 15.17 0
57 25.00 25.00 25.00 19.00 20.50 25.00 0
58 1.00 6.50 6.50 2.00 6.50 4.40 0
59 12.50 16.00 16.00 9.00 16.00 17.80 0
60 20.50 25.00 16.00 11.33 16.00 15.00 0
61 12.50 9.00 6.50 4.00 12.50 10.17 0
62 16.00 16.00 9.00 16.00 16.00 17.50 0
63 16.00 20.50 16.00 19.00 16.00 16.33 0
64 9.00 20.50 9.00 4.00 4.00 9.40 0
65 16.00 20.50 25.00 13.67 16.00 14.83 0
66 12.50 12.50 12.50 13.67 12.50 9.50 0
67 16.00 16.00 16.00 16.00 16.00 17.80 0
68 9.00 12.50 4.00 3.00 2.50 6.00 0
69 17.00 25.00 16.00 7.33 9.00 16.67 0
70 16.00 25.00 16.00 16.00 12.50 16.00 0
71 12.50 16.00 9.00 13.67 16.00 13.67 0
72 9.00 9.00 16.00 9.00 12.50 13.20 0
73 4.00 6.50 6.50 5.67 6.50 6.00 0
74 16.00 16.00 9.00 11.33 16.00 16.00 0
75 16.00 12.50 12.50 9.00 16.00 14.60 0
76 6.50 9.00 6.50 7.33 9.00 7.67 0
77 25.00 25.00 16.00 19.00 25.00 20.50 0
78 9.00 20.50 6.50 7.33 6.50 9.67 0
79 16.00 25.00 16.00 16.00 16.00 16.00 0
80 16.00 12.50 16.00 13.67 12.50 12.50 0
81 20.50 12.50 20.50 13.67 16.00 15.17 0
82 6.50 9.00 6.50 11.33 9.00 10.50 0
83 25.00 25.00 16.00 13.67 16.00 17.83 0
84 25.00 20.50 12.50 9.00 12.50 16.40 0
85 9.00 16.00 12.50 13.67 16.00 15.17 0
86 9.00 12.50 12.50 13.67 9.00 9.00 0
87 16.00 25.00 16.00 16.67 16.00 21.40 1
88 16.00 25.00 25.00 25.00 25.00 25.00 0
345
APPENDIX G
Table G1
Spearman’s Rho Correlation Data Between Demographics Response Items and All
Variables
346
Number of
Hold Years of QM ASQ
CQM? Experience Certifications
BOK1 Spearman’s Rho -.064 .027 -.143
Sig. (2-tailed) .561 .800 .185
N 86 88 88
Sig. (2-tailed) .760 .053 .733
N 86 88 88
BOK3 Spearman’s Rho .075 .125 -.007
Sig. (2-tailed) .495 .247 .946
N 86 88 88
BOK4 Spearman’s Rho -.218(*) -.106 -.172
Sig. (2-tailed) .044 .327 .110
N 86 88 88
Sig. (2-tailed) .527 .981 .301
N 86 88 88
BOK6 Spearman’s Rho .059 .099 .006
Sig. (2-tailed) .590 .361 .959
N 86 88 88
Sig. (2-tailed) .852 .120 .883
N 86 88 88
ISO1 Spearman’s Rho -.020 -.001 -.091
Sig. (2-tailed) .856 .993 .400
N 86 88 88
ISO2 Spearman’s Rho -.038 -.142 -.102
Sig. (2-tailed) .732 .188 .346
N 86 88 88
ISO3 Spearman’s Rho -.142 .015 -.161
Sig. (2-tailed) .193 .890 .134
N 86 88 88
Sig. (2-tailed) .648 .788 .463
N 86 88 88
ISO5 Spearman’s Rho .054 .057 -.059
Sig. (2-tailed) .620 .600 .586
N 86 88 88
MB1 Spearman’s Rho -.075 .088 -.195
Sig. (2-tailed) .496 .418 .070
N 85 87 87
347
Number of
Hold Years of QM ASQ
CQM? Experience Certifications
Sig. (2-tailed) .478 .137 .311
N 83 85 85
MB3 Spearman’s Rho -.238(*) -.149 -.258(*)
Sig. (2-tailed) .028 .168 .016
N 85 87 87
Sig. (2-tailed) .663 .203 .264
N 85 87 87
MB5 Spearman’s Rho .000 .037 -.109
Sig. (2-tailed) 1.000 .731 .313
N 85 87 87
Sig. (2-tailed) .636 .276 .054
N 85 87 87
Sig. (2-tailed) .149 .837 .102
N 85 87 87
348
Years of Post- Employees

Secondary Number of Under
Education Employees Direction
BOK1 Spearman’s Rho -.219(*) .245(*) .054
Sig. (2-tailed) .040 .022 .615
N 88 88 88
Sig. (2-tailed) .335 .133 .962
N 88 88 88
BOK3 Spearman’s Rho -.102 .287(**) .200
Sig. (2-tailed) .343 .007 .062
N 88 88 88
BOK4 Spearman’s Rho -.161 .213(*) -.028
Sig. (2-tailed) .135 .047 .792
N 88 88 88
Sig. (2-tailed) .108 .002 .272
N 88 88 88
Sig. (2-tailed) .207 .008 .230
N 88 88 88
BOK7 Spearman’s Rho -.001 .313(**) -.029
Sig. (2-tailed) .992 .003 .786
N 88 88 88
ISO1 Spearman’s Rho -.143 .155 .114
Sig. (2-tailed) .183 .150 .291
N 88 88 88
Sig. (2-tailed) .621 .093 .922
N 88 88 88
Sig. (2-tailed) .250 .100 .937
N 88 88 88
ISO4 Spearman’s Rho -.243(*) .257(*) -.069
Sig. (2-tailed) .023 .016 .522
N 88 88 88
Sig. (2-tailed) .735 .237 .824
N 88 88 88
MB1 Spearman’s Rho -.124 .309(**) .025
Sig. (2-tailed) .251 .004 .817
N 87 87 87
349
Years of Post- Employees

Secondary Number of Under
Education Employees Direction
MB2 Spearman’s Rho -.050 .306(**) -.039
Sig. (2-tailed) .647 .004 .723
N 85 85 85
MB3 Spearman’s Rho -.134 .109 .001
Sig. (2-tailed) .216 .315 .990
N 87 87 87
MB4 Spearman’s Rho -.170 .268(*) -.029
Sig. (2-tailed) .115 .012 .790
N 87 87 87
MB5 Spearman’s Rho -.009 .213(*) .089
Sig. (2-tailed) .934 .048 .413
N 87 87 87
MB6 Spearman’s Rho -.011 .341(**) .054
Sig. (2-tailed) .922 .001 .620
N 87 87 87
MB7 Spearman’s Rho -.125 .226(*) .066
Sig. (2-tailed) .250 .035 .543
N 87 87 87

Dco Dissertation 6-24-04

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Dco Dissertation 6-24-04

Hochgeladen von

Copyright:

Verfügbare Formate

ASSESSMENT OF CURRENT QUALITY MANAGEMENT PRACTICES

The School of Technology

Indiana State University

Terre Haute, Indiana

of the Requirements for the Degree

© Darren C. Olson 2004

ratings were based upon a 5-point Likert scale.

After applying a squared transformation to the data, ANCOVA was used to

determine if there were significant correlations between independent and dependent

improving quality management efforts.

Participants also responded to demographics questions. Among the responses,

I wish to express my deepest appreciation to my wife, Elizabeth, and children,

LIST OF TABLES ............................................................................................................. x

LIST OF FIGURES ......................................................................................................... xv

CHAPTER 1. Introduction ................................................................................................ 1

The Context of the Problem ................................................................................... 1

Statement of the Problem ....................................................................................... 7

Significance of the Study ........................................................................................ 7

Research Objectives ............................................................................................ 10

CHAPTER 2. Review of Literature ................................................................................ 12

Quality and Quality Management ........................................................................ 12

Defining Characteristics of Quality ......................................................... 12

Activities Used to Assure Quality ............................................................ 17

Organizing to Achieve Quality ................................................................. 24

Quality Managers and Professional Practice .......................................... 29

ASQ’s Vision for the Future .................................................................................. 38

Anticipating Needs ................................................................................... 39

Models of Organizational Quality ........................................................... 43

CHAPTER 3. Method ..................................................................................................... 57

Restatement of the Problem ................................................................................. 57

Restatement of Objectives .................................................................................... 57

Research Design .................................................................................................. 58

Phase I: Review of Literature .................................................................. 59

Phase II: Develop the Study ..................................................................... 63

Questionnaire Refinement Using a Modified Delphi Process ...... 64

Time Testing the Revised Questionnaire....................................... 67

Phase III: Administer the Study and Analyze Results ............................... 68

Administer the Study ..................................................................... 68

Analyze the Results ...................................................................... 71

CHAPTER 4. Results ...................................................................................................... 72

Checking Normality Assumptions ......................................................................... 75

Skewness and Kurtosis Z scores .............................................................. 89

Normality Tests ........................................................................................ 91

Data Transformation ........................................................................................... 98

Analyze the Data for Model Fit ............................................................ 121

Analysis of the Refined Model Using ANCOVA ................................... 146

Analysis of Demographics Data ........................................................................ 168

Is the Respondent a Certified Quality Manager? (CQM) ...................... 171

Number of Years, QM Experience (EXP) .............................................. 173

Number of ASQ Certifications Held (NCERT) ...................................... 174

Responsibility Level (RESP) .................................................................. 176

Education Level (EDU) .......................................................................... 183

Number of Employees, Respondent’s Division or

Organization (EMPL) ............................................................................ 186

Number of Employees Under Respondent’s Direction (DIRECT) ......... 195

NAICS Code Categories of Respondents’ Employers (NAICS) .............. 196

ASQ Region (REGION)........................................................................... 202

Respondent Gender (GEN) .................................................................... 207

CHAPTER 5. Discussion .............................................................................................. 212

Conclusions ........................................................................................................ 212

Descriptive Statistics .............................................................................. 213

Normality Assumptions and Hypotheses ................................................ 218

Normality Assumptions .............................................................. 218

Hypotheses One and Two ........................................................... 219