Quality Engineering

ISSN: 0898-2112 (Print) 1532-4222 (Online) Journal homepage: http://www.tandfonline.com/loi/lqen20

Industrial statistics: The challenges and the

research

David M. Steinberg

To cite this article: David M. Steinberg (2016) Industrial statistics: The challenges and the
research, Quality Engineering, 28:1, 45-59, DOI: 10.1080/08982112.2015.1100453

To link to this article: http://dx.doi.org/10.1080/08982112.2015.1100453

Published online: 29 Jan 2016.

Submit your article to this journal

Article views: 45

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=lqen20

Download by: [University of Arizona] Date: 23 February 2016, At: 14:59

 QUALITY ENGINEERING
, VOL. , NO. , –
http://dx.doi.org/./..

Industrial statistics: The challenges and the research

David M. Steinberg
Department of Statistics and Operations Research, Tel Aviv University, Tel Aviv, Israel

ABSTRACT KEYWORDS
Industrial problems have stimulated an enormous amount of valuable statistical research, from the statistical practice; design of
t-test to advanced statistical tools for quality. Industry continues to generate challenging problems experiments; SPC; reliability;
for statistical design, modeling, and analysis. Useful articles are published in our journals, often stim- data science
ulated by industrial applications. Nonetheless, there is concern that research in industrial statistics is
falling well short of its potential for providing interesting problems, that some of the most exciting
problems are not getting space in our journals, and that few statisticians working in industry are pub-
lishing research. This article endeavors to map out the current state of research in industrial statistics,
Downloaded by [University of Arizona] at 14:59 23 February 2016

to describe major issues that need to be addressed, and to discuss whether the research is on target
to meet those challenges.

Historical perspective r Exploratory data analysis was stimulated by prob-

There is good reason to expect a symbiotic link between
industry and statistical research. Industrial settings
generate many challenging statistical problems. And as
Box (1984) elegantly argued, real problems are often
the catalyst for innovative research.
Many important statistical ideas and methods have
their roots in industrial problems. To name just a few
examples:
r The t-test grew out of Gosset’s desire to analyze
small samples of data at the Guinness Brewery.
r The rank-sum test was developed by Frank
Wilcoxon from his need for distribution-free
methods to analyze data at American Cyanimid.
r Statistical process control was Walter Shewhart’s
response to monitoring and improving produc-
tion at Bell Telephone Company.
r The need for efficient munitions testing in World
War II led Abraham Wald to invent the sequential
probability ratio test.
r The management philosophy of W. Edwards
Deming, and the role of statistics in management
and quality, arose from his close association with
industry.
r Ridge regression addressed problems of correlated
predictors faced by Arthur Hoerl and Robert Ken-
nard at Du Pont.
lems John Tukey encountered at Bell Laboratories.
r Response surface methodology resulted from
George Box’s collaborations with chemists at
Imperial Chemical Industries.
The accomplishments listed above, which span the
first seven decades of the last century, are evidence of
the impact that industrial problems have had on statis-
tical research. Stu Hunter (1983) described the stim-
ulating environment of statistics in the early 1950s:
“For someone interested in the applications of statis-
tics in the sciences and engineering the excitement of
the period went far beyond the usual industrial applica-
tions of control charts and acceptance sampling plans.
… The desire to exchange ideas and experiences among
the industrial statisticians was intense” (pp. 3–4). Some
of those statisticians were convinced that existing jour-
nals did not provide a satisfactory forum for their ideas
and methods. This energy came to the fore at a meet-
ing early in 1958. Hunter (1983, p. 4) quoted from
the notes of that meeting: “A new journal should be
started to present applications of statistics in the physi-
cal and engineering sciences.” The American Statisti-
cal Association (ASA) and the American Society for
Quality Control agreed to jointly sponsor the journal
and Technometrics was established, with Hunter as its
editor.

CONTACT David M. Steinberg dms@post.tau.ac.il Department of Statistics and Operations Research, Tel Aviv University, Tel Aviv , Israel.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/lqen.
©  Taylor & Francis
 46 D. M. STEINBERG

Prior to launching Technometrics, the ASA pub-

lished a single technical journal—the Journal of the
American Statistical Association. This initial decision
by the ASA to sponsor a more focused research jour-
nal was directed toward industrial and engineering
problems, providing further evidence of the impact of
industrial problems on statistical research.
The inaugural issue of Technometrics was published
in February 1959. Of 23 articles published in the first
volume, only 9 included authors from academia. The
remaining articles were written by researchers in indus-
try or at national laboratories. Those authors came
from a variety of sectors, including the chemical indus- Figure . Manufacturing and service jobs in the U.S. economy
try, telecommunications, and the National Bureau of (from the U.S. Bureau of Labor Statistics web site).
Standards.
Several subject areas dominated the initial research
Downloaded by [University of Arizona] at 14:59 23 February 2016

“a detailed study of a subject, especially in order to dis-

on industrial statistics: how to control and improve
cover (new) information or reach a (new) understand-
quality, accelerating development with designed exper-
ing” (http://dictionary.cambridge.org/dictionary/engl
iments, etc. The research proposed and refined many
ish/research). Thus, research includes new theory, new
of the tools that are now deeply ingrained in Six
methods, assessment of methods (including assess-
Sigma’s define–measure–analyze–improve–control
ment of their value for practice), and creative appli-
cycle: design of experiments (DOE), reliability and
cation of existing methods (for example, to new
metrology as fundamental to measurement and analy-
problems). It also embraces theoretical constructs for
sis, and statistical process control (SPC) and acceptance
enhancing thinking and application. Articles that illus-
sampling for control of processes and output.
trate existing methods are of great value to the profes-
sion as case studies, but if they do not include any inno-
Aims and scope vative ideas, I will not include them as research.

My goals in this article are to summarize the cur-

rent state of research in industrial statistics, to describe
some of the problems facing industrial statisticians that
need more research, and to comment on whether the
research is effectively aligned with the challenges. Four
basic questions will guide the presentation:
1. What is driving statistical research today?
2. Are industrial challenges affecting the research
agenda in statistics?
3. Are statistical challenges in industry being bet-
ter addressed by other research communities?
4. Is the statistics research community missing
important opportunities for impact in industry?
Some preliminary comments are in order to stake
out the playing field and impose some ground rules.
Research
My focus is on statistical research, not on application of
statistical methods and thinking in industry. The Cam-
bridge Dictionary (online version) defines “research” as
Industry
More specifically, my focus is on research in “industrial
statistics.” The Cambridge Dictionary defines “indus-
try” as “the companies and activities involved in the
process of producing goods for sale, especially in a fac-
tory or special area” (http://dictionary.cambridge.org/
dictionary/english/industry). So “industrial statistics”
refers to the use of statistics to promote the develop-
ment, testing, production, and marketing of goods and
the processes that support these efforts. I will take a lib-
eral view of what is industry so that “goods” will include
the production of services as well as physical items. My
choice reflects the marked shift of Western economies
from manufacturing to services during the last 30 years.
Figure 1, from the U.S. Bureau of Labor Statistics web
site, shows the number of U.S. jobs in manufacturing
and services from 1970 through 2011. The ratio of ser-
vice to manufacturing jobs was about 2.5 to 1 in 1970
and since then has steadily increased, to about 7 to

1 in 2011. Given the dominant economic role of ser-
vice production, it seems clear that this sector must be
included in any discussion of industrial statistics.
Caveat
I will exclude two industrial sectors from this article:
biostatistics (even if the research was done within the
pharmaceutical industry) and financial engineering.
Although both are areas that currently provide many
jobs in the private sector, they are quite distinct in terms
of research from what I regard as industrial statistics.
Moreover, both have their own publishing outlets and
they rarely publish in journals with an industrial statis-
tics focus. A closer look at the pharmaceutical industry
may help to make the distinction more precise: statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016
tical research on the design and analysis of Phase III
clinical trials is excluded; statistical research related to
identifying treatments and improving processes clearly
is within the scope of the article.
Industrial statistics in 2015
The use of statistics in industry is widespread. Qual-
ity improvement programs, Six Sigma, etc., have been
important contributing factors. The increasing avail-
ability of data is also generating a need for statistics (see
next section). The situation was aptly summarized by
Gerry Hahn and Necip Doganaksoy in the May 2008
Technometrics panel discussion on the future of indus-
trial statistics: “We live in a golden age of industrial
statistics” (Steinberg 2008, p. 104), a sentiment shared
by most of the other panelists.
As a counterpoint to the “golden age,” panelists also
noted that statisticians are not always involved. Sallie
Keller-McNulty stated that “the current status of statis-
tics in industry is strong; however, the status of statis-
ticians in industry is possibly at an all-time low. Statis-
tics has become an integral part of all of business and
industry, but statisticians have not. … Over the last 10–
20 years, we have seen a serious decline of major sta-
tistical forces in industry” (p. 105). One reason is the
positive trend to democratization: easy-to-use software
has enabled engineers, scientists, and others to handle
routine statistical work on their own. Another is that
individuals trained in other domains, including com-
puter science, informatics, and industrial engineering,
may be the first to address problems that involve data
and often end up being the primary source of statistical
QUALITY ENGINEERING 47
expertise. Many companies still have strong statistical
groups and some (especially in information technology
[IT] and Internet companies) have had strong growth,
but others have been downsized or eliminated.
Big data, data science and industry
“Big data” and “data science” are more than just buzz-
words. Business, industry, science, and technology have
an unprecedented volume, velocity, and variety of data
at their disposal and a matching need for skilled indi-
viduals who can help them convert that data into com-
petitive advantage. The data deluge has led to many
articles in the popular press (for example, “The Age of
Big Data” in the New York Times, February 11, 2012,
or “Data Crunchers Now the Cool Kids on Campus” in
the Wall Street Journal, March 2013), and it prompted a
special issue of the leading journal Science in February
2011.
The McKinsey Global Institute released a widely
cited report in June 2011 entitled Big Data: The Next
Frontier for Innovation, Competition and Productivity
(Manyika et al. 2011). The McKinsey report “suggests
that we are on the cusp of a tremendous wave of inno-
vation, productivity, and growth, as well as new modes
of competition and value capture—all driven by big
data as consumers, companies, and economic sectors
exploit its potential” (p. 2). The report points to strong
evidence of the major economic impact of big data and
predicts that big data will make major contributions
to the world economy. To realize that potential, com-
panies will need to hire staff with an increasing level
of sophistication in handling, analyzing, and making
decisions based on data. The report cautions that “a sig-
nificant constraint … will be a shortage of … people
with deep expertise in statistics and machine learning
… a talent gap of 140K–190K positions in 2018 [in the
United States]” (p. 10).
The McKinsey report describes statistics as one
of the basic tools required to convert data into use-
ful information. That view is echoed by Provost and
Fawcett (2013), two of the leading figures in the
“big data” community: “With vast amounts of data
now available, companies in almost every industry are
focused on exploiting data for competitive advantage.
… We highlight data science as the connective tissue
between data-processing technologies (including those
for ‘big data’) and data-driven decision making. … A
large portion of what has traditionally been studied
 48 D. M. STEINBERG

within the field of statistics is fundamental to data sci-
ence” (p. 51).
The London Workshop on the Future of Statistics
devoted one chapter in its report to “big data.” The
report noted a spectrum of opinion about big data, with
some arguing that fundamental changes are needed to
equip statisticians to work effectively today and oth-
ers claiming that it is little more than a fad. In indus-
try, many major companies are recruiting data scien-
tists for their technical workforces. The term remains
somewhat amorphous, and although some knowledge
of statistics is generally required, other skill sets and
training are often equally or more important in screen-
ing applicants. The focus on data science in industry, to
date, has only increased the extent of industrial statis-
tics carried out by people with limited training in statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016
provide is well aligned with their needs. Such a caustic
tone in a professional editorial should set off warning
bells.
As I have already emphasized, the goal of this article
is to examine the state of research in industrial statis-
tics. I do so from a viewpoint that statistics is funda-
mental to converting data into knowledge, in line with
the statement from Provost and Fawcett. As such, the
widespread interest in big data and its potentially criti-
cal role in business and industry point to a ripe area for
statistical research of great relevance to industry.
Industrial statistics research in 2015
In this section I will summarize the current research

in industrial statistics. The summary is based pri-

tics.
marily on a survey of articles published in statistics
Jordan and Lin (2014) provided an insightful view
journals in 2014 with a pronounced interest in and
of big data and statistics. One of their primary points
a solid track record of relevance to industrial prac-
was that statisticians have often ignored the challenges
tice. These include Technometrics, Journal of Quality
of big data and have been largely absent from lead-
Technology, Quality and Reliability Engineering Interna-
ership roles. They write that statisticians have a great
tional, Applied Stochastic Models in Business and Indus-
deal to contribute to the utilization of big data. How-
try, Quality Technology and Quantitative Management,
ever, they add that this will require openness to address
and, of course, Quality Engineering. The above journals
large, unstructured problems. Like Keller-McNulty,
span a range from more theoretical (e.g., Technomet-
they point to statisticians being “on the outside look-
rics) to very applied and pedagogical (e.g., Quality Engi-
ing in” (Jordan and Lin 2014, p. 59). One of the telling
neering). I will also report on some engineering jour-
pieces of evidence noted in their article—when the U.S.
nals that encourage and publish statistical research of
National Science Foundation assembled a panel of 100
relevance to industry, like IIE Transactions and IEEE
experts in 2012 to formulate policy related to big data,
Transactions on Reliability, some general statistics jour-
not a single statistician was included.
nals that emphasize applications, and some venues with
The perception of statistics as less than relevant to
a strong industrial foothold.
big data is not limited to the industrial setting. Mal-
Many subject-matter journals in a variety of fields
ley and Moore (2013), the coeditors of the journal Bio-
also publish articles with interesting applications of
Data Mining, published an editorial entitled “The dis-
statistics in industry. A full survey of such periodi-
connect between classical biostatistics and the biolog-
cals is beyond the scope of this article. Moreover, most
ical data mining community.” They write that “statis-
of those articles illustrate known statistical methods
tics departments and journals still strongly emphasize
rather than developing new ones. How to highlight
a very narrow range of topics and methods and tech-
valuable articles that do breach new ground within the
niques, all driven by a tiny handful of results, many
statistics research community is a challenge that we
dating from the 1930s … the older zombie methods
should address.
persist in the statistics literature and teaching.” Malley
and Moore’s editorial is extreme. Many of their claims
are exaggerated and others, in my opinion, are wrong
(e.g., the implication that we should stop teaching What is being published?
regression and go straight to neural nets). But we also The research published in industrial statistics journals
need to take a closer look at the problems faced by continues to be dominated by the same topics that had
our colleagues (whether in biology or industry) and leading roles in the 1960s: DOE, SPC, and reliability.
to make sure that what we teach and the expertise we Statistical modeling is also a common theme, with the

nature of the model often simulated by specific applica-
tions. Within these primary areas, many new problems
are being studied—for example, designing experiments
for computer simulators, SPC with complex data, and
reliability for products with smart chips that provide a
vast new array of field data.
My literature review found that interesting applica-
tions are driving an increasing number of published
articles. However, few of these problems are from
industry, and not many industrial statisticians are pub-
lishing research. See Hoerl (2000) for earlier comments
on the need to enhance contact between industrial
practitioners and the research community. Table 1 gives
a summary of published articles in 2014 for a number of
journals. For each, I computed the number of industrial
authors and the fraction of articles that were stimulated
Downloaded by [University of Arizona] at 14:59 23 February 2016
by industrial problems. (Two technical notes: (1) I clas-
sified employees of national labs as “industrial” and (2)
I gave half credit to some articles that were “one step
removed” from a generating problem.)
The main stimulus to research is, not surprisingly,
other research. When we read articles, we often think
of an alternative, and perhaps better, way to attack the
problem, or we see that the ideas can be extended to
address other questions. Thus, “threads” of research
grow. Ideally (but not always), they retain roots to the
applied concerns that led to the original research or
extend their applicability to additional problems.
In the following subsections, I will give a brief sum-
mary of recent research in industrial statistics. In each
area, I will highlight several articles that I found partic-
ularly innovative and relevant to industry.
Statistical process control
Recent work on SPC and monitoring has been stim-
ulated by new and interesting types of data that are
collected. Whereas classic SPC charts track a single
characteristic (a mean value or a defect rate), today’s
quality data are often multivariate and have important
structural content. The monitoring of profiles is a good
example. Woodall et al. (2004) coined the term and dis-
cussed an application in which the sequence of data
from each object digitized its geometric shape. There
has been much additional research in recent years, with
extensions to general settings in which data describe a
relationship that characterizes an item.
Wang, Wang, and Tsung (2014) described a
method for monitoring a process that produces a
QUALITY ENGINEERING 49
two-dimensional surface. Their work was motivated
by the need to monitor the thickness of silicon wafers
in a semiconductor fab. The proposed control chart
combines ideas from spatial statistics to model the
thickness data with methods from profile charting.
Vander Wiel et al. (2014) developed an approach for
monitoring an electric power grid using data from pha-
sor measurement units, a technological breakthrough
for collecting online data from the grid. These data
can be used to detect problems in the grid, such as a
line going down or a large load being dropped. The
approach proposed here provides an estimate of the
current topology of a power grid and an algorithm for
the design problem of where to place the phasor mea-
surement units in a grid. The estimation procedure
requires an understanding of the data signals that char-
acterize problems that occur in the grid.
Design of experiments
The two dominant topics in DOE in recent years have
been screening experiments and the design of com-
puter experiments.
Jones and Nachtsheim (2011) proposed a clever
method for screening continuous input factors that
gives the design sensitivity to pure linear effects and
pure quadratic effects. The linear effects are not biased
by any second-order effects and a full second-order
model can be fitted in any subset of three factors. The
appeal of these designs is that they achieve these wider
screening properties with only a few more runs than
would be required by a two-level screening design,
which has no ability at all to detect pure quadratic
effects. Subsequent work by Xiao, Lin, and Bai (2012)
and Yang, Lin, and Liu (2014) showed that the designs
are related to the concept of conference matrices, which
provide a useful base for construction. See the article by
Jones in this volume for up-to-date ideas on “definitive
screening designs.”
Computer experiments involve special considera-
tions because they typically have no experimental
error, so that blocking, randomization, and replica-
tion become irrelevant. Most applications have rejected
standard factorial designs, preferring “space-filling”
designs that spread out the design points in a more-
or-less uniform fashion. Joseph et al. (2015) proposed
the concept of “minimum energy designs”; designs in
this interesting and useful class spread out experimen-
tal sites in the design region and are also able to adapt
 50 D. M. STEINBERG

Table . Industrial collaboration in and direct stimulation of articles published in .

Authors Articles
Journal Total Industrial % Total Industrial %

Annals of Applied Statistics      

Applied Stochastic Models in Business and Industry      
IEEE Transactions on Reliability     . 
IIE Transactions      
Journal of the American Statistical Association: Case Studies and      
Applications
Journal of Quality Technology      
Journal of Machine Learning Research      
Quality Engineering     . 
Quality and Reliability Engineering International     . 
Statistical Analysis and Data Mining      
Technometrics      

Authors from national laboratories are included here as “industrial”authors. The count of “industrial articles”gave a full count for an article directly stimulated by an
industrial problem and a half count for a problem closely, but indirectly, stimulated by an industrial problem. The summary for Applied Stochastic Models in Business
and Industry does not include the special issue for George Box.The summary for IIE Transactions does not include special issues on scheduling and logistics, design
and manufacturing, public health, and health care surveillance; those issues had articles with minimal statistical content.
Downloaded by [University of Arizona] at 14:59 23 February 2016

to settings where some subregions prove more interest-
ing than others. See the article by Joseph in this volume
for more details.
Of special note is the article by Albrecht et al. (2013),
which grew out of an industrial application and led the
authors to new and fertile ground. The article proposed
the use of dimensional analysis in the design of experi-
ments. The authors showed how exploiting dimension-
less quantities could lead to identification of the funda-
mental factors driving a process. This led to analyses
with a wider range of generalizability. Moreover, as the
fundamental factors are composites of the original fac-
tors, the dimensionality of the experimental space can
be reduced, with gains in cost and efficiency of exper-
imentation. For some months preceding the writing of
this article, Albrecht et al. (2013) topped the “down-
load” list from Technometrics. It led to a useful sequel
by Shen et al. (2014) illustrating the ideas on additional
examples.
Reliability
A number of themes have been at the forefront of recent
research in reliability. These include life tests, degrada-
tion models, use of field data, and Bayesian methods.
The use of field data, and in particular the availabil-
ity of field data, was the stimulus for work by Hong and
Meeker (2010). The goal they addressed was to fore-
cast eventual warranty costs for a product. An impor-
tant predictor of warranty costs is actual product use,
but traditionally this was known only for units that had
failed and been returned for repair. Modern sensor and
smart chip technology now provide use levels for many
units that were shipped to customers and never had any
failures. Hong and Meeker showed how to exploit those
data to obtain substantially more accurate estimates of
warranty costs. See Meeker and Hong (2014) for a sum-
mary of other applications where new data collection
methods have paved the way to improved reliability
analysis.
Application of Bayesian methods is effectively
illustrated by Alzbutas and Iešmantas (2014). Their
research concerned how to incorporate the age and
degradation status of components and systems in reli-
ability analysis; it was motivated by the study of elec-
trical instrumentation and control systems in nuclear
plants. They discussed aspects of model building and
verification and the use of Bayesian model aver-
aging for making predictions. The method is also
able to handle high-reliability components with few
failures.
Zhu, Yashchin, and Hosking (2014) described inter-
esting methodology that they developed to assess the
reliability of a municipal water system. Data on equip-
ment failures were available but only for a particu-
lar time window; failures prior to that time simply
were not recorded. The resulting “window-censored”
data require special treatment for statistical analysis
and reliability predictions. In this article, the authors
develop a useful model, apply it to reliability estima-
tion, and study its properties.
Modeling
“Modeling” refers to quite a diverse set of articles
where the primary innovation is how to represent a
particular type of data. General areas covered include

Gaussian processes, dimensional analysis, computer
experiments, spatiotemporal data fusion, unusual lon-
gitudinal profiles, screening in drug discovery, online
virtual market data, variable selection, uncertainty
quantification, and count data. Many more specific top-
ics were studied in 2014: binary yield data, nanoparticle
growth patterns and particle clustering, infrared trans-
missivity data, transportation matrices, exposure to
health threats, traffic volume from mobile phone mon-
itoring, monitoring a nuclear accident, retail demand,
price of a good, and return on assets. Almost all of this
research is driven, directly or indirectly, by applications
that required the development of new tools.
Motivated by a problem in textural analysis of fab-
ric samples, Taylor, Eckley, and Nunes (2014) studied
a model for two-dimensional lattice processes based
Downloaded by [University of Arizona] at 14:59 23 February 2016
on a locally stationary wavelet process. Texture is both
location-dependent and multiscale, making wavelets
a natural modeling choice. Of particular concern was
whether a stationary model would be suitable for the
application and a main contribution of the article is a
test of stationarity.
Chuang and Oliva (2014) developed a model for
demand in a retail chain. Estimation of retail demand
is critical to decisions about procuring, shipping, shelv-
ing, and inventory control. Their research was moti-
vated by fieldwork to improve sales performance in a
U.S. retail chain. A key concern was to estimate the
probability that demand would outstrip inventory. The
authors noted that prior work modeled demand via the
Poisson distribution. Their data showed much greater
dispersion, which led them to investigate alternative
models for count data.
A major goal in semiconductor fabrication is to
improve yield—the fraction of manufactured chips that
function successfully. Hwang and Lee (2014) addressed
the problem of how to set specification limits for
critical chip dimensions for high-yield production.
They explained why naïve modeling of yield versus
critical dimensions could give incorrect answers and
proposed a novel method in which yield probabil-
ities also involve hidden (or latent) variables. The
approach provides accurate target and specification
limits.
Prediction and classification
Traditionally the industrial statistics literature has not
featured many articles on prediction or classification.
QUALITY ENGINEERING 51
Prediction sometimes was addressed in the context
of articles on regression (e.g., Mallows 1973) or time
series (e.g., Box and Jenkins 1968) but was not a fea-
tured topic; classification was largely ignored. (For
example, in the first 10 volumes of Technometrics, there
were only three articles on classification.) In recent
years there has been increased interest in these topics.
Prediction has been a major target for analytics and
classification has been at the forefront of research in
data mining and machine learning.
Recommender systems are algorithms that use data
to help match individuals to products, services, or other
people. As an example, when I order a book from Ama-
zon.com, the web site “recommends” other books that
might interest me. Shi et al. (2014) address the prob-
lem of constructing a data-based recommender sys-
tem that requires fusing data from a variety of different
sources and coping with a high fraction of missing data.
Their work was motivated by the problem of construct-
ing social network graphs from a large demographic
database at AT&T in which there is a high fraction of
missing data. The database is distributed and leads to
90 different network graphs. The algorithm proposed
in the article was able to fuse the information from the
separate graphs and to effectively “fill in” the missing
data and predict which individuals are connected.
Open markets have made the generation and sale
of electricity a challenging business enterprise. Short-
term forecasts of electricity demand are an important
input when producers make production and pricing
decisions. Carpio, Juan, and López (2014) attacked the
forecasting problem with a multivariate exponential
smoothing scheme that exploits dimension reduction.
They illustrated the usefulness of the model by com-
paring daily forecasts to actual demand in several day-
ahead markets.
Industrial research challenges
What are the major challenges arising in industry
today that require statistical research? The landscape is
varied and so are the issues that we face. Some are
technical and others are organizational. I cannot claim
any special expertise in mapping out the challenges
faced by industry. So I polled a number of industrial
colleagues, representing diverse areas including micro-
processors, consumer products, IT, pharma, aerospace,
and defense. Most of what follows reflects the feedback
I received from them.
 52 D. M. STEINBERG
Table . Technical challenges for modern industrial statistics.
Modeling: General Issues Prediction and predictive Model selection
analytics
Modeling: Big data and Analysis of large data sets Analysis of streaming data Making the most of small
small data
Modeling: Specific issues Computer models— Machine learning and
tuning and validation data mining
DOE Computer experiments Fast, high-volume
experiments
Data science/IT Assessing data quality Quantifying uncertainty
SPC Profile monitoring Image monitoring
Reliability Multilayer, multiscale Life tests with multiple
surveillance accelerating factors
Software Bayesian methods
I begin with several important nontechnical issues
that my colleagues cited as barriers to widespread appli-
cation of statistical methods. They included issues of
teaching and training for nonstatisticians, better soft-
Downloaded by [University of Arizona] at 14:59 23 February 2016
ware (and specifically software for Bayesian analysis),
more involvement in decision making, avoiding deci-
sions based on limited data, “upgrading” statisticians
from consultant to collaborator, statistical input to
industry manuals, and involvement in large, complex
problems.
Some of the concerns above have close ties to inno-
vation, an essential component in industrial success
but an area where statistics has not been a major part
of the dialog. Søren Bisgaard forcefully argued in his
2005 Youden address (Bisgaard 2012) that quality man-
agement and methods should be regarded as provid-
ing the framework and tools for systematic innovation.
Box and Woodall (2012) provided further commen-
tary on the interplay of statistics, quality, and innova-
tion, emphasizing lateral thinking (de Bono 1970) and
experimentation. Many constructive ideas were raised
in a panel discussion in Quality Engineering (Jensen
2012). The Harvard Business Review highlighted inno-
vation in December 2014; the article by Thomke and
Manzi (2014) in that issue is completely devoted to the
use of statistical experiments for assessment of busi-
ness innovations. More research by statisticians on how
our unique understanding and skills support innova-
tion would be helpful to move this area forward.
Now let me move on to technical challenges. Table 2
presents a list of topics where my colleagues would like
to see more research. I have grouped them into six gen-
eral categories. Traditional areas like DOE, SPC, and
reliability continue to have high priority. Modeling is
also a major concern and relates to a wide variety of
problems; I have divided the modeling topics into three
groups.
Model averaging Fully Bayesian Modeling “real” data
analysis
samples
Image analysis Clustering Data with complex error
structure
Bayesian methods Choice experiments Experiments with a
nonconstant
background
Anomaly detection Text mining
The list of topics is largely self-explanatory, but a few
comments are in order to give perspective. The sub-
stantial interest in predictive analytics in recent years is
at the top of the list for general issues in modeling and
is also reflected in additional issues like model selection
and model averaging.
The call for modeling “real data” reflects a con-
cern that too many research papers continue to rely
on assumptions that just do not characterize the data
encountered in industry. This view was expressed
starkly by Bert Gunter in the 2008 panel discussion
in Technometrics on the future of industrial statistics
(Steinberg 2008, p. 125):
I fear that Technometrics has evolved from primarily mak-
ing connections to the real, hard, and complex questions
of scientific practice to primarily producing artificial for-
mulations of those questions suitable for compact “solu-
tion” by mathematical characterization. … To under-
stand what is useful and not merely wrong in industrial
statistical practice, we need to pay much more attention
to the messy details that make up reality.
I don’t share Gunter’s opinion that most of our
published research (whether in Technometrics or other
journals) has become completely cut off from real prob-
lems. But I do share the concern that many of the most
challenging and exciting problems arising today are not
getting space in our journals and that we need better
theory to guide us in attacking such problems.
Applications in IT companies involve many simple
comparative experiments. Known as A/B tests, these
experiments compare alternative landing pages at
Internet sites (see Scott [2015] and the ensuing dis-
cussion by Agarwal [2015] for details). What is special
about these experiments is that they involve very large
sample sizes, they are carried out sequentially, it is diffi-
cult to know what characteristics might need blocking
(hence the “nonconstant background”), and it is

desirable to let the data direct new subjects to more
successful landing pages during the course of the
experiment.
The reference to “multi-layer, multi-scale surveil-
lance” in reliability relates to the availability of comple-
mentary data sources that can be used for assessment.
The article by Hong and Meeker (2010) described ear-
lier is a good example of how to use such data.
Statistical engineering
Hoerl and Snee (2010a, 2010b, 2012) adopted the term
“statistical engineering” (SE) to describe “the study of
how to best utilize statistical concepts, methods and
Downloaded by [University of Arizona] at 14:59 23 February 2016
tools and integrate them with information technol-
ogy and other relevant sciences to generate improved
results” (Hoerl and Snee 2010b, p. 123). They argued
that statistical engineering is essential to solving large,
challenging, unstructured problems and implored the
statistical community to move into leadership roles
where they will have an impact on such problems. Their
proposal touched off much discussion; see the special
issue of Quality Engineering devoted to statistical engi-
neering in 2012.
What is the interface between statistical engineering
and statistical research? The panel discussion in Qual-
ity Engineering (Anderson-Cook and Lu 2012) offers
some ideas. Hoerl and Snee (p. 140) pointed to aca-
demic research as essential to give SE a needed theoret-
ical framework. Vining (p. 142) wrote about the need to
develop a theory for the use of statistical thinking and
methods to solve complex, unstructured problems. The
need for external funding of SE research was stressed by
Montgomery (p. 142) and Vining (p. 141) and the need
for publication venues by DeHart and Van Mullekom
(p. 143, 144).
The paper by Kenett (2015) on a life cycle view of
statistics and its ensuing discussion are useful contri-
butions to the theory of applied statistics and are rele-
vant to the development of a theory of SE. DiBenedetto,
Hoerl, and Snee (2014) give a constructive discussion
of problem solving in diverse areas. But they are out-
liers from the larger research cloud, which has yet to
embrace the challenge set down by Hoerl and Snee.
Establishing a coherent theory is still more a vision than
an achievement.
My colleagues did not mention SE per se as a
major topic on the research agenda, but some of their
QUALITY ENGINEERING 53
comments (e.g., involvement in large, complex prob-
lems) clearly intersect with SE.
Statistics and the service sector
I stressed early in this article the importance of includ-
ing the service sector as a component of industrial
statistics. In my survey of recent research and the “wish
list” in Table 2, the service sector has a very limited role.
Rodriguez (2000) pointed to the great opportunities for
SPC in working with service sector data. Yet research
in industrial statistics remains focused on manufac-
turing, despite the dramatic changes that have taken
place in the composition of the Western economies.
In his Youden address, Bisgaard (2012) took an espe-
cially perceptive look at this trend and how it should
affect our research agenda. He concluded that “the field
of quality and quality technology need to adapt to a
rapidly changing economic environment. We are in
a global knowledge economy dominated by services
and high knowledge content products and processes”
(pp. 35–36).
Research specifically stimulated by and directed to
service industries is breaking in to industrial statistics
journals but at a slow pace. It is far from a sufficient
response to the sense of urgency described by Bisgaard
or to the challenging statistical problems in the service
sector. The lack of research in these areas led Doug
Montgomery, the editor of Quality and Reliability Engi-
neering International, to begin 2014 with an appeal to
the quality community to be active participants in the
knowledge economy and to submit manuscripts based
on challenges arising there.
Industry–Academia collaboration
One of the most successful avenues from indus-
trial problems to published research is collaboration
between industry and academia. For many involved,
such collaboration is a win–win arrangement: those in
industry can benefit from the latest research and can
often enjoy more complete solutions than they would
have time to develop and justify on their own; aca-
demics are tuned in to interesting and relevant prob-
lems and have an opportunity to see their research
make a direct impact. In the Technometrics discus-
sion, Bill Meeker encouraged students to spend time
as industrial interns; his own career is a good example,
with summers at Bell Labs providing many problems
for his research.
 54 D. M. STEINBERG
Most of the industrial colleagues who wrote to me
stated that they have colleagues in academia who are
very happy to collaborate with them and work on prob-
lems that arise in industry. A few departments have
industrial affiliate programs (e.g., Stanford and Virginia
Tech).
My summary of the 2014 literature suggests that the
extent of university–industry collaboration is not high.
Bill Meeker went on to write that he has had only lim-
ited success in getting his industrial partners to fund
research by students. That has also been my own expe-
rience. Sometimes the barriers to industry–academia
collaboration are legal: companies that insist on not
divulging proprietary information, even when it is fun-
damental to getting out good research on the prob-
lems studied, and universities who regard themselves
Downloaded by [University of Arizona] at 14:59 23 February 2016
as full partners in any IP that results from the collabo-
ration. See Siegel, Waldman, and Link (2003) and Siegel
et al. (2004) for a critical view of the role of univer-
sity technology transfer offices; they found that these
technology transfer offices often discouraged collabo-
ration. The recent review article by Bozeman, Fay, and
Slade (2013) does not point to any dramatic improve-
ment since the works by Siegel et al.
Is industrial statistics on the statistics agenda?
The London Workshop on the Future of Statistics
convened in November 2013. The workshop brought
together about 100 leaders in the profession to discuss
the challenges in the years ahead. The London Work-
shop Report, one of the fruits of that meeting, is aimed
at a wide reader base, not just statisticians. It begins
with seven short “success stories” to help set the tone
for what statistics can contribute. These include the
development and widespread use of randomized clin-
ical trials, the application of Bayesian statistics, and
the successful use of statistical arguments in the court-
room. Oddly, the quality revolution and its statistical
underpinnings—to me one of the most influential suc-
cess stories we can relate—are not mentioned at all. The
final item is devoted to analytics—in sports and poli-
tics. The strong business and industrial roots of analyt-
ics are ignored.
The absence of industrial statistics from the Lon-
don Workshop is not an isolated event. The statisti-
cal research community has become increasingly dis-
connected from industry. At major conferences like
the Joint Statistical Meetings, industrial topics com-
prise only a small fraction of the sessions (again exclud-
ing clinical trials in pharmaceuticals). General interest
journals like Annals of Applied Statistics or the “Case
Studies and Applications” section of the Journal of the
American Statistical Association have few articles stim-
ulated by industrial problems. Major statistics depart-
ments are making an effort to establish a foothold in
machine learning and data mining, which are highly
relevant to statistical problems in industry associated
with big data, information technology, and the service
sector. However, they rarely teach courses with a direct
focus on industrial topics or in the classical areas of
industrial statistics. Most graduates will complete their
M.Sc. or Ph.D. studies with no exposure to the statis-
tical aspects of quality improvement or to many of the
topics studied in a Six Sigma Black Belt course. (In fact,
I suspect that many will graduate never having heard
about Six Sigma.) Even in departments that have been
leaders in the field, interest in industrial careers is wan-
ing. For example, Bill Woodall (private communica-
tion) related to me that there is less interest among stu-
dents at Virginia Tech in pursuing industrial careers
(with the exception of the financial and pharmaceutical
branches).
Some bright spots
In my review of industrial statistics journals, I found
that most of the articles were in one of the established
categories, such as DOE or reliability. There were also
articles that broke fresh ground and introduced inter-
esting new problems. Invariably those articles showed
direct contact with a stimulating applied problem. I
want to highlight a few of those articles.
A second source for interesting problems and new
territory is special issues that have appeared in some of
the journals. A good example is the section on anomaly
detection published in Technometrics in 2009. One of
the 2015 issues will have a special section with arti-
cles on system informatics. The discussions in Qual-
ity Engineering on statistical engineering (Anderson-
Cook and Lu 2012) and innovation (Jenson 2012) were
also effective in publicizing new topics with impor-
tant implications for industrial statistics. The January
2015 issue of Applied Stochastic Models in Business and
Industry compiled articles on enhancing the impact
of statistics. Effective use of special issues should cer-
tainly be encouraged. Journal space is limited and

editors cannot afford to allot too much to special
issues, so it is important to choose special issue topics
carefully.
Another stimulating source of new topics is the
annual knowledge discovery and data mining (KDD)
conference, in particular the “industry and govern-
ment” track. (Full information for 2014 is available
at http://www.kdd.org/kdd2014/.) Let me mention just
a few of the interesting titles from the 2014 confer-
ence: “Beating the news” with EMBERS: Forecasting
civil unrest using open source indicators; modeling
mass protest adoption in social network communi-
ties using geometric Brownian motion; activity ranking
in LinkedIn feed; corporate residence fraud detection.
There are many more. It is clear that the KDD confer-
ence is attracting many of the interesting IT applica-
Downloaded by [University of Arizona] at 14:59 23 February 2016
tions (and industrial writers on them) that the “main-
stream” industrial statistics journals would like to pub-
lish.
I will now proceed to a few of the sentinel articles
that I read.
Cyber security is a major concern in our
information-based global society. One of the methods
of cyber attack is “masquearading”—when a hacker
succeeds in entering a network using a legitimate
login–password combination. An important security
goal is to identify such an attack from the system
commands (mouse clicks and keyboard entries) of the
phony user. Motivated by an application, Skudlarek
and Yamomoto (2014) proposed an analysis method
for identifying anomalous user behavior. They demon-
strated a number of advantages for their algorithm.
An especially interesting aspect of this work is the
nonstandard nature of the data (“variety” in the big
data parlance) used to detect anomalies.
Afful-Dadzie and Allen (2014) also wrote a pio-
neering article on application of statistical methods to
promote cyber security. Motivated by a live applica-
tion, their work exploits Markov decision processes to
study and improve maintenance policies for a computer
system that may be subjected to a cyber attack. The
method uses online data that are collected by the sys-
tem and makes a status evaluation that could result in
a clear decision as to the status of the system or a rec-
ommendation to collect additional data.
Data quality can be a problem in many applications.
Yet there are not many systematic tools for ensuring
data quality. Motivated by an integrated case study of
an aircraft maintenance database, Jones-Farmer, Ezell,
QUALITY ENGINEERING 55
and Hazen (2014) showed how to use ideas from statis-
tical process control to monitor and improve the data
quality. This article extends well-established methods
into a new and important arena.
Discussion
It is time to take stock and draw conclusions: what is
the current status of research in industrial statistics?
Although there are many bright spots, there is much
room for improvement and many missed opportuni-
ties.
Among the bright spots: We are producing research
at an ever increasing rate. There are many publications
each year that are relevant and useful for advancing sta-
tistical methods and practice in industry. My indus-
trial colleagues wrote that the literature is addressing
many of the topics on their priority list; most of them
observed that they are finding useful articles for their
needs. Many interesting statistical modeling problems
are being published.
A good example is the research on computer exper-
iments. Scientists and engineers are making increas-
ing use of computer models as an experimental plat-
form. For many computer experiments, the results are
deterministic and therefore might be regarded as out-
side the realm of statistics. The realization that these
experiments do have important statistical aspects but
also require different methods due to their unique fea-
tures has spawned a great deal of useful research in the
last 25 years. There is still much to learn. Several of my
industrial colleagues reported on the added value that
research has had for them in applications.
Among the warning signs: Most of the research con-
tinues to be in SPC, DOE, and reliability. The industrial
statistics journals have established a deserved reputa-
tion for promoting these areas, so it is not surprising
that they continue to be major publication domains.
Nonetheless, the continued emphasis on traditional
topics may also be a sign of stagnation. The journals are
not attracting enough articles on interesting and excit-
ing problems that fall outside these traditional areas.
Many challenging statistical questions are arising in
the service sector and in IT companies; despite over-
tures, the industrial statistics journals are receiving very
few submissions on these issues. Few industrial statisti-
cians are contributing coauthors. Articles on methods
abound but articles devoted to principles of practice are
unusual.
 56 D. M. STEINBERG

Let me return to the four questions that were posed
early in this article.
What is driving statistical research today?
Based on my summary of journal articles from 2014,
as well as my past experience as an editor, the primary
motivating force is the academic tenure and promo-
tion system. A large fraction of the authors in our jour-
nals are in academics and they are publishing because
they are being paid (and promoted) to publish. This
situation is neither new nor inherently wrong. How-
ever, there are some pitfalls. One is the lack of publish-
ing activity by statisticians working in industry. They
do not have the same incentive to publish and some-
times are discouraged from publication due to con-
Downloaded by [University of Arizona] at 14:59 23 February 2016
doing so in alternative venues; the strong industrial
statistics journals have published only a handful of
these articles. Research by academics is far more often a
response to previously published research (incremental
innovation) than work addressing new and challeng-
ing problems (breakthrough innovation). The evidence
from our publishing record is that most academics are
not sufficiently in touch with real industrial problems.
Are statistical challenges in industry being better
addressed by other research communities?
I noted above that the traditional industrial statis-
tics journals are not attracting many articles on new
and challenging industrial problems. An immediate

reaction is that such articles are simply being pub-

cerns about sharing information or intellectual prop-
lished elsewhere. Moreover, it is possible that lack of
erty that has competitive value. A second is that most of
receptiveness by the existing journals led authors to
the academics (based on my survey) are not conducting
approach or even to establish other journals where they
research that was stimulated by real problems. A third
found more willing acceptance of their work. A classic
is that it is often easier to publish “standard” research,
example is machine learning—statistics journals were
so that the incentive system may discourage research
slow to publish papers in the area and it quickly became
on more innovative, but risky, topics.
the province of the computer science research commu-
The published statistical research is increasingly
nity, which established its own journals in the area. In
driven by the interface with challenging scientific and
the last 10 years there has been more research by statis-
technological problems. This is particularly evident in
ticians, but the field remains largely identified with
journals like the Annals of Applied Statistics, which has
computer science.
grown into a major publishing venue in less than 10
I began my review of 2014 journals convinced that
years. Most of the articles there grew out of actual prob-
I would find many interesting papers stimulated by
lems, with methods for better statistics on those prob-
industrial problems in other journals. That view proved
lems at the forefront of the article.
mistaken. None of the journals I reviewed stood out as
a venue for addressing industrial challenges. My review
Are industrial challenges affecting the statistical was, of course, limited in coverage. I will be interested
research agenda? to hear of good journals that I missed. The one excep-
tion, noted earlier, was the industry and government
The answer is largely “no.” Other areas appear much
section of the KDD conference. Much of that research is
more dominant in current statistical research. One
published in the conference proceedings but often not
of the striking features from my review of Annals of
in research journals.
Applied Statistics is the very small number of articles
with roots in industrial problems. The fact that indus-
trial statistics was almost totally ignored in the London
Is the statistics research community missing
Workshop reflects its status in the professional hierar-
important opportunities for impact in industry?
chy. What is remarkable is that industry is leading the
way in big data, streaming data, and new types of data The answer is an emphatic “yes.”
and should be a hotbed for statistical research. Thus far, Industry is blazing the big data trail. Analytics is
though, this has not paid many dividends in innovative a major topic on the statistical agenda in industry
statistical research. and business. Where is the accompanying statistical
As noted above, few industrial statisticians are pub- research? Statistical work in the service economy is still
lishing research. Those who are publishing are often only a thin stream in the research literature.

My industrial colleagues complained about the need
for more research on statistical practice and the chal-
lenges posed by realistic data. Research on statistical
practice will often be conceptual and messy; it does
not fit well into the precise mathematical templates
that govern most of our research and that promotion
committees find easy to evaluate. The limited amount
of foundational research on statistical engineering is a
good case in point.
In today’s world of big data and data science, a cru-
cial issue is data quality. No one should have more to
say about data quality than the statistics community.
Yet there is little published work in the area. The article
cited earlier by Jones-Farmer, Ezell, and Hazen (2014)
is one of the exceptions that highlight the rule. Snee and
Hoerl (2012) give constructive suggestions in a Statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016
tics Roundtable article. Kenett and Shmueli (2014) are
among the few who have begun the challenging task
of building a conceptual framework for data quality; in
fact, they take a more encompassing view, assessing the
quality of the information. See also the discussion to
their article. Much more work is needed in this area.
In summary
There is much good research being conducted and pub-
lished in industrial statistics. Most of that work pro-
vides incremental advances in areas that have long
been on the research front burner. Stu Hunter wrote
about the exciting environment of industrial statistics
in the 1950s. He and his colleagues then were learning,
implementing, and advancing completely new ideas:
response surface methods, cumulative sum charts, new
methods for analyzing reliability data, among others.
When I completed my studies in the early 1980s, the
quality engineering ideas of Genichi Taguchi had a sim-
ilar invigorating effect, as we attempted to understand
(and improve) the unique and innovative ways that
Taguchi wed engineering, quality, and statistics. Today
there are fascinating statistical challenges in industry.
Products become increasingly complex and data for
monitoring, testing, and improvement become increas-
ingly rich and varied. There is a need for novel think-
ing and methods to exploit this environment. We have
not been successful in getting those challenges into our
research journals and high on our research agenda. As
a community, we need to catalyze the process to keep
our journals and our research focused on cutting-edge
QUALITY ENGINEERING 57
problems and to enhance our participation in break-
through innovation.
About the author
David M. Steinberg is Professor of Statistics at Tel Aviv
University. He has a Ph.D. in statistics from the University of
Wisconsin–Madison. He was awarded the 2013 Box Medal by
the European Network for Business and Industrial Statistics. He
was the editor of the leading journal Technometrics from 2008
to 2010. Professor Steinberg’s primary research area is experi-
mental design, with an emphasis on the design and analysis of
industrial experiments and on experiments run on computer
simulators. He is involved in numerous applications in a wide
variety of fields. Professor Steinberg was president of the Israel
Statistical Association from 2013 to 2015 and is a member of
ASQ.
Acknowledgments
I want to thank the organizers of the Stu Hunter Research Con-
ference for inviting me to take on the challenging task of writing
about the current state of research in industrial statistics.
Many colleagues and friends contributed input and ideas that
helped me write this article: William Brenneman, Roger Hoerl,
Brad Jones, Ron Kenett, Bill Myers, Jose Ramirez, Phil Scinto,
Steve Scott, Winson Taam, Amit Teller, Xavier Tort-Martorell,
Theresa Utlaut and Bob Wilkinson. I am indebted to them for
sharing their thoughts and experiences with me. This work
would never have come to fruition without their help.
References
Afful-Dadzie, A., and T. T. Allen. 2014. Data-driven cyber-
vulnerability maintenance policies. Journal of Quality Tech-
nology 46:234–50.
Agarwal, D. 2015. Discussion of “Multi-armed bandit exper-
iments in the online service economy.” Applied Stochastic
Models in Business and Industry 31:46–47.
Albrecht, M. C., C. J. Nachtsheim, T. A. Albrecht, and R. D.
Cook. 2013. Experimental design for engineering dimen-
sional analysis. Technometrics 55:257–70.
Alzbutas, R., and T. Iešmantas. 2014. Application of Bayesian
methods for age-dependent reliability analysis. Quality and
Reliability Engineering International 30:121–32.
Anderson-Cook, C. M., and L. Lu. (eds.). 2012. Statistical
engineering—Forming the foundations. Quality Engineer-
ing 24:110–32.
Bisgaard, S. 2012. The future of quality technology: From a man-
ufacturing to a knowledge economy and from defects to
innovations. Quality Engineering 24:30–36.
Box, G. E. P. 1984. The importance of practice in the develop-
ment of statistics. Technometrics 26:1–8.
Box, G. E. P., and G. M. Jenkins. 1968. Some recent advances in
forecasting and control, I. Applied Statistics 17:91–109.
 58 D. M. STEINBERG
Box, G. E. P., and W. H. Woodall. 2012. Innovation, quality engi-
neering and statistics. Quality Engineering 24:20–29.
Bozeman, B., D. Fay, and C. P. Slade. 2013. Research collabo-
ration in universities and academic entrepreneurship: The-
state-of-the-art. Journal of Technology Transfer 38:1–67.
Carpio, J., J. Juan, and D. López. 2014. Multivariate exponen-
tial smoothing and dynamic factor model applied to hourly
electricity price analysis. Technometrics 56:494–503.
Chuang, H. H.-C., and R. Oliva. 2014. Estimating retail
demand with Poisson mixtures and out-of-sample likeli-
hood. Applied Stochastic Models in Business and Industry
30:455–63.
de Bono, E. 1970. Lateral thinking: Creativity step by step. New
York, NY: Harper & Row.
DiBenedetto, A., R. W. Hoerl, and R. D. Snee. 2014. Solving jig-
saw puzzles: Addressing large, complex and unstructured
problems. Quality Progress 50–53.
Hoerl, R. W. 2000. Discussion. Journal of Quality Technology
Downloaded by [University of Arizona] at 14:59 23 February 2016
32:351–55.
Hoerl, R. W., and R. D. Snee. 2010a. Closing the gap: Statistical
engineering links statistical thinking, methods, tools. Qual-
ity Progress 43 (5):52–53.
Hoerl, R. W., and R. D. Snee. 2010b. Further explanation: Clar-
ifying points about statistical engineering. Quality Progress
43 (12):68–72.
Hoerl, R. W., and R. D. Snee. 2012. Leadership—Essential for
developing the discipline of statistical engineering. Quality
Engineering 24:162–70.
Hong, Y., and W. Q. Meeker. 2010. Field-failure and warranty
prediction based on auxiliary use-rate information. Techno-
metrics 52:148–59.
Hunter, J. S. 1983. The birth of a journal. Technometrics 25:3–7.
Hwang, J. Y., and H. C. Lee. 2014. Parametric yield modeling
using hidden variable logistic regression. Journal of Quality
Technology 46:323–39.
Jensen, W. (ed.). 2012. Statistics to facilitate innovation: A panel
discussion. Quality Engineering 24:2–19.
Jones, B., and C. J. Nachtsheim. 2011. A class of three-level
designs for definitive screening in the presence of second-
order effects. Journal of Quality Technology 43:1–15.
Jones-Farmer, L. A., J. D. Ezell, and B. T. Hazen. 2014. Applying
control chart methods to enhance data quality. Technomet-
rics 56:29–41.
Jordan, J. M., and D. K. J. Lin. 2014. Statistics for big data: Are
statisticians ready for big data? International Chinese Statis-
tical Association Bulletin 26:59–66.
Joseph, V. R., T. Dasgupta, R. Tuo, and C. F. J. Wu. 2015. Sequen-
tial exploration of complex surfaces using minimum energy
designs. Technometrics 57:64–74.
Kenett, R. S. 2015. Statistics: A life cycle view. Quality Engineer-
ing 27:111–21.
Kenett, R. S., and G. Shmueli. 2014. On information quality.
Journal of the Royal Statistical Society, Series A 177:3–27.
Kenett, R. S. and Shmueli, G. (eds.). (2015). A special issue on:
Actual impact and future perspectives on stochastic mod-
elling in business and industry. Applied Stochastic Models in
Business and Industry 31:1–2.
The London Workshop Report. Statistics and Science: A report of
the London workshop on the future of the statistical sciences.
http://www.worldofstatistics.org/wos/pdfs/Statistics&Scie
nce-TheLondonWorkshopReport.pdf
Malley, J. D., and J. H. Moore. 2013. The disconnect between
classical biostatistics and the biological data mining com-
munity. BioData Mining 6.
Mallows, C. L. 1973. Some comments on Cp . Technometrics
15:661–75.
Manyika, J., M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Rox-
bough, and A. H. Byers. 2011. Big data: The next frontier for
innovation, competition and productivity. McKinsey Global
Institute.
Meeker, W. Q., and Y. Hong. 2014. Reliability meets big data:
Opportunities and challenges. Quality Engineering 26:102–
16.
Montgomery, D. C. 2014. Lean six sigma and promoting innova-
tion. Quality and Reliability Engineering International, 30:1.
Provost, F., and T. Fawcett. 2013. Data science and its relation to
big data and data-driven decision making. Big Data 1:51–59.
Rodriguez, R. N. 2000. Discussion. Journal of Quality Technol-
ogy 32:364–65.
Scott, S. 2015. Multi-armed bandit experiments in the online
service economy. Applied Stochastic Models in Business and
Industry 31:37–45.
Shen, W., T. Davis, D. K. J. Lin, and C. J. Nachtsheim. 2014.
Dimensional analysis and its applications in statistics. Jour-
nal of Quality Technology 46:185–98.
Shi, X., J.-F. Paiement, D. Grangier, and P. S. Yu. 2014. GBC:
Gradient boosting consensus model for heterogeneous data.
Statistical Analysis and Data Mining 7:161–74.
Siegel, D. S., D. Waldman, L. E. Atwater, and A. N. Link. 2004.
Toward a model of the effective transfer of scientific knowl-
edge from academicians to practitioners: Qualitative evi-
dence from the commercialization of university technolo-
gies. Journal of Engineering and Technology Management
21:115–42.
Siegel, D. S., D. Waldman, and A. N. Link. 2003. Assessing the
impact of organizational practices on the relative productiv-
ity of university technology transfer offices: An exploratory
study. Research Policy 32:27–48.
Skudlarek, S. J., and H. Yamamoto. 2014. Unsupervised anomaly
detection within non-numerical sequence data by average
index difference, with application to masquerade detection.
Applied Stochastic Models in Business and Industry 30:632–
56.
Snee, R. D., and R. W. Hoerl. 2012. Inquiry on pedigree: Do you
know the quality and origin of your data? Quality Progress
65–68.
Steinberg, D. M. (ed.). 2008. The future of industrial statistics:
A panel discussion. Technometrics 50:103–27.
Taylor, S. L., I. A. Eckley, and M. A. Nunes. 2014. A test of sta-
tionarity for textured images. Technometrics 56:291–301.
Thomke, S., and J. Manzi. 2014. The discipline of business
experimentation. Harvard Business Review 92:71–79.
Vander Wiel, S., R. Bent, E. Casleton, and E. Lawrence. 2014.
Identification of topology changes in power grids using

phasor measurements. Applied Stochastic Models in Business
and Industry 30:740–52.
Wang, A., K. Wang, and F. Tsung. 2014. Statistical surface mon-
itoring by spatial-structure modeling. Journal of Quality
Technology 46:359–76.
Woodall, W. H., D. J. Spitzner, D. C. Montgomery, and S.
Gupta. 2004. Using control charts to monitor process and
product quality profiles. Journal of Quality Technology 36:
309–20.
Downloaded by [University of Arizona] at 14:59 23 February 2016
QUALITY ENGINEERING 59
Xiao, L., D. K. J. Lin, and F. Bai. 2012. Constructing defini-
tive screening designs using conference matrices. Journal of
Quality Technology 44:2–8.
Yang, J., D. K. J. Lin, and M.-Q. Liu. 2014. Construction of
minimal-point mixed-level screening designs using confer-
ence matrices. Journal of Quality Technology 46:251–64.
Zhu, Y., E. Yashchin, and J. R. M. Hosking. 2014. Parametric
estimation for window censored recurrence data. Techno-
metrics 56:55–66.