Analytics MAYJUNE 2013

H T T P : / / W W W. A N A LY T I C S - M A G A Z I N E .
O R G
DRIVING BETTER BUSINESS DECISIONS
MAY/JUNE 2013
BROUGHT TO YOU BY:
The
Business
Analytics
Revolution
Four killer competencies of
analytically mature corporations.
ALSO INSIDE:
Profit Center
INFORMS VP
Andrew Boyd on
winning the war
with analytics
Historians vs. Futurists

Who provides best info for
todays decision-makers?
Marketing Analytics
Leveraging behavioral,
attitudinal data for action.
Lost in Translation
Best practices for analyzing
multi-lingual text.
INS IDE STORY
Dutch project earns Edelman

According to the classic tale Hans
Brinker, or the Silver Skates, a Little Dutch
Boy once saved Holland by sticking his finger in a hole in a dike, thereby stemming the
flow of water and averting disaster.
A modern-day retelling of the tale took
place during the 2013 Franz Edelman
Award Competition, only this time it was
fact, not fiction. A consortium of Dutch organizations, headed by the Dutch Delta
Program Commissioner, used high-end
analytics such as operations research to
develop and implement a new method
for calculating the most efficient levels of
flood protection for the Netherlands.
By identifying which dikes to fortify
and to what extent, the massive project
considerably improved the system of
dikes that protect the Netherlands in an
era of rising seas while simultaneously
saving the nation eight billion Euros in
unnecessary expenditures.
The project won the prestigious Edelman Award, besting other outstanding
analytics work from finalists Baosteel,
Chevron, Dell, Kroger and McKesson.
Presented by the Institute for Operations
Research and the Management Sciences
(INFORMS), the Edelman Award is widely
considered the Super Bowl of high-end analytics. It honors the best application of O.R.
following a nearly yearlong competition that
2
A N A LY T I C S - M A G A Z I N E . O R G
begins with a nomination process, continues

with a vetting process and concludes at the
spring INFORMS Conference on Business
Analytics & Operations Research with a series
of finalists presentations and judges deliberations, all capped by the naming of the winner
at the Edelman Awards Gala.
When asked at the Oscar-like gala to
sum up the complexity of the Dutch dike
project and the sophistication of the analytical work involved, Edelman judge Graham
Rand quipped, Its a little bit more than putting your finger in the dike.
For more on the Edelman and other
awards presented in conjunction with the
INFORMS conference in San Antonio, Texas, in April, see page 64.
INFORMS, the worlds leading association for professionals in the fields of operations research, management science and
analytics, used the conference as a site to
administer its inaugural Certified Analytics
Professional (CAP) examination. Along
with helping analytics professionals boost
their careers and elevate themselves in
the job market, the CAP program is also
designed to help hiring managers identify
qualified analytical talent. For more on the
CAP program, see page 22.
PETER HORNER, EDITOR

peter.horner @ mail.informs.org
W W W. I N F O R M S . O R G
C O N T E N T S
MAY/JUNE 2 013
Brought to you by
FEATURES
24 THE BUSINESS ANALYTICS REVOLUTION
By Randy Bartlett and Girish Malik
Four killer competencies of analytically mature corporations and how
to achieve them.
36
36 CASE STUDY: MARKETING ANALYTICS

By Jeff Ahlquist and Kshira Saagar
Comprehending the complete customer: leveraging behavioral and
attitudinal data for actionable analytics.
52 HISTORIANS VS. FUTURISTS
By Gary Cokins
Which group provides more useful information for todays
decision-makers?
56
QUOTABLE
52
56 TEXT ANALYTICS: LOST IN TRANSLATION

By Christopher Broxe and Fiona McNeill
Part one of two-part series on best practices for analyzing
multi-lingual text.
Despite all the value-add that

business analytics brings to the table,
companies have hardly been able to
realize its true potential.
Found in The Business Analytics Revolution on p. 24
64
Brought to you by
REGISTER FOR A FREE SUBSCRIPTION:

http://analytics.informs.org
INFORMS BOARD OF DIRECTORS
76
DEPARTMENTS
2 Inside Story
8 Profit Center
12 Analyze This!
16 Forum
22 Career Builder
64 Analytics Honors
72 Five-Minute Analyst
76 Thinking Analytically
Analytics (ISSN 1938-1697) is published six times a year by
the Institute for Operations Research and the Management
Sciences (INFORMS). For a free subscription, register at
http://analytics.informs.org. Address other correspondence to
the editor, Peter Horner, peter.horner@mail.informs.org. The
opinions expressed in Analytics are those of the authors, and
do not necessarily reflect the opinions of INFORMS, its officers,
Lionheart Publishing Inc. or the editorial staff of Analytics.
Analytics copyright 2013 by the Institute for Operations
Research and the Management Sciences. All rights reserved.

President Anne G. Robinson, Verizon Wireless

President-Elect Stephen M. Robinson, University of
Wisconsin-Madison

Past President Terry Harrison, Penn State University

Secretary Brian Denton,
University of Michigan

Treasurer Nicholas G. Hall, Ohio State University

Vice President-Meetings William Bill Klimack, Chevron

Vice President-Publications Eric Johnson, Dartmouth College

Vice President
Sections and Societies Paul Messinger, University of Alberta

Vice President
Information Technology Bjarni Kristjansson, Maximal Software
Vice President-Practice Activities Jack Levis, UPS
Vice President-International Activities Jionghua Judy Jin, Univ. of Michigan

Vice President-Membership

and Professional Recognition Ozlem Ergun, Georgia Tech

Vice President-Education Joel Sokol, Georgia Tech

Vice President-Marketing,

Communications and Outreach E. Andrew Andy Boyd,
University of Houston

Vice President-Chapters/Fora Olga Raskina, Con-way Freight
INFORMS OFFICES
www.informs.org Tel: 1-800-4INFORMS

Executive Director Melissa Moore

Meetings Director Teresa V. Cryan

Marketing Director Gary Bennett

Communications Director Barry List

Headquarters INFORMS (Maryland)

7240 Parkway Drive, Suite 300
Hanover, MD 21076 USA
Tel.: 443.757.3500
E-mail: informs@informs.org
ANALYTICS EDITORIAL AND ADVERTISING

Lionheart Publishing Inc., 506 Roswell Street, Suite 220, Marietta, GA 30060 USA
Tel.: 770.431.0867 Fax: 770.432.6969

President & Advertising Sales John Llewellyn

john.llewellyn@mail.informs.org
Tel.: 770.431.0867, ext.209

Editor Peter R. Horner

peter.horner@mail.informs.org
Tel.: 770.587.3172

Art Director Lindsay Sport

lindsay.sport@mail.informs.org
Tel.: 770.431.0867, ext.223

Advertising Sales Sharon Baker

sharon.baker@mail.informs.org
Tel.: 813.852.9942
PRO FIT C EN T E R
Winning the war with

analytics
The military didnt call
it analytics but the
thinking was distinctly
analytical. And it was
truly revolutionary.
BY E. ANDREW BOYD
From 1941 to 1943, a small group of British and

American scientists, almost entirely without military
experience or knowledge, revolutionized the way
wars are run and won. So begins a new book about
what is widely considered the birth of analytics.
The military didnt call it analytics. The many different scientists who participated in the war effort were
doing what the military deemed research into operations, giving rise to the name operations research
(for more on the relationship between analytics and
operations research, see What is INFORMS and
why should I care? Analytics, March/April, 2012). But
the thinking was distinctly analytical. And it was truly
revolutionary.
Throughout the First and Second World Wars,
German U-boats were a threat to Allied merchant
ships and the war effort. The question asked of these
early analytics practitioners was, at least on the surface, quite straightforward: Whats the most effective
way to sink U-boats? Issues ranged from optimal
search patterns to the deployment of depth charges.
The book Blacketts War by Stephen Budiansky describes how mathematics and data were for the first
time used to drive decisions that had previously been
made by military personnel. The scientists methods proved extremely successful, showing ways to double or triple
the effectiveness of the faltering Allied
campaign against the German U-boats.
But as anyone whos practiced analytics knows, technical accomplishments
are only part of the story. Getting technical people and non-technical decisionmakers to communicate is just as vital,
if not more so. As Budiansky points out,
the scientists not only faced a general
distrust of intellectuals but the acrimony
that accompanied civilians telling military
officers how to do their jobs. The thought
that analytics gestated in the traditional
military establishment of the mid-twentieth century is almost unimaginable until
one considers the circumstances: the
world was at war. And Budiansky leaves
no doubt that analytics was vital to the
Allied victory.
I like to think that companies are engaged in healthy competition, not war, but
sections of Blacketts War couldnt help
but evoke thoughts of companies Ive
worked with over the years. Some are
driven by analytics. Others, not so much.
But what Ive noticed is that deep down,
companies that want to use analytics
are seeking to do something even more
fundamental than use data: theyre seeking to apply a more rational approach to
decision-making.
A NA L Y T I C S
A well-worn story I like to tell is of a

client who once informed me he used a
ceiling strategy when making forecasts.
Having never heard of this technique I
took the bait and asked, Whats a ceiling
strategy?
I lean back in my chair and look at
the ceiling, he told me, and I make a
forecast.
Thankfully he was joking (or so I
think), but the story underlined what the
quest for analytics is all about. Is looking
at the ceiling any way to make a forecast?
What about the alternative of something
as simple as looking at comparable sales
from the prior year? Why not make adjustments based upon more recent sales
trends? These are the questions competent, reliable, successful businesses are
asking. Analytics is about math and data,
but only as a means to an end. The end is
to make better, more rational decisions.
And like the Allied forces, the use of
analytics is becoming increasingly necessary for companies who want to win the
war. Poorly targeted sales campaigns
are like poorly targeted depth charges:
they fail to reach their objective.
Subscribe to Analytics
Its fast, its easy and its FREE!
Just visit: http://analytics.informs.org/
M A Y / J U N E 2 013
PRO FIT C EN T E R
Its an unbelievably
exciting time for analytics
practitioners and the
companies they work
for. How is it possible to
provide so much benefit
and have so much fun at
the same time?
Help Promote Analytics

Its fast and its easy! Visit:
http://analytics.informs.org/button.html
10
I cant say I was there to witness the

birth of analytics, but Ive been fortunate to watch it grow. Apple was founded in 1976, the year before I graduated
from high school. The Apple 1 came
standard with 4 kilobytes of memory,
enough to hold the characters in this
column about four times over (forget
holding any software to manage those
characters). At the time it was unimaginable that we could process anything
like the amount of data we now routinely work with on our home computers, much less on arrays of servers.
Astonishing is an understatement. Its
an unbelievably exciting time for analytics practitioners and the companies
they work for. How is it possible to
provide so much benefit and have so
much fun at the same time?
This is my last Profit Center column. Much as Ive enjoyed offering my
perspectives on analytics, I fear they
may soon lack a freshness that you, the
reader, deserve. I hope to reappear in
the not-too-distant future with a new column based on analytics case studies. It
would involve less ruminating and more
reporting a nice respite for this analytical mind.
Andrew Boyd, senior INFORMS member and
INFORMS VP of Marketing, Communications
and Outreach, has been an executive and chief
scientist at an analytics firm for many years. He
can be reached at e.a.boyd@earthlink.net.
AN ALY ZE T H I S !
Course puts students in the

analytics game
There is no substitute for
giving people a chance to
learn by doing ... Like
analytics projects
themselves, this
development is less linear
than what we
are accustomed to.
BY VIJAY MEHROTRA
12
This past semester, I taught a class for 2nd year

MBA students called Analytics Consulting Projects.
Students were assigned to teams and then matched
with clients who have real, meaningful problems that
required data management, modeling and analysis.
The objectives of this course are to prepare these students for careers as analytically capable managers
by enabling them to:
recognize how analytics projects are organized
and fit into a broader business context;
identify appropriate analytical techniques,
software tools and modeling assumptions
associated with particular projects;
understand how to work effectively across the
entire information value chain from initial project
definition and data analysis to final analyses and
recommendations; and
collaborate effectively with colleagues and clients
with different skill sets, functional competencies
and working styles.
While I had supervised many students doing analytics projects as independent study courses, this was
the first time that I had taught an actual class like this
one. As such, I was a bit nervous about how things
would go, both for the students (who would be challenged by both the projects technical content and by
the consulting process) and for myself

(as I had put my reputation on the line to
secure several of the client projects).
As part of course content, I had wanted to provide the students with a framework for structuring their project work
and managing their progress. As such,
I spent a lot of time hunting around for
something that both resonated with my
own experience and also appeared to be
somewhat generalizable. Hunting around
the Internet, I eventually found the Data
Analytics Lifecycle from EMC Corporation. In an excellent series of blog posts
[1] that talk about his experience applying this methodology in his own work,
EMC Fellow Steve Todd helps bring this
framework to life, and I decided to incorporate this model into my course.
I thought of this lifecycle again the other day when I received an advance copy
of Keeping Up With the Quants: Your
Guide to Understanding + Using Analytics, a book by Tom Davenport and Jinho
Kim due out this June. As in his previous
books with Jeanne Harris (Competing
on Analytics, Analytics at Work), Davenport once again seeks to provide the
managers and executives with a perspective on the capability of analytics. Davenports partner this time is Kim, a Wharton
Ph.D. and professor of statistics at the
National Defense University in Korea,
and their book opens by clearly stating
A NA L Y T I C S
the authors goals: We would like you to

be an intelligent consumer of [advanced
analytics], helping to frame the decision,
asking questions about the data and the
methodology, working to understand the
results, and using them to improve outcomes for your organization. While this
sounds straightforward, because of my
recent experience developing and delivering the Analytics Consulting Projects
course, I immediately recognized it as a
deceptively ambitious set of objectives.
The first half of the book was organized around an analytics framework that
consists of six steps:
Step 1: Problem Recognition
Step 2: Review of Previous Findings
Step 3: Modeling
Step 4: Data Collection
Step 5: Data Analysis
Step 6: Results Presentation and Action
Though the verbiage is somewhat different, this structure is roughly parallel to
the EMC framework, which also explicitly
includes iterative loops backward from
every step, as this type of reworking/refinement is a common element of many
analytics projects.
Indeed, a structured approach to
thinking about analytics, embodied in this
six-step process, is one of the most significant ideas that Davenport and Kim are
trying to get across to their readers. Every
M A Y / J U N E 2 013
13
ANALY ZE T H I S !
chapter includes multiple stories told explicitly through this six-step lens. Some
of these work beautifully. For example, I
enjoyed reading about a company called
Transitions Optical that had successfully
modeled and analyzed the impact of various marketing efforts to support better
resource allocation. It was also fascinating to get the (admittedly condensed) description of how Bill Fair and Earl Isaac
came up with the now-ubiquitous FICO
score. And the story of how Australian
authorities solved an insider trading case
using bank records and network theory
was downright inspiring.
However, other stories were squeezed
uncomfortably into this six-step frame and
as such felt neither credible nor directly
relevant to the books purpose. In several cases, the authors use this framework
to describe landmark academic research
efforts (Murray and Gottmans theory of
marital conflict, Snowdons study of predictors of Alzheimers Disease based on
data gathered from a population of nuns)
that spanned many years and undoubtedly evolved in a far less linear manner.
In another instance, the authors shoehorn the story of the Houston Rockets
decision to trade for Shane Battier [2]
into their six step process, which felt like
a gross oversimplification and ignored
the critical role of the NBA salary cap on
how/why that trade was made.
14
About halfway through the book,

Keeping Up With the Quants suddenly
began to remind me of a really interesting college radio station (Welcome to
KUWQ, broadcasting live from the center
of the analytics universe.). KUWQ is inveterately eclectic, featuring pop records
(no analytics book is complete without
an obligatory Gary Loveman reference),
golden oldies (a wonderful story about
a discovery by Archimedes), and some
wonderfully weird tunes (uberGeek Garth
Sundems fictitious Fido index for determining ones suitability for pet ownership
[3]). KUWQ also makes up for its occasional lapses and omissions by offering
up songs, albeit ones that seem a little
raw and unfinished, on several important
topics (the connection between creativity
and analytics, how managers can bolster
their own analytic thinking and capabilities, the value of relationships and trust
in enabling data-driven decision making)
that are just not getting the air time they
deserve on more commercial channels.
In the end, in addition to delivering a
solid project framework, some wonderfully instructive stories and a lot of other
interesting related information, Davenport and Kims new book also reminds us
that much of what makes analytics work
in practice is simply very hard to summarize and convey efficiently. The plain truth
is that analytics, even from a managerial
perspective, is a difficult sport to understand from the sidelines.

Books like this one can certainly be
helpful, but there is no substitute for giving people a chance to learn by doing
and providing encouragement, feedback and coaching along the way. Like
analytics projects themselves, this development is less linear than what we are
accustomed to, typically featuring significant discovery, reflection and revision
and requiring some trust in the process.
In the end, both my students and I
learned a great deal from this springs
A NA L Y T I C S
Analytics Consulting Projects course,

and I can hardly wait to teach this class
again next year. And if youve got a project that you might want a group of my
students to tackle, please let me know. I
am pleased, and proud, to report that we
now have a number of very successful
reference clients that we can put you in
touch with.
Vijay Mehrotra (vmehrotra@usfca.edu) is
an associate professor in the Department of
Analytics and Technology at the University of San
Franciscos School of Management. He is also an
experienced analytics consultant and entrepreneur,
an angel investor in several successful analytics
companies and a longtime member of INFORMS.
M A Y / J U N E 2 013
15
FORUM
Problem solving with data

Many collect data, few know what to do with it. The core skill business
professionals need to know.
Imagine a world where

marketers, product
managers, sales people
and operations folks
actually know how to use
the power of the data they
collect on a daily basis.
BY PIYANKA JAIN
16
McKinseys Big Data report, published in May

2011, lists a shortage of talent in the big data space.
Interestingly, the shortage of big data business professionals (1.5 million by 2018) is about 10 times
that of data scientist (140,000 by 2018). Although the
shortage prediction is for 2018, this 10-1 ratio exists
today. That means a lot of people are collecting data,
but few really know what to do with it. This huge gap
between business professionals and data analysts is
paralyzing the business world.
Now imagine a world where marketers, product
managers, sales people and operations folks actually
know how to use the power of the data they collect on
a daily basis. In this world we remove the knowledge
deficiency and provide an amazing amount of competitive power to the business with the data.
As business professionals, we naturally have
some ability to dissect the data we collect, but when it
comes to more detailed analysis, most business professionals conjure up images of complex algorithms
and code and then tune out. Business professionals
need not be afraid of what big data can deliver. There
is so much to learn with the information we collect on
our potential and current customers. Those insights
enable us, as marketers, to be the best

at what we do.
Lets go through a basic decision
making process most of us have or will
go through at some point in our lives
buying a car. I want to provide an analytical and non-analytical approach for you
to gain perspective.
BUYING A CAR
The analytical (data-driven) approach:
You start by nailing down your constraints
time, money, five feature requirements
(must haves) and five wish lists (good to
have). Perhaps good mileage is a high
priority in your features list while low
emission is a lower priority for you. Based
on all of these criteria (which are unique
to you), you narrow down your choices in
vehicle options to a short, final list.
You test drive the finalists, consider
the biggest priorities on your requirements list and begin to grade each vehicle with a 1-5 score on each requirement
and wish. The requirements are graded
with an extra point and the wish lists stay
at the numbers they were given. Then
take the average of requirements and the
wish list. You now find out what your next
vehicle purchase will be. This is analytics.
There is a process by which you came to
the best and most appropriate choice of
car based on your needs and car facts.
Analytics is fact-driven decision-making.
A NA L Y T I C S
The non-analytical approach: This process may start by test driving cars, any
type whatsoever, irrespective of any criteria, and you either begin creating your
own criteria as you go along maybe
rejecting some car and loving others,
based on what you feels good.
So what is the advantage of analytical over non-analytical approach? I was
recently talking to a friend who was complaining about the mileage on the new
car he recently purchased for his long
commute. He seemed very unhappy with
the $100 a week he spent on gas. That
didnt seem unreasonable to me. So I felt
compelled to ask him if he had changed
jobs and if his commute was even longer
after buying the car. That turned out to be
not true. Then I asked if the car is giving
him lower mileage than expected or advertised something that may be feeding
his complaint. Well, it turned out that was
also not the case.
Finally I asked him why he didnt buy
a car with higher mileage instead of the
one he bought, especially since he was
going to use it for his long commute and
when he knew the gas cost was a constraint. He answered by saying he didnt
know the cost would be this high or that
it would be so burdensome to him. Most
importantly, he said he really liked the
feel of the car when he drove it. Could
he have gotten a car, which he liked the
M A Y / J U N E 2 013
17
FORUM
Many of us base our

decisions on facts. The
process of making datadriven decisions in our
day-to-day business life
isnt a whole lot different.
feel of while still making sure it met his

must haves? You bet! But that requires
the analytical approach to buying a car.
Using data to drive decisions delivers
a significantly higher chance of making
a good, long-lasting decision over non
data-driven approach.
Can most of us envision ourselves
using this kind of analytical approach to
buying a car, buying a house, choosing a
career or choosing a school for our kids?
Yes, most of us do. Many of us base our
decisions on facts. The process of making data-driven decisions in our dayto-day business life isnt a whole lot
different.
Say you are a marketing manager at
an e-commerce company selling shoes
(imagine Zappos). Spring is here and you
have a marketing budget of $100,000.
You have a million or more customers and
prospects, and you have to decide where
to spend that $100K to get the best ROI
possible. Should you spend it toward acquisition, i.e., driving new traffic to your
site, or should you spend that toward
engagement of current base or both? If
you focus on acquisition, which channel
or combination of channels should you
Join the Analytics Section of INFORMS
For more information, visit:

http://www.informs.org/Community/Analytics/Membership
18
choose? If you focus on engagement,

should you go out to the entire base or
a subset? Should you customize your
offering by segments and if so, how? At
the end of the day, you want to make a
choice that aligns best with your company/departments priorities and gains you
the best ROI. But the question is, how to
make the best choice now?
A non-analytical approach may look
like doing what was done last spring
(status quo) or choosing projects from
last quarter or going with projects that
A NA L Y T I C S
you believe to be the best. Just like in

the car-buying example, unless you are
specifically focusing on maintaining ROI
(your expected success criteria) in mind,
you would likely not get the best ROI
from your effort. You will execute some
average marketing campaign, but not the
one that makes the most sense for your
goals.
An analytical approach to this marketing
campaign may be gathering insights from
past campaigns what worked, what didnt,
what gave the best ROI. Lets say, you find
M A Y / J U N E 2 013
19
FORUM
that your organic acquisition is at par with

the competition and you decide to invest in
engagement of the current customer base,
and habituation of the prospects or light users. Now, you would go back to past campaigns and see what worked. Lets say
you find that certain customer segments
(call them loyalists) purchase irrespective
of marketing to them (you know it because
you used a control in the last set of campaigns) and you also find other segments
that respond to marketing. Now you have
clues as to which segment not to market to
and which segment to saturate toward optimizing the ROI.
Can you or any business professional do this? Most definitely! All it takes is
a little understanding and practice of a
data-to-decisions framework (BADIR, for
example). You can use simple techniques
to optimize your day-to-day decisions.
These simple methods dont need complex tools. As long as you have access
to data through some data tool (such as
Tableau, Spotfire, Microstrategy, Business Object, etc.), you can download the
data into excel to analyze it (or analyze it
within the data tool).
Today, many business professionals

depend on their analytics counterpart
to help make key product and marketing decisions, analyze future launch and
past campaigns, find the best target segments, etc., but those analytic resources
are increasingly scarce and in high demand. But marketing professionals dont
need to find themselves in a lurch without these analytics professionals. Learning some of the basics of the data game
will help marketers derive some of the
best tactical options for future campaigns
through past consumer behavior.
If you are a business professional in a
role where you are making decisions in a
day-to-day workflow, then it is imperative
that you equip yourself with skills to solve
problems using data analysis. Make sure
you are not left behind.
Piyanka Jain (piyanka@aryng.com) is president
and CEO of the analytics consulting company
Aryng. She often keynotes at business and
analytics conferences on in-sourcing of analytics
via developing internal organizational capabilities
people, process and tools. Her prior roles
include head of NA business analytics at Paypal
and senior marketing positions with Adobe. She
is an INFORMS partner. This article is based on a
blog of Aryngs analytics tips series for business
professionals.
Request a no-obligation INFORMS Member Benefits Packet

For more information, visit: http://www.informs.org/Membership
20
CAREER B U I LDE R
INFORMS confers first

Certified Analytics
Professionals
Thirty-eight individuals comprise the
the inaugural cohort to earn the Certified Analytics Professional (CAP)
designation conferred by INFORMS,
the worlds leading association for professionals in the fields of operations
research, managementscience and
analytics.
The CAP designation reflects a
combination of education and experience, coupled with confirmation of soft
skills (i.e., communication, presentation).
The certification process is capped by
a rigorous, 100-question examination,
which was formally administered for the
first time April 7 in conjunction with the
INFORMS Conference on Business Analytics & Operations Research in San Antonio, Texas.
The names of all of those who earn
the CAP designation will be housed in a
CAP registry on the INFORMS website.
Prior to April 7, the exam had been
taken by pilot tester subject matter experts (SMEs), as well as other SMEs involved in setting the preliminary passing
22
score. The passing rate of those who took

the first exam was 77 percent.
Comments from test-takers indicate the
exam covers a fair representation of the endto-end analytics process, from business and
analytics problem formation, through data
challenges, to methodology and model selection, deployment of the selected model
and lifecycle management.
Along with helping analytics professionals boost their careers and elevate
themselves in the job market, the CAP
program is also designed to help hiring
managers such as Jim Williams, manager
of advanced analytics at Land OLakes,
Inc., identify qualified analytical talent.
I thought certification would be a useful differentiator as Im sifting through
applicant pools looking to find talent. And

the best way to validate my hunch was to
go see it for myself, Williams says when
asked what motivated him to take the
exam.
And his thoughts on the exam now
that hes taken it and passed it?
I thought it was quite comprehensive, Williams says. I would say it
reflects the end-to-end skills that a consultant would need to execute a complete
consulting engagement. It also probes
relatively deeply in several of those areas. Im confident its going to be a very
useful instrument to help hiring managers differentiate applicants.
INFORMS will administer the exam
at least four more times in 2013, including June 23 in Chicago (preceding the
INFORMS Healthcare Conference),
July 13 in McLean, Va. (Booz Allen Hamilton offices), Aug. 3 in Bellevue, Wash.
(University of Washington Bothell Eastside Leadership Center), Oct. 3 in Boston (following Predictive Analytics World)
and Oct. 5 in Minneapolis (preceding the
INFORMS Annual Meeting). INFORMS
is in discussion to add more U.S. sites in
2013 and plans to migrate to computerbased testing in 2014 that will offer exams internationally.
For more information on CAP including sample questions, a study guide
and other details, click here.
A NA L Y T I C S
Thirty-eight earn CAP

The following individuals comprise the
first group to earn the Certified Analytics
Professional (CAP) designation from
INFORMS:
Alan L. Austin (Provo, Utah)

Zahir Balaporia (Green Bay, Wis.)
Russell R. Barton (University Park, Pa.)
Donald Buckshaw (Ellicott City, Md.)
Thomas W. Chestnutt (Encinitas, Calif.)
Thomas W. Fletcher (Buford, Ga.)
Arnold Greenland (Bethesda, Md.)
Terry P. Harrison (University Park, Pa.)
Steven Harrod (Dayton, Ohio)
Christopher Michael Heiden (Columbus, Ohio)
Amanda Humbert (Grove City, Ohio)
Lisa Kart, (Austin, Texas)
Zain Khandwala (Columbus, Ohio)
Diego Klabjan (Evanston, Ill.)
Srikanth Krishnamurthy (Malden, Maine)
Russell P. Labe, Jr. (Pennington, N.J.)
Robert M. Lucas (Cary, N.C.)
Irvin J. Lustig (Short Hills, N.J.)
Paul R. Messinger (Edmonton, Alberta, Canada)
Rami Musa (Claymont, Del.)
Scott Nestler (Carlisle Barracks, Pa.)
Hua Ni (Chantilly, Va.)
Elizabeth Nielsen (Durham, N.C.)
Ranganath S. Nuggehalli (Timonium, Md.)
Subhashish Samaddar (Atlanta, Ga.)
Sam L. Savage (Stanford, Calif.)
Satjeet Singh (Lewisville, Texas)
Michael Anthony Smith (Arlington, Va.)
Alan Taber (Cinnaminson, N.J.)
Jonathan Taylor (Greenbelt, Md.)
Deepak Tirumalasetty (Modesto, Calif.)
Catherine Truxillo (Raleigh, N.C.)
Zachary Waltz (Arlington, Va.)
Glenn Wegryn (Cincinnati, Ohio)
Steven P. Wilcox (Rockville, Md.)
James T. Williams (St. Paul, Minn.)
Kevin Windham (Fairfax, Va.)
Gigi Yuen-Reed (Tampa, Fla.)
M A Y / J U N E 2 013
23
UPG RADE T H E C O R PO RAT I O N
The Business
Analytics Revolution
Four killer competencies of analytically mature
corporations.
BY RANDY BARTLETT (LEFT) AND GIRISH MALIK
Welcome to the Business

Analytics (BA) Revolution.
We live in an age of fastpaced
decision-making,
assorted market disruptions and information and misinformation overload. CEOs
used to make about six or eight critical
decisions involving the organization and
products per year. Now, they make that
number of comparable decisions every month or so. Meanwhile, torrents of
data relentlessly rush in, generated by a
digital revolution, which is characterized
24
by hyper-connectivity and real-time decision-making. It all begs the question:

Where can we make the time to absorb
this much information? or is it another
case of I appreciate that its there, but I
am not sure what to make of it.
Despite all the value-add that business
analytics brings to the table, companies
have hardly been able to realize its true
potential. Such an attainment implies incorporating more facts into decision-making, facing the information deluge from
big data and more nimbly adapting to the
Figure 1: Report card.

changing environment. By upgrading the
corporation, we can better integrate business analytics into how we do business
and thereby compete on BA. This article
will clarify where we are in the BA revolution, how to measure our BA maturity and
what killer competencies we can realize.
We hope this information will enable corporations and leaders to upgrade.
By upgrading the corporation, we mean
more than just adding quants to a conventional infrastructure. We are past the point
where the usual additive approach will keep
up. We need a holistic upgrade. Upgrades
are needed in five major areas: culture, organization, people, statistics and data. Lets
start by recognizing where we are in this
technological revolution.
A NA L Y T I C S
ADAPTING TO A TECHNOLOGICAL
REVOLUTION
The introduction of the automobile provides a familiar example of the
changes needed to fully leverage a new
technological innovation. Three areas of
change culture, infrastructure and the
evolution of the BA toolset are salient
to the discussion.
First, the introduction of the automobile met with strong cultural resistance,
particularly from 1900 to 1930. Road fatalities, which included children playing in
the street, met with outrage. Eventually,
most people accepted the fatalities in exchange for the benefits.
The resistance to BA is centered on
change rather than fatalities. Implementing
M A Y / J U N E 2 013
25
BA requires people to change and requires

a change in people. Decision-makers need
to: (a) plan for their information needs, (b)
learn how to better incorporate statistical
results into decision-making, and (c) digest
a haystack of information by judging the
accuracy and reliability of the results. This
can be much more difficult than the earlier
state of affairs when decision-makers could
rely upon their years of industry experience
and the opinions of their peers to draw
conclusions.
While that experience is still relevant,
it has a shorter shelf life and must be tempered with up-to-the-moment information.
For managers, the need is to embrace more
specialization from everyone and to identify
credible business analysts and business
quants. They need to change their game
from checkers to chess.
Second, fully leveraging the automobile required constant improvements to
the infrastructure. This innovation led to
the gradual development of a full-blown
ecosystem consisting of roads, gasoline
stations, mechanics, traffic rules, traffic
signs and drivers licenses. (In the United
States, the last state to require a drivers
license did so in 1954 and five years later
that state required an examination.)
Likewise, organizations need a very
different infrastructure to incorporate
business analytics. Corporations need
to ensure that the data are put to the
26
best possible use and feed into optimal

decision-making at all levels. While data
warehouses have become more accessible and user-friendly, the greatest organizational challenge is to extend the
BA team. We need to include senior
management, probably a CAO (Chief
Analytics Officer), analytics-based decision-makers and an effective business
analytics leader who can complete the
QLQT (Quant Lead Quant Team). We
must focus on the need for specialization
and placing the right people in the right
roles.
Certifications are becoming available in
the United States, and they will help managers identify business quants and leaders of quant teams from the other chess
pieces. To help with discernment, ASA
launched the PSTAT in 2010 and INFORMS launched CAP this year. In a
similar vein, Michael Rappa of North Carolina State University launched the first master of science in analytics (MSA) degree,
which emphasizes practical and soft skills.
Third, automobiles have evolved
from their less sophisticated ancestors.
We have moved from wagon wheels to
vulcanized rubber tires and now automobiles can park themselves, navigate,
monitor tire pressure and talk. BA needs
to evolve too. We need to: 1) expand the
tool set, and 2) adapt the tools for corporations and for big data.
Expand the tool set: We need to start by

leveraging more of the tools, which have
already been built for scientific applications. For example, we need to more frequently pursue the data we need rather
than always accepting whatever is convenient. In many situations, we can collect and generate more applicable data
by leveraging statistical sample designs,
designed experiments and planned simulations. These tools enable starting with
the business problems and information
needs and aggressively pursuing the
best information.
Similarly, quality control is seldom used
outside of manufacturing even though almost every corporation tracks several KPIs
and can readily apply these tried-and-tested
techniques. Furthermore, there are numerous data reduction techniques, which can
readily subdue big data for specific applications, yet they remain mostly idle amid the
hurried excitement.
Adapt the tools: Many of our ancestral scientific tools were customized to
investigate causality within the confines
of small, purposefully planned samples
and in a methodical decision-making
environment. There was a time when
we had to deliberately misapply diagnostics designed for coefficient estimation because other applications lacked
corresponding diagnostics. In recent
28
decades, researchers have enhanced

statistical techniques to better cover the
corporate problem mix. This mix contains
more data visualization, prediction, ranking and clustering (grouping) relative to
coefficient estimation, which we perform
to investigate causality.
Business datasets are frequently big
and they tend to be conveniently collected
rather than planned. This difference emphasizes the need for validation techniques
and clarity around the difference between
statistical significance and relative significance. Finally, the business decision-making environment is always fast-paced and
sometimes lowbrow. This environment has
created a conflict between being a steward
of pristine information and playing more of
an influencer role.
In expanding the tool set and adapting the tools, we will inevitably improve
and broaden BA applications. One key
application requiring better coverage
is decision-making. We need to rethink
what tools should be used in making and
evaluating decisions. This effort can be
further facilitated by changes in leadership to make the corporation more quantoriented and thereby widen the breadth
of business analytics as a whole.
By juxtaposing the BA revolution with
past technological revolutions, we glean
insight into where we are now and where
we are going.
MEASURING BUSINESS ANALYTICS

MATURITY
Measuring a corporations BA maturity can provide a map of strengths and
weaknesses, generating a great deal of
insight. A proper measurement must be

objective and include sufficient expertise. The review team should be led by
experienced external business quants
who can benchmark capabilities relative
Table 1: Business analytics maturity model.

Maturity Dimension
Developing
Culture
Degree to Which BA is
Integrated Into How We
Do Business
Spotty
BA Recognized at
Specialization
Organization
Adaptation of Structure to
Facilitate Analytics-Based
Decision Making
BA Thought To Be
Additive Kept in Closet
Somewhere
Thought Given to Location

of Quants; Cross functional
analytics teams
Senior Management
Cosmetic support for BA
Mid-level Advocate within

one LOB
Decision Makers
Opinion-based decision
making
Leverages analytics when

convenient
Business Analysts and

Business Quants
Only BAs; no BQs
A few BQs in one LOB
Directors of Analytics
No leadership, only
management
Off-topic manager delegates
Statistical Qualifications
of Decision Makers and
Business Quants
Absent
Recognized and appreciated

by some in the corporation
Statistical Diagnostics
Absent
Employed by BQs only
Statistical Review of
Decisions and Statistical
Results
Absent
Occasional
Data Collection
Only uses data that

finds us
Seeks data available

through vendors
Data Software
Very Basic
Enables Descriptive Data

Analysis
Data Management
Incoherent Data
Dictionaries & Data
Encyclopedia
Readable Data Dictionaries

& Data Encyclopedia
People
Statistics
Data
30
Lagging
to competitors (low hurdle) and needs

(high hurdle). At present, holistic benchmarking of any kind is difficult to find;
perhaps, TGaS Advisors is the closest.
Table 1 categorizes five areas to watch
for measuring BA maturity: culture, organization, people, statistics and data.

Table 1 provides insight into several
aspects of BA maturity. So what does this
investment buys us?
Phase
Leading
Pioneering
In Leadership & Among Decision

Makers
BA Woven Into Corporate Fabric
Decision Makers & Quants

Collaborate; Enterprise-wide
representation
Quants in Leadership Roles
Enterprise-wide Advocate; several

mid-level advocates
Hires CAO; Makes Analytics-Based Decisions as an

example to others
Planning information needs;

understands some diagnostics
Analytics-Based Decision Making; Mastery of

statistical diagnostics; ability to delegate decisions
Manager of BQs delegates
BAs and BQs brainstorm about innovative solutions
Leader has data analysis training

and experience
Business Analytics Leader; completing a QLQT
Leadership can identify quants
Enterprise-wide recognition and appreciation of

PSTATs and CAPs
Institutionalized
Understood by decision makers too
Common
Institutionalized
Proactively seeks data for future

needs
Data collection is part of business strategy
Robust mechanisms to generate

data-driven insights
Integrated systems to seamlessly consume datadriven insights
Data treated as Corporate Asset

Business treated as customer by IT
CAO or other Quant signs off on warehouse
A NA L Y T I C S
M A Y / J U N E 2 013
31
Figure 2: Four killer BA competencies.
32
FOUR KILLER COMPETENCIES

OF ANALYTICALLY MATURE
CORPORATIONS
only; analytically mature corporations

can pursue the information they need.
The rewards for BA maturity are enhanced competencies, as successfully

demonstrated in the field. Figure 2 presents four killer competencies possessed
by corporations that compete on analytics. The arrows indicate the desired progression, e.g. passive corporations must
satisfy themselves with convenient data
Killer competency No. 1: pursuing information. Most of the data we use is generated either in the course of doing business
or through customer behavior. There are no
corresponding planned business applications for this convenient data. When a new
business need is identified, we scour the
data encyclopedia (portfolio of data choices
with background extolling their virtues) looking for best information. In practice, there
are serious gaps. For example, one common unmet need is competitor information.
When there is a gap, we want to be in a
position to do more than use the best information available as a substitute. We want
the super power to collect or generate information customized to fit the business
needs proactively pursuing information.
This requires that the expertise to design
samples, experiments and simulations be
in the hands of people who understand the
business and have earned our trust.
Killer competency No. 2: quant leadership. There are times when we want
to wield the power of a large Quant Led
Quant Team (QLQT). This requires a
business analytics leader who understands what the team can do and how
it functions. We want a leader who has
adequate training, practice experience,
business acumen and a grasp of the soft
skills. We may need to find or develop
this person, who might be standing right
in front of us. The natural people to identify talent are the other BA leaders: CAO
(Chief Analytics Officer) and analyticsbased decision-makers.
As previously mentioned, many corporations are not realizing the true power of
BA. They stop short by encumbering the
quant group with an off-topic manager
A NA L Y T I C S
possessing every thinkable skill except

BA practice experience. This compromise
means that the corporation receives only
diluted quant expertise. This is like having
a manager of finance, legal or accounting
who has only two semesters of relevant
training and no practice experience. The
quant team needs its own overall leader.
Otherwise, each quant is left to lead themselves, and they lack cohesion, guidance
and focus. If a large quant team is managed
and not led, this is a sign of three possibilities: 1) the corporation is not ready for BA;
2) the corporation has more quant than it
knows what to do with; and/or 3) the corporation can not identify an appropriate leader.
Killer competency No. 3: quant involvement. Statistics is a specialization.
It is challenging to blend domain knowledge with the BA expertise needed to
provide greater depth and breadth of
analytics solutions. Integrating BA into
the business requires a high degree
of interaction between the quants and
other professionals, such as analyticsbased decision-makers. By fostering
more interaction between quants and
M A Y / J U N E 2 013
33
Figure 3: Moving up the BA maturity curve by developing killer competencies.
decision-makers, and providing leadership, we can expect greater synergy.

This implies that quants present their
own work, without an interpreter, and
that they review analytics-based decisions and all analytics information flowing into decision-making.
Business analytics is, in sum, a science as well as an art. What we need is a
set of deft craftsmen who can then renovate and help their corporations build analytics capabilities, achieve the desired
mission and vision and, most importantly,
be more customer and competitor ready.
34
Killer competency No. 4: analyticsbased decision-making. Finally, the most

powerful of the BA superhuman/supercorporate capabilities is to make smarter
decisions by basing them upon facts.
This requires sophisticated decisionmakers who are properly supported.
These decision-makers are driven to plan
their information needs, are capable of
discriminating differences in information
quality, and are willing to wade into an
ocean of information and misinformation
if that is what it takes. They live in a fastpaced, decision-making environment,
and they strive to incorporate analytics into their decision-making as much

as possible while maintaining the pace.
They understand what qualifications go
into a good quant. Finally, they welcome
direct interactions with quants and receiving feedback on better planning for,
and leveraging of, the information.
By pushing their way up the business
analytics maturity curve, corporations
can develop killer competencies.
CONCLUSION
The world hates change, yet it is the
only thing that has brought progress.
Charles Kettering
Much as other technologies have
revolutionized the way we live, business
analytics is transforming the way corporations compete. To harness BAs true
potential and make the critical transitions
from data to information to insights to
analytics-based decision-making, corporations need to adapt. The key is to
recognize and accept the current BA maturity and plan how to holistically change,
or rather upgrade, the corporation in five
main categories: culture, organization,
people, statistics and data.
The first steps are to provide the mandate and enlist the leaders who can make it
happen. The likely change agents include
A NA L Y T I C S
a Chief Analytics Officer (CAO) watching

the horizon, a business analytics leader in
the trenches with the quants (QLQT) and
analytics-based decision-makers. Some
corporations can leverage their Six Sigma
team, which is an accelerator for change.
Measuring BA maturity will provide a blue
print listing the areas needing attention.
These upgrades enable competing on business analytics using the four killer competencies listed above: pursuing information,
quant leadership, quant involvement and
analytics-based decision-making.
Randy Bartlett (Randy.Bartlett@
BlueSigmaAnalytics.com), Ph.D., is the author
of A Practitioners Guide to Business Analytics
and a business analytics leader with Blue
Sigma Analytics. He has more than 20 years of
experience providing and performing advanced
business analytics. His activities include organizing
analytical resources, reviewing advanced analytics
and providing analytical advancements.
Girish Malik, PMP, MBA, is an IT services
professional with more than 10 years of experience
across program management, client relationship
management and technology consulting. He has
managed customer expectations and delivery
teams for BI and analytics projects across domains
ranging from financial services and retail to telecom
and the public sector.
REFERENCES
1. Bartlett, Randy, 2013, A Practitioners Guide
to Business Analytics: Using Data Analysis
Tools to Improve Your Organizations Decision
Making and Strategy, McGraw-Hill (ISBN9780071807593).
2. Bennis, Warren, 2009, On Becoming A
Leader, 4th Edition, Basic Books.
3. Lewis, Michael, 2009, The Man Who Crashed
The World. Vanity Fair, August.
M A Y / J U N E 2 013
35
MARKE TIN G A NA LY T I C S
Comprehending
the complete
customer
Leveraging behavioral and attitudinal data for
actionable analytics.
BY JEFF AHLQUIST (LEFT) AND KSHIRA SAAGAR

36
Which came first attitude or

behavior? This is a question
researchers, marketers and
product engineers are no
longer trying to answer. Instead, the two
elements are increasingly being fused to
create a composite view of the customer.
Behavioral inputs are detailed, accurate
and real-time, while attitudinal inputs are
measurable KPIs for a business. The
insights resulting from integration can
potentially influence a wide range of decisions at an individual customer or segment level.
Successful integration of behavioral
and attitudinal data inputs requires some
investments in people, process and technology. However, the payoffs can be
endless and start with busting myths of
self-reported perceptions with the reality
observed in actual user behavior. This
paper examines business cases to apply
behavioral and attitudinal data elements
together, steps involved in establishing a link between the two, technological and organizational requirements or
constraints and last but not the least, the
analytical engine to drive this movement.
WHY IS IT IMPORTANT TO
INTEGRATE BEHAVIOR AND
ATTITUDE?
Understanding how the usage and
behavior of a person drives his or her
A NA L Y T I C S
perceptions and attitudes about a product or service has always been the holy
grail of primary marketing research and
analytics. However, the search for this
grail has been made easier and more
likely in recent times, thanks to the following enablers:
1. Technology, which is helping to
capture various signals in a costeffective manner
2. Analytics, which is becoming core to
business decisions, driving the need
for more data sources
3. Consumers, who are not shy to
generate and share their digital,
behavioral and social footprints
4. Culture, which encourages
experimentation and data discovery,
which is also becoming prevalent in
organizations
Organizations are continuously
trying to improve the understanding
of their end customers to enhance
design and better prioritize engineering and marketing efforts. Traditionally, primary surveys have been the
chief medium used by organizations
to understand customer perception of
products. However, with the advent
of digital media and its vast ability to
understand customer behavior in its
granularity, it has augmented the traditional approach by incorporating
M A Y / J U N E 2 013
37
Traditional
Descriptive Data (Who?)
- Attributes
- Characteristics
- Self-reported info
- (Geo) demographics
Attitudinal Data (Why?)

- Opinions
- Preferences
- Needs and Desires
360o view
Interaction Data (How?)
- E-mail / chat transcripts
- Call center notes
- Web Click-streams
Behavioral Data (What?)

- Transactions
- Payment History
- Usage Information
High Value, Dynamic

Figure 1: Holistic view of a consumer answers not just the who and why questions but also
the how and what.
behavioral data along with the traditional data sources to provide a bigger
picture.
Figure 1 indicates that traditional data
sources only provides a partial view of
the customers interaction with an organization and only answers the who and
the why questions. However, integration of behavioral data (usage logs and
interactions) gives a holistic view of the
consumer, thus answering the how and
what questions as well.
38
WHAT IS STOPPING EVERYONE

FROM DOING THIS?
Finding a needle in a haystack is akin
to deriving insights out of big data, or
vice versa. The difficult but not impossible task of behavioral and attitudinal data
integration comes with its own share of
challenges and roadblocks, including:
1. Big data processing. Behavioral data
representing a considerable size of the
sample population and encompassing its
complete machine usage data is going to

run into petabytes of data. Handling, processing and analyzing such voluminous
data requires one to look beyond the
traditional approaches and architectures
available for data management. Solution: Big data platforms such as Hadoop.
2. Asking the right questions. Sometimes a traditional hypotheses-driven approach might not be a suitable solution
considering that the problem mandates
understanding the scope of the issue and
A NA L Y T I C S
then framing the right questions upfront.

Solution: Discovery-driven framework.
3. Business consumption. Academic
solutions to a problem are always exotic and exciting. Nonetheless, the key
aspect of such an exercise is to make it
easily implementable and business ready
in terms of consumption and action. The
biggest challenge would be convincing
business groups of the validity of such
an approach and quantifying its impact
and findings in a manner that is easily
M A Y / J U N E 2 013
39
as against the
hypothesesdriven approach.
Why?
- How to overcome our biases for structured data?
Traditionally, lack
- Hypothesis vs. discovery driven approaches
of structured data
- What customer problems can we solve and how?
has often been a
deal breaker in
the hypothesesHow?
- Storage
- Manipulation
driven approach,
- Appreciation of quantitive techniques
while an abundance of unstructured data is used
What?
as a springboard
- Descriptive data
- Attitudinal data
in the discovery- Interaction data
driven approach.
- Behavioral data
Looking
at
Capturing Data
Figure 2, it is apTools & Techniques for the analysis
parent that it is
Objective of exercise
relatively
easy
to have the reFigure 2: Datasets, skillsets and mindsets are all requited to solve
quired datasets
difficult problems efficiently.
and the skillsets
to process data,
consumable and intuitive. Solution: Setbut it is paramount to also have the
ting up a channel for frequent and easily
right mindset to be able to solve the
consumable insights.
problem efficiently. And this is where
the discovery-driven approach plays a
MAKING INTEGRATION A REALITY
major role.
When solving innovative and unThus, this paper focuses on a failchartered analytical and business
fast, learn-fast discovery-driven initiative
problems, it becomes imperative to
to tackle new and rapidly evolving probadopt the right problem solving aplems. The approach aims to start with
proach. This brings forth the dilemma
the data story rather than the business
of using a discovery-driven approach
problem.
Mindset
Skillset
Dataset
40
MARKE TIN G A N A LY T I C S
Figure 3: Closed-loop system to effectively integrate, analyze and incorporate behavioral and
attitudinal elements.
CASE STUDY
The program and its constituents: The
case study an integration program of behavior and attitude was undertaken for a
product-based company that was interested in understanding a customers usage
behavior and sentiments expressed toward
its products in order to better design new
features and improve targeting strategies.
The program involved end-to-end enabling
of a closed-loop system to effectively integrate, analyze and incorporate behavioral
and attitudinal elements (people, process
& technology) as shown in Figure 3.
The program was brought to fruition as a result of the marriage of two
42
different data sources: machine log

data from selected respondents and
survey data from the same set of selected respondents.
Machine log data contained behavioral data with spools of information at a
user-event level and at the lowest possible granularity. This data constituted the
biggest chunk of the analysis (running
into terabytes of data) and eventually led
to the big data challenge.
Survey data contained attitudinal
data where the same set of respondents
was surveyed on a host of satisfaction
and perception related metrics. Respondents were asked to rate the technology
provider and its key products under consideration across a list of dimensions.
This was done to gauge their true sentiments of and toward the products and
the brand as a whole, and was captured
through longitudinal surveys across time.
Challenges: The most critical element of
the entire data exercise was to integrate
the disparate data sources. This was accomplished by establishing a common
identifier for both the respondents and
the machine users, thereby establishing a
way to bridge their multiple inputs. Availability of a common element is but one of
the many key steps in this program. The
next and most important step is actually
integrating them, which is a huge challenge both in terms of logistics and logic.
From a logistics standpoint, handling a
large amount of behavioral machine data
requires new data paradigms that are unconventional but efficient. This called for
big data constructs (e.g., Hadoop, Hive and
R). Data were stored on Hadoop clusters
(distributed computing applications), which
store and process big data efficiently. R was
the statistical program used to interface with
the Hadoop cluster to perform statistical
analyses on the big data.
In terms of logic, integration poses the
challenge of identifying the necessary
data to be considered for the analysis. It is
imperative to only consider manageable
A NA L Y T I C S
subsets of the entire dataset as against

the entire population of telemetry data.
Analysis and solution: The program
was planned as a four-phase analysis
covering various dimensions of the synergy produced out of integrating behavior
and attitude. It was designed to reflect and
recognize a customers product lifecycle
journey, which also has four stages (acquisition, service, engagement & retention and penetration & growth). The focus
was not only on integrating and analyzing
the data, but also deriving insights from a
product lifecycle perspective.
The objectives of each of the four
phases of the program include:
1. Behavioral profiling of various
attitudinal segments
2. Understanding linkage between
behavioral and attitudinal data
3. Behavioral segmentation of the user
base, with an attitudinal validation
4. Event triggers and longitudinal
analysis
Phase I: The objective of this phase of
the analysis was to quickly understand
and appreciate the data at hand. With the
linkage between behavioral and attitudinal data previously unexplored, it was
considered supremely imperative to get
to know the data better and get a pulse
on users behavior.
M A Y / J U N E 2 013
43
Active management of welcoming /

on-boarding a new customer and
beginning a relationship
Goal: Initiating users into the
environment
New
Product/Service
Development
Customer
Lifecycle/Loyalty
Management
Brand Equity
Aquire
Developing a
partnership with
the customers
to become an
inextricable part of
their performance.
Goal: Develop a
loyal base/brand
champions
Penetrate
&
Grow
Serve
Management and
assessment of the
product or service
after the acquisition
is complete and the
product usage has
begun.
Goal: Achieving
operational
excellence
Churn
Management
Engage
&
Retain
Needs
Assessment
Market
Segmentation
Identifying avenues for enhancing

engagement with existing customers,
via cross-sell/up-sell
Goal: Understanding customers
long-term strategic goals
Figure 4: Customers product lifecycle journey.

Multiple data slices were performed at
various levels on the integrated datasets. A
set of key questions and hypotheses were
addressed at this phase, which helped
44
obtain a better understanding of the consumers mindset and also helped break
down age-old myths and misconceptions
about certain segments of consumers. The
questions were designed and prioritized in

such a way so as to deliver maximum business impact. In summary, the objective of
this phase was to illuminate actions and
attitudes that drive customer use and advocacy connecting the who (descriptive
profile of the user) to what (behavioral
data on use and machine profile details)
with insights into the why (perceptions
and opinions about the users experience).
Some of the key questions for the
phase were:
A NA L Y T I C S
Are there differences in usage

behavior between different
categories of product owners?
How does level of engagement with
the brand or product affect usage
behavior?
What is the impact of the consumers
lack of awareness of product
updates?
How do we remove barriers for
migration of users between product
versions?
M A Y / J U N E 2 013
45
Phase II: The objective of this phase

was to establish a linkage between the
behaviors and perceptions in other
words, understand the causal relationships that exist between the two disparate sources.
With the axiom that correlation is not
necessarily causation in mind, the analysis was carried out with utmost care to
establish relationships that were both
correlated and causal in nature. Two key
questions emerged:
1. What is the relationship between user
activity, user profile and attitudes?
Business hypotheses:
Use of more features of a product
will increase suite satisfaction and
likelihood to recommend.
Certain customer segments and
groups post higher activity than others.
Quick finding: Usage is not necessarily
the key indicator of satisfaction; however
the version used has a strong linkage
with satisfaction.
2. How does use and perceptions of

competitor products impact product usage and attitudes?
Business hypotheses:
Use of a competitor product is
associated with low use or rejection
of product in consideration.
Quick finding: Use of competitive products does not erode product-in-considerations usage
The next two phases were more focused on leveraging the understanding
and insights from the initial two phases
and creating actionable customer segments and targeting business rules,
which could be readily implemented.
Phase III was dedicated to the creation of behavioral segments formed
out of the usage patterns. These behavioral segments were then compared against existing segmentation
schemas, to validate and enrich the
targeting strategies.
Phase IV was focused on longitudinal
analysis of the integrated data, i.e., trying

46
It was a myth-buster by
nature as it brought down
many age-old misbeliefs.
The program helped create
new knowledge about
extant segments and
busted old
myths around nonmarketing to a few user
segments.
48
to understand how behavior affected attitude at one

point and how the same attitudes affected behavior
in a future point in time thereby trying to understand the virtuous cycle of synergy between usage
and perception.
Results and impact: The program was a powerful
and prolific initiative, in that it provided many actionable insights and eye-openers, including:
1. It was a myth-buster by nature as it brought down
many age-old misbeliefs. The program helped
create new knowledge about extant segments and
busted old myths around non-marketing to a few
user segments. This re-look at non-marketed user
segments from a behavioral perspective revealed
that they were not avoiding the brand, but were
actually super-users of a particular product within
the brand, which had them wrongly classified as
avoiders.
2. It helped re-affirm some business beliefs by adding
some science and data validation to insights. The
programs data shows that version used is a high
predictor of activity level. Users on the most recent
versions are more active than previous version
users, and both user groups far surpass activity
levels among remaining oldest version users.
3. It also provided design inputs and product
enhancement ideas from the horses mouth.
Usage of a certain feature was analyzed,
revealing that those who used that feature
typically followed usage of another feature or
product. This highlighted potential to package
features together, thereby heightening
engagement.
4. It delivered on its promise of

establishing a bridge between
behavior and attitude. Users with
high satisfaction and likelihood to
recommend show higher activity than
less satisfied users. But the usage
of those who consider the product to
be easier to use is not significantly
different from those who have lower
perception toward ease of use.
5. It also helped perform competitive
analysis of products from a usage
standpoint. Among cross-users
of both suites,
competitive products
turned out to be
complementary to
the product under
consideration.
Combined users of
both products were
more active than those
of individual products.
the business objective of the

exercise upfront and also have a
clear idea of the potential business
impact of each objective.
2. Estimate infrastructure needs,
especially for big data. The exercise
dealt with nearly two billion rows of
data. To be able to do that, getting
infrastructure right is absolutely
essential. Infrastructure here denotes
both the ability to capture such
voluminous data and also the ability
to store and process them.
WHAT CAN BE LEARNED

FROM THIS EXERCISE?
To implement a similar
initiative in any organization, one needs to be cognizant of the following factors:
1. Define business
objectives in
advance. It is
paramount to define
A NA L Y T I C S
M A Y / J U N E 2 013
49
3. Devise strategies for optimal

infrastructure. Experiments
usually tend to go overboard with
allocated infrastructure. That is
when it becomes imperative to be
perpetually cognizant of the volume
of data at hand and finding optimal
ways to process more data.
4. Have right people and partners in
place. Datasets and skillsets are in
abundance, but the most critical piece
of the puzzle is the mindset. Having
the right partners in place ensures
that the thought process is enhanced
and the output is consumable.
5. Enable right communication to
drive right consumption. The
cognitive bias against an experiment
or a new project can be removed
only with consistent and engaged
communication with the target
audience. It is important to establish a
communication channel with the right
frequency easily consumable content.
CONCLUSION
Ever since the first group of analysts set out to comprehend a complete

50
picture of a customers usage behavior

and their attitude toward a product, the
analysts relied heavily on surveys to understand customer perceptions with little
or no visibility into how customers use or
interact with products.
In reality, however, it is usage that influences perceptions that in turn leads to
customer actions such as repurchase and
recommendation. Thus, it is imperative to
not look at behavior and attitude from individual lenses, but understand them using a
combinatorial approach that links behavior
and attitudinal data. The sooner organizations realize the significance of this exercise,
the better are their chances of influencing
and/or modeling a customers usage and
brand perception. With advancement in
technology and with right analytical partners
in place this integration exercise would be
one of the most worthwhile investments for
any company looking to understand its customers better.
Jeff Ahlquist, director of market research at
Microsoft, manages the Microsoft Market Share
Practice along with the behavioral analytics
program within the research team. Prior to joining
Microsoft, Ahlquist worked primarily in the areas
of business strategy, marketing and product
management for technology startups.
Kshira Saagar, an associate manager with
Mu Sigma, has considerable experience in
analytics consulting with multiple Fortune 500
clients. His experience spans across technology,
pharmaceutical and retail industries where he
works closely with client teams and business
executives in creating, operationalizing and driving
consumption of analytics.
MAN AG E ME NT SO F T S K I LLS
Historians vs.
Futurists
Which group provides more useful information for
todays decision-makers?
BY GARY COKINS
52
Futurists
enjoy
taking
out their crystal ball and
projecting future innovations, but they are typically
wrong. For example, George Orwells
book, 1984, which was published in
1949, did not come close with its projections. And in the 1960s, I recall a
Walt Disney television show describing automobiles that required no driver
and were guided by a magnet-like strip
imbedded in the streets or highways
roadbed. Nice try.
In contrast, historians research the
past to determine what lessons might be
learned and applied today. For example,
historians examine the judgments, policies and actions of past U.S. presidents
and international government leaders to
assess what actions may best serve citizens today. The recent movie Lincoln is
an example.
But which group futurists or historians provides more useful information? Futurists make us think by being
provocative. Historians allow us to reflect on what worked or did not work in
the past.
This question is relevant for todays
organizations because many enterprises fail to successfully execute their
executive teams plans and allocate an
appropriate mix and level of resources
to complete those plans. This involves
A NA L Y T I C S
strategy and budgeting two disciplines

that are widely criticized today.
HISTORICAL LESSONS APPLICABLE
TO STRATEGY EXECUTION AND
BUDGETS
In the book, The Art of Action, author Stephen Bungay reflects on lessons
from war and military campaigns that can
be applied to leadership skills and planning. He specifically addresses how an
organization can implement and achieve
the formulated strategy and plans of its
executive team.
Bungays premise is that the leaders of almost all organizations can
define reasonably good strategies.
Where executives often fall short is in
leading their organization to execute
their strategy. Bungay describes this
problem as a gap and advises how to
close the gaps.
His assertion is that, similar to military campaigns in war, when a strategy
encounters the real world, three types of
gaps appear. He describes gaps in terms
of expected results and reality, particularly

M A Y / J U N E 2 013
53
MAN AG EME NT SO F T S K I LLS
related to outcomes, actions and plans.

Gaps result from the complex and unpredictable environments that all organizations
deal with and are made more severe by globalization. The three gaps are:
The Knowledge Gap The
difference between what we would
like to know and what we actually
know.
The Alignment Gap The
difference between what we want
people to do and what they actually
do.
The Effects Gap The difference
between what we expect our actions
to achieve and what they actually
achieve.
Based on knowledge as a historian of
military practices, Bungay observes that
a key to successful strategy execution is
delegating more decision-making authority to managers and employee teams.
EMPOWERING MANAGERS AND
EMPLOYEE TEAMS
Bungay recounts lessons from
the 19th century Prussian army.
Following an unexpected military defeat, the Prussian military reformed

its tactics. Lower-level officers were
given more flexible command to
make decisions. What mattered was
that they fully understood the battle
mission. Allowing the officer corps
to make more decisions resolved a
problem: The higher-ranked military
leaders were farther from the battlefield and less aware of the current
situations. Officers could pursue local actions as they saw fit.
The Prussian army solution was the
institutionalization of military genius with
centralized and elite generals, and increased accountability of the field officers
with rewards based on their performance
and outcomes. This reform was successful, and the army conquered other
countries.
In terms of todays managerial methods, the parallels of the Prussian army
reforms are the application of balanced
scorecard methodology and the adoption
of Beyond Budgeting concepts, first
written about by Robin Fraser and Jeremy Hope.

54
The balanced scorecards primary

feature is the development of a strategy map that visually displays a dozen or more cause-and-effect-linked
strategic objectives. Using four sequenced components (referred to as
perspectives), the linkages move
from employee learning, growth and
innovation to process improvement
initiatives to customer loyalty objectives, which all impact the outcome of
financial objectives. The KPIs reported
in the balanced scorecard are derived
from the strategy map. The KPIs monitor the progress toward accomplishing
the strategic objectives and, with each
KPI assigned targets, the foundation
for accountability is established and
alignment with the mission and strategy is achieved.
The Beyond Budgeting concept
views the annual budget as a fiscal exercise done by accountants that is disconnected from the executive teams
strategy and is usually insensitive to
forecasted volume and product/customer mix. It acknowledges that budgeting annual line-item expense limits
are more like shackling handcuffs to
managers; they may need to justifiably
spend more than was planned for and
approved many months ago in order
to take advantage of newly emerged
opportunities.
A NA L Y T I C S
This method advocates abandoning

the annual budget, which quickly becomes obsolete. It proposes removing
the budgets controls by giving managers the freedom of decision rights, including hiring and spending decisions
without requiring approval from superiors. It invokes controls by monitoring
non-financial KPIs against the targets defined by the executives in the
balanced scorecards strategy map.
Managers do not escape accountability, and there are consequences. The
time frame is not annual, but rather
dynamic.
HISTORIANS VS. FUTURISTS
The message here is not that organizations shouldnt be researching emerging and imminent new technologies and
methods, such as analytics and big data.
The message is that granting decision
rights to managers but holding them
accountable with consequences is effective at closing the three gaps. And this
is a lesson learned from historians.
Gary Cokins (gcokins@garycokins.com) is
the founder of Analytics-Based Performance
Management LLC, an advisory firm. He is an
internationally recognized expert, speaker and
author in advanced cost management and
performance improvement system. He previously
served as a principal consultant with SAS. For
more of Cokins unique look at the world, visit
his website at www.garycokins.com. Cokins is a
member of INFORMS.
M A Y / J U N E 2 013
55
TEX T AN ALY T I C S
Lost in translation
Part one of two-part series on best practices for
analyzing multi-lingual text.
BY CHRISTOPHER BROXE (LEFT)

AND FIONA MCNEILL
The phrase, lost in translation takes on a special

meaning when it comes to
text analysis. Nuances in
language can indicate the homeland of
the author, like housecoat or bathrobe
and pop or soda, even with the same
mother tongue. However, when text is
examined across different languages the
same phrase can have altogether different meanings. For example, a Swedish
tourist asking snlla, kr mig till ett roligt stlle of a taxi driver in Copenhagen,
56
Denmark (i.e. Please take me to a fun

place) might be driven to a nearby cemetery because roligt in Danish means
a calm, peaceful place. Other languagespecific idioms abound, such as the
Swedish phrase Ingen ko p isen
(meaning no worries) will inevitably be
translated into no cow on the ice. Likewise, the common English phrase, Its
raining cats and dogs, is bound to be
problematic once translated.
Translating to a common language
can at best confuse the meaning and at
Figure 1: Defining the Web page structure with an interface can be done without knowing the
language.
worst completely butcher the authors intentions. Drawing conclusions from text
that is translated from one language to a
common language can lead to misguided results and inaccurate conclusions.
So what do you do? How can you make
conclusions from texts that are written in
different languages?
Unstructured text retrieval in a native
language can readily be done, even without knowing the language that the text
is written in. Analysis is best done by a
native language speaker who can train
and refine statistical and linguistic models to appreciate inherent meanings and
preserving the authors intentions. The
outputs of such models can then readily
A NA L Y T I C S
be consolidated into one common language: results.

This first of a two-part series describes a method for multi-lingual document retrieval, illustrated with examples
using SAS software.
HOW TO ADDRESS MULTI-LINGUAL
CONTENT
Analytically based information retrieval lets you create distinct content streams
(a.k.a. pipelines) for any language, site,
source or combination thereof in such a
way that each stream contains instructions related to specific language content. So while the text analysis models
are uniquely defined to the language
M A Y / J U N E 2 013
57
associated with the text in order to preserve the meaning such as the expressed sentiment, concepts and facts
[1] the application of multiple pipeline
models to documents written in different
languages can be done by any non-native speaker or even by machine.
Given that Web pages are defined
in the universal language of HTML, one
can readily identify the body of text,
title, author and other core elements of
a page generically and simply direct
the software to retrieve the desired section of the HTML. Figure 1 illustrates the
results from a point-and-click interface
used to defined desired HTML (or XML)
fields that would be retrieved from a Web
page when it is crawled [2]. As you can
see, although the language used in the
body of the text is Arabic, the author and
title follow a standardized format so even
without knowing what the text says, the
structure of the page can be readily defined to different aspects of the content.
XPath expressions [3] are the query
language that typically operates behind
such software interfaces, forming the
code base for retrieval activities. And
given that Web pages can readily change,

it is worthwhile to define the XPath expressions at the most generic level possible.
For example, changing the XPath expression: /html/head/meta[14]/@content to
/head/meta[@name=author]/@content
will ensure you will always capture the author, even if the author is not present as
the 14th element of the meta definition.
After tweaking the XPath expressions
to be as generic as possible, a marked
up template to an Arabic Web page is
created without needing to understand a
word of Arabic, as illustrated in Figure 1.
The same can be done in any language
although it is important to keep in mind
that for the analysis of text data that is,
when focus turns from retrieving the text
to understanding its meaning a native
language speaker is highly valuable to
ensure that the meaning intended within
that specific language is identified and
understood [4].
The ability to easily template a Web
page with markers defining the content
can be done for multi-lingual tweets as
illustrated in Figure 2. XPath expressions
are used to identify different aspects of

58
Figure 2: Arabic (on the left) and English (on the right) tweets, marked up to identify the content
components.
the tweet that are of interest and these

become metadata that is added to the
document once it is retrieved by the
system.
Most NLP-based retrieval systems
will include a built-in facility to detect
language so the appropriate markup
template is applied to any given input
document, Web page, tweet, etc.
The definition of the HTML file or XML
file (i.e., the different fields that are of
interest and which have been identified
in the XPath expressions) can be saved
and included as an early document
A NA L Y T I C S
processing activity that is executed as part

of content stream/pipeline processing. If
your main interest is to crawl somewhat
structured information, for example .pdf,
.doc or .txt files, then this markup definition step isnt necessary given that such
document types are fairly standardized
and most technologies are predefined to
identify the respective content elements.
Many files on the Web are actually fairly
structured according to pre-defined standards and, if this is the case, the technology can automatically detect the body
content of that file. A HTML/XML markup
M A Y / J U N E 2 013
59
Figure 3: Reviews from Tripadvisor.com page in both Swedish and English.
facility is simply an alternative that can

be used when the Web pages are very
unstructured and the exact location of
certain items is desired. Furthermore,
if unstructured pages are repeatedly
crawled, creating customized templates
that can be reused for every page language becomes a valued time saver.
That being said, other situations
where template markup facility can be
very useful include discussion forums
60
and review sites that have more than

one comment on each page and in more
than one language. As illustrated in Figure 3, a page from the Tripadvisor.com
site includes both Swedish and English
reviews of hotels.
Tripadvisor.com includes commentaries from travelers from around the
world. If we were to simply crawl this
page, wed retrieve a confusing review that would need to be untangled
Figure 4: Swedish hotel reviews are uniquely identified using a markup template.
by data processing. In fact, there are

actually many different reviews on this
page/URL, so rather than creating additional data processing work, each
one of these reviews can be separated
into different, unique documents as
the site is being crawled. This is also a
requirement for downstream text analysis, as each review would need to be
contained as a distinct element to be
analyzed. In Figure 4, we can see how
a markup facility can distinguish between the different reviews and identify
unique documents, one for each commentary about the hotel, thereby formatting the content for text analysis.
As shown in Figure 4, 10 different document bodies exist, one for each of the
A NA L Y T I C S
Swedish reviews contained on the Web

page. Ensuring that each is expressed
uniquely is important to the text analysis that would decipher the concepts,
themes and sentiment contained in the
reviews. Once the desired content has
been defined, a series of document processing steps are employed to generate
the desired output from system crawls.
Note that such crawls can be external
system crawls from the Web or internal
M A Y / J U N E 2 013
61
file system crawls, such as intranet or internal social platform retrieval activities,
or a combination of both. Some steps
that might be included in the complete
document processing routine include:
markup matcher used to identify
the desired content from the pages
(created using XPath expressions)
extract_abstract puts text of
varying lengths into a field called
abstract so that a quick overview of
the document is created
add field (occurring three times)
a placeholder for user-defined
fields used for downstream related
processing [5]
language identification used to
to store the automatically detected
language as a field for any type of
filtering activity
export to files used to retain the
exported XML (or text) for access as
training documents for downstream
text analytics processing such as
building sentiment or categorization
linguistic rules
filter used, in this example, to
ensure that the analysis of the
documents is occurring in the native
language [6]
send a final step, this last
document processor directs the
resultant document to matched (via
filter) instances of the text analytics
62
engine for identifying and extracting

concepts, sentiment, facts, etc.
These steps would retrieve text in
the native language format, isolate
them as distinct files and identify the
language retaining the intended meaning of the text. We typically retrieve
such files in order to analyze their contents. How that is done in this framework is the subject of the second article
in this series.
Christopher Broxe is a business solutions manager
and Fiona McNeill (Fiona.McNeill@sas.com) is a
global marketing manager at SAS.
NOTES
1. Text analysis models and how they can be
included in this processing methodology is
addressed in the second article in this series,
Processing multi-lingual text for insight
discovery.
2. The SAS Crawler contains a Markup Matcher
facility for point-and-click definitions of text
structure.
3. XPath code can automatically be generated
from point-and-click activities of the user.
4. The SAS text analytics technologies natively
support an extensive number of languages.
5. An example of a useful field to create for
this type of content would be the type source,
such as News for documents from the BBC or
Aljazeera or Microblog value for tweets from
Twitter.
6. Given that the language identification
processor outputs a field called language,
a filter can readily be created to say
language=ARABIC. It may be that a tweet in
Korean or Polish is captured, so this filter would
ensure that only Arabic text analytics processing
is done on Arabic documents and not to the
Korean ones.
ANALY TIC S H O N O R S
INFORMS presents
practice-oriented awards
The Institute for Operations Research and the Management Sciences
(INFORMS), the worlds premier organization for advanced analytics professionals, presented several practice-oriented
awards at the INFORMS 2013 Conference on Business Analytics & Operations
Research in San Antonio, Texas, in April.
The list includes the Franz Edelman
Award, the INFORMS Prize and the UPS
George D. Smith Prize, as well as the
Daniel H. Wagner Prize (administered by
CPMS, the practice section of INFORMS)
and the Innovative Applications in Analytics Award (presented by the Analytics

Section of INFORMS).
FRANZ EDELMAN AWARD
The Dutch Delta Commissioners office won the 2013 Franz Edelman Award
for Achievement in Operations Research
and the Management Sciences. Using
advanced analytics techniques including
simulation, the award-winning team identified three critical, dike-protected areas to
bolster the Netherlands flood protection
system while simultaneously saving the
country nearly 8 billion in unnecessary
expenditures.
Delta Commissioner Wim Kuijken congratulated
the team on winning this important,
prestigious award.
The Edelman-winning team included the Dutch Delta Commissioner of
Almost 60 percent
Holland, CPB Netherlands Bureau for Economic Policy Analysis, Delft
University of Technology, Deltares, HKV Consultants, the Ministry of
of the Netherlands
Infrastructure and the Environment (The Hague) and Tilburg University. is vulnerable to
64
flooding, he said. The objective of the

Dutch Delta approach is not to respond to
a flood disaster but toavoid it. This innovative project has contributed significantly
to improved flood protection in the Netherlands while at the same time steering
public funds in the most efficient way.
The Delta Commissioner of Holland
collaborated with CPB Netherlands Bureau for Economic Policy Analysis, Delft
University of Technology, Deltares, HKV
Consultants, Ministry of Infrastructure
and the Environment, The Hague and
Tilburg University on the winning submission, entitled Economically Efficient
Flood Standards to Protect the Netherlands against Flooding.
The Franz Edelman competition, now
celebrating its 42nd year, attests to the
contributions of operations research and
analytics in the profit and non-profit sectors. Since its inception in 1972, cumulative dollar benefits from Edelman finalist
projects have reached $190 billion.
Protection against flooding is a vital
issue in the Netherlands since more than
half of the country is either below sea level or susceptible to flood risk. Each year
the Dutch government spends about 1
billion on protection by dikes and dunes.
About 3,500 kilometers of primary dikes
protect the Netherlands from flooding.
Initially, the countrys second Delta
Committee recommended increasing
A NA L Y T I C S
all protection standards by at least a

factor of 10, a costly step given limited
funds. Using operations research techniques, the team determined that it was
efficient to limit increased standards
to only three critical regions. These
results, accepted as a basis for policy
by the minister of Infrastructure and
Environment, led to 7.8 billion less investment costs while strengthening the
countrys defense against the kind of
devastation that cost massive damage
in the United States during Hurricane
Sandy and resulted in thousands of
deaths in Holland during a major storm
in the early 1950s.
The other finalists in the 2013 Franz
Edelman Award Competition included
teams from Baosteel, Chevron, Dell,
Kroger and McKesson.
For more information, click here.
INFORMS PRIZE
INFORMS awarded its annual INFORMS Prize to Ford Motor Company,
the automaker that has been using analytics throughout its organization to
strengthen its position within the American automotive industry.
Analytics and operations research
(O.R.) was a major enabler of our turnaround and our ongoing success as a
data-driven company, says Bob Shanks,
executive vice president and chief
M A Y / J U N E 2 013
65
Fords focus on
data-driven
decisions was reinforced
with the arrival in
2006 of CEO Alan
Mulally, who guided
the companys restructuring.
Analytical tools and
Ford Motor Company earned the INFORMS Prize for its effective use of
O.R. in organizational decision-making.
the operations research team supfinancial officer, Ford Motor Company.
ported many decisions in this period, and
Receiving the INFORMS Prize is recoga number of critical applications were denition of the significant role and impactof veloped, including:
analytics at Ford.
a dealer vehicle recommendation
The INFORMS Prize Committee
system,
noted that, Ford not only perfected the a detailed econometric model
moving assembly line, but it also brought
enabling the study of what-if
O.R., management science and financial
analyses of inventory, production,
discipline to the company early on, helppricing and sales, and
ing to transform it into a data-driven and
a strategic sourcing model to
successful modern organization.
restructure the Ford auto interiors
The INFORMS Prize honored Ford
division.
for its effective integration of operations
The prize committee characterized
research into organizational decisionthe level of engagement by senior execmaking and for repeatedly applying the
utives and support of the analytics work
principles of O.R. in varied, novel and
inside the company as impressive.
lasting ways.
UPS GEORGE D. SMITH PRIZE

66
INFORMS awarded its 2013 UPS

George D. Smith Prize to the Naval Post
Graduate School, located in Monterey,
Calif.
The UPS George D. Smith Prize is

awarded to an academic department
or program for effective and innovative
preparation of students to be good practitioners of operations research.
The Naval Postgraduate School
trains its students in making important
line and policy research that is meant
to ensure military preparedness, proper military planning, strong execution
during wartime, and overall enlightened policy planning, said Donald
Smith of Monmouth University and
A NA L Y T I C S
prize committee chair. Its graduates

help make important decisions at multitude of levels. The program is unique
in its strong and inexorable link to its
sponsor, the U.S. Department of Defense, which keeps NPS students and
faculty focused on applying methods to
solve problems facing the military and
other diverse branches of government,
with a focus improving operational efficiency and effectiveness.
The Lehigh Universitys Enterprise
System Center and Department of
M A Y / J U N E 2 013
67
Garrett Meyer and Kathryn

Moretti of the U.S. Coast
Guard and Fernando Ordonez of Universidad de
Chile.
Every day the U.S Coast
Guard faces challenges of
evolving asymmetric threats
in the Maritime Global ComRandy Stashick (second from left), global vice president of
engineering at UPS, presents the UPS George D. Smith Prize to mons and also within the
the Naval Postgraduate School.
ports and waterways that
make up the United States
Industrial and Systems Engineering and
Maritime Transportation System.
the Massachusetts Institute of TechnolThis research addresses a nationally
ogys Leaders for Global Operations Proimportant problem, the modeling of which
gram were finalists.
is quite complex, says Allen Butler, presFor more information, click here.
ident of Daniel H. Wagner Associates
and chair of the competition committee.
DANIEL H. WAGNER PRIZE
The team has provided a superior soluThe 2012 Daniel H. Wagner Prize
tion using innovative mathematical ap(officially announced in November 2012)
proaches, including a response model to
was presented to the U.S. Coast Guard
allow for potentially irrational behavior on
and its research partner, the University
the part of the adversaries. They deserve
of Southern Californias Viterbi School of praise for the impact of this work and the
Engineering. The winners used analytics
originality of their methodology.
and operations research to ramp up proThe objective of the work was to detection of Americas ports and waterways
velop and deploy a decision support
from unpredictable terrorist attack.
system, known as PROTECT (Port ResilA Deployed Quantal Response
ience Operational/Tactical Enforcement
Based Patrol Planning System for the
to Combat Terrorism), to aid the U.S.
U.S. Coast Guard is by Bo An, Milind
Coast Guard in its mission. The system
Tambe, Eric Shieh and Rong Yang of the is based on an attacker-defender game
University of Southern California, Craig
model of the interaction between Coast
Baldwin, Joseph DiRenzo, Ben Maule,
Guard patrols and attackers.
68
Given the finite patrolling resources, an

optimal use of limited resources takes into
account the relative importance of the different targets protected and the presence of
adversaries that can observe security measures before deciding to attack.
Previous work on security games
assumes the human attackers are perfectly rational. PROTECT addresses this
limitation by drawing from what is known
as quantal response (QR) theory to represent the adversarys behavior. PROTECT has been successfully deployed
in the port of Boston and is currently undergoing testing at the port of New York,
with the potential for nationwide deployment. Results show that the model better handles real-world uncertainties than
a perfectly rational model and is able to
perform better than human generated
patrolling strategies.
INNOVATIVE APPLICATIONS IN
ANALYTICS AWARD
A team led by Cynthia Rudin of MIT
won the 2013 Innovative Applications
in Analytics Award from the Analytics
Section of INFORMS for the entry, Analytics for Power Grid Distribution Reliability in New York City, conducted in
conjunction with researchers from MIT,
Columbia University and Con Edison
(one of the largest investor-owned energy companies in the United States).
Mark Grabau of IBM and Elizabeth
Riczko of Westfield Insurance placed
second with their work, Insurance Agency Productivity and Prospecting, while
an all-GE Research team led by David
Toledano and Marcia Peterson finished
third for the entry, GE Uses Real-Time
Data to Forecast Near-Term Hospital
Throughput and Bed Needs.
The award criteria values innovating
application and integration of a range of
analytics techniques.
The prize-winning work the first major
effort to use analytics for preemptive maintenance and repair of an electrical distribution network is a large-scale, multi-year
effort between scientists and students at
Columbia and MIT and engineers from Con
Edison, which operates the worlds oldest
and largest underground electrical system.

70
FIVE- M IN U T E A N A LYST
Empty Inbox
How long should one

go without receiving an
e-mail before concluding
that something strange
has happened and call
IT support. The bloom of
spring always reminds
me of the beauty of the
Poisson process.
BY HARRISON
SCHRAMM
72
I arrived recently at my office on a Tuesday morning to find that there were no messages in my box
from the previous evening. Now, I had left the office at
about the same time that most of my colleagues (who
generate the lions share of my e-mail), and as I rarely
get e-mails from other time zones, I didnt think too
much about this. As 7:30 a.m. became 8 a.m., I began
to suspect that something was indeed wrong with my
e-mail. Our question today is: How long should one
go without receiving an e-mail before concluding that
something strange has happened and call IT support.
The bloom of spring always reminds me of the beauty
of the Poisson process, and heres the result:
The simplest model for arrivals (of any sort) is to just
say that they arrive as a Poisson process, with a fixed
rate parameter, , and consequently, the distribution of
inter-e-mail arrivals is exponential. We can find the critical time for a given confidence level
by solving the
equation
. While straightforward, there are a
few devils lurking in the details. In a typical nine-hour day
(including lunch), I receive around three e-mails per hour,
so
, and I will declare that my inbox is broken if I
go longer than roughly one hour without e-mail. This is a
simple model, and it is probably a bit too simple for most
applications. Lets explore some refinements:
1. The Markov property assumes lack of

memory and independence. However,
our experience lets us know that the emails that arrive in our inboxes can be
highly correlated. For example, an email from our boss tasking us and a coworker for a project can lead directly to
other e-mails from co-workers. Additionally, if I send e-mails to a large group of
people, I may get several out of office
replies, which themselves may be evidence that my e-mail is working properly.
You may partially control for correlation
A NA L Y T I C S
not considering the actual e-mails, but

rather the conversation; Gmail does this
automatically, Outlook does not.
2. E-mail has strong time-of-day effects.
The peak times for e-mail (as determined
by an informal survey) are at the beginning of the day, immediately after lunch
and immediately before quitting time. The
lulls in the e-mail traffic are during lunch,
the mid-afternoon circadian trough. There
are also strong day-of-the week effects
as well, with Monday mid-morning being
M A Y / J U N E 2 013
73
FIVE- M IN U T E A N A LYST
a big e-mail time. Now, if we observe our

e-mail box at the cutoffs between these
events, we are OK. However, we run into
difficulty if we take a long lunch, for example, and are away for parts of two periods We can revise our model by moving
from the basic Poisson process to a nonhomogeneous Poisson Process, where
we replace the fixed rate parameter, ,
with a rate function
and the probability that the next inter-arrival time will
be greater than T is
. For
example, if I get two e-mails per hour between 8 a.m. and 10 a.m., and one e-mail
per hour between 10 a.m. and noon, then
,
and if I want to know the probability that
we receive zero e-mails between 9 a.m.
and 10:30 a.m., we will find the answer is
approximately 8 percent.
3. The aforementioned e-mail arrival distribution is also useful if you are trying to
escape from your desk to visit the gym
without feeling like you have missed
something important.
4. There is strong dependence between

e-mail accounts. For example, if your
server has failed, then not only will you
not have e-mail, but many of your colleagues will not either. Depending on your
arrangement, this can be correlated by
job title, last name, geographic location or
some other factor. Additionally, you might
consider your electronic communications
in total; after working hours, the amount
of your e-mail on your company account
may dwindle, but it is made up for by increased traffic on your personal account.
Finally, a word on the selection of
.
Many statistics students (and professors!)
blindly use a value of .05, which implicitly
means that they will be wrong 1 in 20 times.
However, in the case of calling the IT department, this really depends on your relationship with the IT folks, how many times
you have called before, whether you want
them to mine your account for the problem,
and whether you think you caused the problem yourself.
Harrison Schramm (harrison.schramm@gmail.com) is
an operations research professional in the Washington,
D.C., area, and is a member of INFORMS.

74
THIN K IN G A N A LY T I CA LLY
Submarines vs. battleships
Figure 1: Where should you move the

friendly submarines?
Naval warfare is a complicated undertaking due

to the varying capabilities and vulnerabilities of both
friendly and enemy ships. Deciding who should attack who is a critical decision that can determine the
outcome of the battle.
Figure 1 shows a map of 15 blue, friendly submarines and 15 red, enemy battleships. Your goal is to
move each submarine so that it occupies the same
cell as a battleship. When a submarine occupies the
same cell as a battleship, the battleship is destroyed.
Each submarine can only destroy one battleship. Battleships cannot move.
Use Pythagorean theorem to calculate the distance between cells. For example, the distance between cells A1 and B3 is 2.236 km.
QUESTION: What is the minimum total distance
the submarines need to travel to destroy all of the
battleships?
Send your answer to puzzlor@gmail.com by July
15. The winner, chosen randomly from correct answers, will receive an Analytics Driving Better Business Decisions T-shirt. Past questions can be found
at puzzlor.com.
BY JOHN TOCZEK
76
John Toczek is the director of Decision Support and Analytics for

ARAMARK Corporation in the Global Risk Management group. He
earned his bachelors of science degree in chemical engineering at Drexel
University (1996) and his masters degree in operations research from
Virginia Commonwealth University (2005). He is a member of INFORMS.

Analytics MAYJUNE 2013

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Analytics MAYJUNE 2013

Hochgeladen von

Copyright:

Verfügbare Formate

H T T P : / / W W W. A N A LY T I C S - M A G A Z I N E .

DRIVING BETTER BUSINESS DECISIONS

Historians vs. Futurists

INS IDE STORY

Dutch project earns Edelman

begins with a nomination process, continues

PETER HORNER, EDITOR

DRIVING BETTER BUSINESS DECISIONS

36 CASE STUDY: MARKETING ANALYTICS

56 TEXT ANALYTICS: LOST IN TRANSLATION

Despite all the value-add that

DRIVING BETTER BUSINESS DECISIONS

REGISTER FOR A FREE SUBSCRIPTION:

ANALYTICS EDITORIAL AND ADVERTISING

Winning the war with

From 1941 to 1943, a small group of British and

A well-worn story I like to tell is of a

Help Promote Analytics

I cant say I was there to witness the

Course puts students in the

This past semester, I taught a class for 2nd year

the consulting process) and for myself

the authors goals: We would like you to

About halfway through the book,

perspective, is a difficult sport to understand from the sidelines.

Analytics Consulting Projects course,

Problem solving with data

Imagine a world where

McKinseys Big Data report, published in May

enable us, as marketers, to be the best

Many of us base our

feel of while still making sure it met his

Join the Analytics Section of INFORMS

For more information, visit:

choose? If you focus on engagement,

you believe to be the best. Just like in

that your organic acquisition is at par with

Today, many business professionals

Request a no-obligation INFORMS Member Benefits Packet

INFORMS confers first

score. The passing rate of those who took

applicant pools looking to find talent. And

Thirty-eight earn CAP

Alan L. Austin (Provo, Utah)

UPG RADE T H E C O R PO RAT I O N

BY RANDY BARTLETT (LEFT) AND GIRISH MALIK

Welcome to the Business

by hyper-connectivity and real-time decision-making. It all begs the question:

Figure 1: Report card.

UPG RADE T H E C O R PO RAT I O N

BA requires people to change and requires

best possible use and feed into optimal

UPG RADE T H E C O R PO RAT I O N

Expand the tool set: We need to start by

decades, researchers have enhanced

UPG RADE T H E C O R PO RAT I O N

MEASURING BUSINESS ANALYTICS

insight. A proper measurement must be

Table 1: Business analytics maturity model.

Thought Given to Location

Cosmetic support for BA

Mid-level Advocate within

Leverages analytics when