Granular Metrics,

Feedback and Experimentation

Arnoldo Hax
Alfred P. Sloan Professor of Management
Contributions of the Delta Model

Opening the The Best Three Strategic Options:

mindset to new Product does z Best Product
The Triangle strategic not always win
position z Total Customer Solutions
z System Lock-in

Linking Execution is Execution is captured through Three

Strategy with not the Adaptive Processes:
Adaptive Execution problem, z Operational Effectiveness
Processes linking to
z Customer Targeting
strategy is
z Innovation
whose roles need to change to achieve
different Strategic Positions
Measuring Good Aggregate performance metrics need to
success financials do reflect each of the Adaptive Processes
Aggregate not always and their role based upon the strategic
Metrics lead to good position
results z Product performance
z Customer performance
z Complementor performance
Granular Discovering Managing by Business is non-linear. Performance,
Business is non-linear. Performance,
Metrics and performance averages particularly bonding, is concentrated.
drivers leads to below Granular metrics allow us to focus on
Feedback average underlying performance drivers, to
performance detect variability, to explain, to learn, and
to act.
 The Delta Model and Granular Metrics
Metrics

Select Performance Indicators

Act from Variability Detect Variability

Define program Drill down to detailed

Define structured tests segmentation
Rollout Isolate variability

Learn from Variability Explain Variability

Hypothesize cause and Identify performance
effect drivers
Evaluation Correlate drivers with
variability
Drivers of variability for selected performance indicators
Performance indicators Drivers of variability
Product cost and quality z Scale
z Density-
Density-e.g. concentrat
concentration of serv
service
ice
z Location
cation
z Labor productivity-
productivity- practices and training
training
z Equipmen
Equipment product
productivity
ivity-- design
design
z Process design
z And
And so on
Customer profit z Customer size
z Customer revenue
z Tenure
z Acquisition cost
z Channel
Channel mix
z Customer care supp
support
z Customer investments in relation
relationshi
ships
z And
And so on
Complementor contribution z Size of releva
relevant complementor products
z Complementor investment in bu business
z Relative size in customer value chain
contribution tto
z Complementor contributio o customer economics
z Exclusivity of relationship
relationship
z And
And so on
Business segment economic z ROI (return on
on investment) profitability
value z Risk-
Risk- volatility and covariance
covariance
z Option value
z Investment base
z Cashflows
Cashflows
Experimentation as the basis for effective change
change

The middle road: Unacceptable risk,

risk,

Large lower returns, or as a starting point.

point.

unacceptable when Highly desirable as

as

first mover endpoint

endpoint

advantage is high
Size of
Change
Ineffective: lower Testing: the
returns, or relevant area for
unacceptable when experiments
first mover leading to large
Small advantage is high change

Slow Fast
Speed of
Change
 Cost De-Averaging:
Using Granular Metrics To
Drive Performance
 1. The Value Chain of the Local Business Data Circuit

Order
Switching Transmission Billing Marketing Etc.
fulfillment

Average Cost = $395/order

2. Individual Order Cost 3. The 80-20 Cost Effect

$1,000 100

80
Cost/order

% of costs
60

40

20
$100
Most Orders Least
Expensive Expensive 0 20 40 60 80 100
% of orders

THE BEHAVIOR OF COST - THE CASE OF BUSINESS DATA SERVICES

Figure by MIT OCW.

The drivers of the cost of order-fulfillment
fulfillment

Cost of order
fulfillment

Location of Activities of Manpower

Equipment
work centers order fulfillment productivity
 Order Fulfillment

70
Competitor 1 - Houston Company
Headcount required to process

Competitor 1 - Chicago Competitors

1,000 orders per month

New York Client performance

}
Chicago curve
Tulsa Charlotte
39%
Best-in-class
gap
Competitor 1 performance
Denver Scale = 72%
Competitor 1 Kansas City Log/log graph
San Francisco
Competitor 2
San Jose Competitor 3
Competitor 2 -
Jacksonville
1 Indianapolis
1,000 2,000 5,000 12,000 27,000 60,000

Orders/Month

T HE BE H AV I O R O F CO S T BY L O CAT I O N

Figure by MIT OCW.

 Customer
ready
Pass access
test $100 (per
Fast access leg)
'Defect-free Path' facilities test
Automatic
switch map
Valid design

1 MILLION POSSIBLE ORDER PATHS

Access
available Pass test
Facilities
available
Access
not
available
Order 10x cost
difference
Access
available
Facilities
not
available Access
not
available Skin test
Invalid design

Manual
'Defect-prone Path' switch map $2,000 (per
Fail
facilities
access leg)
Fail access
test test
Customer
not ready

THE BEHAVIOR O F C O ST B Y FIV E A C TI V I TIES IN TH E VAL UE CHAI N

Figure by MIT OCW.

The behavior of cost by type of defects
defects

Client

Averag
% of Cost/ e cycle Total
orders order time cost
Defect-free 71% $100 5 Days 18%

Business Data
24.0% $900 25 Days 60%
Service Orders
Focus of
Business Data program
Service Orders developmen
t
Business Data 5.0% $1,800 45 Days 22%
Service Orders

Average 1 $395 12.3 Days

 Craftsmen Efficiency
10
Total hours/cleared fault

8 Fault not fixed correctly

& repeats
6 S:1
Pair throw: reroute around
fault
4 Request assistance
Refer fault to someone else
2 (specialist)
Fix fault
0
A B C D E F G H I J K L M N
Craftsmen

100
Cumulative cost (%)

80

The Need for

Granular Metrics

0
0 25 100
Cumulative craftsmen (%)

THE BEHAVIOR OF COST BY INDIVIDUAL WORKERS

Figure by MIT OCW.

 Line Fault Concentration Records Errors Concentration

100 100

80 80

% of Records Errors
100% of demand pair
% of Line Faults

100% of records
60 reclamations are 60 errors are concentrated
concentrated in 17% in 19% of equipment
of junctions
40 40

20 20

0 0
0 20 40 60 80 100 0 20 40 60 80 100
% of Junctions % of Equipment

THE BEHAVIOR OF COST BY TYPE OF EQUIPMENT

Figure by MIT OCW.

 Benchmarks
Cost/Unit

Cost Drivers

Best Practice Sessions Director/Process Reviews

Standard
Select
Supervisor
Associates
a b

Individual Performance

Individual Reviews Supervisor Review District/Center Reviews

Ind. Actual Target Supervisor Actual Target
Standard
- -
- -
- -
- -
a b c Total Total
Supervisor 'Supervisor Coaching'

OPERATIONAL EFFECTIVENESS: ADAPTIVE FEEDBACK USING GRANULAR METRICS

Figure by MIT OCW.

 Profit De-Averaging:
Using Granular Metrics To
Drive Performance
 THE NEED FOR GRANULAR METRICS

3x
2x
1x 35
Cumulative percentage of customers 85% of Profits
0
10 20 30 40 50 60 70 80 90 100
Profit Margin

-x 46% of Profits
-2x
40% of Profits
-3x
21% of Profits
-4x
-5x
-6x
-7x

P RO FIT MA RG IN CO N T RIBU T ION

Figure by MIT OCW.

The drivers of customer profitability in the credit card business
s

Profit by customer

Revenue per Number of Balance level Months of

customer customers per month tenure
 75th Percentile +
1.5* Box Size ('Upper
$70 limit of connected data')
$60
Monthly Profit / Customer

$50
$40 75th Percentile
$30
$20
Median
$10
$0
$(10) 25th Percentile
$(20)
25th Percentile -1.5*
$10-$100 $100-$200 $200-$400 $400+
Box Size ('Lower limit
Revenue/Customer of connected data')

VA R I A B I L I T Y O F P R O F I T S B Y U S A G E L E V E L S

Figure by MIT OCW.

 Company Scale Customer Share Customer Tenure

30 30
Profit Margin (ROS)

20 20
10 10
22% 0 0
18% -10 -10
15% -20 -20
-30 -30
-40 -40
-50 -50
5MM 10MM 20MM 30MM $1000 $2000 $3000 $4000 10 20 30

Number of Customers Balance Level $/Month Months of Tenure

FURTHER E X A MI N AT I ON O F P R O F I T DR IVER S

Figure by MIT OCW.

 }
5x

Tailor offers to individual customers

(i.e., increase response & profit with differential
offers tailored to customer's preferences)
Cumulative Profit

}
3.5x
Offer one product to customers, based on
both response & profitability
(i.e., acquire customers likely to have good volume/
margin/lifetime characteristics)

}
x
Offer one product to those who will respond
(i.e., maximize response for a given offer, assuming average economics)
0
150,000 300,000
} Offer one product to
everyone
Cumulative Number of Customers Solicited Untargeted solicitations
yield a negative profit

T H E E F F E C T S O F C U S TO M E R TA R G E T I N G

Figure by MIT OCW.

 Customers targeting behaviors
behaviors

Behaviors Typical competitors Capital One

z Offer 75 products 5,000 products
z Test/year 30 15,000
z Service tier 4 13
z Key metric ROE, Delinquency NPV
z Targets High response likelihood High NPV potential
z Solicitations/year 50 million 400 million
z Segmentation frequency Biannual Ongoing
z Risk orientation Risk averse Price for risk
z Data sources 80 50,000
z Staff analysts 20 150
Capital One- achieving a Total Customer Solutions through
Customer Targeting

Variation 2
Variation 1
Select Further Offer Core offer
Offers improvement Creation

Interest
Interest rates
rates
Fees
Promotions

Mass
marketing
Full life
Screen for Variation 2
cycle customer
profitability profitability Variation 1
Test cells

Collect
customer
response
 Success metrics
metrics

Typical competitor Capital One

z Customer NPV $62 $427
z Acquisition cost $77 $67
z Activation 58% 79%
z Percent with revolving balances 25% 40%
z Fee income per account $55 $73
z Charge-off per account $71 $54
z Tenure 4 years 6 years
 Value Created by Business Unit
60
100% of
42% more value -21% of value created -21% of value created
value
created
% of Total Company Value Created
created
20

15

10

0
i) 24% of
-5 i) 20% of new i) 36% of new capital i) 20% of new capital
new
capital ii) 25% of R&D ii) 22% of R&D
capital
-20 ii) 32% of R&D
ii) 21% of
R&D
-40
0 25 50 75 100
% of Equity Investment

1) 6 BUs contribute 100% of value created

2) 16 more BUs contribute another 42% of value created
3) Those 22 BUs are worth $ 15.5 billion, on an equity investment of only $4.2 billion
4) 44% of new capital investment and 53% of R&D spending over the next three years is going to BUs
earning economic profits
5) 56% of new capital investment and 47% of R&D spending are going to BUs that will be generating
economic losses

SOURCES OF VALUE CREATION

Figure by MIT OCW.