Beruflich Dokumente
Kultur Dokumente
Spring Term 2, 2009‐2010
Identifying Important Uncertainties
(Tornado Charts, SensIt)
1
Today we will describe a tool for identifying the "most
important" uncertainties in a problem.
Iteration
Source: Strategic Decisions
Monte Carlo Group. Used with permission
Simulation
• "Tornado charts" are used in decision tree and Monte Carlo analysis (as
well as other contexts) to focus attention and effort on the key
uncertainties.
2
Example: Introducing Gatorade in China
• Gatorade was Quaker Oats' largest brand worldwide; acquired by Pepsi in
2000.
• Gatorade has a strong presence in the U.S. and Canada and a growing
presence in other parts of the world: Latin America, Europe, and now Asia.
• Four flavors of Gatorade (Lemon, Orange, Peach, and Grapefruit) are sold
in glass bottles and cans in China.
• “Some estimates show as many as 300 million of the 1.3 billion people
here could afford the likes of Crest toothpaste and Gatorade.” WSJ
1/3/2000. But the number could be much lower.
• Coke costs a quarter: Can they charge 50 cents for Gatorade?
3
An Economic Evaluation Model:
Financial Assumptions:
Average price/unit $0.50
Inflation rate 5.00% per year
Per unit materials $0.15
Labor FTEs per million units 5
Labor rate $8,000 per year
Shanghai facility rental $1,000,000 per year
Marketing--Variable 20% of revenue
Marketing--Fixed $850,000 per year
SG&A Variable 20% of revenue
SG&A Fixed $5,000,000 per year
Discount rate 10% per year
Gatorade in China Projections
All amounts in 000s
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Inflation factor 1.00 1.05 1.10 1.16 1.22 1.28 1.34 1.41 1.48 1.55
Unit sales 26,737 68,911 120,402 178,939 240,288 299,290 351,481 394,303 427,298 451,503
Revenue $13,368 $36,178 $66,372 $103,572 $146,036 $190,989 $235,509 $277,412 $315,657 $350,214
Cost of goods
Ingredients/ Bottling materials $4,011 $10,853 $19,911 $31,072 $43,811 $57,297 $70,653 $83,224 $94,697 $105,064
Labor $1,069 $2,894 $5,310 $8,286 $11,683 $15,279 $18,841 $22,193 $25,253 $28,017
Shanghai facility rental $1,000 $1,050 $1,103 $1,158 $1,216 $1,276 $1,340 $1,407 $1,477 $1,551
Gross profit $7,288 $21,380 $40,048 $63,057 $89,327 $117,137 $144,676 $170,588 $194,230 $215,582
Net income (loss) -$3,909 $767 $7,050 $14,856 $23,802 $33,275 $42,632 $51,392 $59,324 $66,421
NPV $142,704
4
The sales forecast is based on a standard new product diffusion
model, a variation on the Bass model.1
• Customers try the product and some of them become repeat customers; the base
case assumes 7% of "triers" become repeat customers.
o Trial purchases assumed to be of a single unit.
o Repeat purchasers are assumed to purchase 6 units per month.
• Potential market size = 100,000,000 people
• The fraction of the market who try the product for the first time each month is a
linear function of the number of people who have tried the product:
4.0%
3.5%
Proportion of population
3.0%
who tries in month
2.5%
2.0%
Slope = "Coef f icient of imitation" = 3%
1.5%
1.0%
0.5%
Intercept = "Coef f icient of innovation" = 0.5%
0.0%
0% 20% 40% 60% 80% 100%
Proportion of population that has alre ady trie d
1
Bass, F.M., "A new product growth model for consumer durables," Management Science (1969).
5
The diffusion model in equations:
Base Assumption Description
m 100 million Market size: the number of people that might try the product.
Coefficient of innovation: describes how quickly innovators will try
a .5% per month
a new product.
Coefficient of imitation: describes the strength of the word‐of‐
b 3% per month
mouth effect in creating new triers.
Fraction of trial purchasers in month k‐1 who convert to repeat
c 7%
purchasers in month k.
Average units per trial: the number of units a trial purchaser will
t 1
buy in a month.
Average units per repeat purchase: the average number of units a
r 6
repeat purchaser will buy in a month.
⎛ Y (k − 1) ⎞ Q1 (k) = tT (k)
T (k) = ⎜ a + b ⎟(m − Y (k − 1))
⎝ m ⎠
Q2 (k) = rcY (k − 1)
where
T(k) = number of trial purchasers in month k.
Y(k) = cumulative number of trial purchasers up to month k.
Q1(k) = sales to trial purchasers in month k.
Q2(k) = sales to repeat purchasers in month k.
6
Results from the diffusion model:
New product diffusion model
All amounts in 000s
100,000
45,000
40,000 Cumulative trial purchasers
35,000 75,000
30,000
25,000 50,000
20,000
15,000 25,000
Cumulative repeat purchasers
10,000
5,000
0
0
0 12 24 36 48 60 72 84 96 108 120
0 12 24 36 48 60 72 84 96 108 120
month M onthly trial purchasers month
7
Tornado Analysis: First, we assess base, low, and high values for
each uncertain input.
• Assess:
o Low values are 10th percentiles: there is a 1‐in‐10 chance that the true
value is below this amount.
o Base case values are 50th percentiles or medians: there is a 50‐50
chance that the true value will be above or below this amount.
o High values are 90th percentiles: there is a 1‐in‐10 chance that the true
value is above this amount.
• Consistency in the definition of the scenarios is required for comparability:
o Most people use 10th, 50th, and 90th percentiles.
o Some suggest using 5th, 50th, and 95th percentiles instead of 10‐50‐90s.
8
Rough estimates will do – we will refine the ranges for the
important uncertainties later.
• Ranges for the example:
Ranges
Input cell 10th %ile 50th %ile 90th %ile
Average price/unit $0.50 $0.30 $0.50 $0.60
Inflation rate 5.00% per year 0.00% 5.00% 15.00%
Per unit materials $0.15 $0.13 $0.15 $0.17
Labor FTEs per million units 5 3 5 7
Labor rate $8,000 per year $7,000 $8,000 $9,000
Shanghai facility rental $1,000,000 per year $900,000 $1,000,000 $1,400,000
Marketing--Variable 20% of revenue 12% 20% 25%
Marketing--Fixed $850,000 per year $750,000 $850,000 $1,000,000
SG&A Variable 20% of revenue 12% 20% 25%
SG&A Fixed $5,000,000 per year $3,500,000 $5,000,000 $6,500,000
Total potential customers 100,000,000 50,000,000 100,000,000 350,000,000
Coefficient of innovation 0.50% per month 0.01% 0.50% 1.00%
Coefficient of imitation 3.00% per month 1.00% 3.00% 10.00%
Percent trial purchasers => rep 7% 3% 7% 20%
Average units per trial purchase 1 1 1 3
Average units per repeat purch 6 3 6 25
Existing trials purchasers 800,000 650,000 800,000 1,000,000
9
Next we vary each input, one at a time, to see how changes
affect the bottom line.
• Set all non‐changing variables at their base value.
• Results summarized in a "tornado" chart:
SensIt 1.31 Academic
• Caveats: Cell ranges must be contiguous.
Output cells must be on the same worksheet as the input cells
No apostrophes in file or worksheet names.
11
SensIt Output ‐‐ Table:
NPV
Corresponding Input Value Output Value Percent
Input Variable Low Output Base Case High Output Low Base High Swing Swing^2
Average units per repeat purchase 3 6 25 $49,618 $142,704 $732,254 $682,636 38.0%
Total potential customers 50,000,000 100,000,000 350,000,000 $48,272 $142,704 $614,452 $566,179 26.1%
Percent trial purchasers => repeaters 3% 7% 20% $36,320 $142,704 $488,455 $452,135 16.7%
Average price/unit $0.30 $0.50 $0.60 ($68,569) $142,704 $248,341 $316,910 8.2%
Coefficient of innovation 0.01% 0.50% 1.00% ($16,568) $142,704 $198,596 $215,164 3.8%
Coefficient of imitation 1.00% 3.00% 10.00% $79,387 $142,704 $250,733 $171,346 2.4%
Inflation rate 0.00% 5.00% 15.00% $104,394 $142,704 $260,936 $156,542 2.0%
Marketing--Variable 25% 20% 12% $98,689 $142,704 $213,129 $114,440 1.1%
SG&A Variable 25% 20% 12% $98,689 $142,704 $213,129 $114,440 1.1%
Per unit materials $0.17 $0.15 $0.13 $107,492 $142,704 $177,917 $70,424 0.4%
Labor FTEs per million units 7 5 3 $114,535 $142,704 $170,874 $56,340 0.3%
SG&A Fixed $6,500,000 $5,000,000 $3,500,000 $131,545 $142,704 $153,864 $22,319 0.0%
Labor rate $9,000 $8,000 $7,000 $133,901 $142,704 $151,507 $17,606 0.0%
Average units per trial purchase 1 1 3 $142,704 $142,704 $157,692 $14,987 0.0%
Shanghai facility rental $1,400,000 $1,000,000 $900,000 $139,729 $142,704 $143,448 $3,720 0.0%
Existing trials purchasers 650,000 800,000 1,000,000 $141,783 $142,704 $143,923 $2,140 0.0%
Marketing--Fixed $1,000,000 $850,000 $750,000 $141,588 $142,704 $143,448 $1,860 0.0%
12
SensIt Output – Tornado Chart:
13
The impact of uncertainty about a variable is measured by its
"swing."
• Swing = (Value in high case) – (Value in Low Case)
= Width of the bar in the Tornado chart
• Variables are sorted by their swing in the Tornado chart
• Qualitatively, swing is determined by two things:
o How uncertain we are about the variable:
No uncertainty ⇒ No swing
o How sensitive the value measure (e.g., NPV) is to changes in the
variable:
No sensitivity ⇒ No swing
14
Often most of the uncertainty in value is due to uncertainty in a
small set of variables.
• The "percent of variance explained" is a rough measure of the impact of
one variable on the uncertainty in value:
o Variance for a variable is roughly proportional the square of the swing.
(Exact if all variables have a normal distribution.)
o Total variance is roughly equal to the sum of variances due to the
individual variables. (Assumes additivity and independence.)
(Swing for this var)2
"Percent variance" for variable = (Total of Squared Swings)
• Impact on uncertainty in value is proportional to the square of the swing.
o Little is lost in ignoring uncertainty in variables with small swings.
o May be able to capture 95% of variance with just a few variables.
15
What if two or more variables are highly correlated?
• Worry about it: If two or more variables are highly correlated, you should
vary them simultaneously rather than independently:
o Define scenarios that jointly specify high, medium, and low values for
each variable and do sensitivity over the scenarios.
o Define a multiplier that changes all of the dependent variables
simultaneously.
• Don't worry about it: If one variable has little impact on its own, it is not
likely to have much of an impact when moved with others.
o Correlation with another variable can increase a variable's impact, but
the increase will (typically) be less than doubling the "percent
variance" associated with the variable.
• Worry later: If both variables are important, we will model the uncertainty
in more detail later and can model the dependence then.
16
SensIt's – Spider Output (Table):
NPV
Corresponding Input Value Input Value as % of Base Output Value
Input Variable Low Output Base Case High Output Low % Base % High % Low Base High Swing
Average units per repeat purchase 3 6 25 50.0% 100.0% 416.7% $49,618 $142,704 $732,254 $682,636
Total potential customers 50,000,000 100,000,000 350,000,000 50.0% 100.0% 350.0% $48,272 $142,704 $614,452 $566,179
Percent trial purchasers => repeaters 3% 7% 20% 42.9% 100.0% 285.7% $36,320 $142,704 $488,455 $452,135
Average price/unit $0.30 $0.50 $0.60 60.0% 100.0% 120.0% ($68,569) $142,704 $248,341 $316,910
Coefficient of innovation 0.01% 0.50% 1.00% 2.0% 100.0% 200.0% ($16,568) $142,704 $198,596 $215,164
Coefficient of imitation 1.00% 3.00% 10.00% 33.3% 100.0% 333.3% $79,387 $142,704 $250,733 $171,346
Inflation rate 0.00% 5.00% 15.00% 0.0% 100.0% 300.0% $104,394 $142,704 $260,936 $156,542
Marketing--Variable 25% 20% 12% 125.0% 100.0% 60.0% $98,689 $142,704 $213,129 $114,440
SG&A Variable 25% 20% 12% 125.0% 100.0% 60.0% $98,689 $142,704 $213,129 $114,440
Per unit materials $0.17 $0.15 $0.13 113.3% 100.0% 86.7% $107,492 $142,704 $177,917 $70,424
Labor FTEs per million units 7 5 3 140.0% 100.0% 60.0% $114,535 $142,704 $170,874 $56,340
SG&A Fixed $6,500,000 $5,000,000 $3,500,000 130.0% 100.0% 70.0% $131,545 $142,704 $153,864 $22,319
Labor rate $9,000 $8,000 $7,000 112.5% 100.0% 87.5% $133,901 $142,704 $151,507 $17,606
Average units per trial purchase 1 1 3 100.0% 100.0% 300.0% $142,704 $142,704 $157,692 $14,987
Shanghai facility rental $1,400,000 $1,000,000 $900,000 140.0% 100.0% 90.0% $139,729 $142,704 $143,448 $3,720
Existing trials purchasers 650,000 800,000 1,000,000 81.3% 100.0% 125.0% $141,783 $142,704 $143,923 $2,140
Marketing--Fixed $1,000,000 $850,000 $750,000 117.6% 100.0% 88.2% $141,588 $142,704 $143,448 $1,860
• Note: Spider plots do not include variables whose base case value is 0.
17
SensIt's – Spider Output (Plot):
SensIt 1.42 Academic Version
$900,000
$800,000
Average units per repeat purchase
$700,000
Total potential customers
Percent trial purchasers => repeaters
$600,000
Average price/unit
$500,000 Coefficient of innovation
Coefficient of imitation
$400,000 Inflation rate
NPV
Marketing‐‐Variable
$300,000
SG&A Variable
Per unit materials
$200,000
Labor FTEs per million units
$100,000 SG&A Fixed
Labor rate
$0 Average units per trial purchase
Shanghai facility rental
($100,000) Existing trials purchasers
Marketing‐‐Fixed
($200,000)
‐50.0% 0.0% 50.0% 100.0% 150.0% 200.0% 250.0% 300.0% 350.0% 400.0% 450.0% 500.0%
Input Value as % of Base Case
18
Example: Risk analysis for a large offshore oil project
• Study done in 1995.
• Significant oil reserves had been
discovered, but uncertainty
about reserves remained.
• Development requires building a
production platform, drilling
many wells, and building a
transportation system.
• The owner of the field was
looking for a partner to share
expenses and revenues.
• There were complex tax and
royalty considerations.
• The purpose of the study was to
value and look at the risks
associated with the investment.
19
Uncertainty about production was described by low, medium,
and high scenarios with different amounts of reserves.
Year 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Years from Start 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
20
Uncertainty about future oil prices was similarly described using
low, medium and high scenarios.
30
High Scenario
Oil Price ($/bbl WTI)
25 Mid Scenario
20
Low Scenario
15
10
5
1995 1997 1999 2001 2003 2005 2007 2009 2011 2013 2015
Year
• These corporate price forecasts (for WTI) had to be adjusted for the quality
and location of the oil produced as well as exchange rates.
21
Modeling cost uncertainties:
• Engineers had prepared detailed cost estimates for drilling, facilities, and
other costs for each year, in each production scenario.
• Uncertainty about these costs was captured by "multipliers."
Example: Facilities construction costs:
o High scenario: All costs multiplied by 1.30 (increase by 30%)
o Mid scenario: All costs multiplied by 1.00.
o Low scenario: All costs multiplied by .85
Ranges provided by executive team.
• There was more uncertainty about drilling costs, because of the new
technologies being used and the potentially difficult conditions
o Multiplier range: Low = .80; Mid = 1.0; High = 1.5
22
The tornado analysis helped focus the analysis on the "most
important" uncertainties.
SensIt - Sensitivity Analysis - Tornado
Production Scenario 1 3
Number of Tankers 1 2
23
Tornado charts in Crystal Ball:
• CB has a tool for creating Tornado charts.
o Must define distributions ("assumptions") before doing Tornado
analysis in Crystal Ball.
o You can however define "dummy" distributions to match specified
percentiles and then use CB's tornado tool.
o See CB documentation.
• CB can also generate "sensitivity reports" that look a bit like Tornado
charts but are quite different.
o Report correlations between input uncertainties ("assumptions") and
outputs ("forecasts") after running a simulation.
o Bars are rank correlation coefficients or contribution to variance.
o Not recommended.
24
Summary
• Tornado charts help identify the most important uncertainties in a
problem.
o Build a deterministic model of the decision problem (i.e., identify the
relationships among the elements), and
o See how changes in the input uncertainties impact the bottom line.
• The tornado chart helps you:
o Understand the workings of the model:
- Be sure you can explain the results of the tornado chart!
- Helpful in debugging!
o Decide which variables you should focus your attention on (i.e., assess
probabilities from experts, gather more data, ...)
25