Beruflich Dokumente
Kultur Dokumente
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/
info/about/policies/terms.jsp
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content
in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
For more information about JSTOR, please contact support@jstor.org.
INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Interfaces.
http://www.jstor.org
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
Interfaces
Vol.
34, No.
iirfJH.
3, May-June
2004,
pp.
180-190
DOI
i0.1287/inte.l040.0078
Vendor
Waste
Allocating
in the Hanford
Risks
Cleanup
J.M. Keisler
of Management
Sciences
and
Department
100 Morrissey
of Massachusetts
Boston,
University
Information
Systems,
M/5-230,
Boulevard,
of Management,
College
Boston, Massachusetts
02125,
jeff.keisler@umb.edu
and
Decision
Information
Sciences
900, Argonne,
Building
Pacific
W. A. Buehring
Division,
National
9700
Argonne
Laboratory,
Illinois 60439-4832,
wabuehring@anl.gov
P. D. McLaughlin,
Northwest
National
South
Cass
Avenue,
Cass
Avenue,
M. A. Robershotte
Richland,
Laboratory,
{peter.mclaughlin@pnl.gov,
99352
Washington
mark.robershotte@pnl.gov}
R. G. Whitfield
and
Decision
Information
Sciences
900, Argonne,
Building
risks
allocating
as
outsourcing
could
incorrectly
increase
develop
tank waste
the Hanford
vendors
a way
National
Argonne
Illinois 60439-4832,
to manage
remediation
react
would
to
proposed
costs
9700
Laboratory,
rgwhitfield@anl.gov
risk. We
determine which
how
view
may
Organizations
Division,
we
their
decision-analytic
ones
and
of Energy's
In the model,
in terms
of
(TWRS).
system
risk allocations
1995
Between
dramatically.
developed
and which
used
South
1998,
we
used
this
(DOE's) privatization
an
actions
assessment
and
their
to
approach
it should bear. We
found that
to
approach
initiative
for
to
predict
considered
protocol
We
pricing.
the impact of allocating each major risk to potential vendors, to the DOE, or to both and identified the risk
to vendors plus the costs of any
the DOE's total cost?its direct payments
allocation that would minimize
residual
risks
it
have
costs
increased
Key
words:
decision
This
Allocating
because
risks
inappropriate
the vendor
paper
risks;
analysis:
was
refereed.
to the vendor
not
risk allocation,
History:
accepted.
vendor would
government:
take
acceptable
have
inappropriate
allocating
adequate
would
risk-reduction
increased
costs
because
the
With
the
improved
to the DOE.
agencies.
of Energy
the mid
1990s, the US Department
was
its
nuclear
of
the cleanup
In (DOE)
planning
in
State.
at
site
waste
tanks
the Hanford
Washington
of
the
because
It faced well-documented
problems,
National
Northwest
(PNNL), established
Laboratory
a large privatization
team near the Hanford
site to
was
con
initiative.
to
this
Its
first
task
support
major
tract with one or more vendors
for the development
of the cleanup.
The DOE
phase
a
issue
had
(RFP),
request for proposals
contract terms.
evaluate and select bids, and negotiate
on a 1995 draft
Comments
from potential
vendors
RFP (TWRS Privatization
that risk allo
1995) showed
a major
concern:
initia
the privatization
cation was
on the contractor's
tive depended
being able to obtain
were
about spending,
concerned
The antici
issues, and radioactivity.
in the tens of billions of dollars.
costs and risks, in 1994 the DOE
a privatization
initiative
it called
what
undertook
as
have
would
contractors
under which
(vendors)
ious
stakeholders
and
environmental
was
pated cost
To minimize
much
decision-making
The DOE,
possible.
demonstration
to draft and
financing,
not accept
as
authority and responsibility
with
the Pacific
in partnership
avoid
being
and
certain
locked
into bad
180
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
contract
terms.
et al.: Allocating
Keisler
Vendor
Interfaces 34(3), pp. 180-190, ?2004
Waste
Cleanup
Price
RFP
Mitigation
Exogenous Amelioration
(DOE)(Vendor)
(DOE)(Vendor)
Mai
interactions with
ipated in the assessments,
managed
the privatization
the results with
team, and integrated
a
the rest of the privatization
We
developed
plan.
to
the
deter
DOE
decision-analytic
approach
help
mine how it should allocate risks within
the RFP, that
is, to what extent should the DOE accept risks and to
what extent should it pass risks on to the vendor?
and assessed
the
We defined
alternative
allocations
risks resulting from them to quantify
their cost impli
for the different
incentives
cations, while
accounting
and risk-bearing
and
the
DOE
its poten
of
capacity
a
to
tial vendors,
RFP
and
successful
further
leading
our
efforts to institutionalize
approach.
in
The problem of how to allocate risks is prevalent
all sorts of contracting
and using RFPs that
situations,
can cause inefficient
outcomes.
allocate
risks poorly
in
Our conceptual
of those
many
approach
applies
situations.
Conceptual
Approach
a
Allocating
it motivates
age that risk.
the other will
Vendor,
l
Figure 1: Stylized decision trees were used to represent the cash flows
resulting froma contract.For the DOE,the firstnode is the DOE'sdecision
about how to allocate risks in the RFP.The second node is the potential
vendor's decision (an uncertaintyfromthe DOE'sperspective) aboutwhat
price to bid (to be predicted using a vendor-responsemodel). The third
node represents the DOE'sdecisions taken to mitigate risks. The fourth
node consists of the vendor's decisions taken to mitigate risks (inde
pendent of the previous node). Following this are uncertain exogenous
events, afterwhich the DOEacts to ameliorate the outcomes for the events
that have occurred, afterwhich the vendor does the same (independent
of the previous node). The DOE'send-pointvalues are determined by the
contractualpayments itmakes to the vendor alongwith the costs incurred
due to the risks itbears and the events that occur. For the vendor, the first
node is an uncertaintyabout how the DOEdecided to issue the RFP.The
second node is the vendor's decision aboutwhat price to bid. The third
node is a decision node about itssteps tomitigate risks. The fourthnode
is an uncertaintyabout the DOE'ssteps tomitigate risks (independentof
the vendor's
choices
tain exogenous
about
events.
The
of uncer
decisions
as a styl
could be viewed
point of view, the problem
tree with nodes
ized decision
its deci
representing
sions and nodes representing
uncertainties
(Figure 1).
From
the vendor's
of view,
the DOE has
point
chosen
its risk allocation. We simplified
the
already
tree and assumed
vendor's
decision
that the vendor's
its chance of getting
the contract,
goal is to maximize
that is, to bid as low a price as possible,
subject to
the constraint
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
Keisler
Assessment
of Individual Risks
change),
may
(the vendor
(4) Permitting
obtain necessary
permits),
C (probably
(5) Waste
envelope
waste
to
difficult
process),
cally
uncontrollable
we
as would
to each outcome
abilities
the DOE, unless
reasons
not
to
there were
the
(for example,
special
a
cer
DOE might
be far more
trustworthy
regarding
or the vendor
tain risk than the vendor
believed,
more
be
about
risks and
different
concerned
might
events than would
the DOE). We also obtained verbal
be likely
of the steps each party would
descriptions
outcomes
to take to ameliorate
of risks
the negative
under
each allocation.
Finally, we obtained
of what
the
verbal descriptions
out
each
be
for
each
side
for
consequences
come of the risk, assuming
both sides had taken the
we assessed
this
From
information,
steps.
predicted
over the
or
estimates
distributions
point
probability
conse
side.
financial
each
The
for
consequences
were
in terms of the net present
defined
quences
at the current
value
(NPV) of cash flows
computed
as increments
from a
risk-free
rate, and expressed
diffi
base case in which
the project proceeds without
that outcomes
of one
culty. In some cases, we noted
would
not
be
the most
able
to
techni
(6) Appropriation
(Congress may not appropriate
for planned
funds adequate
activities),
and decommissioning
(D and
(7) Decontamination
D may be more costly than expected),
(8) Other
et al.: Allocating
182
circumstances
(lawsuit,
tornado), and
earthquake,
sabotage,
risk (the contract
(9) Not-to-exceed
(NTE) price
could be a literal price limit, as
pricing arrangement
in the draft RFP, or could be a target price that allows
selected adjustments).
a background
risk level for the
We also assessed
risk each
that is, the residual
DOE and the vendor,
bear even if all nine risks resolved to their base
would
risk would
determine
the relevance of other risks; for
if the project stopped early, later risks would
example,
not affect cash flows.
The following
of one such assessment,
summary
assessment
D
and D risk, illustrates the
the
for
namely
alternate
and the
of
the
allocations
process
defining
that we
cases.
discussions
We
each risk through
characterized
with
the PNNL privatization
staff experts. We used a
common
the assessments.
For each
form to organize
we
the risk
three
allocations:
alternative
defined
risk,
was
to
to be borne primarily
be
the
shared,
DOE,
by
con
or to be borne primarily
vendor.
We
the
by
both
and
these allocations
structed
using
qualitative
to
alterna
definitions
match
reasonable
quantitative
include in the RFP. This
the DOE might
tive phrasings
even
that
for
for risks allocated pri
meant,
example,
to
the
the
DOE
could still bear some
vendor,
marily
risk does not neces
risk.
shared
Similarly,
tangible
sitate a completely
arrange
symmetric
risk-bearing
so
ment.
The specific vendor was not yet known,
scenarios
risk-management
eters
approximately
and Buehring
and
the
resulting
param
assessments.
Results
of Interviews
to Characterize
One
of the
Risks (D and D)
Risk Name:
Decontamination
and
decommission
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
Keisler
Vendor
Interfaces 34(3), pp. 180-190, ?2004
DOEtakes risk
Risk allocation
DOE is responsible
Description
Waste
for D and D.
$5M:10%
-$50 M :40%
-$150M:50%
Probability
distributionfor
No cost
183
Shared risk
Vendor
sets aside
Vendor
takes risk
limited funds,
Probability
distributionfor
cost to DOE
Cleanup
excess
if vendor
reaches
the
$2 M:30%
-$40 M :60%
-$140 M: 10%
$3 M: 30%
-$10 M: 70%
$5 M: 50%
-$50
M.-25%
-$100 M: 25%
cost to vendor
Table 1:We summarized the risk-assessment interviews inthe formof quantitativedescriptions of the risksused
as inputsto the simulationmodel. The description for the D and D riskshown here is typical.
and D cost
than expected.
Risk Allocation
Characteristics)
(Representative
The DOE bears the risk: The DOE states up front that
it is responsible,
and the vendor
allocates no money
for D and D.
is defined,
and the ven
Shared risk: Responsibility
a
dor establishes
medium-sized
sinking fund and gets
D
if
is
back
D
and
under
the DOE
money
budget;
covers
overage.
and Prevention
Strategies
strict
bears the risk: The DOE establishes
in planning
the vendor must
and
follow
and have close cooperation
with
environ
operation
mental
regulators.
some
Shared risk: The DOE monitors
the vendor
in planning;
the vendor
is responsible
what
for oper
ation with
and
The DOE,
the vendor,
checkpoints.
some
of
the regulators
coordinate
these
together
activities.
The vendor
the risk: The DOE
bears
intervenes
in
extreme
all
actions
the
vendor's
take
cases;
only
into account
their impact on future D and D costs,
and the vendor works hard to establish
rapport with
regulators.
Likelihood
of Occurrence
cost = 10 percent,
The DOE
the risk: Low
bears
cost = 40 percent, high cost = 50 percent.
medium
cost =
Shared risk: Low cost = 30 percent, medium
=
60 percent, high cost
10 percent.
The vendor bears the risk: Low cost = 25 percent,
cost = 50 percent, high cost = 25 percent.
medium
Costs are lowest when
the risk is shared and high
est when
the DOE bears the risk, because
both the
DOE
have
and
the
can
the vendor
to
incentive
do
influence
so under
outcomes
and
a shared-risk
scenario.
Modeling
We
risk-allocation
defined
four main
for
strategies
further analysis
Each
consisted
of
(Table 2).
strategy
nine
allocations
All
each
of
risks.
for
the
of
specific
the realm of possibility.
these strategies were within
Prior to engaging us to work on the problem,
the DOE
was heading
toward something
close to the vendor
bears
strategy
(Strategy 3). This strategy is quite sim
ilar to the 1995 draft RFP, and even after we started
a senior DOE official declared
that the DOE
work,
ven
would
"shift all risks to the vendor."
Potential
turn
in
had
dors
indicated
that just about every risk
should be borne by the DOE
(Strategy 1), although,
the DOE was skeptical of such a position.
predictably,
As we analyzed
individual
risks, the analysis
(and
other factors) influenced
the RFP. By the time we fin
ished analyzing
the various
strategies,
Strategy 2 had
Strategy 1 2 3
DOE
Name
Risk name
Interestrate
Inflation
Changes in law
Permitting
Waste streamC
Appropriation
DandD/RCRA
Sabotage
Earthquake
Tornado
Not to exceed
DOE
DOE
DOE
DOE
DOE
DOE
DOE
Lawsuit
DOE
DOE
DOE
DOE
DOE
Shared
Vendor
Vendor
DOE
Vendor
Vendor
DOE
Shared
Vendor
Vendor
Vendor
Vendor
Vendor
Shared Vendor
Shared
DOE
Vendor
DOE
Vendor
DOE
Vendor
Shared
Vendor
Vendor
Vendor
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
Keisler
Strategy
Vendormean cost increment
Vendor standarddeviation
DOEmean cost increment
DOEstandarddeviation
1 ($)
2 ($)
157
55,876
336,731
133,409
32,660
61,780
107,625
82,517
3 ($)
106,152
94,655
60,400
70,675
Table 3:We generated key summary statistics from the risk simulations
(figures inthousands).These results described the impactof risksharing,
butwe still needed to incorporatethemain cost, direct payments fromthe
DOEto the vendor.
to produce
for each risk. For each
profiles
we
the
increments
from the base
summed
iteration,
case due to each risk. The result, after accounting
for
a frequency
was
over
distribution
risk interactions,
ations
costs in the
the total increments
(in which
higher
to reduced profits) to the base case,
chart correspond
for both the vendor and the DOE,
including payments
for each of the strategies
considered
(Table 3).
As a measure
of the reliability
of the simulation
we used the stan
results under repeated simulations,
of the average cost for a given strategy,
dard deviation
was
which
$2 million.
approximately
Vendor
Response
the price the vendor would
then had to predict
under
each
strategy, which we could combine
charge
with
the DOE's risk profile to obtain a net-cost distri
bution for the DOE.
conversations
with
the privati
Based on extensive
zation
financial
task leader for the TWRS and with
we developed
external
financial
consultants,
equa
set a price in
tions to predict how the vendor would
to any given set of risks and risk-allocation
response
these equations using experts'
decisions. We validated
specific changes to the risk pro
predictions
regarding
estimates were
file for which
fairly easy
subjective
that
to provide. We then had other experts confirm
realistic. Our
the parameter
values we used were
was
to codify
rather than to
purpose
judgments
We
firm
describe exactly and in detail what any particular
would
do.
in the financial-risk
used
One
variable
important
for which we
the probability
of default,
model was
as a proxy the probability
that the vendor would
stake.
loss larger than its equity
after-tax
? i<
we
h
that
the
calculated
probability
Specifically,
-(j + k), where
h is the baseline NPV
of vendor
cash flows dis
counted at the risk-free rate,
used
face
an
et al.: Allocating
184
from
i is the net impact on vendor
cash flows
at the
all risks, also in terms of dollars discounted
the probability
dis
risk-free rate (we approximated
to the
tribution on i by fitting a normal distribution
simulation
results),
; is the amount of equity the vendor puts up, and
tax benefit
k is the relative
in
of reduced
profits
cases compared
to the baseline.
unfavorable
We determined
cash flows by entering
the pro
a
terms
into
and
financial
price
posed
financing
on
stan
and
based
Paananen
model
1995)
(Weimar
and
the
dard accounting
pri
practice
developed
by
to predict
the DOE's
vatization
finance task manager
costs in a deterministic
and the vendor's
base case
and to clarify the literal implications
of contract terms
(Figure 2).
We assumed
that the vendor would
raise prices
to being
to take on additional
in response
forced
that are pre
risk in the following
ways:
(1) Risks
would
downside
decrease
the expected
dominantly
the
vendor
raise
would
the
return;
price to bring the
to
return
baseline
back
its
value.
(2) Risks
expected
it harder
that increase the probability
of default make
so we assume
to obtain debt financing,
the interest
rate for debt financing
to the degree
of
corresponds
risk and the percentage
of equity
we
assume would
increase
which
risk increases.
required, both of
when
downside
are concerned with
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
Vendor
Keisler
Interfaces 34(3), pp. 180-190, ?2004
Simulation
Strategy
output
Waste
Cleanup
Summary
statistics
185
Financing
terms
Pro forma
financi?is
Objectives
DOE RFP
risk
allocation
Figure2: This stylized influencediagram represents the financial relationships in the model. The model pre
dicts price from risk allocation and other assumptions. The influencediagram has two features thatmake it
unconventional?the circular relationship involvingprice and probabilityof default and the fact thatmean, vari
ance, and probabilityof default are summarystatistics derived from the probabilitydistributionover the vendor
cost increment(derived fromend points of the decision tree inFigure 1).
In other words,
the project's
the solution
lies where
pro forma NPV including debt costs is exactly 0 when
at the predicted
IRR requirement.
discounted
to Represent
Used
Vendor
Financial
Equations
to
Risks
Response
Programmatic
a few key consid
After obtaining
advice concerning
the following
erations, we postulated
relationships.
The financial experts based their estimates
of param
eter values on their knowledge
of other debt offerings,
in which
situations
debt
(which
including
ratings
to default
have a known
relation
rates) and corre
terms were available.
sponding
The required after-tax
internal rate of return (IRR)
is given by
IRR = k1+k2x
A + fc3xB,
where
(1)
=
kx 0.03 (the risk-free rate),
= 0.0275
k2
(the increase in IRR required to accom
and
modate
of average downside),
$10 million
= 0.025
to
rate
variance
interest
of
(the
k3
sensitivity
in NPV of cash flows).
In this equation
and in the ones that follow,
A denotes
in the vendor
cost distribu
the variance
in dollars2 x 109), and
tion (expressed
B denotes
cost increment
the vendor mean
(in
dollars xlO5) from the simulation
results.
If the probability
the maximum
cent,
of default
(P) is less than five per
debt fraction (DF) is given by
0.95
(the maximum
case with
zero
debt
where
fraction
(2)
for
of
percent
hypothetical
probability
default),
= 1.5
of the debt fraction to the
(the sensitivity
k5
of default),
probability
= 3.23
to the mini
(the variance
k6
corresponding
mum
vendor
risk case, including
the assumed
level
of background
risk), and
= 3.0
of the debt
(the exponential
k10
sensitivity
to variance). We used
fraction
the last part of the
in particular
to calibrate predicted
financ
equation
were
aware
We
ing terms for experts'
judgments.
that it lacked a further theoretical
basis, but given
our time constraints, we used it, albeit
In
cautiously.
we
later applications,
this
element
of
the
implemented
model more elegantly.
If the probability
is greater than or equal
of default
to five percent,
the debt fraction is given by
DF = fc4- k5 x P - 0.03 x (A/ke)m
-fc7x(P-T),
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
where
(3)
Keisler
= 16
of debt fraction
to
k7
(marginal
sensitivity
rate for risky projects),
default
and
T = 0.05 (the probability
of default
corresponding
to financiers'
threshold of great concern). This thresh
to a downgrading
old is analogous
of the debt on the
as the
project, which we assessed
point
subjectively
at which
as
the project would
from
business
change
into what
The minimum
usual
given
financiers
required
would
debt
view
interest
as risky.
rate (DR)
by
DR = M + k8 x P? Y + k9 x A/k6,
is
= 0.0737
(the T-bond
yield
at the
where
time
(4)
of
the
study),
y =
to be near
the maximum
0.07, considered
allowable
of
default,
probability
P = the probability
of default
(computed
by the
model),
= 0.138
of the sensitivity
of the
(the coefficient
k8
debt interest rate to the probability
of default),
and
=
of the debt interest rate to
k9 0.015 (the sensitivity
in cash flows).
the variance
Where
the financial
advisors
identified
possible,
in
the
for
market;
analogous
financing
packages
a
a
with
bonds
have
histori
example,
given
rating
cal annual default
rate and a known
risk premium
over T-bonds.
a
bond
By finding
rating correspond
= 0.07 and
a linear
ing to y
assuming
relationship
between
rate, one can estimate
yield and default
k8.
some parameters
We derived
directly from such mar
some using
ket data, estimated
subjective
judgment
informed
such
and estimated
such
data,
others,
by
as k10,
what
about
by obtaining
expert
judgments
terms ought
to be for a small number
of
financing
artificial
scenarios
and
the
values
specified
fitting
to mimic
those expert judgments. We compared
the
a
of
the
for
wide
of
cases,
model,
range
implications
et al.: Allocating
186
with
the intuitions
and expectations
of the financial
and of the privatization
team management.
experts
We concluded
that the financing
terms and the cor
case were
reason
for
baseline
the
price
responding
able. The model's
predicted
impact of risk allocations
on
same order as the finan
terms
is
of
the
financing
cial advisors'
estimates
of those impacts
for several
were
reference
that
to
risks
particular
fairly easy
these parameters
would
judge. Our experts
thought
over
reasonable
results
the
of
range
provide
strategies
we examined
for considering
the
and, in particular,
incremental
of
to
the
risk-allocation
impact
changes
strategy.
Numerical
Results
Strategy
Vendor
financial measures
3.4%
5.17%
8.23%
11.6%
14.3%
30.3%
86.6%
82.7%
30.6%
9.3%
9.8%
11.6%
$602,255
$636,530
$1,397,921
$336,731
$133,409
$938,986
$107,625
$82,517
$744,155
$60,400
$70,675
$1,458,321
DOEfinancialmeasures
DOE mean
cost
increment
DOEstandarddeviation
DOEtotalcost
Table 4: Key financialmeasures of interestto the vendorwere trackedforeach strategy.When the vendor bears
all risks (Strategy3), the difficultyof financingthe project results in the DOEhaving tomake much higher pay
ments to the vendor.Although the DOEcouldminimize its direct payments to the vendor by accepting all risks,
thiswould usually lead to higher total costs.
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
Vendor
Keisler
Interfaces 34(3), pp. 180-190, ?2004
Waste
Cleanup
would
Frequency
in simulation
results
to the vendor
risks, the DOE would make payments
than
that would
be six percent
($34 million)
higher
to
but
the
the
DOE
those under Strategy
1,
savings
lead to a net cost
from impacts of residual risks would
over
20
for
the
DOE
of
($194 million).
percent
savings
-$200M
Frequency
in simulation
in which
to the strategy
the vendor bears
Compared
2 requires the DOE to
all risks (Strategy 3), Strategy
risks that are 78 percent
pay costs due to residual
to the
but
the
total cost savings
($47 million)
higher
it
because
DOE are nearly 50 percent
($714 million)
results
-$200M
results
-$200M
$200M
would
Frequency
in simulation
the shared-risk
its potential
costs,
the vendor
strategy,
but its mean costs
has somewhat
do not increase
greater
much.
spread
and
for
In analyzing
the results
the strategies
we
the
risks
demonstrated
each,
that, indeed, sharing
all the risks to the
would
lead to savings. Allocating
not do enough
the vendor would
DOE would mean
all the risks to the
to keep costs low, while
allocating
cause the vendor
to demand
too high
vendor would
a risk premium;
the low
the
DOE
would
obtain
thus,
Frequency
in simulation
these
which
resembled
the
vendor
DOENet Cost
Frequency
in simulation
Risk
Shared
the worst
results
Frequency
in simulation
$1.5B
DOENetCost
$0.5B
U
$1B
DOENet Cost
$L5B
we generated
for each risk-allocation
strategy
Figure 4: The histograms
dominates
the two extreme risk allo
show that sharing risk stochastically
cost and its risk.
cations and so reduces both the DOE's expected
changes-in-law,
In
risks.
partic
all
risks
and
shared,
as
well
as
many
other
varia
that we
at the DOE's
tions (not shown)
considered
the DOE
and the vendor
share all
request. When
cost increment
to the DOE
is higher
risks, the mean
to the vendor,
than in Strategy
2, as is the payment
so Strategy
worse
6 is $24 million
than Strat
5 had a mean
cost increment
to the
egy 2. Strategy
than Strategy 2, in exchange
DOE of $5 million more
in payments
to the ven
for a $15 million
reduction
of
million
$10
dor, for an additional
savings
along
a slight reduction
in the standard deviation
with
of
$2 million.
and
results
rate,
appropriation
to the vendors
risks
ular, by completely
allocating
that they believed were under the DOE's
control, for
D
and
and
the
D,
appropriation,
permitting,
example,
assume
increase its cost: vendors would
DOE would
For
$0.5B
interest
the
and
waste-stream-C,
constructed
5,
Strategy
2, except that the DOE and
Strategy
shared
results
$0.5B
we
allocations
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
188
Keisler
the allocation
of the other risks, the
By changing
DOE would
obtain
cost
relatively minor
potential
increases or cost savings. This implied that if the ven
dors seemed very concerned
about risks for which
the model
little increase in costs (for exam
predicted
an additional
$50 million
ple, if the vendor demanded
to accept waste-stream-C
then
DOE
the
could
risk),
on those risks (for
make
concessions
accept
example,
the risk rather than pay the premium).
Such risks
or
have
value
for
vendor
the ven
the
might
symbolic
dor might have a more pessimistic
view of the situa
tion than it merited.
if the vendors were
Conversely,
more
a
to
risk
than
the model
pre
accept
willing
to
the
DOE
could
obtain
concessions
from
dicted,
try
to reduce its total cost. The entire portfo
the vendor
as a unit.
lio of risks matters
and must be considered
For example,
of
the
impact
although
shifting the NTE
risk entirely to the vendor
from Strategy 2 (in which
it
is shared) is $41 million,
the cost of shifting the NTE
risk to the vendor when
the vendor
already bears all
be much
other risks would
In
larger ($198 million).
can bear a little risk comfort
the vendors
other words,
a certain threshold, vendors will not
ably, but beyond
assume
risk without
additional
the DOE
penalizing
dramatically.
The final RFP was
Decision-Support
System
the RFP, we
it
anticipated
bidders'
and
suggestions
the bid-selection
and negoti
requests quickly during
some parts of the model
ation phases. We improved
to speed the DOE's
and to take advantage
response
of insights
the
then created
way. We
gained
along
a
rather
than a one
(DSS)
system
decision-support
When
would
the DOE
issued
to evaluate
need
off model.
used
to characterize
basic templates
common
to many procurement
to make
it easier to
We developed
four
risks that would
be
contracts,
specifically
and ongo
termination,
of
the
contracted
stage
one type of risk tem
permitting,
processing,
early
risks. For each
ing financial
project, users must
complete
for several risks of that type. The DOE
plate, possibly
to successfully
would
have
resolve
risks from each
move
on
to
to
next
the
This
fact led to
stage
stage.
the next feature. We modeled
risk interactions
using
a
a cell entry of 1 or 0
in which
matrix,
precedence
indicates whether
the DOE must
resolve
successfully
the risk in the corresponding
column before going on
row. This
the risk in the corresponding
it easy to add individual
risks without
and allowed us to
the rest of the simulation
in a more compact and compre
interactions
to encounter
matrix
made
affecting
represent
hensive
way.
not directly
the debt fraction but
calculate
it from a debt-service-coverage-ratio
derives
found in practice.
This structure
facili
requirement
was
more
tated assessments
which
(with
everyone
of the judgmental model
comfortable)
relating project
to financing
statistics
terms. We ended by
summary
and automating
the optimization
sub
streamlining
routine for finding the vendor's minimum
price.
It does
instead
The
answered
system
negotia
resulting
an hour, instead
in
half
about
questions
a day or more
the original model
could
on
the
of
the
take, depending
complexity
question. To
some of the improvements,
we relied on stan
make
to make models
dard methods
faster, more flexible,
In particular, we restructured
and more user-friendly.
to use modular
the model
and automated
components
tors' what-if
of the half
most
tasks. We also made
improvements
conceptual
more
in
that made
the model
than it was
efficient
our earlier efforts; we recommend
such improvements
as a starting point
for similar efforts others might
undertake.
We
structure
Organizational
Impact
The insights
from our analysis
within
team
the privatization
The DOE managers
presented
of
mary
diagram
(Department
produced
and within
consensus
the DOE.
sum
the conceptual
Privatization
Energy
1998, ?5, Figure 4.6) on risk shar
Group
Working
of energy, Hazel O'Leary.
ing to the then secretary
to provide
One purpose
of our effort was
the DOE
with
evidence
about
the potential
of
consequences
that
vendors
alone
should
bear
the
the
risk,
insisting
and we
that doing
this would
showed
successfully
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
Keisler
Vendor
Interfaces 34(3), pp. 180-190, ?2004
Waste
Cleanup
have
attract
report
ing
require
excessive
to compensate
price
for
the
risk
reviews
with
decision
and
construction
will
and
wastes
Hanford-specific
and
arrangements
technology
and
to
prior
The
operations.
time to verify
allow
equity
points
to
proceeding
design
on
and
debt
optimize
technical
phase
performance
included
the following
lessons
learned:
nificant
would
some
but
For
risks
example,
performance
and preexisting
areas
such
as waste
characterization
conditions.
The DOE
group (Department
privatization
working
of Energy Privatization
1998, Case
Working
Group
a
on
for the
report
study 6) prepared
privatization
of
Its
list
learned
from
this
of
lessons
energy.
secretary
included
the
project
following:
Risk Allocation. This is the single most
ture
of
the
to be
"deal"
established.
important
There
must
be
fea
that
it can
performance
for risks
that
furnished
control?for
DOE
risks.
The
it can
control?for
items
and minimum
and
technical
example,
to be
needs
responsible
example,
waste
government
quantities.
The overall
lead to estab
report, which
helped
lishment
of the DOE's Office
of Privatization
and
recommen
Contract Reform,
contained
the following
dation
Group
of Energy Privatization
(Department
1998, ?2, recommendation
5):
community
between
ensur
draft RFP.
Afterword
often on an
1998, the DOE used the model
Through
in finalizating
ad hoc basis
the RFP and selecting
and negotiations
vendors
vendor
with
the remaining
into what was called
(of two initially selected) going
Phase 1B-1 (the first stage was Phase 1A). In this case,
the DOE
used
the model
to evaluate
was
whether
the ven
and to
reasonable
proposed
an
al.
et
1998). Over
agreement
negotiate
(McLaughlin
to
the rest of the demonstration
the start
phase up
risk allocation
of
full-scale
mal
strategy
almost all risk
$5 billion less than strategies allocating
to either the DOE or the vendor.
The larger privatization
effort had a disappointing
reasons
were beyond
for
that
the scope
end, however,
of the RFP risk-allocation
decisions. About a year after
contractor-operated
approach.
an
balance
a
This represented
turnaround
from the
complete
view
that the vendors
should
all
risks, which
carry
in response
comments
to the
had led to discouraging
dor's
requirements.
The flexibility
that the design phase added proved
new
to risk allocation
Our
important.
approach
kept
con
to
that
viable
up
privatization
point. Our work
vinced
the DOE to take a more businesslike
attitude
toward risk. The DOE's
report to Congress
(Depart
ment
of Energy Privatization
1998,
Working
Group
?3.2)
the business
a proper
Working
Summary
Our
new
have
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
et al.: Allocating
190
Keisler
That
businesses
others?certainly
and probably
involved
McLaughlin,
Whitfield.
governmen
this approach.
considering
Government
should
the Hanford
consider
agencies
a rather extreme
case.
In
sit
many
cleanup
simpler
a
to
could
this
uations,
help
approach
bureaucracy
overcome
to share risks. The DOE's doc
its reluctance
umented
lessons learned apply in such cases. Compa
nies
also
government
doing business with
agencies may
as a way
find our approach
to help the
useful
in nego
understand
the
agency
company's
positions
tiations that include the allocation
of risks.
tization
and
TWRS
cooperation,
Richmond
and
of
standing
Carol
Sohn
bers
of
programmatic
interest
and
complex
for her
the
DiPrinzio
their
of
the
We
improve
risk
topics.
and
support
including
assistance.
We
response
thank
Bill
who
experts
our under
We
other
thank
mem
Kearns
Paul
thank
and
A.
Raymond
Services
of Scully Capital
in
assistance
contractor
his enthusiastic
paper.
matter
team,
invaluable
ment
this
subject
and
interviews
privatization
for their
Straalsund,
Jerry
on
Lerchen,
Megan
to conduct
us
allowed
the
development
We
model.
also
for
and
refine
thank
Mark
assistance
running
scenarios.
We
thank
two
anonymous
funded
by
the US
ber DE-AC06-76RL
Department
of Energy,
contract
num
O1830.
References
of
Department
Immobilization
27,
February
www.hanford.
(
to Congress:
Treatment
and
Report
Radioactive
Tank
Waste.
Retrieved
of Hanford
2002 www.hanford.gov/docs/twrs-atp/toc.html
Energy.
Keisler,
on
report
estimating
allocation
decisions
Energy
Argonne,
1996. Summary
of the technical
risk
of key programmatic
impact
on Phase
1 bids and U.S. Department
of
the
costs. ANL/DIS/TM-41,
IL.
Argonne
National
Laboratory,
and Science
Cambridge,
Privatization.
TWRS
Privatization.
DE-RP06-96RL13308,
under
of Energy
1995.
contract
of Negotiation.
DE-AC06-76RLO
Harvard
Univer
MA.
Privatization
TWRS
Request
for Proposals
Request
for Proposals.
November.
(draft). DE-RP06-96RL13308,
1996. TWRS
Privatization
February.
market
1995. Financial
of Phase
Weimar,
M.,
J. Paananen.
analysis
on file in TWRS Privatization
1 for TWRS privatization.
Report
June 9.
Backup Documentation
Library,
costs may
nuclear
delay
Skyrocketing
1. Retrieved
Seattle
Times,
May
2002
seattletimes.nwsource.com/news/local/html98/
27,
.html.
hanf01m_20000501
C.
at
comments
his
H.
sity Press,
Welch,
Acknowledgments
We thank Mark Weimar
P. D., M. A. Robershotte,
R. G.
W. A. Buehring,
1998. Analysis
of the risk allocation
for TWRS priva
IB. PNNL-12028
(limited distribution).
phase
Prepared
for US Department
1830, November.
Raiffa,
2000.
Hanford.
cleanup
February
B. Mellinger,
Senior
George
Manager,
Program
Pacific
Northwest
902 Bat
National
Laboratory,
teile Boulevard,
PO Box 999, Richland, Washington
"I was The Deputy
of the
99352, writes:
Manager
Team (WIT) at the Han
Disposal
Integration
of Energy
ford, Washington
(DOE) site
Department
from 1995-1998.
The WIT was
the Pacific North
west National
team (also referred
to as
Laboratory
the Privatization
the
for assisting
Team) responsible
DOE with developing,
and
implementing,
managing
the contracting
for the cleanup
of massive
strategy
amounts of nuclear waste
stor
stored in underground
Waste
Buehring, McLaughlin,
did in fact do the work
tization team in the roles
This content downloaded from 80.87.90.164 on Sat, 23 May 2015 10:27:58 UTC
All use subject to JSTOR Terms and Conditions
received
enabled
in response
to
the initiative
bids