@RISK: A BEGINNERS GUIDE TO RISK MODEL OUTPUTS (PART I CONFIDENCE

TABLES)

http://www.haironfirepm.com/2012/10/10/risk-a-beginners-guide-to-risk-model-outputs-part-i-
confidence-tables/

BY ROSHAN

In my previous post, Developing a Simple Monte Carlo Estimate Contingency Model, I

discussed how to develop a simple Monte Carlo estimate contingency model using Palisade
Corporations Excel add-in @Risk.

As with any analysis, the results are not useful unless they are able to be interpreted &
presented accurately to stakeholders. In this three part tutorial I would like to further expand
that model and show you how you can auto generate Histograms, Cumulative Frequency
Curves, Confidence Tables and Tornado Charts. If you are a new user to @Risk or Cost Risk
Analysis, I recommend reading my previous article before going any further.

Confidence Table (aka Contingency Interval)

When reviewing the output of a cost risk analysis there is no right answer, but there is one
good question, What level of risk is the team prepared to accept? One way to present
diminishing risk stated in terms of contingency is by using a Confidence Table.

In our last post our estimate was summarized to 40 lines items that resulted in a total
estimate value of $117.5M. Our estimate model was constructed and the estimate risk total
was subtotaled in cell I41.
In this post we have created a new tab in the same sheet called output. For our Confidence
Table we will use a Contingency Interval of 10%. What this means is we will present the
confidence level of our estimate in terms of 10% incremental percentiles. Or said another
way for a 10% confidence level, the estimate contingency should be X, a 20% confidence
level should be Y etc.

Note: We have renamed cell I41 in our example to Result.

To build our confidence table we will be using the built in @Risk function =RiskPercentile(Cell
Reference, Percentile) which takes two input parameters.

Cell Reference is typically assigned to a @Risk defined output/input cell. In our simple
model we want to point this cell to output named cell Result
Percentile is the confidence percentile we want to display. This value must be >= 0
and <=1
The confidence table shown above states the ascending cumulative frequency. Or said
another way, it is the probability of being less than or equal to the x-axis value (or bin). So in
this case a P50 confidence value means the model resulted in $123.1M or less 50% of the
time.

The confidence value can also be stated in terms of descending cumulative frequency or the
probability of being greater than or equal to the x-axis value. Typically when we take about
project estimate contingency we want to know the contingency requirement stated in terms of
less than or equal to the estimate value. However, if you are asked to develop the
descending table simply use the function =RiskPercentileD(Cell Reference, Percentile). You
will notice it is essentially the invert of the ascending frequency curve.

What level of risk do you need to protect for?

Lets look again at the original question, What level of risk is the team prepared to accept?
As a management team (in many cases company standards outline risk requirements) the
team must decide on a confidence level at which they are comfortable to execute the project.
As no two projects are the same, this discussion must include key variables such as contract
type (e.g. cost reimbursable vs. lump sum), labor availability, regional politics, market
conditions and contract language just to name a few.

In Part II of this tutorial we will go into how to auto generate Histograms & Cumulative
frequency curves using @Risk.

Let us know what you think of this tutorial in the comments section below.

@RISK: A BEGINNERS GUIDE TO RISK MODEL OUTPUTS (PART II HISTOGRAMS &

CUMULATIVE CURVES)

BY ROSHAN

The final result of our first @Risk post, Developing a Simple Monte Carlo Estimate
Contingency Model concluded with a standard @Risk histogram. In Part I of this series we
explored how to develop a risk confidence table using the @Risk function =RiskPercentile().

In this tutorial I would like to go through the process of using the =RiskResultsGraph() to
develop output reports using Histograms & Cumulative Curves.

Risk Results Graph Function

To generate a output graphs in @Risk we will use the =RiskResultsGraph(cell reference,

Location Cell Range, Graph Type, Excel Format, Left x delimiter)

Cell Reference is typically assigned to a @Risk defined output/input cell. In our simple
model we want to point this cell to output named cell Result
Location Cell Range is an optional input that specifies where the output graph will be
displayed
Graph Type is a whole number, between 1 and 13, that represents 13 pre-defined
graph types. As this tutorial is a beginners guide we will focus on 6 of the 13 graphs
as some are more complicated and need a range of inputs vs. a single cell. In this
tutorial series we will focus on the bolded line items below
o 0 for histogram
o 1 for cumulative ascending graph
o 2 for cumulative descending graph
o 3 for tornado graph of regression sensitivity results (will discuss in Part III
of this tutorial)
o 4 for tornado graph of correlation sensitivity results
 o 5 for a summary graph of 1) the output range that includes cellRef or 2) the
results for each cell in cellRef (where cellRef is a multi-cell range)
o 6 for a box plot of 1) the output range that includes cellRef or 2) the results for
each cell in cellRef (where cellRef is a multi-cell range)
o 7 for a graph of a theoretical distribution function
o 8 for a histogram of a simulated input overlaid with its theoretical distribution
o 9 for a histogram with a cumulative ascending overlay
o 10 for a histogram with a cumulative descending overlay
o 11 for tornado graph of mapped value sensitivity results
o 12 for a scatter plot graph of the results for each cell in cellRef (where cellRef is
a multi-cell range)
o 13 for histogram using relative frequency
o Excel Format is a True or False input where True outputs the results as a
native excel graph or False (Default) outputs as a meta file image.
o Left x delimiter is an optional input that gives a position

Before we dive right into the outputs we want to customize our graphs to the P50 & P85
points in the legend and along the x-axis. We also want to set our numbers as currency and
our default color from red to blue. To customize our graph settings use the Application
Settings in the @Risk menu tab. See settings screen capture below:
Histograms

In our post How to create a histogram we discussed how to take a large set of data,
organize it into bins and eventually plot the results in a histogram. The @Risk simulation
follows the same process however the data set is created over a number of different
iterations based on a specified range. This repetitive process over bound ranges is known as
the Monte Carlo Method.

In our example we will be using the following input conditions,

=RiskResultsGraph(Result,K1:P12,0,FALSE,0) to output a meta file image in the yellow
region shown below. The equation we used in cell K1 is shown below the yellow shaded
region in cell K13.
The histogram frequency plot takes the aggregated results of the analysis and displays the
output as a histogram where the y-axis is the number of values that fall within in x-axis bin
or frequency grouping. In the example above, if we were to sum all the incremental values it
would equal the number of iterations simulated during the analysis (1,000 iterations).

The 50% and 85% values shown are a function of the cumulative frequency curves and
shown on the histogram because of our Simulation Graph Settings.

Cumulative Frequency curves

In Part I of this tutorial we went through the process of developing Ascending & Descending
Cumulative Frequency Tables (aka Confidence Levels). To display these results in a graphic
format we will use Graph Type 2 & 3 in the =RiskResultsGraph function.

Using Graph Type 2 will result in an ascending Cumulative Frequency Chart. The equation
we used in cell R1 is shown in cell R13 in the graphic below. Our results indicate 50% of the
time our Estimate value is $123.M or less & 85% of the time or less our estimate value is
$126.2M or less.
Using Graph Type 3 will result in a descending Cumulative Frequency Chart. The equation
we used in cell Y1 is shown below the yellow shaded area in Cell Y13. As we stated in Part I
of this tutorial we typically do not use the descending chart for estimate contingency results.

Combining the two Results

A typical risk report overlays the Histogram and the Cumulative Frequency Curves as they
are directly correlated. A change in the histogram will change the cumulative frequency
values. Therefore it is normal practice to present the two curves overlaid in the final result. To
do this we will use Graph Type 9 for Ascending and Graph Type 10 for descending

In Part III of this tutorial we will look at Tornado (or Regression) charts to identify our primary
risk drivers and discuss mitigation.

What are you experiences with risk reports & presentations? Let us know your thoughts in
the comments section below.

@RISK: A BEGINNERS GUIDE TO RISK MODEL OUTPUTS (PART III TORNADO

(REGRESSION) CHARTS)
In Part I & Part II of this series we discussed the importance of the management team to
develop a methodology on the amount of cost risk they are willing to accept in an estimate.
We also went through the process of developing Confidence Tables, auto generating
Histograms & Cumulative Frequency Charts using the = RiskResultsGraph() @Risk function.

The last part of this series is focused on Tornado (Regression) Charts or Graph Type 3 {If
this doesnt make sense to you please go back and re-read Part II)}. Tornado charts are one
of the most important outputs of a risk output as their primary purpose is to identify the
primary drivers impacting the results of the model. In most cases this can be translated in to
the highest risk activities.