Beruflich Dokumente
Kultur Dokumente
Summary of conclusions
The best product choice depends on the factors you consider most important:
Approach
The primary data source for this review is this1 independent and up-to-date overview of Excel risk
analysis add-ins on Wikipedia. This review only includes the Wikipedia information where there is a
difference between the software products. Aside from which technical capabilities each product has,
user-experience is also extremely important and a strong measure of the quality of the software as a
whole so, where possible, links have been provided to official videos produced by each developer so you
can compare the approaches. Please do take a look at them. This review is split into the following
sections:
Functionality
Technical specifications
Simulation controls
Reporting results
Help file, technical support and training
Implementation within Excel
Precision of the algorithms used in the software
Pricing
1
Last viewed 13 July, 2014
The three types of random variables that you tend to see in risk analysis modeling are:
Distributions
Correlation structures
Time series
Distributions - there are something like 150 distribution types that have a practical use and are not
simply a rehashed or renamed version of another one. You could say that 90% of users only need to use
10% of the types of distribution that are known, but which 10% varies enormously between the types of
problems that the users focus on. You can view videos of how the different products fit distributions to
data here: Crystal Ball3, ModelRisk, @RISK.
Correlation structures (copulas) are a very important and much overlooked component of risk modeling.
Failure to incorporate appropriate correlation will usually result in an underestimation of the risk. All of
the products reviewed effectively use a Gaussian copula which allows simulating a correlation structure
between any number of variables4. ModelRisk also provides the Clayton, Gumbel, Frank and Student
2
The table was removed from general view because of Wikipedias policy of not including original research (you can track the discussion that
resulted in its removal on this page).
3
Official video not found
4
In fact, @RISK and Crystal Ball uses rank order correlation, a simulation technique from 1982 that closely resembles a Gaussian copula. See
Iman, R. L. and Conover, W. J., (1982). 'A Distribution-Free Approach to Inducing Rank Order Correlation Among Input Variables', Commun
Statist-Simula Computa 11(3) 311-334.
Correlating two Uniform distributions using the Gaussian copula with, for example, a 90% correlation
would produce a scatter pattern that looks like this:
But the different copulas will produce markedly different patterns for the same 90% correlation, for
example (reading from the top left: Clayton, inverted Clayton, Frank, Gumbel, inverted Gumbel,
Student):
For example, the Clayton copula (top left plot) is often used by banks for loan defaults because these are
strongly correlated in times of recession, but not in better times. The selection of correlation shape can
have a very significant effect on the tails of results when the correlation level is moderate (e.g. 10% to
50%). You can view the way ModelRisk fits copulas to data here, and for unusual patterns here.
Time series models are used to forecast variables across several periods, like exchange rates, inflation,
sales volumes. They incorporate at a minimum the trend and the level of randomness around that trend.
More sophisticated models incorporate characteristics like sudden jumps, cycles of high and low periods
of volatility, etc. The more sophisticated models are usually fit to data using software. You can view
videos of how the different products fit time series to data here: Crystal Ball Suite, ModelRisk, @RISK.
Source: Wikipedia
Notes
Distribution fitting a fundamental requirement unless your risk analysis model is only based on subjective estimates of uncertainty
Correlation fitting another fundamental requirement unless your risk analysis model is only based on subjective estimates of
uncertainty, or all your variables are unconnected (independent)
Time series fitting a fundamental requirement if you want to make forecasts based on historic data
Optimizer allows you to determine the optimal values of variables you control based on some target(s) like maximizing the mean
profit. All three products use the industry leader OptQuest optimizer, and @RISK includes other optimizers
Database connectivity useful if you have databases with large datasets that you want to fit distributions, correlation structures or
time series and that will update as new data are added to the database
VBA calls useful if you want to automatically run simulations, or functions that will internally perform simulations or probability
calculations, etc.
C++ calls for very advanced modelers who want to build their own specialist, high performance simulation programs
Six Sigma functionality a moderately useful feature for Six Sigma specialists that analyzes simulation results using Six Sigma
metrics
Probability calculations a very useful capability for advanced probability modelers because, amongst other reasons, one can
combine calculation and simulation together to create more precise and faster models.
User Defined Function error analysis an error message appears in the spreadsheet cell. For example:
Extreme vale modeling very useful in certain circumstances - many risk analysis issues are driven by extremes, like predicting the
size of the largest wave or wind gust, the lowest temperature, the largest insurance claim, the longest blackout, etc.
Expert elicitation tools to help the modeler get the most precise uncertain estimates from subject matter experts. If this is useful
to you, you should investigate each tools capabilities because there is a great deal of variation in what they offer
Data previsualizer allows you to review and explore the data before you start fitting distributions, time series, or correlations.
Again, a large variation in what is offered, so worth investigating the differences if this is important
Differentiation and integration a very advanced capability for either (i) building a system of ordinary differential equations with
uncertain inputs; or (ii) performing one-dimensional numerical integrations. Useful for scientists, engineers and finance/insurance
people mostly
Simulation controls
This section compares the controls that are available to run simulations.
Source: Wikipedia
Notes:
Lock/unlock sampling switches off the Monte Carlo simulation, replacing random values with a predictable value like the median.
Useful in model auditing as one can compare values between different versions of a model
Apply sample to model allows the user to review the results data and force Excel to show a particular selected sample. This lets
you check, for example, why a particularly high or peculiar value was produced by the model. It also demonstrates to an auditor that
specific scenarios appear within the results, which is especially useful for banks and insurance companies
Random number generator advances in random number generators have made this essentially irrelevant. Nearly every product
uses the Mersenne Twister by default.
Spreadsheet interpreter this creates a compiled version of the Excel model and then runs this instead of within Excel. The result is
a much faster simulation time. Mostly useful if one has a large model that needs optimizing. The drawback is that not all Excel
models can be compiled.
Sampling method MCS = Monte Carlo simulation, LHS = Latin Hypercube sampling. We describe in a LinkedIn blog why LHS is no
longer important here.
Reporting results
Risk analysis is a decision-making tool. That means that it is vital to be able to share the results of a risk
analysis effectively. The simulation results from a risk analysis model can be complex for a decision-
maker to understand, so a wide variety of customizable graphical reports is essential.
The greatest difference between the products is how they organize the storage and presentation of
results, and particularly how these results can be shared with people who are not licensed users of their
products, which is explained below.
Results files can only be opened by another Crystal Ball user, though the charts can be copied as bitmaps
into Word, PowerPoint, etc.
ModelRisk
Results are shown in a separate application called the Results Viewer which opens at the end of the
simulation. The user can create and save an electronic report and that file can then be distributed to
others, without the need to share the model itself. The file can be opened by anyone who has installed
the ModelRisk Results Viewer, which is available free of charge from Vose Softwares web site here.
Aside from saving on licenses, the main advantages of this approach is that the report is electronic and
completely customizable labels and color schemes can be edited by the reviewer, new graphs can be
created, plots can be interrogated for say the 95th percentile instead of the 90th, and the whole
report can be resaved if desired. Plots can also be copied into Word, PowerPoint and other applications.
Results files can only be opened by another @RISK user, though the charts can be copied as bitmaps into
Word, PowerPoint, etc.
For the majority of risk analysis problems, the modeling techniques are not that complex, but some
basic training is really essential to avoid making the most common mistakes. Online videos that teach
the basics of driving the software are a good start, and a comprehensive help file saves a lot of time, but
if you are making multi-million dollar decisions based on a risk analysis model then we recommend
investing in 1-2 days of face-to-face, or at least online, training. We strongly recommend that the
training you purchase is not just about how to use the software. Instead, the course should focus on the
types of problems that you face, and the content tailored to show you the features of the software, the
particular distributions, etc. that you will need. The examples really need to be tailored to your industry
and problems to be effective, since people find it difficult to extrapolate from an example in another
field.
Help file, support @RISK @RISK @RISK Crystal ModelRisk ModelRisk ModelRisk
and training Standard Professional Industrial Ball Standard Professional Industrial
UDFs linked to help file Yes Yes Yes No Yes Yes Yes
Online videos Yes Yes Yes No Yes Yes Yes
Online training Yes Yes Yes No Yes Yes Yes
Onsite training Yes Yes Yes No Yes Yes Yes
ZH, EN, FR, ZH, EN, FR, DE, ZH, EN, FR,
Language versions EN, JA, ES EN EN EN
DE, JA, PT, ES JA, PT, ES DE, JA, PT, ES
UDFs linked to help file this means that when you open a dialog for a function, there is a link that will take you directly to the
relevant help file topic. Excel makes this quite easy for developers to do, and automatically adds a hyperlink to the dialog box, so it is
hard to see why this feature should not be implemented
ModelRisk: =VoseTriangle(0,40,100)
@RISK: =RiskTriang(0,40,100)
The use of UDFs makes it possible to trace through the logic of the model using Excels Formula Auditing
tools as illustrated below:
=SUM(SQRT(A1), A2)
Excel will evaluate A1 and A2, then evaluate SQRT(A1), and finally evaluate the SUM.
These rules are critical to having a predictable calculation outcome. However, this presents problems for
developers in creating functions that perform some operation on a distribution. For example, consider
the following @RISK function:
=RiskCompound(5, RiskLognorm(10,1))
The function sums 5 independent samples from a Lognormal(10,1) distribution. However, if we follow
Excels rules, the RiskLognorm function which is a random sampling function - must be evaluated
before the RiskCompound function. So, wed get something like this as an intermediary calculation:
=RiskCompound(5, 1.032)
in which case the RiskCompound would just be adding five 1.032 values together. To avoid this, @RISK
suppresses the evaluation of the RiskLognorm function. This causes problems of predicting how the
function behaves. For example, it is not immediately evident how the RiskCompound function should
behave in these circumstances:
=RiskCompound(5, RiskUniform(RiskUniform(2,3),4))
=RiskCompound(5, A2), where A2: =RiskNormal(A1,2)
ModelRisks approach is to use different functions when defining a distribution, called Objects, not the
same function used for sampling. Their equivalent function works as follows:
= VoseAggregateMC(5, VoseLognormalObject(10,1))
The LognormalObject function is used to define the distribution we are intending to use. Object
functions allow ModelRisk to comply with Excels evaluation rules. They are used in a variety of
ModelRisks tools.
Perpetual license + 1st year support $895 $1,195 $1,213 $1,595 $1,645 $2,195 $2,295 $2,433
Cost of additional year's support $161 ? $218 ? $296 ? $413 $438
# CPUs supported All 2 ? 2 All All All ?
Single click function view Yes No No No Yes No Yes No
Distribution fitting No No Yes Yes Yes Yes Yes Yes
Links to Microsoft Project No No No Yes No Yes No No
Aggregate modeling No No No Yes Yes Yes Yes No
Markov Chain simulation No No No No Yes No Yes No
Multivariate time series No No No No Yes No Yes No
Dynamic sorting No No No No Yes No Yes No
Optimizer No No No No Yes Yes Yes Yes
Stochastic data sharing No No No No No No Yes No
Specialist financial tools No No No No No No Yes No
Specialist insurance tools No No No No No No Yes No
Bayesian model averaging No No No No No No Yes No
Notes
Some additional factors have been included that are not already described in previous sections. The choice is somewhat subjective
but focused on features useful to a modeler.
5
https://shop.oracle.com/pls/ostore/product?p1=oraclecrystalball&p2=&p3=&p4=&sc=ocom_crystalball
6
www.vosesoftware.com/purchasing.php
7
www.palisade.com