Quantitative Risk Management For Homeland Security

Quantitative Risk Management
Tom Tuduc
1
tomtuduc@webarches.com 09/25/08
Introduction
 Risk is ubiquitous - We are all familiar with risks:

 RISK board games and video games.
 Download risk-free product trials
 Buy products that reduce risks of illness
 Terrorist threats
 Take calculated risks (video clip ReturnOfTheKing)
 Avoid running the risk of ...
 Risk attitude
 Eliminating risk by getting more information and/or controlling
outcome (video clip ValueInfoControl)
 Who is at risk, high risk groups (video clip highrisk)
2
Summary
 Homeland Security is complex and include uncommon and/or hypothetical
uncertainties. It takes both qualitatively and quantitatively models to
consider hundreds of intelligences with different credibility and accuracy.
 Topics:
1. How Influence Diagrams/Decision Analysis help experts communicate

and model Homeland Security decisions, threats, and countermeasures
visually, qualitatively, and quantitatively.
2. How Decision Analysis enable calculations that maximize security, make

decision policies, quantify insights of each threat factor, and the worth of
additional information and control on each factor.
3. A review of several examples in the literature: influence diagrams in

terrorist threat countermeasures, early warming systems, toxin containment
policies, and intrusion-aware information systems.
4. Security categories, Application trends, technology integration

possibilities, and online resources
3
Table of Content
 Introduction
 Summary
 Table of Content
 What is Security Analytics?
 Security Risk Methodology - the Four Steps
 Risk Management: Dealing with Uncertainty  Homeland Security Infrastructures & Assets
 Example of Security Application Areas
 Characteristics of complex risk problem,  Homeland Security - System View
 Decision Analysis & Influence Diagrams  Homeland Security - Decision View
 Tradeoffs & Risk Preference  Infrastructure Elements
 Differences between Trees and Diagrams  Homeland Security
 Certainty Equivalence, Utility & Risk Premium Decision Analysis & Influence Diagrams Exam
 Risk taking
 Risk averse
 Example 1
 Tutorial Example  Overarching Influence Diagram
 Best Policy and value of Control  Example 2: Site Profiler
 Risk Profiles  Architecture & Influence Network
 Gaining Insights  Example 3 – Using Analytica
 Sensitivity Analysis
 Similar security ROI starting point  Security Categories
 Similar Intrusion Detection problem  Where are the numbers
 A more complex party problem  Tools & Resources
 A more complex security ROI problem  Conclusion
 Complex Intrusion-Aware Model
4
Overview
 DEFINITION: Security Analytics (Table1) are the use of analytics to
optimize security and security ROI.
 Applications:
– Model
– Processes
– Policies
– Systems
Probability Dynamic
Statistics
Stochasti programming
c
Graph theory Markov Game

theory
Information
Utilit
theory Negotiations
y
theor
y 5
Security Methodology - the Four Steps
 1. Determine risk:
– Assets and risk to assets.
– Making security ROI known.
 2. Analyze risk: *
– Qualitative
– Quantitative: Analytics
 3. Design and Implement: policies, architectures, technologies, trainings,
and countermeasures
 4. Management: Monitoring, audits, and evaluate
* "One of the major problems is that security risk assessment and the benefits of using the
results of risk assessment cannot be measured in any sufficiently accurate to provable
way... Positive benefit is absence of unknown possible loss" Tom Peltier, "Risk Analysis
Vs. Security Controls." NetSec 2002
6
Risk Management: Dealing with Uncertainty
Fundamental Approaches
 Frequentist
– Based on hundreds or thousands of events.

– Probability lies objectively in the world, not in the observer.
 Bayesian
– Based on personal experience.
– Probability is different for people having different past experiences.
7
Example of Security Applications
 Security ROI
 Risk assessment and management
 Knowledge management and Information retrieval (1)
 SPAM filtering (2)
 Intrusion Detection Systems
 Other examples: Search engines, portfolio management, polling, etc.
(1) 21 US agencies with 200,000 employees has deployed Autonomy, a knowledge

management tool based on Bayes and Shannon theorems, for homeland security functions
(Business Weekly, 31 October 2002.)
(2) Baysian-based SPAM filters: http://www.webarches.com/filters.html
8
Characteristics of complex problems
 Many uncertainty/probabilities cannot be obtained from empirical

frequency distributions because the events are uncommon and/or
hypothetical.
 Probabilities come from expert opinions with different experience of the
same problem
 In a closed-loop system, the probabilities improves over time with
repeated cycles. Time is a luxury not always available.
9
Decision Analysis/ Influence Diagram (DAID)
 Advantages
risks, decisions, threats, and

model
countermeasures
communicate visually, qualitatively, and quantitatively.
10
Modeling
 Decisions: made by the decision maker

 Uncertain events: events with discrete outcomes or probability functions
 Consequences: values resulting from the decisions and uncertain events
outcomes
 Risk Preferences: how the decision maker feels about the consequences (1)
 Objectives: direction and value, i.e. eliminate risk areas, maximizing ROI,
minimizing loss of data and/or resources.
(1) Will the real risk-preference stand up: A popular misconception is that security
managers in private sectors are risk-averse and overspend on security. IDC research
data shows otherwise. A typical organization of 5,000 employees, on average spends
$1 million on security products ($200/person, or $500 for each $1 million in
revenues).
11
Tradeoffs & Risk Preference
 Conflicting objectives: A policy may be optimal for one objective, but

not for all objectives, i.e. how much expected loss of data availability is
an agency willing to accept to increase data integrity to 100 percent.
 Tradeoffs (conflicting objectives): 10 percent loss in data integrity is
equivalent to 50 percent loss in data availability
 Risk Preference: which Risk Profile is your organization's

•High risk-taker: black
•Low risk-taker :blue
(video clip riskProfiles)
12
Differences between Decision Trees and Influence Diagrams
 Influence diagrams show

dependencies among variables
clearly: good visuals for
communication and qualitative
relationship.
 Influence diagrams are compact -
one or two order of reduction in
node representation in typical
problems.
 Decision trees show details of
possible paths/scenarios:
relatively good visual for small
problems. Best for quantitative
calculations.
 Decision trees show asymmetric
outcome trees.
13
14
Certainty Equivalence, Utility & Risk Premium
 Common decision rule: maximize expected value, often expected

monetary value. However, this is not realistic for the risk-averse.
 Better decision rule: Expected Value with minimum risk variance
(portfolio investment)
 Best decision rule: maximize expected utility. Utility is found by
presenting simple lotteries to decision makers.
 Certainty Equivalence: taking monetary equivalence instead of
playing the lottery.
 Risk premium: EV of lottery - CE of lottery
15
RISK TAKING
 Risk premium = EV - CE or -$2.

 CE is larger than EV
 Buying a lot of superlotto tickets is risk taking
16
RISK AVERSE
 Even though EV is higher now (50 versus 23), Certainty equivalence is lower (25 versus -5)
 Risk premium = 50 - 5 = $55.
 CE is less than EV
This is analogous to hiring a consultant, or outsourcing instead of performing a function internally.
17
Tutorial Example
18
19
Best Policy and Value of Control
The Influence Diagram and Decision Tree show the Location Decision is
made independent of knowing the weather
Conclusion:
 If we don't know what the weather will be, we should locate it on the
porch because that has the highest payoff of $38 million.
 Best case saving: (60-38) or $22 million. This is Value of Control *
* The Department of Energy benefits by eliminating security-update risks (Value of Control)

when it required Oracle to deliver its 9i database with all security features and to take
responsibility of maintaining security updates. This is an unusual but excellent example of
cyber-security practice.
20
Risk Profiles
 Locating the party by the pool can give negative utility if it rains. But if it’s sunny it’s the
best decision. If it’s cloudy, it might rain.
21
Gaining Insights
 1. If we know what the weather will
be, we can make a better decision.
Thus the new expected payoff is now
$47 million, instead of $38 million.
 2. If we want to ask a security expert

(clairvoyant) about what the weather
will be, we should only pay a
maximum of $9 million (new
expected payoff - old expected
payoff)
– New expected payoff: (.2*45) +
(.5*40) + (.3*60)= 47
– Old expected payoff: $38
 3. New Value of Control: new best

case saving is (60- Expected Value)
= 60-47 = 13
22
Sensitivity Analysis
23
Similar Security ROI starting point
 Budgets: basic security (firewall, VPN, antiVirus), audits, realtime intrusion

detection, advanced access control, encryption, etc.
24
Similar Intrusion Detection problem
25
26
27
Complex Intrusion-Aware Model
TRIAD (Trustworthy Refinement through Intrusion-Aware Design): an intrusion-

aware model developed at CMU/SEI (TECHNICAL REPORT CMU/SEI-2003-
TR-002)
SUMMARY
PROBLEM: Military and business systems face increasingly sophisticated and
coordinated computer network attacks. Existing security system development
are typically isolated solutions resulting in patchwork designs that are not robust
under attack.
TRIAD, a model, helps IT decision makers to formulate and maintain a coherent

and justifiable survivability strategy that addresses mission-compromising
threats. TRIAD uses DAID to model the dynamics of fraud and authentication.
TRIAD's goals are:
 Develop a development methodology for security systems to resist, recognize,

recover from, and adapt to mission-compromising attacks.
 to provide a documented response to the primary threats to the mission;
 to provide a justification for and the limitations of the system design;
 to support the design and implementation of the desired system behavior across
multiple systems and multiple development teams; and
 to support maintenance and evolution as the system operations and threat
environment evolve over time.
28
29
Homeland Security Infrastructures & Assets
 Critical Infrastructures  Key Assets

– Agriculture – National Monuments
– Food – Dams
– Water – Nuclear Power Plants
– Public Health – Government Facilities
– Emergency Services – Commercial Key Assets
– Government
– Defense Industrial Base
– Information and
Telecommunications
– Energy
– Transportation
– Banking and Finance
– Chemical Industry and
Hazardous
– Materials
– Postal and Shipping
30
Homeland Security- System View
31
Homeland Security- Decision View
32
Infrastructure elements
33
Homeland Security
Decision Analysis & Influence Diagrams Examples
2. Probabilistic Modeling of Terrorist Threats: A Systems Analysis

Approach to Setting Priorities Among Countermeasures
3. Site Profiler, a system being used in bio-terrorism early warning
systems, passenger and cargo profiling, vulnerability assessments,
threat warnings and dissemination.
4. Using Analytica: Toxin Containment Model and Analysis
5. TRIAD (Trustworthy Refinement through Intrusion-Aware Design): an
intrusion-aware model
6. Others: GIS and Decision Analysis Journal, COPLINKS (Search and
match given incomplete information), Paper "Warning and Response in
Homeland Security“, and Sandia/CA’s Weapons of Mass Destruction
Decision Analysis Center
34
Example 1 & Influence Diagram
Probabilistic Modeling of Terrorist Threats: A Systems Analysis Approach to
Setting Priorities Among Countermeasures. Elisabeth Paté-Cornell and Seth
Guikema. Department of Management Science and Engineering. Stanford
University. Military Operations Research, Vol. 7, No 4, pp. 5-20 December
2002.
SUMMARY
PROBLEM: assess the benefits of risk reduction by different countermeasures

and their costs
OBJECTIVE:
- Prioritize the protection of US infrastructures, networks and socio-economic
components
- Discover most effective means of reducing the overall threat, i.e. the
disruption of the terrorists’ supply chain
- Prioritize intelligence information that needs to be gathered given accuracy,
time, and constraints.
35
36
Example 2 – Architecture and Diagram
Site Profiler, a system developed by Bryan Ware, Anthony Beverina, Lester Gong,
and Brian Colder at Booz Allen Hamilton and Digital Sandbox. Site Profiler is
used in bio-terrorism early warning systems, passenger and cargo profiling,
vulnerability assessments, threat warnings and dissemination. Site Profiler
applies DAID to combined data from various sources.
SUMMARY
PROBLEM: Build a system to sift through massive amount of information to
determine terrorist risk
OBJECTIVE: Determine the following:
 how likely a terrorist will attempt attacks including tactic, weapon, delivery
system against an asset
 how likely the terrorist will succeed
 consequences of successful attacks
CHALLENGES:
 High volumes of data.
 Disparate sources of data and information
 Diverse forms of information
 Significant organizational friction among producers, owners, and consumers of
information
37
38
39
Example 3 – Using Analytica
Using Analytica to model and analyze the cost and benefit of Toxin
Containment (Adopted from Analytica’s Toxic Emission Control)
SUMMARY
PROBLEM: Determine costs and benefits of containing an airborn toxin
that is potentially fatal.
Objective: Maximize the expected benefit, defined as benefits(1) less the

cost(2) to contain toxin.
(1) Cost to contain toxins depends on the containment level

(logarithmic)
(2) Benefits as the reduced mortality multiplied by the value of a life
40
Problem: how much to contain and eliminate certain toxins
including the option of reducing them by zero.
41
42
Security Categories
 Access Controls, Authentication  Media Protection Safes
 Anti-eavesdropping  Media Security
 Anti Virus  Physical/Facility Security - Anti-
 Virus protection/detection Theft Devices
 Automated Patch Management  Physical/Facility Security
 Biometrics Authentication of -Entrance Control Systems
users/terminals  Physical/Facility Security -
 Business Continuity & Disaster Environmental Controls
Recovery  Physical/Facility Security - Power
 Content Delivery Network Security Management
 Email spam filters
 Risk Management Risk Analysis
 Encryption
 Security Incident Management
 Extranet Security Integration
 Single Sign On
 Firewalls and Internet Security
 Software Controls
 Intrusion Detection & Network
 Telecom & Remote Access
Monitoring Security
 Media Security Destruction Devices
 Wireless Security
43
Where to find statistics
 1. Symantec Internet Security Threat Report Volume IV - Every six months.

– During the first half of 2003, Symantec saw a 50% increase in confidential data
attacks using backdoors.
– In the past six months, Web application vulnerabilities increased 12 percent,
malicious codes were up 20 percent, and worms and viruses increased 19 percent
 2. Computer Security Institute/ FBI Computer Crime and Security Survey - Yearly
 3. @Stake Advisories and Research Labs (see Table below)
44
Tools
 Traditional Decision Analysis  Risk management tools:

and/or Influence Diagrams:
Analytica, DLP,LHS,
Analytica, DATA, Decide, Fuldek,SAPHIRE, SETS, SANET,
SABLE, FTAP, SEATree,
DecisionPro, DPL, Expression Stepwise,
Tree, Precision Tree, Risk
Detective,
Supertree/Sensitivity, TreePlan.
45
The End
"[T]he U.S. Air Force …is faced with a multitude of decisions-

programmatic, technical, personnel, strategic, and yes,
cultural - that we must make based on knowledge of, and
respect for, the relevant underlying data. In that spirit …
operations research and decision analysis are and will
continue to contribute to national security decision-
making."- Secretary of the Air Force James G. Roche,
OR/MS December 2002
46

Quantitative Risk Management For Homeland Security

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Quantitative Risk Management For Homeland Security

Hochgeladen von

Copyright:

Verfügbare Formate

Quantitative Risk Management

 Risk is ubiquitous - We are all familiar with risks:

1. How Influence Diagrams/Decision Analysis help experts communicate

2. How Decision Analysis enable calculations that maximize security, make

3. A review of several examples in the literature: influence diagrams in

4. Security categories, Application trends, technology integration

Graph theory Markov Game

– Based on hundreds or thousands of events.

(1) 21 US agencies with 200,000 employees has deployed Autonomy, a knowledge

(2) Baysian-based SPAM filters: http://www.webarches.com/filters.html

 Many uncertainty/probabilities cannot be obtained from empirical

risks, decisions, threats, and

communicate visually, qualitatively, and quantitatively.

 Decisions: made by the decision maker

 Conflicting objectives: A policy may be optimal for one objective, but

(video clip riskProfiles)

 Influence diagrams show

 Common decision rule: maximize expected value, often expected

 Risk premium = EV - CE or -$2.

This is analogous to hiring a consultant, or outsourcing instead of performing a function internally.

* The Department of Energy benefits by eliminating security-update risks (Value of Control)

 2. If we want to ask a security expert

 3. New Value of Control: new best

 Budgets: basic security (firewall, VPN, antiVirus), audits, realtime intrusion

TRIAD (Trustworthy Refinement through Intrusion-Aware Design): an intrusion-

TRIAD, a model, helps IT decision makers to formulate and maintain a coherent

 Develop a development methodology for security systems to resist, recognize,

 Critical Infrastructures  Key Assets

2. Probabilistic Modeling of Terrorist Threats: A Systems Analysis

PROBLEM: assess the benefits of risk reduction by different countermeasures

Objective: Maximize the expected benefit, defined as benefits(1) less the

(1) Cost to contain toxins depends on the containment level

 1. Symantec Internet Security Threat Report Volume IV - Every six months.

 Traditional Decision Analysis  Risk management tools:

"[T]he U.S. Air Force …is faced with a multitude of decisions-

Das könnte Ihnen auch gefallen