1

INSE 6320 -- Week 2 Session 1

Risk Analysis for Information and Systems Engineering

Descriptive Statistics
Discrete Probability Distributions
Continuous Probability Distributions
Stochastic Processes

Dr. Babak Khosravifar Concordia University

 2

What does Risk Management mean?

One wants to modify the profit distribution in order to

satisfy the preferences of the decision maker
Probability
0.20
OR
0.18

0.16 INCREASE THESE

FREQUENCIES
REDUCE 0.14
THESE 0.12
FREQUENCIES
0.10

0.08

0.06

0.04

0.02

0.00
-300 -200 -100 0 100 200 300 400 500 600 700 800

Profit x
OR
BOTH!!!!
 3
Random Variables and Probability Density Functions
A random variable is a quantity whose value is not known exactly but its probability distribution is known. The
value of the random variable will vary from trial to trial as the experiment is repeated. The variables
probability density function (PDF) describes how these values are distributed (i.e. it gives the probability that
the variable value falls within a particular interval).
Continuous PDFs
f(x) All values between 0 f(x)
and 1 are equally likely Smallest values
are most likely

0 1 0
Uniform distribution x Exponential distribution x
(e.g. soil texture) (e.g. event rainfall)
0.3
f (x)

0.2 0.25 Probability that x = 2

A Discrete PDF 0.15
0.1 Only discrete
values (integers)
are possible
0 1 2 3 4 x
Discrete distribution
(e.g. number of severe storms)
 4
Mean and Variance of a Discrete Random Variable
 5

Expected Returns
Expected returns are based on the probabilities of possible outcomes
n
E ( R ) = pi Ri
i =1
In this context, expected means average if the process is repeated
many times
The expected return does not even have to be a possible return
 6

Example: Expected Returns

Suppose you have predicted the following returns for stocks C and T in
three possible states of nature. What are the expected returns?

State Probability C T

Boom 0.3 0.15 0.25

Normal 0.5 0.10 0.20
Recession ??? 0.02 0.01

RC = .3(.15) + .5(.10) + .2(.02) = .099 = 9.99%

RT = .3(.25) + .5(.20) + .2(.01) = .177 = 17.7%
 7

Variance and Standard Deviation

Variance and standard deviation still measure the volatility of returns

Using unequal probabilities for the entire range of possibilities

Weighted average of squared deviations

n
2 = pi ( Ri - E ( R))2
i =1
 8

Example: Variance and Standard Deviation

Consider the previous example. What are the variance and standard
deviation for each stock?

Stock C
s2 = .3(.15-.099)2 + .5(.1-.099)2 + .2(.02-.099)2 = .002029
s = .045

Stock T
s2 = .3(.25-.177)2 + .5(.2-.177)2 + .2(.01-.177)2 = .007441
s = .0863
 9

Another Example
Consider the following information:

State Probability ABC, Inc.

Boom .25 .15

Normal .50 .08
Slowdown .15 .04
Recession .10 -.03

What is the expected return?

What is the variance?
What is the standard deviation?
 10

Another Example
Consider the following information:
State Probability ABC, Inc.

Boom .25 .15

Normal .50 .08
Slowdown .15 .04
Recession .10 -.03

What is the expected return?

E(R) = .25(.15) + .5(.08) + .15(.04) + .1(-.03) = .0805
What is the variance?
Variance = .25(.15-.0805)2 + .5(.08-.0805)2 + .15(.04-.0805)2 + .1(-.03-.0805)2 = .00267475

What is the standard deviation?

Standard Deviation = .051717985
 11

Binomial Distribution
In many Geographic studies, we often face a situation where we deal with a random
variable that only takes two values, zero-one, yes-no, presence-absence, over a given
period of time. Since there are only two possible outcomes, knowing the probability of one
knows the probability of the other.

P(1)=p
P(0)=1-p=q

If the random experiment is conducted n times, then the probability for the event to happen
x times follow binomial distribution:

n x n - x n!
P( x) = p q = p x q n - x
x x!(n - x)!

Where n! the factorial of n. e.g. 5!=5*4*3*2*1=120.

 12
Binomial Distribution Example

For example, the presence-absence of drought in a year directly influence

the profit of agriculture due to irrigation costs added in a dry year. Suppose a
geographer is hired to do risk analysis for an Ag. Company whether a piece
of land is profitable for agriculture. Past experience shows that irrigation can
be afforded only one year in five. According to weather record, 4 out of the
last 25 years suffered from drought in the area.

Let 1 denote drought presence, and 0 denote drought absence, then

P(1)=4/25=0.16, so P(0)=1-0.16=0.84.

For 5 years, there are six possibilities of drought occurrence: 0, 1, 2, 3, 4, 5.

 13

Drought occurrence probability in 5 years

5!
P(0) = 0.160 0.845-0 = 0.418
0!(5 - 0)!
5!
P(1) = 0.161 0.845-1 = 0.398 The probability of profitable agriculture
1!(5 - 1)!
is summation of probabilities of no
5! drought and one drought in five years,
P(2) = 0.16 2 0.845-2 = 0.152
2!(5 - 2)! i.e. 0.418+0.398=0.816
5!
P(3) = 0.16 3 0.84 5-3 = 0.029 This the risk is 18.4% in five years.
3!(5 - 3)!

5!
P(4) = 0.16 4 0.845-4 = 0.003
4!(5 - 4)!

5!
P(5) = 0.165 0.845-5 = 0.000
5!(5 - 5)!
 14

Binomial Distribution Example

5!
P(0) = 0.160 0.845-0 = 0.418
0!(5 - 0)!
5!
P(1) = 0.161 0.845-1 = 0.398 The probability of profitable agriculture
1!(5 - 1)!
is summation of probabilities of no
5! drought and one drought in five years,
P(2) = 0.16 2 0.845-2 = 0.152
2!(5 - 2)! i.e. 0.418+0.398=0.816
5!
P(3) = 0.16 3 0.84 5-3 = 0.029 This the risk is 18.4% in five years.
3!(5 - 3)!

5!
P(4) = 0.16 4 0.845-4 = 0.003
4!(5 - 4)!

5!
P(5) = 0.165 0.845-5 = 0.000
5!(5 - 5)!
 15
Poisson Probability Distribution
The Poisson distribution is
e-l l x
f ( x) = x = 0,1, 2,...
x!
Where the parameter l>0 is the mean number of successes in the interval.
The mean and variance of the Poisson distribution are

= l and s 2 = l
 16

Poisson Distribution: Example

Suppose a geographer is assessing the risk of summer wheat yields to devastating
hailstorms in a particular geographic location. Weather records show that in the
past 35 years show that 10 years with no hailstorm, 13 years with one hailstorm, 8
years with 2 hailstorms, 3 years with 3 hailstorms, and 1 year with1 hailstorm.
Assume the occurrence of hailstorm is independent of past or future occurrences
and can be considered random. Then the number of hailstorms happening in any
given year follows Poisson distribution. In the above example, there are 42
hailstorms in 35 years, thus the mean number of hailstorms in a year is 1.2, then

e -1.21.20 e -1.21.21
P(0) = = 0.301 P(1) = = 0.361
0! 1!
e1.21.22 e1.21.23
P(2) = = 0.217 P(3) = = 0.087
2! 3!
e1.21.24
P(4) = = 0.026
4!
 17

Poisson Distribution: Example

Arrivals at a bus-stop follow a Poisson distribution with an average of 4.5 every
quarter of an hour.
Obtain a barplot of the distribution (assume a maximum of 20 arrivals in a quarter
of an hour) and calculate the probability of fewer than 3 arrivals in a quarter of an
hour.

e -1.21.20 e -1.21.21
P(0) = = 0.301 P(1) = = 0.361
0! 1!
e1.21.22 e1.21.23
P(2) = = 0.217 P(3) = = 0.087
2! 3!
e1.21.24
P(4) = = 0.026
4!
 18

Poisson Distribution: Example

The probabilities of 0 up to 2 arrivals can be calculated directly from the
formula
e-l l x
f ( x) = x = 0,1, 2,... with l =4.5
x!

p(1)=0.04999 and p(2)=0.11248

So the probability of fewer than 3 arrivals is 0.01111+ 0.04999 + 0.11248

=0.17358
 19
Normal Probability Distribution
The normal probability distribution is the The normal distribution is
most important distribution for describing 1 -
( x - )2

a continuous random variable.
It has been used in a wide variety of
applications:
Heights and weights of people
Test scores
Scientific measurements
Amounts of rainfall
It is widely used in statistical inference
f ( x) = e 2s 2
- < x <
s 2p
with mean and variance s 2
The normal distribution is: X : N ( , s 2 )
The visual appearance of the normal
distribution is a symmetric, unimodal or
bell-shaped curve as shown in the figure.

The mean value tells us where the

value x is concentrated most.
The variance tell us how the value is
spread. The larger the variance, the
more even the value spreads over a
large range. Is this good or bad?
 20
Calculating Normal Probabilities

We can use the following function to convert any normal random variable to a
standard normal random variable

Some advice: always

draw a picture!
 21
Calculating Normal Probabilities
P(45 < X < 60) ?
mean of 50 minutes and a
standard deviation of 10 minutes

0
 22

Examples:

F (0.76) = 0.776373
F (1.3) = ?
F (-3) = 1 - F (3) = ?
F (3.86) = ?
 23
Lognormal Distribution Probability Density Function

A random variable X is said to have the Lognormal Distribution with

parameters and s, where > 0 and s > 0, if the probability density
function of X is:
1
1 - 2 ( ln x - )
2

f ( x) = e 2s , for x >0
s 2p
, for x 0
f(x) 0

0 x
 24
Lognormal Distribution - Probability Distribution Function

If X ~ LN(,s),

then Y= ln (X) ~ N(,s)

ln x -
F ( x ) = P ( X x ) = F
s
where F(z) is the cumulative probability distribution function of N(0,1)
 25
Lognormal Distribution
Mean or Expected Value of X
1 2
+ s
X = E(X ) = e 2

Median of X

m edian = e
1
Standard Deviation of X
2
2 + 2 2
s X = e e - 1


 26
Lognormal Distribution - Example
A theoretical justification based on a certain material failure mechanism
underlies the assumption that ductile strength X of a material has a
lognormal distribution.
If the parameters are =5 and =0.1 ,
Find:
(a) x and x
(b) P(X >120)
(c) P(110 X 130)
(d) The median ductile strength
(e) The expected number having strength at least 120, if ten different
samples of an alloy steel of this type were subjected to a strength test.
(f) The minimum acceptable strength, If the smallest 5% of strength
values were unacceptable.
 27
Lognormal Distribution Example Solution
s 2
(a) +
X = E( X ) = e 2

=e 5.005

= 149.16

2 +s 2 s 2
s X = e (e - 1)
= 223
= 14.933
 28
Lognormal Distribution Example Solution
(b)
P( X > 120) = 1 - P( X 120)
ln120 - 5.0
= 1 - P( Z )
0.1
= 1 - F (-2.13)
= 1 - 0.0166
= 0.9834
 29
Lognormal Distribution Example Solution
(C)
ln110 - 5.0 ln130 - 5.0
P(110 X 130) = P( Z )
0.1 0.1
= P ( -2.99 Z -1.32)
= F ( -1.32) - F ( -2.99)
= 0.0934 - 0.0014
= 0.092
(d)

X 0.5 = median = e = e = 148 .41 5
 30
Lognormal Distribution Example Solution
(e) Let Y=number of items tested that have strength of at least 120
y=0,1,2,,10

p = P( X > 120)
= 1 - P( X 120)
ln120 - 5.0
= 1 - P( Z )
0.1
= 1 - F (-2.12)
= 1 - 0.0170
= 0.983
 31
Lognormal Distribution Example Solution

(f) The value of x, say xms, for which P( X < xms ) = 0.05 is
determined as follows:

ln xms - 5.0
and P( Z < ) = 0.05 ,
0.1
,
P( Z < -1.64) = 0.05
so that
ln xms - 5.0
= -1.64 ,
0.1
therefore xms = 125.964
 32
Exponential Distribution
A random variable X is defined to be exponential random variable (or
say X is exponentially distributed) with positive parameter if its
probability density function is given by:
l e- l x if x 0, l > 0
f ( x) =
0 if x < 0

- l x
f ( x) dx = l e - l x
Note: dx = - e =1
- 0 0

Thus, f(x) is a probability density function.

The cumulative distribution function:

x
F ( x) = P( X x) = f (t ) dt = 0
-
0
For x < 0, F ( x) = 0 dt = 0
-
x x
For x 0, F ( x) = l e- lt dt = -e- lt = 1 - e- l x
0 0
- l x
1 - e if x 0
F ( x) =
0 if x < 0
 33
Exponential Distribution
Expectation:

E[ X ] = xf ( x)dx = xl e - l x
dx = - x de - l x
- 0 0

Integration by part:

- l x
1 1
E[ X ] = - xe - (-e - l x ) dx = e - l x dx = - e - l x =
0 0 0 l 0 l

Variance:

E[ X 2 ] = x 2 f ( x)dx = x 2 l e - l x dx = - x 2 de - l x
- 0 0

Integration by part:
2 2 1 2
- (-e - l x )2 x dx = 2 xe - l dx =
l 0
E[ X 2 ] = - x 2e - l x xl e - l dx = =
0 0 0 l l l 2
2
1 1 2
Var [ X ] = E[ X ] - ( E[ X ]) = 2 - = 2
2 2

l l l
 34
Exponential Distribution: Example
The lifetime of an alkaline battery (measured in hours) is exponentially
distributed with l = 0.05. Find the probability a battery will last between 10 & 15
hours

P (10 X 15)
= F (15) - F (10)
P(10 X 15) = e - (0.05)(10) - e - (0.05)(15)
= e -0.5 - e -0.75
= 0.1341

There is about a 13%

chance a battery will only last
10 to 15 hours
 35
Gamma Distribution
A continuous random variable X is said to have a Gamma Distribution, if
the probability density function of X is
x
1 a -1
-
b
a x e for x 0,
f ( x; a , b ) = b G( a )
0
Otherwise

where a >0 and b>0

The Standard Gamma Distribution has b= 1
The parameter b is called the scale parameter because values other than
1 either stretch or compress the probability density function.
Important applications in waiting time and reliability analysis. Special cases
include exponential and chi-square distributions
 36
Gamma Function
Definition
For a > 0 , the Gamma Function G (a ) is defined by

G (a ) = x a -1 - x
e dx
0
Properties of the gamma function:

(1) For any a > 1 , G ( a ) = ( a - 1 ) G ( a - 1 )

(2) For any positive integer, n , G ( n ) = ( n - 1 ) !

1
(3) G = p
2
 37
Gamma Density Functions
1
f ( x;a , b ) a = 2, b = 0.5

0.8

0.6 a = 1, b = 1

0.4 a = 2, b = 2

a = 2, b = 1
0.2

0 x
0 2 4 6 8

If X~G(, ), then
Mean or Expected Value: = E ( X ) = a b
Standard Deviation: s = b a
 38
Stochastic Process - Introduction

Stochastic processes are processes that proceed randomly in time.

Rather than consider fixed random variables X, Y, etc. or even

sequences of i.i.d. random variables, we consider sequences X0, X1,
X2, ., where Xt represent some random quantity at time t.

In general, the value Xt might depend on the quantity Xt-1 at time t-1,
or even the value Xs for other times s < t.

Example: simple random walk .

 39
Stochastic Process - Definition

A stochastic process is a family of time indexed random variables Xt

where t belongs to an index set. Formal notation, {X t : t I } where I is
an index set that is a subset of R.
Examples of index sets:
1) I = (-, ) or I = [0, ]. In this case Xt is a continuous time
stochastic process.
2) I = {0, 1, 2, .} or I = {0, 1, 2, }. In this case Xt is a discrete
time stochastic process.
We use uppercase letter {Xt } to describe the process. A time series,
{xt } is a realization or sample function from a certain process.
We use information from a time series to estimate parameters and
properties of process {Xt }.
 40
Probability Distribution of a Process

For any stochastic process with index set I, its probability

distribution function is uniquely determined by its finite dimensional
distributions.

The k dimensional distribution function of a process is defined by

FX t
1
,..., X t k (x1 ,..., xk ) = P(X t
1
x1 ,..., X tk xk )
for any t1 ,..., t k I and any real numbers x1, , xk .

The distribution function tells us everything we need to know about

the process {Xt }.
 41
Moments of Stochastic Process

We can describe a stochastic process via its moments, i.e.,

E ( X t ), E (X t2 ), E ( X t X s ) etc. We often use the first two moments.

The mean function of the process is E ( X t ) = t .

The variance function of the process is Var ( X t ) = s t2 .

The covariance function between Xt , Xs is

Cov( X t , X s ) = E (( X t - t )( X s - s ))
The correlation function between Xt , Xs is
Cov ( X t , X s )
r ( X t , X s ) =
s t2 s s2

These moments are often function of time.

 42

Counting process
A stochastic process {N(t) : t 0} is a counting process if N(t) represents
the total number of events that occur by time t.
eg, # of persons entering a store before time t, # of people who were
born by time t, # of goals a soccer player scores by time t.

N(t) should satisfy:

N(t)>0
N(t) is integer valued
If s<t, then N(s)< N (t)
For s<t, N(t)-N(s) equals the number of events that occur in (s, t]
 43

The Poisson Process

1st Event 2nd Event 3rd Event 4th Event
Occurs Occurs Occurs Occurs
X1 X2 X3 X4

1 2 3 4
time
t=0 S1 = X i S 2 = X i S 3 = X i S 4 = X i
i =1 i =1 i =1 i =1

X1, X2, represent a sequence of positive independent random variables with

identical distribution
Xn depicts the time elapsed between the (n-1)th event and nth event occurrences
Sn depicts a random variable for the time at which the nth event occurs
Define N(t) as the number of events that have occurred up to some arbitrary time t.

The counting process { N(t), t>0 } is called a Poisson process if the inter-
occurrence times X1, X2, follow the exponential distribution
 44

The Poisson Process: Example

For some reason, you decide everyday at 3:00

PM to go to the bus stop and count the number
of buses that arrive. You record the number of
buses that have passed after 10 minutes

Sunday N (t=10 min) = 2

1st Bus 2nd Bus 3rd Bus 4th Bus
Arrival Arrival Arrival Arrival
X1=5 min X2=4 min X3=7 min X4=2 min

time
t=0 S1 = 5 min S2 = 9 min S3 = 16 min S4 = 18 min
 45

The Poisson Process: Example

For some reason, you decide everyday at 3:00

PM to go to the bus stop and count the number
of buses that arrive. You record the number of
buses that have passed after 10 minutes

Monday N (t=10 min) =4

1st Bus 2nd Bus 3rd Bus 4th Bus 5th Bus
Arrival Arrival Arrival Arrival Arrival
X1=1 min X2=2 min X3=4 min X4=2 min X5=6 min

time
t=0 S1 = 1 min S2 = 3 min S3 = 7 min S4 = 9 min S5 = 15 min
 46

The Poisson Process: Example

For some reason, you decide everyday at 3:00

PM to go to the bus stop and count the number
of buses that arrive. You record the number of
buses that have passed after 10 minutes

Tuesday N (t=10 min) =1

1st Bus 2nd Bus
Arrival Arrival
X1=10 min X2=6 min

time
t=0 S1 = 10 min S2 = 16 min
 47

The Poisson Process: Example

Given that Xi follow an exponential distribution then N(t=10) follows

a Poisson Distribution