Uq Course Lecture2

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/319306526
Lecture 2: Elements of Probability Theory
Presentation · August 2017

DOI: 10.13140/RG.2.2.36484.01929
CITATIONS READS
0 255
1 author:
Americo Cunha Jr
Rio de Janeiro State University
119 PUBLICATIONS 181 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Flight Control and Trajectory Planning of Balloon-Multicopter Aerial Vehicles View project
Modeling of Zika virus outbreak in Brazil View project
All content following this page was uploaded by Americo Cunha Jr on 27 August 2017.
The user has requested enhancement of the downloaded file.

Lecture 2
Elements of Probability Theory
Americo Barbosa da Cunha Junior
Universidade do Estado do Rio de Janeiro (UERJ)

NUMERICO – Nucleus of Modeling and Experimentation with Computers
numerico.ime.uerj.br
americo@ime.uerj.br
www.americocunha.org
UNESP Ilha Solteira

22 - 25 de agosto de 2017
Ilha Solteira - SP, Brasil

c A. Cunha Jr (UERJ) Modeling and Quantification of Uncertainties in Physical Systems 1 / 60
Probability in Dimension 1

Random experiment
An experiment which repeated under same fixed conditions produce
differente results is called random experiment.
Examples:
1 Rolling a cube-shaped fare die
2 Choosing an integer even number randomly
3 Measuring temperature

Probability space
The mathematical framework in which a random experiment is de-

scribed consists of a triplet (Ω, Σ, P), called probability space.
The elements of a probability space are:

Ω: sample space
(set with all possible events)
Σ: σ-algebra on Ω
(set with relevant events only)
P: probability measure
(measure of expectation of an event occurrence)

Sample space
A non-empty set which contains all possible events for a certain

random experiment is called sample space, being represented by Ω.
Examples:
1 Rolling a cube-shaped fare die (finite Ω)
Ω = {1, 2, 3, 4, 5, 6}
2 Choosing an integer even number randomly (denumerable Ω)
Ω = {· · · , −8, −6, −4, −2, 0, 2, 4, 6, 8, · · · }

3 Measuring temperature in Kelvin (non-denumerable Ω)
Ω = [a, b] ⊂ [0, +∞)

σ-algebra of events
In general, not all of the events in Ω are of interest.
Intuitively, a σ-algebra on Ω is the set of relevant outcomes for a

random experiment. Formally, Σ is a σ-algebra on Ω if
φ∈Σ
(contains the empty set)
Ac ∈ Σ for any A ∈ Σ
(closed under complementation)
[∞
Ai ∈ Σ for any Ai ∈ Σ
i=1
(closed under denumerable unions)

Examples of σ-algebras
Sample space: Ω = {1, 2, 3, 4, 5, 6}
Are σ-algebras:

Σ = φ, {1, 3, 5}, {2, 4, 6}, Ω

Σ = φ, {1, 2, 3, 4}, {5, 6}, Ω
Σ = 2Ω (set of all subsets)
Are not σ-algebras:

Σ = φ, {1}, {2}, {3}, {4}, {5}, {6}, Ω

Σ = φ, {1, 2}, {3, 4}, {5, 6}, Ω
G. Grimmett and D. Welsh, Probability: An Introduction. Oxford University Press, 2 edition, 2014.

Probability measure
A probability measure is a function P : Σ → [0, 1] ⊂ R such that
P {A} ≥ 0 for any A ∈ Σ

(probability is nonnegative)
P {Ω} = 1
(entire space has probability one)
 
[ ∞  X ∞
P Ai = P {Ai } for any Ai mutually disjoint
 
i=1 i=1
(σ-additivity)
Remark:
P {φ} = 0 (empty set has probability zero)

An example in discrete probability
A fair coin is thrown twice.

The number of faces is of interest.
A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

Probabilidade, SBM, 2016


Probability space 1:
Ω1 = {(H,H), (H,T ), (T ,H), (T ,T )}



Ω1 = {(H,H), (H,T ), (T ,H), (T ,T )}
P1 {(H,H)} = 1/4, P1 {(H,T )} = 1/4,
P1 {(T ,H)} = 1/4, P1 {(T ,T )} = 1/4



Ω1 = {(H,H), (H,T ), (T ,H), (T ,T )}
P1 {(H,H)} = 1/4, P1 {(H,T )} = 1/4,
P1 {(T ,H)} = 1/4, P1 {(T ,T )} = 1/4
Ω2 = {0, 1, 2}



Ω1 = {(H,H), (H,T ), (T ,H), (T ,T )}
P1 {(H,H)} = 1/4, P1 {(H,T )} = 1/4,
P1 {(T ,H)} = 1/4, P1 {(T ,T )} = 1/4
Ω2 = {0, 1, 2}
P2 {0} = 1/4, P2 {1} = 1/2, P2 {2} = 1/4


An example in continuous probability
A point is randomly choosen in a square (side b).

Event A: point lie above main diagonal
Event B: point lie in main diagonal
Event C: point lie outside main diagonal


area of upper triangle 0.5 b2 1

P {A} = = 2
=
area of square b 2



P {A} = = 2
=
area of square b 2
area of main diagonal 0

P {B} = = 2 =0
area of square b



P {A} = = 2
=
area of square b 2
area of main diagonal 0

P {B} = = 2 =0
area of square b
area outside main diagonal b2 − 0

P {C } = = =1
area of square b2

Remarks on probability
Probability zero
An impossible event has probability zero
(e.g. roll a six faces dice, numbered from 1 to 6, and get 7)
Not every event with probability zero is impossible
(e.g. randomly pick a point on the main diagonal of a square)
Probability one
An event whose occurrence is certain has probability one
(e.g. throw a coin and obtain head or tail)
Not every event with probability one occurs
(e.g. randomly pick a point outside square’s main diagonal)

Conditional probability
Consider a pair of random events A and B such that P {B} > 0.
The conditional probability of A, given the occurrence of B, denoted

as P A|B , is defined as
P {A ∩ B}
P A|B = .
P {B}
It follows that

P {A ∩ B} = P A | B × P {B} .

An example in conditional probability
Somebody rolls a pair of six-sided dice.

A = value rolled on die 1
B = value rolled on die 2
What is the probability that A = 2 given that A + B ≤ 5?
Conditional probability — Wikipedia, The Free Encyclopedia, 2017.


A=2


A=2
P {A} = 6/36


A=2 A+B≤5
P {A} = 6/36


A=2 A+B≤5
P {A} = 6/36 P {A+B≤5} = 10/36


A=2 A+B≤5 A | A+B≤5
P {A} = 6/36 P {A+B≤5} = 10/36


A=2 A+B≤5 A | A+B≤5
P {A} = 6/36 P {A+B≤5} = 10/36 P {A | A+B≤5} = 3/10

Independence of events
If the occurrence of an event B does not affect the occurrence of an

event A one has

P A | B = P {A} .

In this way, once P {A ∩ B} = P A | B × P {B} it is true that
P {A ∩ B} = P {A} × P {B} .
Events A and B in which the latter holds are said to be independent.
Remark:
This notion generalizes itself naturally to n events.

An example in events independence
A card is drawn from a deck with 52 unknown cards.
Event 1: Q “queen”
Event 2: ♠ “spade”
Are these events independent?

Probability space 1 (fair deck):
 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 
P1 {Q } = 4/52

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 
P1 {Q } = 4/52
P1 {♠} = 13/52

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 
P1 {Q } = 4/52
P1 {♠} = 13/52
P1 {Q♠} = 1/52

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 
P1 {Q } = 4/52
P1 {♠} = 13/52
P1 {Q♠} = 1/52 = 4/52 × 13/52
|{z} | {z }
P1 {Q} P1 {♠}

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♣,3♣,4♣,5♣,6♣,7♣,8♣,9♣,10♣,J♣,Q♣,K ♣,A♣,
cards =

 2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥, 

 
 2♠,3♠,4♠,5♠,6♠,7♠,8♠,9♠,10♠,J♠,Q♠,K ♠,A♠ 
P1 {Q } = 4/52
P1 {♠} = 13/52
P1 {Q♠} = 1/52 = 4/52 × 13/52
|{z} | {z }
P1 {Q} P1 {♠}
Events are independent.


Probability space 2 (unfair deck):
 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 
P2 {Q } = 28/52

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 
P2 {Q } = 28/52
P2 {♠} = 1/2

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 
P2 {Q } = 28/52
P2 {♠} = 1/2
P2 {Q♠} = 1/2

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 
P2 {Q } = 28/52
P2 {♠} = 1/2
P2 {Q♠} = 1/2 6= 28/52 × 1/2
| {z } |{z}
P2 {Q} P2 {♠}

 

 2♦,3♦,4♦,5♦,6♦,7♦,8♦,9♦,10♦,J♦,Q♦,K ♦,A♦, 


 

2♥,3♥,4♥,5♥,6♥,7♥,8♥,9♥,10♥,J♥,Q♥,K ♥,A♥,
cards =

 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 

 
 Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠,Q♠, 
P2 {Q } = 28/52
P2 {♠} = 1/2
P2 {Q♠} = 1/2 6= 28/52 × 1/2
| {z } |{z}
P2 {Q} P2 {♠}
Events are not independent.

Further remarks on probability
The last example shows that:
Different probability spaces, for the same random experiment,

can produce different predictions
A probability space that does not accurately describe a random

event can produce completely erroneous predictions
The notion of independence strongly depends on the probability

measure employed

Random variable
A mapping X : Ω → R is called a random variable (RV) if the

preimage of every real number under X is a relevant event, i.e.,
X −1 (x) = ω ∈ Ω : X (ω) ≤ x ∈ Σ, for every x ∈ R.

A collection of events in Ω is
mapped to an interval E on the
real line under such mapping.
RV are numerical characteristics of interesting events.
Remark:
A random variable is a function from Ω to R, not a real number.
*Picture from http://theanalysisofdata.com/probability/2_1.htmll

Examples of random variables
1 Rolling die experiment

Ω = {1, 2, 3, 4, 5, 6}

Σ = φ, {1, 3, 5}, {2, 4, 6}, Ω
(
1 if ω is even
X1 (ω) = X2 (ω) = ω 2
0 if ω is odd
(random variable) (not a random variable)
2 Temperature (in Kelvin) measurement experiment

Ω = [a, b] ⊂ [0, +∞) Σ = B[a,b] (Borel σ-algebra)
X (ω) = −459.67 + 1.8 ω
(random variable)

Probability distribution
The probability distribution of X , denoted by FX , is defined as the

probability of the elementary event {X ≤ x}, i.e.,
FX (x) = P {X ≤ x} .
FX is also known as cumulative distribution function (CDF) and has

the following properties:
0 ≤ FX (x) ≤ 1
FX is a monotonic, non decreasing, right continuous function
Z x2
P{x1 < X ≤ x2 } = FX (x2 ) − FX (x1 ) = dFX (x)
x1
Z
dFX (x) = 1
R
FX (−∞) = 0 and FX (+∞) = 1

Probability density function
If the function FX is differentiable, its derivative pX (x) = dFX (x)/dx

is called probability density function (PDF) of X , and one has
Z x
FX (x) = pX (ξ) dξ.
−∞
Note also that:

pX (x) ≥ 0 for every x ∈ R
Z
pX (x) dx = 1
R
Remark:
Intuitively, pX (x) dx can be thought of as the probability of X
falling within the infinitesimal interval [x, x + dx].

Types of random variables
1 Discrete random variable

Distribution is discontinuous.
Assumes a denumerable number of values.
Typically associated with counting processes.
2 Continuous random variable
Distribution is continuous.
Assumes a non-denumarable number of values.
Typically associated with measuring processes.
3 Mixed random variable
Distribution has points of discontinuity.
Assumes a non-denumarable number of values.
A “mixture” of the two previous types.
4 Singular random variable
Distribution is different from the other types.
It has theoretical interest only.

Function of random variable
The function of random variable Y = h (X ), for a random vari-
able X and measurable mapping h : R → R, is also a random
variable, which has its own probability distribution.
Example:
Let h(x) = x 2 and X : Ω → [1, 2] be a random variable such that

0 if x < 1


FX (x) = 1/2 if 1 ≤ x ≤ 2

1

if x > 2.
The composition Y = X 2 is a random variable with distribution


0 if y < 1


√
FY (y ) = 1/2 y if 1 ≤ y ≤ 4

1

if y > 4.

Mathematical expectation operator
The mathematical expectation of a random variable X is defined as
Z
E {X } = x dFX (x).
R
The mathematical expectation is a linear operator since
E {α1 X1 + α2 X2 } = α1 E {X1 } + α2 E {X2 } ,
for any pairs of number α1 , α2 and random variables X1 , X2 .

Theorem (law of the unconscious statistician):
Given a measurable mapping h : R → R and a random variable X
the expected value of h (X ) is given by
Z

E h (X ) = h (x) dFX (x).
R

Statistical moments
n o
The value E (X − b)k is called k-th moment around b of the ran-
dom variable X , for b ∈ R and k = 1, 2, · · · .
n k o
For b = E {X }, E X − E {X } is dubbed k-th central moment
of the random variable X .
n o
When b = 0, E X k is simply called k-th moment of the random
variable X .

Mean value
The mean value of the random variable X is defined as
µX = Z {X }
E
= x dFX (x)
ZR
= x pX (x) dx.
R
(measure of the central tendency)
Remark:
The mean value µX is the constant which best approximate the
random variable X . The error of this approximation is the standard
deviation σX .

Variance
The variance of the random variable X is defined as
n o
σX2 = E (X − µX )2
Z
= (x − µX )2 dFX (x)
ZR
= (x − µX )2 pX (x) dx.
R
(measure of dispersion about the mean)
Note that variance can also be written as

n o 2
σX2 = E X 2 − E {X } .
Remark:
σX2 has the same unit as X 2 .

Standard deviation and variation coefficient
Other second-order statistics of X are the standard deviation

q
σX = σX2 ,
and the variation coefficient
δX = σX /µX , µX 6= 0.
(both are measures of dispersion about the mean)
Remark:
σX has the same unit as X and δX is dimensionless.

Sknewness
The skewness of the random variable X is defined as

( )
X − µX 3
Skew [X ] = E
σX
(measure of asymmetry about the mean)
*Picture from https://en.wikipedia.org/wiki/Skewness

Kurtosis
The kurtosis of the random variable X is defined as

n o
E (X − µX )4
Kurt [X ] = n o2 .
2
E (X − µX )
(measure of “tailedness”)

Entropy
The Entropy of pX is defined as

n o
S (pX ) = −E ln pX (X )
which is equivalent to
Z

S (pX ) = − pX (x) ln pX (x) dx.
R
It provides a measure for the level of uncertainty of pX .
Remark: X
For discrete random variables S (PX ) = − Pk ln (Pk ).
k

Second-order random variables
The mapping X is a second-order random variable if the expectation

of its square (second-order moment) is finite, i.e.,
n o
E X 2 < +∞.
In consequence,
E {X } < +∞,
and hence
n o 2
σX2 = E X 2 − E {X } < +∞.
(mean and variance are also finite)

Probability Distributions

Binomial
Probability Mass Function
Notation: B(n, p)
Support: {0, 1, 2, · · · , n}
Parameters:
n ∈ N — number of trials
p ∈ [0, 1] — success probability
PMF: Cumulative Distribution Function

n
Pk = p k (1 − p)n−k
k
Statistics:
µ = np
σ 2 = n p (1 − p)
Binomial distribution — Wikipedia, The Free Encyclopedia, 2017.

Poisson
Probability Mass Function
Notation: Poisson(λ)
Support: {0, 1, 2, 3, · · · }
Parameter:
λ > 0 — mean
PMF: Cumulative Distribution Function
λk e −λ
Pk =
k!
Statistics:
µ=λ
σ2 = λ
Poison distribution — Wikipedia, The Free Encyclopedia, 2017.

Uniform
Probability Density Function
Notation: U(a, b)
Support: [a, b]
Parameters:
−∞ < a < b < +∞ — boundaries
PDF:
1 Cumulative Distribution Function
pX (x) = 1 (x)
b − a [a,b]
Statistics:
1
µ= (a + b)
2
1
σ2 = (b − a)2
12
Uniform distribution — Wikipedia, The Free Encyclopedia, 2017.

Beta
Notation: Beta(α, β)
Support: [0, 1]
Parameters:
α — shape parameter
β — shape parameter
PDF:
Γ(α + β) α−1 Cumulative Distribution Function
pX (x) = x (1−x)β−1 1[0,1] (x)
Γ(α)Γ(β)
Statistics:
α
µ=
(α + β)
αβ
σ2 =
(α + β)2 (α + β + 1)
Beta distribution — Wikipedia, The Free Encyclopedia, 2017.

Exponential
Notation: Exp(λ)
Support: [0, +∞)
Parameter:
λ — rate parameter
PDF:
Cumulative Distribution Function
pX (x) = λ e (−x/λ) 1[0,+∞) (x)
Statistics:
µ = λ−1
σ 2 = λ−2
Exponential distribution — Wikipedia, The Free Encyclopedia, 2017.

Gamma
Notation: Gamma(k, θ)
Support: (0, +∞)
Parameters:
k — shape parameter
θ — scale parameter
PDF:
Cumulative Distribution Function
1
pX (x) = x k−1 e (−x/θ) 1(0,+∞) (x)
Γ(k) θk
Statistics:
µ =kθ
σ2 = k θ2
Gamma distribution — Wikipedia, The Free Encyclopedia, 2017.

Gaussian
Notation: N (µ, σ 2 )
Support: (−∞, +∞)
Parameters:
µ — mean
σ 2 — variance Cumulative Distribution Function
PDF:
( )
1 (x − µ)2
pX (x) = p exp −
2 π σ2 2 σ2
Normal distribution — Wikipedia, The Free Encyclopedia, 2017.

Log-normal
Notation: ln N (m, s 2 )
Parameters:
m ∈ R — location parameter
s > 0 — scale parameter
PDF:
( ) Cumulative Distribution Function
1 (ln x − m)2
pX (x) = √ exp −
x 2 π s2 2 s2
Statistics:
2
µ = e m+s /2
2 2
σ 2 = (e s − 1) e 2 m+s
Log-normal distribution — Wikipedia, The Free Encyclopedia, 2017.

Cauchy
Notation: Cauchy (x0 , γ)

Parameters:
x0 — location parameter
γ — scale parameter
PDF:
" # Cumulative Distribution Function
1 γ2
pX (x) =
πγ (x − x0 )2 + γ 2
Statistics:
µ = undefined
σ 2 = undefined (not 2nd-order RV)
Cauchy distribution — Wikipedia, The Free Encyclopedia, 2017.

Characterization of a probability distribution
Given the mean µ and standard deviation σ of a random variable.

Is the distribution well-defined?


No! Multiple distributions may have the same statistics.


( )
1 (x − 1)2 1
pX (x) = √ exp − and pX (x) = 1[−2,4] (x)
6π 6 6
√
have µ = 1 and σ = 3.


( )
1 (x − 1)2 1
pX (x) = √ exp − and pX (x) = 1[−2,4] (x)
6π 6 6
√
What type of information determines a distribution?


( )
1 (x − 1)2 1
pX (x) = √ exp − and pX (x) = 1[−2,4] (x)
6π 6 6
√
What type of information determines a distribution?
cumulative distribution function
probability density function (if exists)
quantile function
characteristic function
moment-generating function (if exists)

Probability in Dimension n

Random vector
Let x = (x1 , · · · , xn ) ∈ Rn . A random vector X = (X1 , · · · , Xn ) is a

collection of n random variables Xi : Ω → R that together may be
considered a (measurable) mapping X : Ω → Rn .
A collection of event in Ω
is mapped into a region on
the Euclidean space under such
mapping.
*Picture from http://theanalysisofdata.com/probability/4_1.html

Joint probability distribution
The joint probability distribution of random vector X = (X1 , · · · , Xn ),

denoted by FX , is defined as

FX (x1 , · · · , xn ) = P {X1 ≤ x1 } ∩ · · · ∩ {Xn ≤ xn } .
Thus,
Z b1 Z bn
P {a < X ≤ b} = ··· dFX (x1 , · · · , xn ),
a1 an
in which {a < X ≤ b} = {a1 < X1 ≤ b1 } ∩ · · · ∩ {an < Xn ≤ bn }.
FX is also known as joint cumulative distribution function.

Joint probability density function
If pX (x1 , · · · , xn ) = ∂ n FX /∂x1 · · · ∂xn exists, for any x1 , · · · , xn ,

then it is called joint probability density function of X, and one has
Z x1 Z xn
FX (x1 , · · · , xn ) = ··· pX (ξ1 , · · · , ξn ) dξ1 · · · dξn .
−∞ −∞
Note also that:
pX (x1 , · · · , xn ) ≥ 0 for every (x1 , · · · , xn ) ∈ Rn
Z +∞ Z +∞
··· pX (x1 , · · · , xn ) dx1 · · · dxn = 1
−∞ −∞

Marginal probability density function
The marginal probability density function of Xi is defined as
Z +∞ Z +∞
pXi (xi ) = ··· pX (x1 , · · · , xn ) dx1 · · · dxi−1 dxi+1 · · · dxn ,
| −∞ {z −∞
}
n−1 times
for i = 1, · · · , n.

Conditional distribution
Consider a pair of jointly distributed random variables X and Y .

The conditional distribution of X , given the occurrence of the value
y of Y , is defined as
FX ,Y (x, y )
FX |Y (x | y ) = .
FY (y )
Thus
FX ,Y (x, y ) = FX |Y (x|y ) × FY (y ),
and
pX ,Y (x, y ) = pX |Y (x|y ) × pY (y ).
Remark:
This definition extends naturally to the n-dimensional case.

Independence of distributions
The random variables X and Y are said to be independent if the

realization of X does not affect the probability distribution of Y ,
i.e.,
FX |Y (x|y ) = FX (x).
Therefore, for independent random variable one has
FX Y (x, y ) = FX (x) × FY (y ),
and
pX Y (x, y ) = pX (x) × pY (y ).
Remark:
This definition extends naturally to the n-dimensional case.

Statistics of random vectors
second-order random vector
n o Z
E k X k2 = k x k2 dFX (x) < +∞
Rn
mean vector
Z
mX = E {X} = x dFX (x) ∈ Rn
Rn
correlation matrix
n o Z
[RXY ] = E XYT = xxT dFXY (x, y) ∈ Rn×n
Rn
covariance matrix
n o
[KXY ] = E (X − mX ) (Y − mY )T ∈ Rn×n
= [RXY ] − mX mT
Y
Remark:
Matrices [RXY ] and [KXY ] are symetric positive semi-definite
when X = Y.

Correlation of random variables
The random vectors X = (X1 , · · · , Xn ) and Y = (Y1 , · · · , Yn ) are

said to be uncorrelated if covariance matrix [KXY ] is null, i.e.,
[RXY ] = mX mT
Y.
If two random vectors are independent, then they are uncorrelated.
independence =⇒ uncorrelation
But uncorrelated random vectors are not independent in general.
6 independence
uncorrelation =⇒
Remark:
Uncorrelated random vectors which the joint distribution is Gaussian
are independent.

Notions of Random Processes

Random process
A real-valued random process (also called stochastic process) defined
on probability space (Ω, Σ, P), indexed by t ∈ T , is a mapping
(t, ω) ∈ T × Ω → X (t, ω) ∈ R,
such that, for fixed t, the output is a random variable X (t, ·), while
for fixed ω, X (·, ω) is a function of t (sample function).
*Picture from https://www.vocal.com/noise-reduction/statistical-analysis-random-signals/

Interpretation and classification
Note that, by definition, a real-valued random process X (t, ω) is a

collection of real-valued random variables indexed by a parameter,
and can be thought of as a time-dependent random variable.
According to the nature of the set of indices T , a stochastic process

is classified as:
discrete – if T is countable
continuous – if T is uncountable

Wiener process (Brownian motion process)
Wiener process — Wikipedia, The Free Encyclopedia, 2017.

White noise and colored noise
White noise Colored noise
White noise — Wikipedia, The Free Encyclopedia, 2017.

Finite-dimensional distribution
A random process Xt = X (t, ω) is statistically defined to the order n

by a set of its n-th order joint probability distribution, i.e.,

FXt (x1 , · · · , xn ) = P Xt1 ≤ x1 , · · · , Xtn ≤ xn ,
which is the joint CDF of random variables Xt1 , · · · , Xtn .

Statistics of a random process
second-order random process
n o Z
E Xt2 = x 2 dFXt (x) < +∞
R
mean function
Z
µX (t) = E {Xt } = x dFXt (x)
R
correlation function
Z

corrX (t1 , t2 ) = E Xt1 Xt2 = x1 x2 dFXt1 Xt2 (x1 , x2 )
R
covariance function
n o
covX (t1 , t2 ) = E Xt1 − µX (t1 ) Xt2 − µX (t2 )
= corrX (t1 , t2 ) − µX (t1 )µX (t2 )

Stationary process
A stochastic process is stationary when all the random variables of
that stochastic process are identically distributed.
Stationary process — Wikipedia, The Free Encyclopedia, 2017.

References
J. Jacod and P. Protter, Probability Essentials. Springer, 2nd edition, 2004.
A. Klenke, Probability Theory: A Comprehensive Course. Springer, 2nd edition, 2014.
A. Papoulis and S. U. Pillai, Probability, Random Variables and Stochastic Processes. McGraw-Hill
Europe; 4th edition, 2002.
View publication stats


Uq Course Lecture2

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Uq Course Lecture2

Hochgeladen von

Copyright:

Verfügbare Formate

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Lecture 2: Elements of Probability Theory

Presentation · August 2017

Modeling of Zika virus outbreak in Brazil View project

The user has requested enhancement of the downloaded file.

Americo Barbosa da Cunha Junior

Universidade do Estado do Rio de Janeiro (UERJ)

UNESP Ilha Solteira

2 Choosing an integer even number randomly

The mathematical framework in which a random experiment is de-

The elements of a probability space are:

A non-empty set which contains all possible events for a certain

Ω = {· · · , −8, −6, −4, −2, 0, 2, 4, 6, 8, · · · }

Ω = [a, b] ⊂ [0, +∞)

In general, not all of the events in Ω are of interest.

Intuitively, a σ-algebra on Ω is the set of relevant outcomes for a

Sample space: Ω = {1, 2, 3, 4, 5, 6}

Are not σ-algebras:

A probability measure is a function P : Σ → [0, 1] ⊂ R such that

P {A} ≥ 0 for any A ∈ Σ

A fair coin is thrown twice.

A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

A fair coin is thrown twice.

A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

A fair coin is thrown twice.

A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

A fair coin is thrown twice.

A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

A fair coin is thrown twice.

A. C. Morgado, J. B. Pitombeira, P. C. P. Carvalho, P. J. Fernandez, Análise Combinatória e

A point is randomly choosen in a square (side b).

A point is randomly choosen in a square (side b).

area of upper triangle 0.5 b2 1

A point is randomly choosen in a square (side b).

area of upper triangle 0.5 b2 1

area of main diagonal 0

A point is randomly choosen in a square (side b).

area of upper triangle 0.5 b2 1

area of main diagonal 0

area outside main diagonal b2 − 0

Consider a pair of random events A and B such that P {B} > 0.

The conditional probability of A, given the occurrence of B, denoted

Somebody rolls a pair of six-sided dice.

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

P {A} = 6/36 P {A+B≤5} = 10/36

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

P {A} = 6/36 P {A+B≤5} = 10/36

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

Somebody rolls a pair of six-sided dice.

P {A} = 6/36 P {A+B≤5} = 10/36 P {A | A+B≤5} = 3/10

Conditional probability — Wikipedia, The Free Encyclopedia, 2017.

If the occurrence of an event B does not affect the occurrence of an

Events A and B in which the latter holds are said to be independent.

Events are independent.

The last example shows that: