Beruflich Dokumente
Kultur Dokumente
k
2
1
A certain event of an experiment is the event which appears in each realization of that
experiment. An impossible event of an experiment is the event which never appears in any
realization of that experiment.
2. Sample space
Definition:
i) Outcome or elementary event of an experiment is each logic result of an experiment
which cannot be split in other events. One and only one outcome appears in each
realization of the experiment.
ii) The set of all possible outcomes of a random experiment is called the sample space of
the experiment. The sample space is denoted as (the sample space can be finite,
countable infinite or infinite uncountable set.).
MILE KARANFILOV
3. Random events
Definition:
A random event is a subset of the sample space of a random experiment.
The intersection of the events A and B is an event which appears then and only then two
events A and B appear together.
If the events A and B cannot appear in the same realization of an experiment then they are
called mutually exclusive events. Their intersection is an impossible event, A B = .
The union of events A and B is an event which appears then and only then when at least
one of the two events appears.
The complement of an event A is an event which appears then and only then when the
event A does not appear. It is denoted by .
The difference between two events A and B is an event which appears then and only then
when the event A appears, but the event B does not.
Let A1, A2,, An are events such that =1 = . These sets are called exhaustive sets.
If A1, A2,, An are given events such that Ai Aj = , i j and =1 = then we say
that is presented as a union of mutually exclusive and exhaustive events.
4. Kolmogorov axioms
Definition:
The family is called a -algebra of subsets of , if the following three conditions are
satisfied:
.1. ;
.2. If A , then ;
.3. If Ai , i = 1, 2, then +
=1 ;
Theorem:
For each -algebra , the following properties are satisfied:
i) ;
ii) If Ai , i = 1, 2, then +
=1 ;
iii) If A, B , then A B , A B .
Proof:
i) Using the conditions .1 and .2 from the definition for -algebra, we obtain:
.2
+
=1 = =1
.3
Ai , i = 1, 2, , i = 1, 2, +
=1
.2
+
=1 = =1
+
=1 = A B
On the other side, if we take A1 = A, A2 = B, Ai = , i = 3, 4, in statement ii) of this
theorem, then
+
=1 = A B
Definition:
Let be a -algebra of subsets of . A mapping P : , where is the set of real
numbers, is called a probability, if the following three conditions are satisfied:
P.1. P(A) 0, for each A ;
P.2. P() = 1;
P.3. If Ai , i = 1, 2, and Ai Aj = , for i j, then
+
P (=1
) = +
=1 ( )
Theorem:
The probability P has the following properties:
i) P() = 0;
ii) P (=1 ) = =1 ( )
iii) If A B, then P(A) P(B);
iv) For each A , 0 P(A) 1;
v) P() = 1 P(A);
vi) P(A B) = P(A) + P(B) P(AB).
Proof:
i) P.1 implies that P() 0. Now, we can present as:
=+++
From P.3, we obtain:
P() = P() + P() + P() + = P() + +
=1 ()
MILE KARANFILOV
ii) Let An+1 = An+2 = = . Using i) and P.3, we get that P (=1 ) = =1 ( ).
iii) Since A B, the event B can be presented as:
B = B = (A + )B = AB + B = A + B.
Now, using the property ii), we have:
P(B) = P(A) + P(B) P(A),
since P(B) 0.
iv) For each A, A . Now, from the property iii), we can conclude that
P() P(A) P(), i.e.,
0 P(A) 1.
v) Using the equality = A + A and the property ii), we obtain that
P() = P(A) + P(), i.e.,
1 = P(A) + P().
Therefore,
P(A) = 1 P().
vi) The event A B can be presented as:
A B = A + B.
Using ii), we obtain:
P(A B) = P(A) +P(B)
(1)
On the other side, using the equality
B = B = (A + )B = AB + B,
and ii), we have:
P(B) = P(AB) + P(B).
Therefore, P(B) = P(B) P(AB). By replacing the last expression in (1), we obtain that
P(A B) = P(A) + P(B) = P(A) + P(B) P(AB).
MILE KARANFILOV
()
()
Here, we suppose that all outcomes have an equal probability of appearance. This means
that probability that a point of A will appear is proportional with m(A).
7. Conditional probability
Let (, , P) be a given probability space and B has a positive probability
P(B) 0.
The conditional probability of an event A given an event B, denoted as P(A|B), is defined by
P(A|B)
()
()
, A .
()
()
and P(B|A) =
()
()
Theorem:
For arbitrary n events A1, A2,, An , the probability of their intersection is given by
P(A1A2A3 An1An) = P(A1)P(A2|A1)P(A3|A2A1) P(An|An1 . . .A2A1).
MILE KARANFILOV
8. Independence of events
Let (, , P) be a given probability space and A, B . such that P(A) > 0 and P(B) > 0.
We say that the event A is independent of the event B, if
P(A|B) = P(A).
Theorem:
The events A and B are independent iff:
P(AB) = P(A)P(B).
Theorem:
If A and B are independent events, then the pairs: A and ; and B; and are
independent, also.
Proof:
If A and B are independent events, then P(AB) = P(A)P(B). Now, A = A = A(B + ) = AB
+ A, so the probability of A is given by:
P(A) = P(AB) + P(A).
Using the independence of A and B, we have:
P(A) = P(A)P(B) + P(A), so
P(A) = P(A)P(A)P(B) = P(A)(1P(B)) = P(A)P().
From the last equality we conclude that A and are independent events.
By symmetry we can conclude that and B are independent. The independence of and
is a consequence of the independence of the previous two pairs.
Definition:
The events A1, A2,, An are mutually independent, if for an arbitrary k (2 k n) and for
arbitrary selection of indexes i1 < i2 < < ik, the following equality is satisfied.
P(Ai1Ai2 Aik) = P(Ai1)P(Ai2) P(Aik).
Definition:
The events A1, A2,, An are pairwise independent, if for arbitrary i, j {1, 2,, n} (i j) Ai
and Aj are independent events, i.e.,
P(AiAj) = P(Ai)P(Aj).
MILE KARANFILOV
Now,
+
+
P(A) = P(+
=1 ) = =1 ( )= =1 ( ) (| ).
The random events Hi , i = 1, 2, are called hypotheses.
Their probabilities P(Hi) are called prior probabilities.
Very often, we have to calculate the probabilities P(Hj |A), j = 1, 2, i.e., we have to
evaluate the probability of the hypotheses after the occurrence of the event A. These
conditional probabilities are called posterior probabilities. They can be calculated using
the Bayes theorem.
Bayes theorem
+
Let Hi , i = 1, 2, and HiHj = , for i j and =1
= .
The following formulas are satisfied
P(Hj |A) =
P(j) P(|j)
P()
P(j) P(|j)
= + ( )(| ), j = 1, 2,
=1
Proof:
P(Hj |A) =
( )
P()
P(j) P(|j)
P()
MILE KARANFILOV
MILE KARANFILOV
1 + 2 + + 1 ,
1 <
{
1,
>
Relation between the probability mass function and the cumulative distribution function
p i = P{X = xi} = lim+ ( ) ( )
MILE KARANFILOV
10
(
)( )
pi = P{X = i} =
, i RX.
(
)
Geometric distribution
Lets consider a sequence of independent and equal trials until the event A appears. Let p
= P(A) and q = 1 p. Let X denote the number of trials in the previous described
sequence. The number of trials until the event A appears, can be 1 or 2 or 3 or
Therefore, RX = {1, 2, 3, }.
Ai : the event A appears in the i th trial, i = 1, 2, These events are independent and P(Ai) =
p, for each i = 1, 2, The event {X = i} will occur if the event A does not appear in the first
i1 trials (which means that the event A appears) and the event A appears in the last ith
trial. Therefore,
i1
1
1 )P(
pi = P{X = i} = P(
2
1 Ai) = P(
2 ) P(
1 )P(Ai) = q p, i RX.
A random variable X has a geometrical distribution with parameter p and denote it by
X ~ Geo(p).
Negative binomial distribution
Lets consider a sequence of independent and equal trials until the event A appears exactly
k times. And, p = P(A) and q = 1 p.
X - denotes the number of trials in the previous described sequence.
Therefore, RX = {k, k + 1, k + 2, } and
The event appears 1 times in the first
1
P{X = n} = P {
} = (1)(1 - p)n-k pk
th
1 trials and appears in the last trial
The random variable X has a negative binomial distribution with parameters k and p and
denote it by X ~ NB (k, p).
Poisson distribution
A random variable X has a Poisson distribution, if RX = {0, 1, 2 } and pi = P{X = i} =
11
p(x) = lim
{ < +}
p(x) = lim
( + )()
= F(x)
If p(x) is an integrable function then from the previous equality we find that
F(x) = ()
Here, we suppose that p(x) = 0, for x a or x b.
Definition:
If there exists an integrable function p(x) such that the cumulative distribution function of
X can be presented as
F(x) = ()
(1)
ii) () = 1;
1 = lim () = lim () = ()
+
iii) For a continuous random variable X, the equality P{X = a} = 0 holds for each a .
Therefore,
12
1 = ( ) = = C (b -a)
Therefore, C =
, so
1
,
p(x) = {
(, )
(, )
0,
The cumulative distribution function of X can be found with the following way:
0,
F(x) = { ,
<
>
1,
Exponential distribution
A random variable X has an exponential distribution with a parameter denoted by
X ~ (), if its probability density function is given by
0,
<0
0
,
p(x) = {
0,
1 ,
<0
0
Gamma distribution
A random variable X has gamma distribution with parameters and and denote X ~
( , ), if its probability density function is given by
0,
p(x) = { 1
( )
+
where ( ) = 0
<0
,
, 0
13
p(x) =
1
2
(xa)2
2 2
p(x) =
1
(1+ 2 )
!
1 ! 2 ! k !
1 1 2 2
14
Theorem:
If F(x, y) is a joint cumulative distribution function of the two-dimensional random vector
(X, Y), then for a < b and c < d, the following equality is satisfied:
P{a X < b, c Y < d} = F(b, d) F(b, c) F(a, d) + F(a, c).
Proof:
P{a X < b, c Y < d} = P{X < b, Y < d} P{X < b, Y < c} - P{X < a, Y < d} + P{X < a, Y <
c} =
F(b, d) F(b, c) F(a, d) + F(a, c).
Properties of a joint cumulative distribution function
If F(x, y) is a joint cumulative distribution function of a random vector (X, Y), then F has
the following properties:
FR1) F(x, y) is a non-decreasing function for each of the variables x and y;
FR2) lim (, ) = 0,
lim (, ) = 0,
lim
(, ) = 1
(,)(+,+)
Marginal distribution
The individual probability distribution of a random variable is called marginal probability
distribution.
FX(x) = P{X < x} = P{X < x, Y < +} = lim (, )
+
Conditional distribution
({X = xi },{Y = yj })
{Y = yj }
( , )
( )
, x i R X.
Definition:
The conditional probability mass function of X given {Y = yj} (where P{Y = yj} > 0) is
defined by
pX(xi | yj) = =
( , )
( )
, x i R X.
MILE KARANFILOV
15
Analogous, for Y
pY(yj | xi) =
( , )
( )
yj R Y
F(x, y) = (1 , 2 )2 1
then we say that (X, Y) is a continuous random vector.
Uniform distribution
A random vector (X, Y) has an uniform distribution in the region G 2, if
1
, (, )
0, (, )
p(x, y) = {()
where m(G) is the area of the region G. And wee denote (X, Y) ~ U(G).
Marginal distributions
+
pX(x) = (, )
+
pY(y) = (, )
Definition:
The conditional probability density function of X given {Y = y}, for pY (y) > 0, is defined by
pX(x|y) =
(,)
()
FX(x|y) = ( |) , x
Theorem:
The random variables X and Y are independent if and only if the joint cumulative
distribution function of the vector (X, Y ) can be presented as a product of the marginal
cumulative distribution functions of X and Y , i.e.,
F(x, y) = FX(x)FY(y).
If (X, Y) is a continuous random vector given by its probability density function p(x, y),
then:
p(x, y) = pX(x)pY(y),
x, y .
MILE KARANFILOV
16
MILE KARANFILOV
17