mesure de probabilité

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

5 Aufrufe

mesure de probabilité

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

- DTMC
- Bruce K. Driver - Analysis Tools With Examples
- S.V. Astashkin, E.M. Semenov and F.A. Sukochev- Banach-Saks Type Properties in Rearrangement-Invariant Spaces with the Kruglov Property
- CGUIDE
- Financial Economics pt 2 Modelling in Insurance and Finance
- Alberto Contreras Mayo 2011
- UNIT 8 - Discrete Probability Distribution
- Project Report Probability
- MIB.LECTURE11
- Order Statistics
- 3rd Semester CSE
- Math 244
- Probabilistic Analysis of General Manpower – SCBZ Machine System with Exponential Production, General Sales and General Recruitment
- Airline Ticket Pricing Model
- Statistical Inference for Decision Making
- 6684_01_que_20050622
- Standard Normal Curve
- HW1
- Latih Tubi Mat Tam t5 Set 7
- rnotes

Sie sind auf Seite 1von 21

Peter D. Hoff

This is a very brief introduction to measure theory and measure-theoretic probability, de-

signed to familiarize the student with the concepts used in a PhD-level mathematical statis-

tics course. The presentation of this material was influenced by Williams [1991].

Contents

1 Algebras and measurable spaces 2

2 Generated -algebras 3

3 Measure 4

7 Product measures 12

8 Probability measures 14

1

9 Expectation 16

11 Conditional probability 21

(X ) = measure of whole space. It could be , could be 1.

If we can measure A, we should be able to measure AC .

If we can measure A and B, we should be able to measure A B.

1. X A;

2. A A Ac A;

3. A, B A A B A.

have An A.

Exercise: Show An A.

2

Definition 3 (measurable space). A space X and a -algebra A on X is a measurable space

(X , A).

2 Generated -algebras

that contains C, and is denoted (C).

Examples:

1. C = {} (C) = {, X }

2. C = C A (C) = {, C, C c , X }

Let X = R

C = {C : C = (a, b), a < b, (a, b) R2 } = open intervals

(C) = smallest -algebra containing the open intervals

Now let

G G = open sets G = Cn for some countable collection {Cn } C.

G (C)

(G) (C)

Exercise: Convince yourself that (C) = (G).

Exercise: Let D be the closed intervals, F the closed sets. Show

Hint:

3

(a, b) = n [a + c/n, b c/n]

Generally, for any topological space (X , G), (G) are known as the Borel sets.

3 Measure

measure if it is countably additive, meaning if Ai Aj = for {An : n N} A, then

X

(n An ) = (An ).

n

A measure is -finite if {Cn : n N} A with

1. (Cn ) < ,

2. n Cn = X .

Examples:

(A) = number of points in A

4

A = Borel sets of X

(A) = nk=1 (aH L n L H

Q

k ak ), for rectangles A = {x R : ak < xk < ak , k = 1, . . . , n}.

Theorem 1 (monotonic convergence of measures). Given a measure space (X , A, ),

1. If {An } A, An An+1 then (An ) (An ).

2. If {Bn } A, Bn+1 Bn , and (Bk ) < for some k, then (Bn ) (Bn ).

Exercise: Prove the theorem.

Example (what can go wrong):

Let X = R, A = B(R), = Leb

Letting Bn = (n, ) , then

(Bn ) = n;

Bn = .

Let X() : R (or Rp , or X )

Definition 7 (measurable function). A function X : R is measurable if

{ : X() B} A B B(R).

So X is measurable if we can measure it in terms of (, A).

Shorthand notation for a measurable function is X mA.

Exercise: If X, Y measurable, show the following are measurable:

X + Y , XY , X/Y

5

Probability preview: Let (A) = Pr( A)

Some will happen. We want to know

= (X 1 (B))

i.e. we need X 1 (B) = { : X() B} A}

We will now define the abstract Lebesgue integral for a very simple class of measurable

functions, known as simple functions. Our strategy for extending the definition is as

follows:

For a measurable set A, define its indicator function as follows:

(

1 if A

IA () =

0 else

PK

Definition 8 (simple function). X() is simple if X() = k=1 xk IAk (), where

xk [0, )

Aj Ak = , {Ak } A

Z K

X

(X) = X()(d) = xk (Ak )

k=1

6

Various other expressions are supposed to represent the same integral:

Z Z Z

Xd , Xd() , Xd.

We will sometimes use the first of these when we are lazy, and will avoid the latter two.

Exercise: Make the analogy to expectation of a discrete random variable.

Let X() be a measurable function for which ( : X() < 0) = 0

we say X 0 a.e.

Z

(X) = X()(d) = sup{(X ) : X is simple, X X}

R R R

Exercise: For a, b R, show (aX + bY )d = a Xd + b Y d.

Most people would prefer to deal with limits rather than sups over classes of functions.

Fortunately we can calculate the integral of a positive function X as the limit of the inte-

grals of functions Xn that converge to X, using something called the monotone convergence

theorem.

n a.e. , then

Z Z

(Xn ) = Xn (d) X(d) = (X) as n

With the MCT, we can explicitly construct (X): Any sequence of SF {Xn } such that

Xn X pointwise gives (Xn ) (X) as n .

Here is one in particular:

0

if X() = 0

Xn () = (k 1)/2 if (k 1)/2n < X() < k/2n < n, k = 1, . . . , n2n

n

n if X() > n

7

1. Xn () (mA)+ ;

2. Xn X;

Draw picture

Example:

Let (, A) = ([0, 1], B([0, 1]))

(

1 if is rational

X() =

0 if is irrational

Then Z 1 Z 1

X()d is undefined, but X()(d)

0 0

R

We now have a definition of X()(d) for positive measurable X. What about for

measurable X in general?

Let X mA. Define

X + () = X() 0 > 0, the positive part of X;

Exercise: Show

X = X+ X

X + , X both measurable

X d are both

R R

Definition 11 (integrable, integral). X mA is integrable if X + d and

finite. In this case, we define

Z Z Z

(X) = X()(d) = X ()(d) X ()(d).

+

8

5 Basic integration theorems

lim supn cn = limn (supkn ck )

Theorem 4 (Fatous reverse lemma). For {Xn } (mA)+ and Xn Z n, (Z) < ,

Theorem 5 (dominated convergence theorem). If {Xn } mA, |Xn | < Z a.e. , (Z) <

and Xn X a.e. , then

Proof.

|Xn X| 2Z , (2Z) = 2(Z) <

By reverse Fatou, lim sup (|Xn X|) (lim sup |Xn X|) = (0) = 0.

To show (Xn ) (X) , note

Among the four integration theorems, we will make the most use of the MCT and the DCT:

DCT : If {Xn } are dominated by an integrable function and Xn X, then (Xn ) (X).

9

6 Densities and dominating measures

One of the main concepts from measure theory we need to be familiar with for statistics is

the idea of a family of distributions (a model) the have densities with respect to a common

dominating measure.

Examples:

The normal distributions have densities with respect to Lebesgue measure on R.

Density

Theorem 6. Let (X , A, ) be a measure space, f (mA)+ . Define

Z Z

(A) = f d = 1A (x)f (x)(dx)

A

Proof. We need to show that is countably additive. Let {An } A be disjoint. Then

Z

(An ) = f d

An

Z

= 1An (x)f (x)(dx)

Z X

= f (x)1An (x)(dx)

n=1

Z

= lim gk (x)(dx),

k

Pk

where gk (x) = n=1 f (x)1An (x). Since 0 gk (x) 1An (x)f (x) g(x), by the MCT

Z Z

(An ) = lim gk d = lim gk d

k k

k

Z X

= lim f (x)1An (x)d

k

n=1

k

XZ

X

= lim f d = (An )

k An

n=1 n=1

10

R

Definition 12 (density). If (A) = A

f d for some f (mA)+ and all A A, we say that

the measure has density f with respect to .

Examples:

(normal probability measure).

bution (Poisson probability measure).

Note that in the latter example, f is a density even though it isnt continuous in x R.

Radon-Nikodym theorem

R

For f (mA)+ and (A) = A f d,

is a measure on (X , A),

absolutely continuous with respect to if (A) = 0 (A) = 0.

is dominated by or

.

R

Therefore, (A) = A

f d .

What about the other direction?

suppose . Then there exists an f (mA)+ s.t.

Z

(A) = f d A A.

A

11

In other words

has a density w.r.t.

d

write f = d

.

This helps us with notation when changing measure:

Z Z Z

d

g d = g d = gf d

d

You can think of as a probability measure, and g as a function of the random variable.

The expectation of g w.r.t. can be computed from the integral of gf w.r.t .

Example: Z

x2 1 ([x ]/)dx

g(x) = x2 ;

7 Product measures

We often have to work with joint distributions of multiple random variables living on poten-

tially different measure spaces, and will want to compute integrals/expectations of multivari-

ate functions of these variables. We need to define integration for such cases appropriately,

and develop some tools to actually do the integration.

12

Let (X , Ax , x ) and (Y, By , y ) be -finite measure spaces. Define

Axy = (F G : F Ax , G Ay )

xy (F G) = x (F )y (G)

Suppose f (x, y) is an Axy -measurable function. We then might be interested in

Z

f (x, y)xy (dx dy).

X Y

The calculus way of doing this integral is to integrate first w.r.t. one variable, and then

w.r.t. the other. The following theorems give conditions under which this is possible.

Theorem 8 (Fubinis theorem). Let (X , Ax , x ) and (Y, Ay , y ) be two complete measure

spaces and f be Axy -measurable and x y -integrable. Then

Z Z Z Z Z

f d(x y ) = f dy dx = f dx dy

X Y X Y Y X

Additionally,

1. fx (y) = f (x, y) is an integrable function of y for x a.e. x .

R

2. f (x, y) dx (x) is y -integrable as a function of y.

Also, items 1 and 2 hold with the roles of x and y reversed.

The problem with Fubinis theorem is that often you dont know of f is x y -integrable

without being able to integrate variable-wise. In such cases the following theorem can be

helpful.

Theorem 9 (Tonellis theorem). Let (X , Ax , x ) and (Y, Ay , y ) be two -finite measure

spaces and f in(mAxy )+ . Then

Z Z Z Z Z

f d(x y ) = f dy dx = f dx dy

X Y X Y Y X

Additionally,

1. fx (y) = f (x, y) is a measurable function of y for x a.e. x .

R

2. f (x, y) dx (x) is Ay -measurable as a function of y.

Also, 1 and 2 hold with the roles of x and y reversed.

13

8 Probability measures

Definition 14 (probability space). A measure space (, A, P ) is a probability space if

P () = 1. In this case, P is called a probability measure.

Uncertainty in the outcome leads to uncertainty in the data.

This uncertainty is referred to as randomness, and so X() is a random variable.

function in a probability space.

Examples:

multivariate data: X : Rp

replications: X : Rn

Suppose X : Rk .

For B B(Rk ), we might write P ({ : X() B}) as P (B).

Often, the -layer is dropped and we just work with the data-layer:

(X , A, P ) is a measure space , P (A) = Pr(X A) for A A.

Densities

Suppose P on (X , A). Then by the RN theorem, p (mA)+ s.t.

Z Z

P (A) = p d = p(x)(dx).

A A

Then p is the probability density of P w.r.t. .

(probability density = Radon-Nikodym derivative )

Examples:

1. Discrete:

X = {xk : k N}, A = all subsets of X .

P

Typically we write P ({xk }) = p(xk ) = pk , 0 pk 1, pk = 1.

14

2. Continuous:

X = Rk , A = B(Rk ).

R

P (A) = A p(x)(dx), =Lebesgue measure on B(Rk ).

(

Z w.p. 1/2

Z N (0, 1), X =

0 w.p. 1/2

(b) P = L + 0 , where 0 = #(A {0}) for A A.

space. The support of P is given by

Examples:

Let X = [0, 1], G be the open sets defined by Euclidean distance, and P (Q [0, 1]) = 1.

2. P (Qc [0, 1]) = 0.

15

9 Expectation

In probability and statistics, a weighted average of a function, i.e. the integral of a function

w.r.t. a probability measure, is (unfortunately) referred to as its expectation or expected

value.

surable function of X (i.e. a statistic). The expectation of T is its integral over X :

Z

E[T ] = T (x)P (dx).

Where do you expect a sample from this distribution to be?

Jensens inequality

Recall that a convex function g : R R is one for which

i.e. the function at the average is less than the average of the function.

Draw a picture.

The following theorem should therefore be no surprise:

variable on (R, B(R), P ) such that E[|X|] < and E[|g(X)|] < . Then

g(E[X]) E[g(X)].

i.e. the function at the average is less than the average of the function.

The result generalizes to more general sample spaces.

Schwarzs inequality

R R R

Theorem 11 (Schwarzs inequality). If X 2 dP and Y 2 dP are finite, then XY dP is

finite and Z Z Z Z

| XY dP | |XY | dP ( X dP ) ( Y 2 dP )1/2 .

2 1/2

16

In terms of expectation, the result is

One statistical application is to show that the correlation coefficient is always between -1

and 1.

Holders inequality

A more general version of Schwarzs inequality is Holders inequality.

w (0, 1),

Conditioning in simple cases:

X {x1 , . . . , xK } = X

Y {y1 , . . . , yM } = Y

Pr(X = xk |Y = ym ) = Pr(X = xk , Y = ym )/ Pr(Y = ym )

E[X|Y = ym ] = K

P

k=1 xk Pr(X = xk |Y = ym )

This discrete case is fairly straightforward and intuitive. We are also familiar with the

extension to the continuous case:

Z Z

p(x, y)

E[X|Y = y] = xp(x|y) dx = x dx

p(y)

17

Where does this extension come from, and why does it work? Can it be extended to more

complicated random variables?

Let {, A, P } be a probability space and X, Y random variables with finite supports X , Y.

Suppose A contains all sets of the form { : X() = x, Y () = y} for (x, y) X Y.

Draw the picture.

Let F, G be the -algebras consisting of all subsets of X and Y, respectively.

Add F, G to the picture (rows and columns of X Y-space)

In the Kolmogorov theory, E[X|Y ] is a random variable Z defined as follows:

E[X|Y = y1 ] if Y () = y1

E[X|Y = y2 ] if Y () = y1

Z() = ..

.

E[X|Y = yM ] if Y () = yM

the following:

(Y ) = ({ : Y () F }, F F})

This means we dont need the whole -algebra A to measure E[X|Y ], we just need the

part that determines Y .

Defining properties of conditional expectation

Z X

E[X|Y ] dP = E[X|Y = y]P (Y = y) = xP (X = x|Y = y)P (Y = y)

Y =y x

X Z

= x Pr(X = x, Y = y) = X dP

x Y =y

In words, the integral of E[X|Y ] over the set Y = y equals the integral of X over Y = y.

18

In this simple case, it is easy to show

Z Z

E[X|Y ] dP = X dP G (Y )

A A

In words, the integral of E[X|Y ] over any (Y )-measurable set is the same as that of X.

Intuitively, E[X|Y ] is an approximation to X, matching X in terms of expectations over

sets defined by Y .

E[|X|] < . Let G A be a sub- algebra of A. Then a r.v. E[X|G] s.t.

1. E[X|G] is G-measurable

2. E[|E[X|G]|] <

3. G G, Z Z

E[X|G]dP = XdP.

G G

conditions only specify things a.e. P .

From 1,2 and 3, the following properties hold

For (c), we need to show that E[X|H] is a version of E[Z|H], where Z = E[X|G]

This means the integral of E[X|H] over any H-measurable set H must equal that of Z over

H. Lets check:

Z Z

E[X|H] dP = X dP, by definition of E[X|H]

H

ZH

= E[X|G] dP, since H H G.

H

19

Exercise: Prove (d).

Independence

algebras G and H are independent if P (A B) = P (A)P (B) A G, B H.

This notion of independence allows us to describe one more intuitive property of conditional

expectation.

Intuitively, if X is independent of H, then knowing where you are in H isnt going to give you

any information about X, and so the conditional expectation is the same as the unconditional

one.

Interpretation as a projection

Let X mA, with E[X 2 ] < .

Let G A be a sub--algebra.

Problem: Represent X by a G-measurable function/r.v. Y s.t. expected squared error is

minimized, i.e.

minimizeE[(X Y )2 ]among Y mG

= E[(X Y )2 ] 2E[Z(X Y )] + 2 E[Z 2 ].

This implies

2E[Z(X Y )] 2 E[Z 2 ]

2E[Z(X Y )] E[Z 2 ] for > 0

2E[Z(X Y )] E[Z 2 ] for < 0

which implies that E[Z(X Y )] = 0. Thus if Y is the minimizer then it must satisfy

20

In particular, let Z = 1G () for any G G. Then

Z Z

X dP = Y dP,

G G

so Y must be a version of E[X|G].

11 Conditional probability

Conditional probability

For A A, Pr(A) = E[1A ()].

For a -algebra G A, define Pr(A|G) = E[1A ()|G].

Exercise: Use linearity of expectation and MCT to show

X

Pr(An |G) = Pr(An |G)

if the {An } are disjoint.

Conditional density

Let f (x, y) be a joint probability density for X, Y w.r.t. a dominating measure , i.e.

ZZ

P ((X, Y ) B) = f (x, y)(dx)(dy).

B

R

Let f (x|y) = f (x, y)/ f (x, y)(dy)

R

Exercise: Prove A f (x|y)(dx) is a version of Pr(X A|Y ).

This, and similar exercises, show that our simple approach to conditional probability

generally works fine.

References

David Williams. Probability with martingales. Cambridge Mathematical Textbooks. Cam-

bridge University Press, Cambridge, 1991. ISBN 0-521-40455-X; 0-521-40605-6.

21

- DTMCHochgeladen vonDavid Lee
- Bruce K. Driver - Analysis Tools With ExamplesHochgeladen vonAndrey Ferreira
- S.V. Astashkin, E.M. Semenov and F.A. Sukochev- Banach-Saks Type Properties in Rearrangement-Invariant Spaces with the Kruglov PropertyHochgeladen vonLekkell
- CGUIDEHochgeladen vonJustine Powell
- Financial Economics pt 2 Modelling in Insurance and FinanceHochgeladen vonJohn
- Alberto Contreras Mayo 2011Hochgeladen vonJosé Luis Mb
- UNIT 8 - Discrete Probability DistributionHochgeladen vonAne Aneka
- Project Report ProbabilityHochgeladen vonAli Raza
- MIB.LECTURE11Hochgeladen vonmrtfkhang
- Order StatisticsHochgeladen vonasifzubair
- 3rd Semester CSEHochgeladen vonZuZumon deka
- Math 244Hochgeladen vonRashmi Nattamai
- Probabilistic Analysis of General Manpower – SCBZ Machine System with Exponential Production, General Sales and General RecruitmentHochgeladen vonIOSRjournal
- Airline Ticket Pricing ModelHochgeladen vonCarmen Aid
- Statistical Inference for Decision MakingHochgeladen vonNitin Amesur
- 6684_01_que_20050622Hochgeladen vonChristopher Williams
- Standard Normal CurveHochgeladen vonSridharan Venkat
- HW1Hochgeladen von李協諭
- Latih Tubi Mat Tam t5 Set 7Hochgeladen voncgnash
- rnotesHochgeladen vonalin444444
- Umar Patek....Hochgeladen vonFadzli Makmur
- Selection 2Hochgeladen vonomr7
- ECE316-Yates6-10-WongHochgeladen vonzorothemask
- Binomial 50Hochgeladen vonwongkw5557
- A Possibilistic and Probabilistic Approach to Precautionary Saving FinalHochgeladen vonIrina Alexandra
- Finite MdpHochgeladen vonnaoto_soma
- 03-Introduction to RHochgeladen vonnaman
- Econ250 A2 SolHochgeladen vonDiana Miranda
- KrokhmalHochgeladen vonAnonymous fwgFo3e77

- ISO 55001 Standard Certification Plant Wellness WayHochgeladen vonRoberto Angel Grangeno
- 2 Très Bons LiensHochgeladen vonSaid Mahtaou
- 20150203.fullHochgeladen vonSaid Mahtaou
- frm-part-1-study-plan-nov-2017.pdfHochgeladen vonSaid Mahtaou
- HIHochgeladen vonSaid Mahtaou
- The Matrix Cookbook2012Hochgeladen vonmehdicheraghi506
- Ch32HullOFOD10thEdition.pptxHochgeladen vonSaid Mahtaou
- 9 Binomial WaysHochgeladen vonHendy Kurniawan
- Krishnan-Winter - May 30 2012Hochgeladen vonSaid Mahtaou
- sium teklabHochgeladen vonSaid Mahtaou
- 12-2637Hochgeladen vonSaid Mahtaou
- goodHochgeladen vonSaid Mahtaou
- 2014_ifa_sydney_summary.pdfHochgeladen vonelsobki
- Functional overview 7Hochgeladen vonSaid Mahtaou
- HelloHochgeladen vonSaid Mahtaou

- Feyman's operational calculusHochgeladen vonMoisés Arias
- An Informal Introduction to Stochastic Calculus With ApplicationsHochgeladen vonMarjo Kaci
- The 100 Absolutely Unbreakable Laws of Business SuccessHochgeladen vonAnonymous QlJjisdlLI
- AdemboHochgeladen vonKelvin Guu
- Lectures Part1 probability theoryHochgeladen vonseethamrajumukund
- Probability,Possibility and Fuzzy SetsHochgeladen vonbkjr2008
- (Atlantis Studies in Mathematics 2) Alexander B. Kharazishvili-Topics in Measure Theory and Real Analysis-Atlantis Press _ World Scientific (2009)Hochgeladen vonLe Duy Hien
- Stanford Notes of Probability TheoryHochgeladen vonRómulo Tomé Alves
- Random Process for EngineersHochgeladen vonNatasa Begovic
- Financial Asset Trading By InsidersHochgeladen vonchandi95
- Radon NikodymHochgeladen vonSaid Mahtaou
- Integration Theory& Functional Analysis ComHochgeladen vonDAVID
- Measure TheoryHochgeladen vonMizanur Rahman
- Real Analysis Lecture Notes.pdfHochgeladen vonphoeberamos
- Optimal Transport Old and NewHochgeladen vonKarthik Vazhuthi
- Lecture7 Borel Sets and Lebesgue MeasureHochgeladen vonsourav kumar ray
- Empirics for Growth and DistributionHochgeladen vonJuan Carlos Núñez
- Set TheoryHochgeladen vonmars
- Notes for Applied Mathematics - Todd ArbogastHochgeladen vonVictor Maximiano
- Combine SolutionHochgeladen vons0845
- Real Prjct11Hochgeladen vonMuzna Zainab
- Absolute Measurable Spaces Encyclopedia of Mathematics and Its ApplicationsHochgeladen vonmario
- Wieslaw Kubis and Saharon Shelah- Analytic ColoringsHochgeladen vonSakoMC
- zhc:tsHochgeladen von葉彥均
- BV RAO All-notesHochgeladen vonShubhada Agrawal
- (Courant Lecture Notes in Mathematics 7) S. R. S. Varadhan-Probability Theory-Courant Institute of Mathematical Sciences_ American Mathematical Society (2001)Hochgeladen vonRodrigo
- APA Lecture Notes Part1Hochgeladen vonSheela Shivaram
- hw1_solHochgeladen vonviviva
- Atmnotes 2 HayHochgeladen vonrcpuram01
- ProbabilitProbability With Martingalesy With MartingalesHochgeladen voningramk