Probability & Statistics

Statistics 310 Genevera I. Allen

March 19, 2013

Lecture 19 Outline: 1. Review: Law of Large Numbers & Central Limit Theorem. 2. Point Estimation. Have a random sample from some known distribution, but the population parameters, , of the distribution unknown. , is a statistic (only depends on the data) used to approximate . A point estimate, Intuition: Point estimates for the normal mean, binomial proportion, Poisson mean. 3. Method of Moments Estimation. Idea: Set the population moments equal to the sample moments and solve for unknown parameters. Population moments: M1 = E (X ), M2 = E (X 2 ), . . .. 1 = 1 n Xi , M 2 = 1 n X 2 , . . .. Sample moments: M i=1 i=1 i n n Procedure: If there are k unknown parameters, equate k population moments and sample moments. Solve for the unknown parameters. Example: Normal with unknown mean and variance. Example: Gamma with unknown shape and scale parameters. 4. Maximum Likelihood Estimation. Idea: Estimate the parameters that are most consistent with the data, or for which the data has the highest probability of occurring. Likelihood function: L() = f (x1 , . . . xn ; ) = Log-likelihood function: () = log(L()) =
n i=1 n i=1

f (xi ).

log(f (xi )). = argmax (). Find the value of that maximizes the likelihood: Procedure: (i) Calculate the log-likelihood. (ii) Take the partial derivative of the likelihood with respect to each unknown parameter. (iii) Set the partials equal to zero and solve for the unknown parameters. Example: Normal with unknown mean and variance. Textbook References: Section 6.1. Next Time . . . Point Estimation Properties & Condence Intervals. Read ahead! Textbook reference: Sections 6.1 - 6.2