You are on page 1of 38

Sample Size and Power Calculations

Marcia A. Ciol 04/09/08

What resources do I need?

How long will it take to conduct the study?


I need 50 participants in my study About 5 individuals per year will be enrolled Therefore, it will take 10 years to finish the study I will follow a cohort of 500 individuals A lab test that costs US$100 will be conducted for each person Therefore, I will need US$50,000 just for lab tests

How much money do I need?


Am I going to reach my objective?

I have 2 years to finish my thesis, of which one year is for data collection I think I can get data on 50 people in that year Is 50 a sufficient number of people to test my hypothesis with the significance level I want?

Why to calculate sample size and power?

To show that under certain conditions, the hypothesis test has a good chance of showing a desired difference (if it exists) To show to the funding agency that the study has a reasonable chance to obtain a conclusive result To show that the necessary resources (human, monetary, time) will be minimized and well utilized

What do I need to know to calculate sample size?


Most Important: sample size calculation is an educated guess It is appropriate for studies involving hypothesis testing There is no magic involved; only statistical and mathematical logic and some algebra Researchers need to know something about what they are measuring and how it varies in the population of interest

Factors related to the sample size

Population factor (cannot be controlled by researcher) Characteristics of the study design


Quantities related to the research question (defined by the researcher)

Where do we get this knowledge?

Previous published studies Pilot studies If information is lacking, there is no good way to calculate the sample size!

Population factor

Variance of the measure (outcome) within the population

0.10

0.08

0.06

0.04

0.02

0.00 -20 -10 0 x 10 20 30

0.10

0.08

0.06

0.04

0.02

0.00 -20 -10 0 x 10 20 30

Study Design
Type of response variable or outcome Number of groups to be compared Specific study design Type of statistical analysis

In conjunction with the research question, the type of outcome and study design will determine the statistical method of analysis

Quantities related to the research question (defined by the researcher)

= Probability of rejecting H0 when H0 is true is called significance level of the test

= Probability of not rejecting H0 when H0 is false

1- is called statistical power of the test

Quantities related to the research question (defined by the researcher)

Size of the measure of interest to be detected


Difference between two or more means Odds ratio 2 Change in R , etc

The magnitude of these values depend on the research question and objective of the study (for example, clinical relevance)

Example: test of difference of means in two populations

Researcher fixes probabilities of type I and II errors

Prob (type I error) = Prob (reject H0 when H0 is true) =

Smaller error greater precision need more information need larger sample size

Prob (type II error) = Prob (dont reject H0 when H0 is false) = Power =1-

More power smaller error need larger sample size

Example: test of difference of means in two populations


The equation for sample size is derived from the equation for the statistical test In a t-test the equation for the test is

t=

(x1 - x2) - (m1 - m2) (s12 n + s22 n )12

The derived equation for sample size is n = (z1-/2 + z1- ) 2 (s12 + s22) (m1 - m2)2

Using PASS: t-test example


Question: does exercise help to decrease body weight? Study design: participants will be randomized into two groups (exercise and control) Outcome: change in weight Want to detect: a change of at least 15 pounds Known: from past studies, the standard deviation varies between 10 and 15 pounds.

Example: One-way ANOVA


Number of Groups: 4 Hypothesized means: 35, 20, 25, 18 (possibly from a pilot study) Sample size pattern: same number in each group SD of subjects: 18 (from a previous study) = 0.01 and 0.05 Find power for sample sizes from 5 to 30 per group (increments of 5)

Example: Linear Multiple Regression


Research Question: is depression score an important factor in explaining pain ratings, after adjusting for age and sex? Statistical question: does adding depression score increase the explained variation of pain ratings, in a linear regression model that already has age and sex in it and has R2 =.2? Suppose I may have sample sizes of 20, 30, 50, 70, and 100. What is the minimum R2 change I can detect with power .8?

Other Types of Hypothesis Tests

Different methods of data analysis require different input for sample size calculations

Cox Regression (Survival analysis)

Logistic Regression

Repeated measures

Simple designs may not require complex calculations

Read chapter 2 of Statistical Rules of Thumb, by Gerald van Belle (2002, John Wiley and Sons) Using specialized software is useful if many calculations will be performed

Important to remember

Pilot studies do not need sample size calculation!!! There is no point in doing power analysis after the study is done Sample size is an educated guess, and it works only if:

The study samples comes from the same or similar populations to the pilot study populations The population of interest is not changing over time The difference or association being studied exists

How about Effect Size?

Most common definition

E = m1 - m2

spooled

If we change de value of E, how do we know what we changed in the formula?

Some situations I have encountered

Question: How many more people do I need to enroll in the study (already in progress) to show statistical significance? Answer: It depends If the two populations have the same mean, increasing the sample size will not help! Since when is the objective of a study to find a statistically significant result??

Some situations I have encountered

Researcher is interested in outcome A, which differs very little for two treatments Sample size needed is around 3000!! Researchers changes the outcome to B, where sample size is smaller B does not answer the researchers question and he needs to accept that his new treatment is not really different (clinically speaking) from the already existent treatment

Some situations I have encountered

Researcher is interested in comparing two groups regarding prediction of outcome A by using a regression analysis (using several variables) He uses the only available formula from his statistical book (for a t-test) Wrong! He should find a software that can calculate the sample size appropriately

Summary

Define research question well Consider study design, type of response variable, and type of data analysis Decide on the type of difference or change you want to detect (make sure it answers your research question) Choose and Use appropriate equation sample size calculation