Power analysis Power is the ability to find a statistically significant difference when the null hypothesis is in fact false,

in other words power is your ability to find a difference when a real difference exists. Technically, power is the probability of rejecting the null hypothesis when the specific alternative hypothesis is true. The power of a study is determined by three factors: the sample size, the alpha level, and the effect size. There are two different aspects of power analysis To estimate the sample size needed to detect a specified effect, To estimate the size of the effect that can be detected with a specified sample size In order to do a power analysis, you need to specify an effect size Difference between your null hypothesis and the alternative hypothesis that you hope to detect. When doing basic biological research, you often don't know how big a difference you're looking for, and the temptation may be to just use the biggest sample size you can afford, or use a similar sample size to other research in your field. You should still do a power analysis before you do the experiment, just to get an idea of what kind of effects you could detect. Parameters Minimum effect size: minimum deviation from the null hypothesis that you hope to detect. For measurement variables, you also need an estimate of the standard deviation. This can come from pilot experiments or from similar experiments in the published literature. Your standard deviation once you do the experiment is unlikely to be exactly the same, so your experiment will actually be somewhat more or less powerful than you had predicted.

Cohen, regarded as the deity of power analysis, (1977, 1988) justifies these levels of effect sizes. Effect size Index Small Medium Large t-test on Means t-test on Correlations F-test ANOVA F-test regression Chi-Square Test d r f f
2

0.20 0.10 0.10 0.02 0.10

0.50 0.30 0.25 0.15 0.30

0.80 0.50 0.40 0.35 0.50

http://www.jeremymiles.co.uk/misc/power/ for pradeep http://meera.snre.umich.edu/plan-an-evaluation/step-6-analyze-data http://www.ats.ucla.edu/stat/seminars/Intro_power/ http://meera.snre.umich.edu/plan-an-evaluation/related-topics/power-analysis-statisticalsignificance-effect-size http://statistics.uchicago.edu/~s220e/Lect

Power is the probability of detecting an effect, given that the effect is really there. In other words, it is the probability of rejecting the null hypothesis when it is in fact false. A power analysis is often required as part of a grant proposal. And finally, doing a power analysis is often just part of doing good research. A power analysis is a good way of making sure that you have thought through every aspect of the study and the statistical analysis before you start collecting data. Researchers also need to carefully consider all aspects of the experimental design, the variables involved, and the statistical analysis technique that will be used. The point of this section is to make clear that knowing your research project involves many things, and you may find that you need to do some research about experimental design or statistical techniques before you do your power analysis.

For an independent samples t-test, you will need to know the population means of the two groups (or the difference between the means), and the population standard deviations of the two groups. So, using our example of drug A and placebo, we would need to know the difference in the means of the two groups, as well as the standard deviation for each group (because the group means and standard deviations are the best estimate that we have of those population values). Clearly, if we knew all of this, we wouldn't need to conduct the study. In reality, researchers make educated guesses at these values. We always recommend that you use several different values, such as decreasing the difference in the means and increasing the standard deviations, so that you get a range of values for the number of necessary subjects For a correlation, you need to know/guess at the correlation in the population. This is a good time to remember back to an early stats class where they emphasized that correlation is a large N procedure (Chen and Popovich, 2002). If you guess that the population correlation is .6, a power analysis would suggest (with an alpha of .05 and for a power of .8) that you would need only 16 subjects. There are several points to be made here. First, common sense suggests that N = 16 is pretty low. Second, a population correlation of .6 is pretty high, especially in the social sciences. Third, the power analysis assumes that all of the assumptions of the correlation have been met. For example, we are assuming that there is no restriction of range issue, which is common with Likert scales; the sample data for both variables are normally distributed; the relationship between the two variables is linear; and there are no serious outliers. Also, whereas you might be able to say that the sample correlation does not equal zero, you likely will not have a very precise estimate of the population correlation coefficient

This power tells us that if that difference between means was actually a true difference, then what is the probability of finding it to be significant. Clearly, we are more likely to find as significant those spots with a large difference between groups. Thus, these spots will have a high power. Also, these spots will have a lower p-value, as the greater the difference between spots (all other factors equal), then the lower the p-value. - For a chi-square test, you will need to know the proportion positive for both populations (i.e., rows and columns). Let's assume that we will have a 2 x 2 chi-square, and let's think of both variables as 0/1. Let's say that we wanted to know if there was a relationship between drug group (drug A/placebo) and improved health. In SPSS Sample Power, you would see a screen like this. - For an ordinary least squares regression, you would need to know things like the R2 for the full and reduced model. For a simple logistic regression analysis with only one continuous predictor variable, you would need to know the probability of a positive outcome (i.e., the probability that the outcome equals 1) at the mean of the predictor variable and the probability of a positive outcome at one standard deviation above the mean of the predictor variable. Especially for the various types of logistic models (e.g., binary, ordinal and multinomial), you will need to think very carefully about your sample size, and information from a power analysis will only be part of your considerations. For example, according to Long (1997, pages 53-54), 100 is a minimum sample size for logistic regression, and you want *at least* 10 observations per predictor. This does not mean that if you have only one predictor you need only 10 observations. Also, if you have categorical predictors, you may need to have more observations to avoid computational difficulties caused by empty cells. More observations are needed when the outcome variable is very lopsided; in other words, when there are very few 1s and lots of 0s, or vice versa. These cautions emphasize the need to know your data set well, so that you know if your outcome variable is lopsided or if you are likely to have a problem with empty cells. Three ways to guestimate the values that are needed to do a power analysis: a literature review, a pilot study and using Cohen's recommendations. - Literature review: Sometimes you can find one or more published studies that are similar enough to yours that you can get a idea of the effect size. If you can find several such studies, you might be able to use meta-analysis techniques to get a robust estimate of the effect size. However, oftentimes there are no studies similar enough to your study to get a good estimate of the effect size. Even if you can find such an study, the necessary effect sizes or other values are often not clearly stated in the article and need to be calculated (if they can) based on the information provided. - Pilot studies: There are lots of good reasons to do a pilot study prior to conducting the actual study. From a power analysis prospective, a pilot study can give you a rough estimate of the effect size, as well as a rough estimate of the variability in your measures. You can also get some idea about where missing data might occur, and as we will discuss

later, how you handle missing data can greatly affect your power. Other benefits of a pilot study include allowing you to identify coding problems, setting up the data base, and inputting the data for a practice analysis. This will allow you to determine if the data are input in the correct shape, etc. (please listen to our podcast # for more information on this). Cohen's recommendations: Jacob Cohen has many well-known publications regarding issues of power and power analyses, including some recommendations about effect sizes that you can use when doing your power analysis. Many researchers (including Cohen) consider the use of such recommendations as a last resort, when a thorough literature review has failed to reveal any useful numbers and a pilot study is either not possible or not feasible.

From Cohen (1988, pages 24-27): - Small effect: 1% of the variance; d = 0.25 (too small to detect other than statistically; lower limit of what is clinically relevant) - Medium effect: 6% of the variance; d = 0.5 (apparent with careful observation) - Large effect: at least 15% of the variance; d = 0.8 (apparent with a superficial glance; unlikely to be the focus of research because it is too obvious)

What is power?
To understand power, it is helpful to review what inferential statistics test. When you conduct an inferential statistical test, you are often comparing two hypotheses:

The null hypothesis This hypothesis predicts that your program will not have an effect on your variable of interest. For example, if you are measuring students level of concern for the environment before and after a field trip, the null hypothesis is that their level of concern will remain the same. The alternative hypothesis This hypothesis predicts that you will find a difference between groups. Using the example above, the alternative hypothesis is that students posttrip level of concern for the environment will differ from their pre-trip level of concern. Statistical tests look for evidence that you can reject the null hypothesis and conclude that your program had an effect. With any statistical test, however, there is always the possibility that you will find a difference between groups when one does not actually exist. This is called a Type I error. Likewise, it is possible that when a difference does exist, the test will not be able to identify it. This type of mistake is called a Type II error. Power refers to the probability that your test will find a statistically significant difference when such a difference actually exists. In other words, power is the probability that you will reject the null hypothesis when you should (and thus avoid a Type II error). It is generally accepted that power should be .8 or greater; that is, you should have an 80% or greater chance of finding a statistically significant difference when there is one.

How do I use power calculations to determine my sample size?

Generally speaking, as your sample size increases, so does the power of your test. This should intuitively make sense as a larger sample means that you have collected more information -- which makes it easier to correctly reject the null hypothesis when you should. To ensure that your sample size is big enough, you will need to conduct a power analysis calculation. Unfortunately, these calculations are not easy to do by hand, so unless you are a statistics whiz, you will want the help of a software program. Several software programs are available for free on the Internet and are described below. For any power calculation, you will need to know:

What type of test you plan to use (e.g., independent t-test, paired t-test, ANOVA, regression, etc. See Step 6 if you are not familiar with these tests.), The alpha value or significance level you are using (usually 0.01 or 0.05. See the next section of this page for more information.), The expected effect size (See the last section of this page for more information.), The sample size you are planning to use