26

Analysis of Variance (ANOVA)

members of upcoming events, training opportunities, and awards. Another approach that the SRMS has used to promote interest in survey methods is to award scholarships to students and to honor those who have made important contributions to survey research. For example, the SRMS offers Student Travel Awards to several doctoral students to support their attendance at the ASA annual meeting and attendance at an SRMS short course. In conjunction with other sections of the ASA, the SRMS annually has a competition open to students and postgraduates in survey methodology and related elds, and the winners are given awards to support their attendance at the ASA annual meeting. Pat Dean Brick
See also American Association for Public Opinion Research (AAPOR)

Further Readings

Hess, I. (1990). History of the Section on Survey Research Methods. The American Statistician, 44(2), 98100. Section on Survey Research Methods: http:// www.amstat.org/sections/SRMS/index.html Section on Survey Research Methods. (n.d.). What Is a Survey? Retrieved March 4, 2008, from http:// www.whatisasurvey.info

ANALYSIS OF VARIANCE (ANOVA)

Analysis of variance (ANOVA) is a statistical technique that is used to compare groups on possible differences in the average (mean) of a quantitative (interval or ratio, continuous) measure. Variables that allocate respondents to different groups are called factors; an ANOVA can involve one factor (a one-way design) or multiple factors (a multi-way or factorial design). The term analysis of variance refers to the partitioning of the total variation in the outcome variable into parts explained by the factor(s)related to differences between groups, so-called explained or between variationand a part that remains after taking the factor(s) into account, the so-called unexplained, residual, or within variation. Consider a one-factor example in which the target population contains respondents from four different ethnic backgrounds (e.g., Chinese, Japanese, Korean,

Vietnamese) and the research question is whether these ethnic groups have different average incomes. The null and alternative hypotheses for this example tested with the ANOVA are H0 : m1 = m2 = m3 = m4 and HA : not all mj equal, where mj (j = 1, . . . , 4) denote the population mean incomes for the ethnic groups. The test statistic, denoted by F and following an F -distribution, is based on the ratio of the between variation (the variation between the sample group means) and the residual (within groups) variation. A statistically signicant result is obtained if the former is large compared to the latter. The conclusion that can be drawn from a signicant result is that the mean incomes for the ethnic groups are not all four equal. Of note, no causal conclusions can be made, since this is a nonexperimental study. In a factorial design, for instance, by the inclusion of gender as a second factor in the previous example hypotheses about main and interaction effects can be tested. A signicant main effect of gender implies that the marginal mean incomes of men and women (irrespective of the four ethnic groups) differ. A signicant interaction effect of gender and ethnicity on income implies that the differences in mean income between men and women are different among the four ethnic groups. Some important assumptions underlying the ANOVA are independence of observations and approximately normally distributed residuals, as well as approximately equal residual variances in the subgroups. Note that the practical conclusions that can be drawn from an ANOVA are somewhat limited. The null hypothesis all means are equal is evaluated against the rather uninformative alternative hypothesis stating nothing more than not all means are equal. Rejecting the null hypothesis in an ANOVA does not inform the researcher about which pairs of means differ from each other. Therefore, an ANOVA is often followed by pair-wise comparisons to further investigate where group differences are found. Since several tests are performed in such a case, the alpha level used per comparison is usually corrected to protect for an increased Type I error probability (post-hoc corrections). Several correction methods are developed, but unfortunately it is not always clear which method should be preferred. Another approach for further investigation of differences between specic means or investigation of a specic structure in the group means is contrast testing.

Anonymity

27

A second limitation of ANOVA is that directional testing is not possible. An exception is when the ANOVA is applied to a two-mean hypothesis; the ANOVA is then equivalent to the independent samples t test. However, it is regularly seen that researchers have specic expectations or theories in terms of the order of the population means. For instance, in a four-group ANOVA the actual hypothesis the researcher is interested in may be: m1 < m2 < m3 < m4 . Irene Klugkist
See also Alpha, Signicance Level of Test; Factorial Design; F -Test; Interval Measure; Level of Measurement; Mean; Null Hypothesis; p-Value; Ratio Measure; Signicance Level; Subgroup Analysis; t-Test; Type I Error; Variance

Further Readings

Field, A. P. (2005). Discovering statistics using SPSS (2nd ed.). London: Sage. Klugkist, I., Laudy, O., & Hoijtink, H. (2005). Inequality constrained analysis of variance: A Bayesian approach. Psychological Methods, 10(4), 477493. Tabachnick, B. G., & Fidell, L. S. (2006). Using multivariate statistics (5th ed.). Boston: Allyn & Bacon.

ANONYMITY
Anonymity is dened somewhat differently in survey research than in its more general use. According to the American Heritage Dictionary, anonymity is the quality or state of being unknown or unacknowledged. However, in survey research, the concept is more complex and open to interpretation by the various organizations that conduct surveys. In the form closest to the standard denition, anonymity refers to data collected from respondents who are completely unknown to anyone associated with the survey. That is, only the respondent knows that he or she participated in the survey, and the survey researcher can not identify the participants. More often, anonymity refers to data collected in surveys in which the respondents are de-identied and all possible identifying characteristics are separated from the publicly available data. Many survey research organizations provide data and data summaries to individuals outside their organizations. These data are

considered anonymous if those outside the survey organization cannot identify the survey participants. However, for many surveys dened as anonymous, the survey organization could, if needed, identify the respondents. For example, in a survey that uses pure random-digit dial procedures, limited information about the respondent is available to the survey organization. Through the use of various databases, the organization could possibly determine the household associated with the telephone number. Survey organizations would rarely do that. Survey researchers have developed a number of procedures for designing anonymous surveys. For example, many surveys conducted in classrooms or other gathered events use unnumbered questionnaires and do not contain questions that could identify respondents. For some classroom surveys, identifying information is collected on a sheet separate from the questionnaire. A procedure sometimes used in postal surveys is to include a return postcard along with return envelope. The unnumbered questionnaire is returned in the envelope, and the postcard is sent separately to let the researchers know that the questionnaire has been returned. Survey researchers have developed many techniques for conducting completely anonymous surveys. For example, Internet surveys offer multiple methods for anonymous participation. Some surveys may not require authentication to access the survey. Invitations are sent to potential participants but with no control over who participates nor how often. A more sophisticated recruitment method is to completely separate the database used for authentication from the database that contains the survey responses. Another method is for one organization to send the recruiting requests and a second to collect the data. A similar method can be used for telephone surveys. The telephone numbers can be stored in a database that has no direct link to the survey responses. This method can be used with random-digit dial telephone number samples to further separate the identifying information from the survey responses. However, the procedures for ensuring anonymity can conict with other important survey quality control procedures. For example, sending unnumbered paper questionnaires with postcards in postal surveys allows respondents to return the questionnaires but not the postcard. As a result, follow-up requests cannot be limited to nonrespondents only. Respondents who did not return the postcards may believe their