Recent Advances in Nutritional Sciences

Nonreplication in Genetic Association Studies of Obesity and Diabetes Research1,2

David T. Redden3 and David B. Allison Department of Biostatistics, Section on Statistical Genetics and Clinical Nutrition Research Center, University of Alabama at Birmingham, Birmingham, Alabama

account for population stratication and present our perspective on additional likely causes of nonreplication. Finally, we present our opinion on the advancements in designs needed in obesity and diabetes genetic research.

Obesity and Diabetes Association Studies. In the past

decade, the technological advancements in genomic research have led to a dramatic increase in published association studies. Lohmueller et al. (10) performed meta-analyses for several genetic association studies over a wide range of phenotypes. They reported meta-analyses for nine different markers (ABCC8 Intron 24 3T/C, ABCC8 Exon 22C/T, GYS1 Xbal RFLP, INSR Sstl RFLP, INSR V985M, KCNJ11 E23K, PPARG P12A, SLC2A1 Xbal RFLP, SLC2A2 and Taql RFLP) that had been examined for association with Type 2 diabetes in 50 published association studies. Of the 41 studies designed to replicate the initially observed associations, only 10 produced results in agreement with the original ndings (Table 1). Based on their meta-analysis results, Lohmueller et al. (10) concluded that of the nine potential markers associated with Type 2 diabetes, only three of them (ABCC8 Exon 22C/T, PPARG P12A, SLC2A1 Xbal RFLP) have sufcient replication to support the claim of association. Chagnon et al. (1) provided a comprehensive review of association studies of obesity-related phenotypes. They reported cumulatively 222 association studies of 71 markers with obesity-related phenotypes. Forty-nine associations of markers with obesity-related phenotypes were published in 2002 (1). Markers associated with obesity and BMI published in 2002 include LEPR, GHRL, PPARG, APM1, PPARGC1, CART, ADRB2, SERPINE1, ADRB3, CYP7A1, ADRB1, IGF2, INS, TH, DRD2, GNB3, HTR2A, SAH, AGRP, FIZZ3 and TNRC11 (1). Chagnon et al. (1) provided tables and gene maps detailing the information for each study and also reported that 49 studies published in 2002 showed no association between markers and obesity-related phenotypes. A comparison between signicant and non-signicant studies revealed substantial overlap in markers reported (1). Markers reported as having no association with obesity-related phenotype include PPARG, FIZZ3, GNB3 and ADRB3 (1).

ABSTRACT The objective of this article is to provide an overview of the existing literature concerning the identication of genetic markers associated with obesity and diabetes. Specically, this article will review recent association studies of diabetes and obesity with an emphasis on the need for the replication of ndings. Unfortunately, a substantial number of the published associations between genetic markers and phenotypes, including diabetes and obesity, have not been replicated. Literature that addresses the potential reasons for the nonreplication of association studies (population stratication, publication bias, effect heterogeneity, Type I errors and lack of statistical power) is summarized. Recommendations to improve future association studies are presented. J. Nutr. 133: 33233326, 2003.
KEY WORDS: association studies nonreplication population stratication Type I error statistical power

Downloaded from jn.nutrition.org by guest on November 3, 2013

Over the past decade, numerous research projects have reported associations between nutritional phenotypes (obesity, type 1 and 2 diabetes mellitus and energy expenditure) and regions of the human chromosomes (12). Unfortunately, many of the reported associations have not been replicated in independent research. The nonreplication of these association ndings is a concern and has caused some researchers to question the utility of association methodology in genetic studies (35). When the weaknesses of genetic associations studies are presented, the confounding of association due to population stratication is often emphasized. However, opinions regarding the importance of population stratication in association studies vary greatly (6 9). In this paper, we briey review the current literature regarding markers associated with nutrition-related phenotypes, specically obesity and diabetes. We discuss the growing concern across many research elds regarding the nonreplication of association studies. We review the cited reasons for nonreplication with emphasis on population stratication and its consequences on study design and genetic research. We review recent statistical approaches to

Limited Replication. Although the above examples do not

comprehensively review markers associated with obesity and Type 2 diabetes, they illustrate an issue of current debate in many scientic elds utilizing association methods. A great concern has arisen from the fact that few reported associations between markers and phenotypes are consistently and convincingly replicated. To lend support to this concern, Hirschhorn et al. (2) reviewed over 600 published positive association studies and discovered that only 166 were investigated three or more times. Of those 166 studies, Hirschhorn et al. (2) concluded that only six associations had been consistently replicated [75% percent of all association studies produced signicant (P 0.05) results]. As emphasized by Cardon and Bell (12), this lack of replicability has fueled skepticism regarding the overall usefulness of association methods in genetic research.

1 This work was supported in part by NIH Grants R01DK056366 and P30DK056336. 2 Manuscript received 1 July 2003. 3 To whom correspondence should be addressed. E-mail: dredden@ms.soph.uab.edu.

0022-3166/03 $3.00 2003 American Society for Nutritional Sciences.

3323

3324

REDDEN AND ALLISON

TABLE 1
Summary of the replication of association studies identifying markers associated with Type 2 diabetes1
Number of studies, P 0.05, same direction as original report 1 2 0 1 0 0 4 2 0 Number of studies, P 0.05, opposite direction as original report 1 0 1 0 0 0 1 0 0

Associated marker ABCC8 ABCC8* GYS1 INSR INSR KCNJ11 PPARG* SLC2A1* SLC2A2

Variant associated Intron 24-3T/C Exon 22 C/T Xbal RFLP Sstl RFLP V985M E23K P12A Xbal RFLP Taq1 RFLP

Total number of studies 9 4 3 4 4 6 14 3 3

Number of studies, P 0.05 2 2 1 1 0 0 5 2 0

1 Adapted from Lohmueller et al. (10). * Indicates that meta-analyses using replication studies provide sufcient evidence to support the claim of association between marker and Type 2 diabetes.

Many researchers have proposed reasons for the lack of reproducibility (2,8,10 12). Population stratication, publication bias, effect heterogeneity, lack of Type I error control and lack of statistical power to detect small to moderate effects have all been suggested as potential reasons (6,8,11). Of these possible reasons, population stratication seems to have received the most attention (6).

Population Stratication. Simply stated, population stratication can create confounding leading to spurious results in genetic association studies. We dene a spurious association as any association between allelic variation at a marker locus and phenotypic variation that is neither due to the fact that marker locus causes variation in phenotype nor is linked to a marker locus that causes variation in the phenotype. Population stratication occurs when both of the following conditions are met (6,8,12). First, allele frequencies under investigation must vary among subpopulations. Second, disease prevalence or mean phenotypic value must vary among subpopulations. If researchers are unaware of the subpopulations, subjects can be unknowingly selected from differing subpopulations and a spurious association between genotype and phenotype can be created. Under such circumstances the relationship between allele frequency and disease is confounded by the subpopulation (Fig. 1). With dichotomous outcomes, this situation is an example of Simpsons paradox (13), in which an apparent association between allele frequencies and disease is not present conditional on subpopulation. Two genetic association studies using case control methodology are commonly cited as examples of spurious ndings due to population stratication. Knowler et al. (14) reported an association between an HLA haplotype and diabetes for Pima Indians. When the analysis was repeated stratifying subjects by the amount of European ancestry, the observed association between HLA haplotype and diabetes was not present. The other common example is Blum et al. (15) who reported an association between alcoholism and the dopamine DRD2 allele. Gelernter et al. (16), through a review of all published association studies of the alcoholism and the dopamine DRD2 allele, demonstrated no association between phenotype and allele, indicating that the original nding was likely due to population stratication because of the large ethnic variation in the prevalence of the A1 allele and alcoholism. Because of the concern over potential population stratication leading to spurious results, family-based designs that are

not confounded by population stratication have been highly utilized. Specically, the original transmission disequilibrium test (TDT) introduced by Spielman et al. (17) has become a widely utilized design for genetic association studies because it is not susceptible to confounding by admixture or population stratication. Several authors (6,12) have recently questioned whether concern over population stratication warrants the shift to family-based designs. Cardon and Palmer (6) stated that few reported studies provide clear published examples of the biases created by population stratication. Furthermore, Wacholder et al. (8) indicated that a large bias due to population stratication is a rare occurrence unless a large correlation between allele frequencies and disease prevalence exists across ethnic groups that cannot be accounted for with questionnaire data on ethnic origin. Given the ndings of Wacholder (8), we suggest that population stratication, though an important consideration in design and interpretation, does not account for the majority of nonreplicable association studies currently observed in the literature.

Signicance Level and Statistical Power. Even though population stratication has received much attention, the published literature (6,9,10,12) is now also emphasizing the inadequate control of the Type I error rate and the lack of

FIGURE 1 Illustration of confounding between marker and phenotype due to population stratication. In an association study confounded by population stratication, an association between marker and phenotype is observed (broken line). However, the association is attributable to the unobserved admixture or subpopulations by which marker prevalence and phenotypic values vary together (solid lines).

NONREPLICATION IN GENETIC ASSOCIATION STUDIES

3325

power as potential reasons for nonreplication. Several factors contribute to the problem. First, within association studies, multiple markers can be tested for association with a specic phenotype. As illustrated by Cardon and Bell (12), if 100 independent markers are tested for association with a phenotype using 0.05 signicance without correction for multiple tests, then there is a 99% chance that at least one type I error will be made assuming all null hypotheses are true and all tests are independent. Colhoun et al. (11) cited that the most likely reason for nonreplication is false positive results by chance in the initial positive study. The chances of a false positive result are exacerbated by the fact that statistical tests for multiple loci are assessed in each study and only the positive results are reported. This is referred to as publication bias or the le drawer effect (18). Given the large number of markers available for investigation, extremely careful consideration of signicance levels must be undertaken in the design and interpretation of prospective association studies. Risch (9) and Lohmueller et al. (10) both provided the argument that inadequate power in replication studies may also be contributing to the large number of nonreplications. Under a true polygenic model, Risch (9) argued that the effect size for any single marker would be small to moderate. The meta-analyses conducted by Lohmueller et al. (10), for which they declared sufcient replication, support this conclusion with odds ratios for associated markers and phenotypes varying from 1.07 to 2.28. Given the limited number of individuals a given researcher has available to investigate associations, it is not surprising that numerous published results would fail to be replicated simply because of inadequate sample sizes.

Increasing the Replicability of Association Studies.

Given the aforementioned issues with association studies, the question must be raised as to how to improve their replicability as researchers continue to look for markers associated with diabetes and obesity. Given that the TDT test is robust against population stratication, one might opine that association studies should rely heavily upon that methodology. Unfortunately, when compared with association methods, the original TDT generally suffers because of the complexity of its design (6). First, parents must be recruited and genotyped. This requirement presents difculty for studying late onset diseases. Furthermore, TDT methodologies benet only from having heterozygous parents at marker alleles. Therefore, not everyone genotyped contributes information to the genetic association test. Given these limitations, as well as the greater statistical power of nonTDT association methods under certain circumstances, association studies utilizing nonTDT methodology are necessary. We are not stating that TDT-type approaches should not be used, only that they should not be considered the sole sensible approach to association testing. Utilizing recently developed statistical methods and sound statistical design, the aforementioned issues of population stratication, inated signicance levels and poor statistical power can be addressed.

Identifying and Adjusting for Population Stratication.

Because population stratication is a well-known weakness of association methods, substantial research effort has been focused on adjusting for population stratication. Genomic control methods (19) use multiple-unlinked genetic markers independent of disease (or any dichotomous phenotype) to calculate a correction factor for genetic association studies using case-control methodologies. In general, population stratication leads to an inated Chi-square test that rejects the null hypothesis of no association too often (although the

reverse is possible as we will discuss below). The genomic control approach estimates the ination factor due to population stratication and adjusts the statistical tests appropriately. Cardon and Bell (12) demonstrated via simulation that when candidate allele frequencies differ moderately (54% prevalence in one strata, 46% prevalence in the other), genomic control methodologies preform exceedingly well in protecting against Type I errors induced by population stratication. Interestingly, the genomic control methods do not protect well against inated Type I error rates when small deviations in candidate allele frequencies (51% prevalence in one strata, 49% prevalence in the other) occur. These methods have also been extended for use with quantitative traits (20) such as BMI. An important disadvantage of this approach is that it only corrects for ination in test statistics due to admixture or stratication. Although the fact that admixture and stratication can also suppress or mask real effects is less widely discussed, it has been clearly acknowledged (21). In such cases, this form of genomic control will not be helpful and will actually reduce power. However, TDT-type approaches and the methods of Devlin (19), Satten (22) and Pritchard and Rosenberg (23) do not suffer from this limitation. Devlin (19), Satten (22) and Pritchard and Rosenberg (23) also proposed utilizing multiple-unlinked genetic markers that are presumed independent of disease to provide a formal test for detection of population stratication within case control designs. Pritchard et al. (24) demonstrated that unlinked genetic markers can be used to estimate the proportion of an individuals genome that is derived from unobserved parental populations. Building upon this idea, Hoggart et al. (25) demonstrated that population stratication can be statistically estimated and controlled in association studies by utilizing unlinked genetic markers unrelated to the marker under investigation. These methods, culminating with that of Hoggart et al., offer several important advantages. First, Hoggart et al. allow for measurement error in the estimated admixture. Second, Hoggart et al.s method is also easily adaptable to quantitative phenotypes. Third, these approaches estimate the variance due to admixture and therefore, even in the absence of confounding, may be useful by reducing residual variation in the phenotype and thereby increasing power. Finally, as stated above, these methods are equally valuable whether admixture or stratication are creating spurious associations or masking real associations. Many authors believe that the use of the above methods to detect and control for the population stratication, coupled with careful adherence to standard epidemiologic methods, can greatly increase the acceptance of traditional association methods in genetic studies. We agree with several published papers (6,9,12) that association methodologies are essential to the investigation of genetic associations of complex diseases. However, we are concerned that recently published articles (6,8) may be misinterpreted by researchers in concluding that population stratication is not a concern. Because few examples of spurious results due to population stratication are found does not imply that the potential for spurious results is not a concern. This is true in part because it is not clear that we have conducted careful systematic investigations to determine how much problematic stratication or admixture is present. In the absence of an extremely thorough rigorous search for a phenomenon, lack of evidence cannot be interpreted as nonexistence. Furthermore, the results of Wacholder et al. (8) indicated that for studies of nonHispanic Caucasians of European decent bias due to population stratication may not be a major concern. However, Wacholder et al. (8) indicated that further work is needed to estimate the effect of

3326

REDDEN AND ALLISON

population stratication within other populations. It is crucial that researchers be aware of the potential pitfalls created by population stratication. It is equally important that researchers are aware of the epidemiologic design methods that can protect against population stratication and recently developed statistical methods to account for spurious associations due to population stratication. Although research into genomic control and subpopulation identication methods are not complete, they show initial promise in protection against spurious associations due to population stratication. Furthermore, incorporating these statistical methods into association studies may provide substantial benet. The identication of population substructure will provide greater ability to recognize effect heterogeneity, to identify masking of marker effects by subpopulation pooling and to increase statistical power to detect association by decreasing residual variation. Overall, utilizing statistical methods to identify and control for population stratication in association studies will assist in increasing the validity of results by removing the possibility that unobserved substructures in the data produced a spurious result. Otherwise, association studies without TDT-like analyses, genomic control or admixture estimation must acknowledge that any signicant results may be spurious.

tion studies of markers and nutrition-related phenotypes incorporate these methods and recommendations into their studies. LITERATURE CITED
1. Chagnon, Y. C., Rankinen, T., Snyder, E. E., Weisnagel, S. J., Perusse, L. & Bouchard C. (2003) The human obesity gene map: the 2002 update. Obes. Res. 11: 313343. 2. Hirschhorn, J. N., Lohmueller, K., Bryne, E. & Hirschhor, K. (2002) A comprehensive review of genetic association studies. Gen. Med. 4: 45 61. 3. Gambaro, G., Anglani, F. & DAngelo, A. (2000) Association study designs of complex diseases. Lancet 355: 308 311. 4. Strohman, R. (2002) Maneuvering in the complex path from genotype to phenotype. Science 296: 701703. 5. Holtzman, N. A. (2001) Putting the search for genes in perspective. Int. J. Health Serv. 31: 445 461. 6. Cardon, L. C. & Palmer, L. J. (2003) Population stratication and spurious allelic association. Lancet 361: 598 604. 7. Thomas, D. C. & Witte, J. S. (2002) Point: population straticationa problem for case control studies of candidate gene associations? Cancer Epidemiol. Biomarkers Prev. 11: 505512. 8. Wacholder, S., Rothman, N. & Caporaso, N. (2000) Population stratication in epidemiological studies of common genetic variants and cancer: quantication of bias. J. Natl. Cancer Inst. 92: 11511158. 9. Risch, N. J. (2000) Searching for genetic determinants in the new millennium. Nature 405: 847 856. 10. Lohmueller, K. E., Pearce, C. L., Pike, M., Lander, E. S. & Hirschhorn, J. N. (2003) Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nat. Genet. 33: 177182. 11. Colhoun, H. M., McKeigue, P. M. & Smith, G. D. (2003) Problems of reporting genetic association with complete outcomes. Lancet 361: 865 872. 12. Cardon, L. R. & Bell, J. I. (2001) Association study designs for complex diseases. Nat. Rev. Genet. 2: 9199. 13. Simpson, E. H. (1951) The interpretation of interaction in contingenay tables. J. Royal Stat. Soc. B 13: 238 241. 14. Knowler, W. C., Williams, R. C., Pettitt, D. J. & Steinberg, A. G. (1988) Gm 3; 5, 13, 14 and Type 2 diabetes mellitus: an association in American Indians with genetic admixture. Am. J. Hum. Genet. 43: 520 526. 15. Blum, K., Noble, E. P., Sheridan, P. J., Mongtomery, A., Ritchie, T., Jagadeeswarns, P., Nogami, H., Briggs, A. H. & Cohn, J. B. (1990) Allelic association of human dopamine D(2) receptor gene in alcoholism. J. Am. Med. Assoc. 263: 20552060. 16. Gelernter, J., Goldman, D. & Risch, N. (1993) The A1 allele at the DZ dopamine receptor gene and alcoholism. A reappraisal. J. Am. Med. Assoc. 269: 16731677. 17. Spielman, R., McGinnus, R. & Ewens, W. (1993) Transmission test for linkage disequilibrium: the insulin gene region and insulin-dependent diabetes mellitus (IDDM). Am. J. Human Genet. 52: 506 516. 18. Cook, T. D., Cooper, H., Cordray, D. S., Hartman, H., Hedges, L. V., Light, R. J., Louis, T. A. & Mosteller, F. (1992) Meta-analysis for Explanation. Russell Sage Foundation. New York, New York. 19. Devlin, B. & Roeder, K. (1999) Genomic control for association studies. Biometrics 55: 9971004. 20. Bacanu, S-A., Devlin, B. & Roeder, K. (2002) Association studies for quantitative traits in structured populations. Am. J. Human Genet. 67: 78 93. 21. Deng, H. W. (2001) Population admixture may appear to mask, change, or reverse genetic effects of genes underlying complex traits. Genetics 159: 1319 1323. 22. Satten, G. A, Flanders, W. D. & Yang, Q. (2001) Accounting for unmeasured population substructure in case control studies of genetic association studies using a novel latent-class structure. Am. J. Human Genet. 68: 466 477. 23. Pritchard, J. K. & Rosenberg, N. A. (1999) Use of unlinked genetic markers to detect population stratication in association studies. Am. J. Human Genet. 65: 220 228. 24. Pritchard, J. K., Stephens, M. & Donnelly, P. (2000) Inference of population structure using multilocus genotype data. Genetics 155: 945959. 25. Hoggart, C. J., Esteban, J. P., Shriver, M. D., Bonilla, C., Kittles, R. A., Clayton, D. G. & McKeigue, P. M. (2003) Control of confounding of genetic association in stratied populations. Am. J. Human Genet. 72: 14921504. 26. Allison, D. B., Beasley, T. M., Fernandez, J., Heo, M., Zhu, S., Etzel, C. & Amos, C. I. (2002) Bias in estimates of quantitative-trait-locus effect in genome scans: demonstration of the phenomenon and a method-of-moments procedure for reducing bias. Am. J. Human Genet. 70: 575585.

Controlling Signicance Levels and Power. Colhoun et al. (11) make the clearest recommendations regarding significance levels. They recommended that instead of the traditional 5 102 (0.05), the signicance level of 5 105 (0.00005) be used. This is equivalent to adjusting the traditional 0.05 signicance level for 1000 tests. Of course, the stringent signicance level recommended by Colhoun (11) greatly increases the sample size needed to achieve adequate power. The sample size required to obtain 90% power would roughly increase by threefold. However, this stringent alpha level and large sample sizes would not necessarily be required for replication studies. We further recommend that the number of markers tested in each sample be presented as well as the number of negative associations. We also recommend that for each signicant result, a postanalysis power calculation be performed to indicate the probability of observing the effect size under the conditions of the study. It may be important that such analyses take into account the likely ination of apparent effect that will be observed in studies of multiple polymorphisms in which only statistically signicant effects are selected for follow-up (26). This power calculation will provide rough guidance to groups designing replication studies as to the likelihood of observing a statistically signicant association within their study sample assuming equal population effects. Locating and understanding the genetic factors of complex traits depends completely upon the researchers ability to correctly identify and replicate true associations between markers and disease. The recent advancements in statistical methods to detect and adjust for population stratication offer a new opportunity in the design of association studies. These recent advancements coupled with recommended adjustments in signicance levels and statistical power are needed to produce replicable associations between markers and disease phenotypes. We recommend that all researchers designing associa-