Correlations in Genetic Risk Scores Produced by Direct-to-Consumer Genetic Testing Companies

Correlations in Genetic Risk Scores Produced by Direct-to-Consumer Genetic Testing Companies A thesis submitted to the Graduate School of the University of Cincinnati in partial fulfillment of the requirements for the degree of Master of Science in the Department of Pediatrics of the College of Medicine April 2013 by Brian Douglas Reys B.S. The Ohio State University 2011 Committee Chair: Ge Zhang, MD, PhD Committee Members: Mehdi Keddache, PhD Melanie Myers, PhD, MS, CGC Daniel Prows, PhD

Abstract Background. Direct-to-consumer genetic testing companies provide consumers genetic risk scores for common diseases based on genotype. Single nucleotide polymorphism (SNP)- associated risk estimates published by genome wide association studies are the most common source of genotype-driven risk information for common diseases. However, the risk estimate of any given SNP varies depending on the source population and study design of the original publication. An important factor in establishing clinical validity for genetic testing of common disease is the consistency in genetic risk scoring between direct-to-consumer companies. Such an association however, has not been well described. While small-scale studies looking at individual sample results between direct-to-consumer companies have been performed, to our knowledge, no large-scale studies aiming to measure the consistency in risk scoring have been reported. Methods. A genotyped cohort of 834 individuals was used to calculate the equivalent genetic risk score that would be produced by the direct-to-consumer genetic testing companies 23andMe and DeCODE Genetics for two diseases, type 2 diabetes (T2D) and agerelated macular degeneration (AMD). These scores were compared to comprehensive academic SNP panels to look at the consistency between the risk scoring of different companies and academic literature. Results. Our results showed that although the genetic risk scores calculated based on different SNP risk panels (23andMe, DeCODE Genetics and academic) were significantly correlated, (r 2 = 0.46-0.66 for T2D and r 2 = 0.30-0.70 for AMD), the levels of correlation were far from appropriate to establish the clinical utility of these SNP-based genetic scores. In addition, the ranges of the estimated genetic scores varied substantially among these three different SNP risk panels with a greater number of SNPs utilized roughly correlating with II

increased range in risk score. Conclusion. Significant differences in the number of SNPs used to calculate risk score as well as selection of SNP risk estimates are the primary causes of inconsistency in risk scoring between direct-to-consumer companies. To improve consistency direct to consumer genetic testing companies need to incorporate into their calculations more recently published SNP associations, use consistent SNP effect sizes and use similar numbers of SNPs. III

(This page left blank intentionally) IV

Table of Contents Page # Abstract... II Table of Contents... V List of Tables... VI List of Figures... VI Introduction... 1 Methods... 4 i. DTC Genetic Testing Company Selection... 5 ii. Common Disease Selection... 5 iii. Cohort... 6 iv. Genotyping... 6 v. Genetic Risk Prediction... 7 Results... 8 vi. Results: Type 2 Diabetes... 8 vii. Results: Age-Related Macular Degeneration... 10 Discussion... 11 viii. Study Limitations... 16 ix. Conclusions... 17 References... 20 Tables... 22 Figures... 27 V

List of Tables I. Type 2 Diabetes SNP Reported Risks by SNP II. III. IV. Age-Related Macular Degeneration Reported Risks by SNP Mean and SDs of Genetic Risk Scores for Type 2 Diabetes Mean and SDs of Genetic Risk Scores for Age-Related Macular Degeneration List of Figures I. Venn Diagram of Number of Identical SNPs Shared between Panels II. III. IV. Distributions of Genetic Risk Score by Source for Type 2 Diabetes Correlation Plots between Sources for Type 2 Diabetes Distributions of Genetic Risk Score by Source for Age-Related Macular Degeneration V. Correlation Plots between Sources for Age-Related Macular Degeneration VI

Introduction Direct-to-consumer (DTC) genetic testing has introduced a new frontier to personalized medicine by providing consumers direct access to genetic testing for common diseases without the medium of a healthcare provider. A growing industry with limited regulation and consumer protections, DTC genetic testing has arrived to the field of genetics with both concerns and opportunity [1-3]. Today single nucleotide polymorphism (SNP)-based genetic testing runs the gambit from nutrition and ancestry testing to predicting the inherent genetic risk of developing common diseases [2, 4]. While the SNP-based genotyping technology behind DTC genetic testing for common diseases is well understood, the interpretation of genotypic information into predictive genetic risk remains complex and varied across DTC genetic testing companies [5, 6]. Advocates of DTC genetic testing note that benefits of testing include increased access to testing, improved health awareness and potential health benefits to consumers. In early adoption of personal genomics, individuals were optimistic about using genomic profiling in consultation with physicians to improve health [7]. Further studies gauging interest in personalized genomic testing have shown that the majority of consumers who would consider testing would ask their physician for help with the interpretation of the results [8, 9]. Additionally, DTC genetic testing companies may contribute their own research into the area of common disease; the DTC genetic testing company 23andMe has recently pursued patents for SNPs implicated in Parkinson s disease [10, 11]. 1

Despite notable benefits to DTC genetic testing, critics of DTC genetic testing for common diseases list concerns for the clinical utility, interpretation and consistency of DTC genetic testing results. Given the complex nature of genetic information, a clear understanding of testing results remains important in communicating disease risk. Furthermore consumers may be prone to overestimating the health benefit of testing as well as misinterpreting the results compared to healthcare providers [12]. While consumers who overestimate their risk might seek additional healthcare guidance, alternatively, consumers who underestimate their risk may not seek appropriate healthcare guidance [13]. Likewise it is unclear if DTC genetic testing results will change behavior in the cases of predictive health testing [14]. Possible misinterpretation of results demonstrates the need for healthcare providers or genetic counseling services in the facilitation of interpreting DTC genetic testing results to consumers. However, literature suggests that healthcare providers are not prepared to interpret DTC genetic testing results [15]. A study performed by the Government Accountability Office (GAO) noted inconsistency in disease risk results between DTC genetic testing companies that was concerning given that consumers may treat DTC genetic testing disease risk as clinically significant [16]. Inconsistencies which may provide different risks to an individual for the same disease, cause significant concern for the clinical utility and interpretation of the test and consistency in risk results between companies remains a concern in the field today. Understanding the consistency in disease risk results between major DTC genetic testing companies has been the focus of several recent studies, however the definition and measurement of consistency varies [17-19]. For this study, consistency represents the uniformity in genetic risk score for an individual for a given common disease across DTC genetic 2

testing companies. In 2009, Nature published a study by Ng et al. that compared test results from five individuals DNA samples sent to two DTC genetic testing companies, 23andMe and Navigenics [17]. While the study found a high concordance (99.7%) in the SNP data (genotypic microarray results), generated between the companies, their calculated genetic risk scores differed significantly for the individuals tested. In another study, DNA samples from an individual healthy volunteer were sent to the DTC genetic testing companies 23andMe, DeCODE Genetics (DeCODE) and Navigenics [18]. This study similarly found the concordance of SNP data to be high (>99.6%), but the predictive genetic risk scores between companies were poorly correlated. As an example, the relative risk estimates for rheumatoid arthritis from the three companies ranged between 0.9 (protective) to 1.85 (harmful). These previous studies have focused on comparing the correlation between companies based on a few samples submitted to multiple companies. While this method allows the studies to look across many diseases tested between companies, it provides a narrow view of the possible range of scores provided to consumers who use these services. As each individual s risk is dependent on their genotype, studies with a small sample size are limited to data produced only from their samples genotypes. Our study aims to expand the current description of consistency between major DTC genetic testing companies using a much larger sample size to better predict the range in genetic risk scores for two common diseases. For this analysis, we have assessed the consistency between major DTC genetics testing companies 23andMe and DeCODE, using type 2 diabetes (T2D) and age-related macular degeneration (AMD) as exemplar diseases. T2D is a well-studied common disease displaying multifactorial inheritance with an estimated heritability ranging from 26-77% [20-22]. The DNA 3

and Public Policy Center reports that 9 DTC genetic testing companies and 2 DTC genetic testing through-physician companies currently provide testing for the genetic risk of T2D [23]. Currently there exists between 50-64 SNPs (depending on the rigor of association) in populations with European ancestry known to be associated with T2D, however these SNPs explain less than 10-13% of the known heritability of T2D [22, 24, 25]. Age-related macular degeneration (AMD) is another well-studied common disease. AMD has a significant known SNP contribution to disease risk with SNPs accounting for nearly 25% [22] of AMD s genetic contribution to overall heritability, which is estimated to range from 45-71% [22, 26-28]. In contrast to T2D, the large SNP heritability in AMD is accounted for by less than 30 known SNPs [29, 30]. These diseases are utilized in our study to describe differing scenarios, a disease with a relatively large genetic contribution from a few SNPs (AMD), and a disease with a small genetic contribution from many SNPs (T2D) with small effects. Together these diseases help identify the current range in genetic risk for common diseases estimated by DTC genetic testing companies. Methods To observe the correlation of genetic risk interpretation between DTC genetic testing companies, we generated genetic risk scores equivalent to those provided by DTC genetic testing companies 23andMe and DeCODE for 834 genotyped individuals from a control cohort. The data used for risk calculation was based on available information that DTC genetic testing companies use to predict genetic risk on their websites. The extracted information included which SNPs were used in the calculation as well as the associated risk estimates for each SNP. 4

The data from DTC genetic testing websites was compiled and applied to the genotypic data from our cohort to calculate the genetic risk for the common diseases T2D and AMD. DTC Genetic Testing Company Selection DTC genetic testing companies were chosen based on open availability of genomic risk prediction information on the company website. To determine the companies to include in our study, a list of 27 DTC genetic testing and DTC genetic testing through-physician companies, published by the Genetics and Public Policy Center was obtained [23]. A web search was performed that narrowed the list to four companies based on the availability of the necessary data for genetic risk score prediction, including SNPs used in risk calculation, odds ratio (OR)/relative risk (RR) for the SNPs and genotype. The selected companies 23andMe, DeCODE, Navigenics and Pathway Genomics were then further narrowed to 23andMe and DeCODE, based on company acquisitions and changes in website data access over the course of our data collection period (July 2012 to Nov. 2012). Common Disease Selection AMD and T2D were selected as exemplar diseases to represent the genetic spectrum of common diseases assessed by DTC genetic testing. T2D is a common disease of phenotypic and genetic heterogeneity with many small-effect SNP-associations contributing to a relatively small genetic contribution to disease risk 10-13% [22, 24, 25]. In contrast, AMD is the paradigm disease of SNP-based common disease testing, with only a few SNP-associations representing nearly 25% [22] of the significant genetic contribution (45-71%) [22, 26-28] to heritability. However, despite significant research in the genetics of both these diseases, much of the 5

heritability remains unexplained. As such, T2D and AMD represent many common diseases in which continued research is needed to explain further genetic contribution to disease. Cohort The Cincinnati Genomic Control Cohort was used to provide the genotypic data for the study. The cohort consists of 995 healthy individuals (at the time of enrollment) ages 3-18 from the Greater Cincinnati area, including individuals from the tri-state area of Ohio, northern Kentucky and southeastern Indiana. The cohort population was selected to be representative of the Cincinnati population based on the 2010 US Census data and consists of roughly 80% Caucasian, 20% African American and <1% of individuals of Asian descent. The racial stratification percentages provided are based on parental report. For the cohort study, parents of the participants filled out a form detailing age, race and gender for their child and the participants provided samples of urine, serum, hair and DNA. Each individual in the cohort was genotyped using an Affymetrix 6.0 array. The genotypic information used in our study was obtained de-identified from the cohort. The joint Cincinnati Children s Hospital Medical Center and University of Cincinnati Institutional Review Board deemed this study to be non-human subject research (Study ID: 2012-1621). Genotyping The 995 samples of the Cincinnati Genomic Control Cohort were successfully genotyped using the Affymetrix Human SNP Array 6.0 following the manufacture s protocol. Genotype calls were determined using the CRLMM algorithm [31, 32] among chips that passed the vender suggested quality control (Contrast QC > 0.4). Specifically, contrast QC removed poor quality samples of the 995 raw genotype intensity data (cel files) from the cohort narrowing the cohort 6

to 834 individuals, an 83% pass rate. Genotype calling was performed using CRLMM for the 906,600 SNP markers generated for each individual. Imputation for the SNPs used by DTC genetic testing companies not represented by the Affymetrix 6.0 array was performed using MACH and the Minimac program [33, 34]. The reference haplotypes for the imputation were extracted from the phased genotype calls of all (ALL, N=1092) samples of the 1000 Genomes Integrated Phase I release [35]. Genetic Risk Prediction Genetic risk scores for each individual in the cohort were calculated for T2D and AMD using the equation: ( ) G is the genotype coded as either 0, 1 or 2 for the number of risk allele(s) at SNP i and β is defined as the reported effect size estimate (beta coefficient or log odds ratio) of the risk allele at SNP i. Instead of using odds ratios (ORs), 23andMe uses an adjusted OR and DeCODE uses an adjusted genotype relative risk (RR) in reporting the effect sizes of the risk alleles. For direct comparisons, these risk measures were converted to ORs by accounting for allele frequency and prevalence in control populations. The allele frequencies used in conversion of risk measures to ORs were taken from the 1000 Genomes Project [36], which are based on European ancestry. A disease prevalence of 25.7% for T2D and 6.5% for AMD were used in calculations and obtained from the DTC genetic testing company 23andMe website [11]. The DTC genetic testing company, genetic risk scores were then compared to scores generated using SNPs and ORs extracted from a published academic report for T2D [37] and AMD [30]. A PubMed search using keywords was performed to identify academic papers with 7

comprehensive SNP lists. Two academic papers, one each for T2D and AMD, were selected based on their recent publication (since 2011) and maximum number of SNPs listed (79 T2D and 24 AMD) compared to other academic papers found. Strength of SNP association was not considered, as there is no current consensus in appropriate cutoff value. Results Of the DTC genetic testing companies we reviewed online, two (23andMe and DeCODE) of 27 provided open web access to the SNP and adjusted OR/RR numbers they use to generate genetic risk scores. For T2D, 23andMe uses 11 SNPs, DeCODE uses 21 SNPs and an academic paper by Sanghera et al. listed 33 European SNPs (79 total SNPs). Only the 33 European SNPs were used in the current study, shown in Table 1 [37]. For AMD, 23andMe uses 3 SNPs, DeCODE uses 6 SNPs and the academic paper by Cipriani et al. listed 24 SNPs, see Table 2 [30]. Before any calculations were performed the wide discrepancies in the number of SNPs used between sources for each disease provided strong insight into the likely correlation between risk results provided to consumers. Results: Type 2 Diabetes A Venn diagram for each disease was created to display the overlap in SNPs used between the three sources (academic, 23andMe and DeCODE), Figure 1. Within T2D the academic source shared 6/33 SNPs with both 23andMe and DeCODE, 8/33 with 23andMe, 10/33 with DeCODE and had 21/33 unique SNPs. 23andMe shared 6/11 SNPs with both the academic source and DeCODEme, 8/11 with the academic source, 7/11 with DeCODE and had 2/11 unique SNPs. DeCODE shared 6/21 SNPs with both the academic source and 23andMe, 8

10/21 with the academic source, 7/21 with 23andME and had 10/21 unique SNPs. After accounting for the shared SNPs, there was a range of 2 unique SNPs used by 23andMe to 21 unique SNPs used in the academic paper representing a large gap in the current SNP knowledge and the utilization of all known SNPs in commercial testing. However, linkage disequilibrium was not accounted for between these SNPs and could explain some lack in overlap between the comprehensive panel and the DTC genetic testing companies. The mean genetic risk score and standard deviations calculated for each source are shown in Table 3. The distribution of genetic scores across the three groups (academic, DeCODE and 23andMe) is shown in Figure 2 (A-C). Each of the three sources showed a normal distribution as needed to calculate correlation, and each source had a mean genetic score close to zero (representing population risk), as would be expected from a large random sample. Notably, the academic panel, which included the greatest number of SNPs, had the genetic score mean closest to zero as well as the widest variation in genetic scores as expected given the greater number of SNPs used in the calculation. Pearson correlation was calculated for T2D between the genetic scores generated for each SNP source (academic, 23andMe, DeCODE). The r 2 can be found in the x-y plots showing the correlation between genetic scores estimated based on different risk panels (Figure 3, A-C). The genetic scores for DeCODE and 23andMe compared to the academic panel were similarly significant, r 2 = 0.46, p < 0.05 and, r 2 = 0.42, p < 0.05, respectively (Figure 3 A,B). The correlation between the two DTC genetic testing companies 23andMe and DeCODE (r 2 = 0.66, p < 0.05) was the highest of the three comparisons (Figure 3C). The higher correlation between 9

23andMe and DeCODE was not surprising, given that 23andMe shared 7 of 11 SNPs in common with DeCODE. Significant correlation was found for T2D between all sources; however given the overlap in SNPs used, the correlation was far from satisfactory. While an appropriate correlation for molecular genetic testing has not been established, a correlation between companies of r > 0.99, would be expected from a test used for clinical purposes as suggested by Cary, R.N. et al [38]. In addition, it can be seen in the variation in genetic scores (Figure 2, A-C) between sources that the highest and lowest possible genetic risk score is roughly proportional to the number of SNPs used in the panel, with the academic panel having the widest range of possible genetic score. Results: Age-Related Macular Degeneration A Venn diagram displaying the overlap in SNPs used between the three sources (academic, 23andMe and DeCODE) for AMD is shown in Figure 1B. Within AMD there were zero SNPs shared between all three sources. The academic sources shared 4/28 SNPs with DeCODE and had 24/28 unique SNPs. 23andMe shared 2/3 SNPs with DeCODE and had 1/3 unique SNPs. DeCODE shared 4/6 with the academic sources and 2/6 with 23andMe; DeCODE had no unique SNPs. There were also no SNPs shared between the academic panel and 23andMe. The mean genetic risk score and standard deviations calculated for each source are shown in Table 4. The distribution of genetic risk scores for AMD across the three panels is shown in Figure 4 (A-C). While normal distribution was present for all three sources, 23andMe and DeCODE had multiple peaks (Figure 4 B,C) caused by the relatively few number of SNPs 10

used in calculating their distribution. Across all three sources the mean genetic score was negative ranging from -0.26 to -0.58, indicating that within our cohort there was a general trend toward having a protective genotype. Similarly to the results for T2D, the range of possible genetic risk score was roughly proportional to the number of SNP markers used by the source. Pearson correlation was calculated for AMD between the genetic scores generated for each SNP source (academic, 23andMe, DeCODE). The r 2 can be found in the x-y plots showing the correlation between genetic scores estimated based on different risk panels (Figure 5, A-C) Correlation was calculated between the three sources. The academic source and 23andMe had an r 2 = 0.70, p < 0.05, the academic source with DeCODE had a r 2 = 0.30, p < 0.05, and 23andMe compared to DeCODE had an r 2 = 0.31, p < 0.05. All three correlations were statistically significant. Similar to the results for T2D, while there was significant correlation between sources, the correlation was not as highly significant (r > 0.99) as would be expected from a test used for clinical purposes. For AMD the highest correlation was noted between 23andMe and the academic source, which shared no SNPs in common. This suggests high linkage disequilibrium between SNPs from both sources. The position column of Table 2 suggests all three SNPs used by 23andMe are in high linkage disequilibrium with SNPs used in the academic source (a SNP within 1 cm, ~1,000,000 nucleotides). Discussion 11

Our results confirm that genetic testing for predisposition to common disease is far from consistent between DTC genetic testing companies. While previous studies have shown that consistency between individual samples submitted to multiple DTC genetic testing companies can produce significantly differing results, the consistency in genetic risk scoring between DTC genetic testing companies has been poorly understood. Genetic risk score used in our study is not a predictor of an individual s absolute genetic risk, which accounts for disease prevalence, age, gender and a number of other risk factors. Rather, genetic risk score provides a tool to compare consistency in the genetic risk reported between companies. The results from our study indicate significant inconsistencies in risk scoring between major DTC genetic companies testing for common disease, as well as clear differences in the number of SNPs used by companies and reported in academic literature. When considering the equation for genetic risk score there are three factors that combine to produce genetic risk; the number of SNPs used, the risk effect of each SNP and the genotype. As an individual s genotype stays consistent between companies and that DTC genetic testing companies use genotyping technology with high analytical validity (>99.6%) [17, 18], it can be assumed that genetic risk score inconsistency does not come from genotyping, but rather, the number of SNPs used and the risk effect of each SNP. To address differences in the number of SNPs used between sources our study compiled a list of the SNPs used (including the effect size of each SNP) by DTC genetic testing companies 23andMe and DeCODE. Additionally, SNP markers were collected from academic studies to create a comparison of incorporation of SNP usage by DTC genetic testing companies and the number of SNPs reported in academic literature [17, 18]. Correlation was calculated between the sources to measure the 12

consistency in genetic risk scoring across both DTC genetic testing companies and DTC genetic testing companies and recent academic studies. Our results show that an increased number of SNPs utilized by a company is associated with an increased range of genetic scores provided. Interestingly the number of SNPs shared between two companies was not consistently a good indicator of high correlation as would be expected. For example 23andMe shared 7/11 total SNPs with DeCODE (r 2 = 0.66) and 8/11 total SNPs with the academic source (r 2 = 0.41) with a significantly lower correlation. Such a phenomenon could indicate that SNPs not shared between companies have significant impact on the correlation (in this example the academic source has 25 SNPs not shared with 23andMe) or could indicate that several SNPs are in high linkage disequilibrium between sources. An example of high linkage disequilibrium could be seen in AMD between the academic source and 23andMe, which share no SNPs in common, yet have a high correlation (r 2 = 0.70). A direct comparison of the numbers of SNPs utilized by DTC genetic testing companies suggests that while linkage disequilibrium is likely present, it cannot account for the large gaps in the number of SNPs used between companies. While ideally linkage disequilibrium could be taken into account for all SNPs within the study, confounding factors such as allelic heterogeneity where SNPs may appear to have high linkage disequilibrium (r 2 = >0.8) but actually have significantly different risk effect and much lower linkage disequilibrium. Instances of allelic heterogeneity may be challenging to identify from simple differences in effect size estimation and therefore are often difficult to take into account. Our findings suggest that while the number of SNPs used by a source is critical, the need for all SNPs to be identical between sources is unnecessary. However, if two SNPs have an 13

r 2 = 1.0, then use of the same effect size is critical in calculating risk score. An example of allelic heterogeneity can be seen in our study between two SNPs for the disease AMD, rs#1061147 and rs#1410996. The SNPs rs#1061147 and rs#1410996 lie only ~40kb in separation, suggesting that they may be in high linkage disequilibrium (<1 cm in separation). However, these SNPs have an r 2 = 0.36, not r 2 = 1.0, supporting that they likely represent two different effects contributing to AMD. If a source were to assume the two SNPs were in high linkage disequilibrium, the source might incorrectly utilize only one of the two SNPs in risk calculation rather than both. While, SNPs with high linkage disequilibrium used by different companies can allow the usage of different platforms (Affymetrix vs. Illumina) to produce the same risk score, the effect of allelic heterogeneity, linkage disequilibrium and differences in effect size need to be taken into account. Our study suggests that much of the discrepancy in the number of SNPs used by DTC genetic testing companies lies in the lack of incorporation of newly found SNPs into risk calculation. For example in the calculation of AMD risk, 23andMe uses three SNPs from three different chromosomes, whereas SNPs have been associated with AMD on 12 different chromosomes in the academic paper [30]. The discrepancy in number of unique SNPs used in risk calculation cannot be attributed to linkage disequilibrium where risk information from 9 chromosomes is not accounted for by 23andMe. Standardization in the incorporation of newly discovered SNP-associations is a key factor in increasing the consistency in genetic risk scoring for SNP-based testing. Additionally, clinical validity will not improve with consistency, only with additional research into the genetic cause of disease. However, if only 10-13% of the heritability of T2D is accounted for by the >50 known SNPs, DTC genetic testing companies 14

need to use the full panel of known SNPs to achieve a genetic score that represents 10-13% of known heritability [22, 24, 25]. While this is an intuitive concept, barriers exist to DTC genetic testing companies integrating new SNPs into practice. The DTC genetic testing company 23andMe publishes standards they use to incorporate SNPs into their panel [39], but standards that are meant to protect consumers from additions of poorly-studied SNPs may also cause DTC genetic testing companies to lag behind in the inclusion of new SNPs into disease risk calculation. Barriers to incorporation of new SNPs often include quality of the research performed, the sample size of the study, the strength of associated between the SNP and disease, factors such as population ethnicity, gender and age, as well as others. It would be easy to assume that high standards cause the disparity in number of SNPs incorporated into DTC genetic testing panels. As an example, 23andMe has not updated the SNPs they use for T2D risk calculation since 2010 [11], yet the number of SNP associations discovered between 2007-2008 outpaced the entire preceding decade [40], a trend of rapid discovery that has continued past 2008. However, many of the SNPs listed in Sanghera et al. [37], used in our study as a comprehensive SNP list for T2D, have already met current standards listed by 23andMe for inclusion but have not yet been incorporated by 23andMe [37, 39]. Such findings suggest that rather than standards preventing new incorporation of recently discovered SNPs, DTC genetic testing companies are not keeping up with the rate of discovery. While the focus of this study was on the consistency between DTC genetic testing company risk scores, inconsistency between the effect size estimates reported for SNPs from genome wide association studies (GWAS) remains a concern. Mentioned previously, the risk effect of each SNP is a key component in the calculation of disease risk score. As the effect size 15

estimates reported depend on the study population and study design, there are significant differences between the effect size estimates being reported by different academic sources even for a single SNP. When DTC genetic testing companies select a SNP to incorporate they may choose to average an effect size estimate taken from multiple academic sources or chose the effect size estimate from an academic source with the largest study population. Use of effect size estimates that vary slightly may seem inconsequential, however even slight variations have the potential to adjust risk when many SNPs are used in conjunction for disease risk calculation. An additional consideration not accounted for by the equation that generates genetic risk score is ethnic and genetic population diversity. When population diversity is taken into account, it is known that 96% of GWAS performed up to 2011 have used primarily European populations [41, 42]. While the DTC genetic testing company 23andMe incorporates SNPs for other ethnicities when available, users who are unaware of these important differences may not recognize the importance of their ancestry in the genetic prediction of disease risk. In the case of T2D, 23andMe only uses one SNP for individuals of African ancestry [11]. For consumers of mixed heritage or ethnic backgrounds not widely studied, a clear need remains for additional studies in these populations. In the case of mixed ethnic backgrounds, it is unclear what benefit DTC genetic testing would have for the consumer. Study Limitations This study was limited in the number of diseases compared. Whereas previous studies have looked between companies at multiple diseases, such was not the primary aim of our study. T2D and AMD were chosen as well-studied common diseases each with significant 16

features similar and overlapping with features significant to the predisposition SNP testing for other common diseases. Increasing the number of diseases examined in this study would provide corresponding correlation results for additional diseases, but would not necessarily broaden the study scope. The results of this study remain a confirmation of an intuitive hypothesis that an increased number of SNPs increases the range of possible genetic scores provided for a disease and that consistency in risk scoring between DTC genetic testing companies is far from adequate. While mentioned in this study, an in-depth look at linkage disequilibrium was not considered in comparing the difference/overlapping of risk SNPs used by different risk panels. Linkage disequilibrium is a confounding factor in the study of SNP-based disease associations where SNPs positioned closely on a chromosome may convey similar disease risk. To avoid duplication in selecting risk SNPs (for the academic panel) only one academic publication each was selected to represent T2D and AMD instead of multiple papers. Both of the selected studies, however, cited SNPs from multiple GWAS, so some linkage-disequilibrium between SNPs is expected. Conclusions Despite recent efforts by the FDA to scrutinize the field of DTC genetic testing, many scientific concerns in the field over how best to interpret the clinical validity and utility of SNPbased genetic risk results remain. While samples sent to multiple DTC genetic testing companies may receive differing results, the underlying problem lies not in the genotyping technology used by companies [17, 18], but in the interpretation of genotype into absolute 17

genetic risk. Furthermore, the current consistency, regardless of clinical utility and validity, does not meet standards for clinical testing. Continued research into the SNP-based association to disease and the genetic contribution to disease, remains critical to the forward growth of testing for SNP-based disease risk prediction. In addition, while the number and risk-effects of SNPs associated with disease risk continues to be an important component in the consistency between companies, the current knowledge of complex disease is far from the point of clinical utility. To increase consistency, companies should utilize collective available knowledge in incorporating newly validated SNPs. To do so, however, appropriate standards should be created to provide consistency for SNP incorporation into disease risk prediction. DTC genetic testing companies interested in improving the field of SNP-based diseaserisk testing should consider an open source policy, as practiced by 23andMe and DeCODE in SNP selection and usage. Such a policy would allow companies to compare SNP markers and increase inter-company consistency as well as provide a framework for appropriate SNP inclusion into disease risk panels. In addition, companies should create and openly publish (together or separately) quality standards for SNP and effect size selection in an effort to promote greater consistency between companies. The road for DTC genetic testing is a new one, with future obstacles to overcome. However, as DTC genetic testing companies strive to promote their product to consumers and expand their testing to cover more diseases, the need for improved consistency remains. Even after incorporation of the current SNP knowledge of common disease, the clinical validity and utility of SNP-based genetic risk prediction is a long way from incorporation into medical 18

practice. Only continued research effort into the field of complex disease will reveal the longterm potential of SNP-based testing. With the additions of whole-exome and whole-genome sequencing, SNP-based genotyping technology may quickly be replaced by DTC genetic testing companies and the medical field alike. Regardless of the technology behind the genotyping, consistency in interpretation of genetic information into disease risk will continue to be a topic of consideration well into the future. 19

References 1. Melzer, D., et al., Genetic tests for common diseases: new insights, old concerns. BMJ, 2008. 336(7644): p. 590-3. 2. Goddard, K.A., et al., Awareness and use of direct-to-consumer nutrigenomic tests, United States, 2006. Genet Med, 2007. 9(8): p. 510-7. 3. Magnus, D., M.K. Cho, and R. Cook-Deegan, Direct-to-consumer genetic tests: beyond medical regulation? Genome Med, 2009. 1(2): p. 17. 4. Janssens, A.C., A.A. Wilde, and I.M. van Langen, The sense and nonsense of direct-to-consumer genetic testing for cardiovascular disease. Neth Heart J, 2011. 19(2): p. 85-88. 5. Swan, M., Multigenic condition risk assessment in direct-to-consumer genomic services. Genetics in Medicine, 2010. 12(5): p. 279-288. 6. Amin, N., C.M. van Duijn, and A.C. Janssens, Genetic scoring analysis: a way forward in genome wide association studies? Eur J Epidemiol, 2009. 24(10): p. 585-7. 7. Gollust, S.E., et al., Motivations and perceptions of early adopters of personalized genomics: perspectives from research participants. Public Health Genomics, 2012. 15(1): p. 22-30. 8. McGuire, A.L., et al., Social networkers' attitudes toward direct-to-consumer personal genome testing. Am J Bioeth, 2009. 9(6-7): p. 3-10. 9. Myers, M.F., Health care providers and direct-to-consumer access and advertising of genetic testing in the United States. Genome Med, 2011. 3(12): p. 81. 10. Sterckx, S., et al., "Trust is not something you can reclaim easily": patenting in the field of directto-consumer genetic testing. Genet Med, 2012. 11. 23andMe. Genetic Testing for Health, Disease & Ancestry; DNA Test. 2013 [cited 2013; Available from: https://www.23andme.com/. 12. Leighton, J.W., K. Valverde, and B.A. Bernhardt, The general public's understanding and perception of direct-to-consumer genetic test results. Public Health Genomics, 2012. 15(1): p. 11-21. 13. Kaufman, D.J., et al., Risky Business: Risk Perception and the Use of Medical Services among Customers of DTC Personal Genetic Testing. J Genet Couns, 2012. 21(3): p. 413-22. 14. Bloss, C.S., N.J. Schork, and E.J. Topol, Effect of direct-to-consumer genomewide profiling to assess disease risk. N Engl J Med, 2011. 364(6): p. 524-34. 15. Myers, M.F., et al., Genetic testing for susceptibility to breast and ovarian cancer: Evaluating the impact of a direct-to-consumer marketing campaign on physicians' knowledge and practices. Genetics in Medicine, 2006. 8(6): p. 361-370. 16. Kutz, G., Direct to consumer genetic tests: misleading test results are further complicated by deceptive marketing and other questionable practices, G.A. Office, Editor. 2010. 17. Ng, P.C., et al., An agenda for personalized medicine. Nature, 2009. 461(7265): p. 724-6. 18. Imai, K., L.J. Kricka, and P. Fortina, Concordance study of 3 direct-to-consumer genetic-testing services. Clin Chem, 2011. 57(3): p. 518-21. 19. Kuehn, B.M., Inconsistent results, inaccurate claims plague direct-to-consumer gene tests. JAMA, 2010. 304(12): p. 1313-5. 20. Carlsson, S., et al., Shared genetic influence of BMI, physical activity and type 2 diabetes: a twin study. Diabetologia, 2013. 21. Poulsen, P., et al., Heritability of type II (non-insulin-dependent) diabetes mellitus and abnormal glucose tolerance--a population-based twin study. Diabetologia, 1999. 42(2): p. 139-45. 22. Do, C.B., et al., Comparison of family history and SNPs for predicting risk of complex disease. PLoS Genet, 2012. 8(10): p. e1002973. 20

23. Center, G.P.P. List of DTC genetic testing companies. 2011 August 11, 2011 [cited 2012 2/21/2012]; Available from: http://www.dnapolicy.org/news.release.php?action=detail&pressrelease_id=145. 24. Kwak, S.H. and K.S. Park, Genetics of type 2 diabetes and potential clinical implications. Arch Pharm Res, 2013. 36(2): p. 167-77. 25. Perry, J.R., et al., Stratifying type 2 diabetes cases by BMI identifies genetic risk variants in LAMA1 and enrichment for risk variants in lean compared to obese cases. PLoS Genet, 2012. 8(5): p. e1002741. 26. Fine, S.L., et al., Age-related macular degeneration. N Engl J Med, 2000. 342(7): p. 483-92. 27. Hammond, C.J., et al., Genetic influence on early age-related maculopathy: a twin study. Ophthalmology, 2002. 109(4): p. 730-6. 28. Seddon, J.M., et al., The US twin study of age-related macular degeneration: relative roles of genetic and environmental influences. Arch Ophthalmol, 2005. 123(3): p. 321-7. 29. Fritsche, L.G., et al., Seven new loci associated with age-related macular degeneration. Nat Genet, 2013. 30. Cipriani, V., et al., Genome-wide association study of age-related macular degeneration identifies associated variants in the TNXB-FKBPL-NOTCH4 region of chromosome 6p21.3. Hum Mol Genet, 2012. 21(18): p. 4138-50. 31. Carvalho, B., et al., Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data. Biostatistics, 2007. 8(2): p. 485-99. 32. Scharpf, R.B., et al., Using the R Package crlmm for Genotyping and Copy Number Estimation. J Stat Softw, 2011. 40(12): p. 1-32. 33. Li, Y., et al., Genotype imputation. Annu Rev Genomics Hum Genet, 2009. 10: p. 387-406. 34. Howie, B., et al., Fast and accurate genotype imputation in genome-wide association studies through pre-phasing. Nat Genet, 2012. 44(8): p. 955-9. 35. Abecasis, G.R., et al., An integrated map of genetic variation from 1,092 human genomes. Nature, 2012. 491(7422): p. 56-65. 36. Abecasis, G.R., et al., A map of human genome variation from population-scale sequencing. Nature, 2010. 467(7319): p. 1061-73. 37. Sanghera, D.K. and P.R. Blackett, Type 2 Diabetes Genetics: Beyond GWAS. J Diabetes Metab, 2012. 3(198). 38. Cary, R.N., C.C. Garber, and D.D. Koch, Concepts and Practices in the Evaluation of Laboratory Methods., in Am Assoc Clin Chem. 1993: New York. 39. Naughton, B.W., S., Guidelines on Vetting Genetic Associations, in White Paper 23-03, 23andMe, Editor. 2011, 23andMe. p. 12. 40. McCarthy, M.I., et al., Genome-wide association studies for complex traits: consensus, uncertainty and challenges. Nat Rev Genet, 2008. 9(5): p. 356-69. 41. Bustamante, C.D., E.G. Burchard, and F.M. De la Vega, Genomics for the world. Nature, 2011. 475(7355): p. 163-5. 42. Rosenberg, N.A., et al., Genome-wide association studies in diverse populations. Nat Rev Genet, 2010. 11(5): p. 356-66. 21

Tables Table 1 Type 2 Diabetes SNP Reported Risks by SNP rs# Chr. Position Ref. Alt. Academic 23andMe DeCODE rs10923931 1 120517959 G T 0.71 0.97 0.97 rs2793831 1 120523902 T C - - 1.03 rs340874 1 214159256 T C - - 1.06 rs780094 2 27741237 T C 0.94 - - rs7578597 2 43732823 T C 1.07 - - rs243021 2 60584819 G A - - 1.00 rs7593730 2 161171454 T C 1.07 - - rs7578326 2 227020653 A G 1.08 - - rs17036101 3 12277845 G A 0.92-0.95 rs1801282 3 12393125 C G - - 0.97 rs4607103 3 64711904 C T 1.13 - - rs2877716 3 123094451 T C - - 1.05 rs4402960 3 185511687 G T - - 1.09 rs10010131 4 6292915 A G - 0.84 - rs10012946 4 6293350 T C 0.88 0.90 - rs4457053 5 76424949 G A 1.23 - - rs4712523 6 20657564 A G 1.11 - - rs7754840 6 20661250 G C 1.06 - - rs7756992 6 20679709 A G 1.06 - - rs9472138 6 43811762 C T 0.87 - - rs10244051 7 15063833 T G - 0.90 0.88 rs864745 7 28180556 T C 0.88 - - rs4607517 7 44235668 A G 1.12 - - rs896854 8 95960511 T C 0.88 - - rs13266634 8 118184783 C T 1.38 1.18 1.22 rs564398 9 22029547 T C 1.08 - - rs2383208 9 22132076 A G - - 0.99 rs13292136 9 81952128 C T - - 0.90 rs12779790 10 12328010 A G - 1.16 - rs1111875 10 94462882 C T 0.88 0.98 0.88 rs5015480 10 94465559 C T 1.11 - - rs7903146 10 114758349 C T 0.93 - - rs231362 11 2691471 A G - - 0.98 rs2237892 11 2839751 C T 0.87-0.97 rs5215 11 17408630 C T 1.10 - - rs5219 11 17409572 T C 1.08 1.11 - rs1552224 11 72433098 A C 0.88-0.92 rs10830963 11 92708710 C G 0.89 0.95 0.97 22

rs1153188 12 55098996 T A 0.88 0.98 0.90 rs1531343 12 66174894 G C 0.91-0.91 rs7961581 12 71663102 C T 0.92 - - rs7957197 12 121460686 T A 0.93 - - rs11634397 15 80432222 A G 1.14 1.05 1.09 rs8042680 15 91521337 C A 0.90 - - rs8050136 16 53816275 C A 0.90 - - rs4430796 17 36098040 G A - - 1.12 Summary of SNPs used by DTC genetic testing and academic sources and the risk estimates used for each SNP. rs# denotes the reference SNP ID. Chr is the chromosome of SNP origin. Position is genomic nucleotide position of the SNP based on Build 37 of the Human Reference Consortium. Ref indicates the reference allele based off of the reference allele from the dbsnp database and 1000 genome reference allele from the forward strand. Alt indicates the alternate allele to the reference strand. Odds ratios are listed for the Academic source taken from the academic paper, all odds ratios are listed based on the reference allele risk estimate. *23andMe reports their risk panel in adjusted odds ratios listed in this table. **decode reports their risks as adjusted relative risk reported in this table. A dash indicates the SNP was not used by that source. 23

Table 2 Age-Related Macular Degeneration Reported Risks by SNP rs# Chr. Position Ref. Alt. Academic 23andMe* DeCODE** rs1061147 1 196654324 A C - 2.76 - rs1061170 1 196659237 C T 2.33-0.78 rs1410996 1 196696933 G A 2.50 0.78 rs1329428 1 196702810 C T 2.50 - - rs13095226 3 99396272 T C 0.84 - - rs17778253 3 99435102 T C 0.81 - - rs7690921 4 110578746 T A 1.19 - - rs2285714 4 110638810 C T 0.99 - - rs10033900 4 110659067 T C 1.07 - - rs9332739 6 31903804 G C 1.47-1.06 rs547154 6 31910938 G T - 0.57 0.58 rs641153 6 31914180 G A 0.51 - - rs541862 6 31916951 T C 1.97 - - rs12153855 6 32074804 T C 0.69 - - rs2071277 6 32171683 T C 0.77 - - rs3132946 6 32190028 A G 0.88 - - rs833069 6 43742579 T C 0.92 - - rs943080 6 43826627 C T 0.83 - - rs4711751 6 43828582 T C 1.20 - - rs722782 8 516479 A C 0.60 - - rs10490924 10 124214448 G T 0.33 - - rs3750847 10 124215421 C T - 0.47 1.59 rs12231166 12 100571018 A C 0.82 - - rs10468017 15 58678512 C T 1.10 - - rs493258 15 58687880 T C 0.89 - - rs3764261 16 56993324 C A 0.93 - - rs2230199 19 6718387 G C 0.80-0.76 rs2075650 19 45395619 A G 1.37 - - rs429358 19 45411941 T C 0.68 - - rs9621532 22 33084511 A C 1.10 - - Summary of SNPs used by DTC genetic testing and academic sources and the risk estimates used for each SNP. rs# denotes the reference SNP ID. Chr is the chromosome of SNP origin. Position is genomic nucleotide position of the SNP based on Build 37 of the Human Reference Consortium. Ref indicates the reference allele based off of the reference allele from the dbsnp database and 1000 genome reference allele from the forward strand. Alt indicates the alternate allele to the reference strand. Odds ratios are listed for the Academic source taken from the academic paper, all odds ratios are listed based on the reference allele risk estimate. *23andMe reports their risk panel in adjusted odds ratios listed in this table. **decode reports their risks as adjusted relative risk reported in this table. A dash indicates the SNP was not used by that source. 24

Table 3 Mean and SDs of Genetic Risk Scores for Type 2 Diabetes Academic decode 23andMe mean = 0.01 SD = 0.21 mean = -0.15 SD = 0.14 mean = -0.07 SD = 0.14 The mean and standard deviation of the risk scores calculated for the combined 834 samples for each source contributing to type 2 diabetes are shown here. These numbers are displayed in units of genetic risk score (S). Genetic risk score is the sum of the de-adjusted odds ratios/relative risk multiplied by the genotype for each SNP a source uses, see Methods, Genetic Risk Prediction. 25

Table 4 Mean and SDs of Genetic Risk Scores for Age-Related Macular Degeneration Academic decode 23andMe mean = -0.58 SD = 0.97 mean = -0.26 SD = 0.37 mean = -0.26 SD = 0.46 The mean and standard deviation of the risk scores calculated for the combined 834 samples for each source contributing to age-related macular degeneration are shown here. These numbers are displayed in units of genetic risk score (S). Genetic risk score is the sum of the de-adjusted odds ratios/relative risk multiplied by the genotype for each SNP a source uses, see Methods, Genetic Risk Prediction. 26

Figures Figure 1 Venn Diagram of SNPs Shared between Panels Type 2 Diabetes Age-Related Macular Degeneration A B Figure 1A represents the number of SNPs unique to each source and the number of SNPs shared between sources for T2D; Figure 1B reflects the same findings for AMD. SNPs were compared directly using their rs# values. Linkage disequilibrium was not taken into consideration in this figure, but could potentially account for even greater overlap in the sharing of SNPs between sources. 27

Figure 2 Distributions of Genetic Risk Score by Source for Type 2 Diabetes A B C The distribution of genetic risk score by source. 28

Figure 3 Correlation Plots between Sources for Type 2 Diabetes A. B. 29

C. 30

Figure 4 Distributions of Genetic Risk Score by Source for Age-Related Macular Degeneration A B C The distribution of genetic risk score by source. 31

Figure 5 Correlation Plots between Sources for Age-Related Macular Degeneration A. B. C. 32