Clinical validation of a genetic model to estimate the risk of developing choroidal neovascular age-related macular degeneration

Many diseases of ageing characterised by complex inheritance patterns are progressive; the individual may be asymptomatic in the early stages. One of these diseases, age-related macular degeneration (AMD), is the most common cause of visual impairment and the leading cause of blindness in the elderly population in the developed world. The prevalence of AMD increases with advancing age in all populations studied. Thus, in developed nations such as the USA, UK, Canada and Australia, with increasingly aged populations, the condition affects a progressively larger segment of the population and has become a major public health issue. Early- or late-stage AMD is present in 15 per cent of individuals over the age of 60 years [1]. It is estimated that there are currently 9.1 million patients in the USA with AMD, of which 1.7 million suffer with the vision-threatening late-stage complications of choroidal neovascularisation (CNV) or geographic atrophy [1]. Moreover, it is predicted that the number of cases of early AMD will increase to 17.8 million by 2050 and, if untreated, cases of late-stage blinding AMD will increase to 3.8 million [1]. It has been determined that vision loss from AMD decreases quality of life by 60 per cent, similar to the experience of dealing with a stroke that requires intensive nursing care [2].

The clinical presentation and natural course of AMD are highly variable. The disease may present as early as the fifth decade of life or as late as the ninth decade. The clinical symptoms of AMD range from no visual disturbances in early disease to profound loss of central vision in the advanced late stages of the disease. Some patients never progress beyond early AMD; however, in 10-15 per cent of Caucasian patients with early-stage disease, the condition progresses to an exudative neovascular (or 'wet' form) or geographic atrophic (or 'dry' form) AMD, which threatens vision. The phenotype is characterised by development of subretinal choroidal neovascular complexes, haemorrhage and fibrosis and is typically associated with severe central vision loss [3, 4].

AMD has been one of the success stories of the genome revolution and is probably one of the best characterised of the complex trait diseases in terms of genetic predisposition (for reviews, see Allikmets and Dean [5] and Swaroop et al. [6]). Besides age, genetic background is the most significant non-modifiable risk factor for all stages of AMD, while smoking is the most significant modifiable risk factor [7, 8]. Initial groundbreaking studies established that loci on chromosomes (Chr) 1 and 10 -- in particular the complement factor H (CFH) and the age-related maculopathy susceptibility protein 2 (ARMS2)/high temperature requirement factor A1 (HTRA1) genes, respectively -- are significantly associated with AMD risk and protection in populations of various ethnicities [9–19]. Although the specific role(s) of the Chr 10 genes in AMD pathobiology has not yet been elucidated, the role of the alternative complement pathway, where CFH functions as a major fluid-phase regulator, is well established (see Anderson et al. [20, 21] Gehrs et al. [22, 23] Hageman et al. [24, 25] and Mullins et al. [26] for overviews). Early pathobiological investigations showed dysregulation of the complement cascade to be a critical early predisposing step in the development of AMD. This spurred the discovery of the association of CFH variants with AMD risk. Subsequent genetic investigations revealed additional associations between AMD and risk/protective variants in various complement pathway-associated genes, including complement component 2 (C2), complement factor B (CFB), complement component 3 (C3), complement factor H-related 1 and 3 (CFHR1 and CFHR3) and complement factor I (CFI) [21, 27–38]. Using a genome-wide association approach, a handful of additional AMD-associated loci have been reported recently; these appear to be modestly associated with AMD risk and will probably require replication in additional cohorts to establish their role in AMD pathogenesis [39, 40] (see also Gehrs et al. [23] for a review).

A prerequisite for a new era in genetic testing and diagnosis for AMD is a robust test that accurately captures the impact of consistently replicated AMD risk variants in predicting the risk of developing CNV. Patients with CNV represent an important segment of the AMD population that would benefit from early diagnosis, given the current availability of an effective therapeutic intervention. Jakobsdottir and coworkers [41] recently concluded that the diagnostic value of three variants in the CFH, ARMS2/HTRA1 and C2 genes was not sufficient to discriminate between individuals with and without AMD because of the relatively low sensitivity and specificity of the combined test panel, in combination with the relatively low prevalence of late-stage disease in the general population. They applied a three single nucleotide polymorphism (SNP) test to their cohort of 640 late-stage AMD cases and 142 controls to demonstrate a clinical sensitivity of 74 per cent and a specificity of 69 per cent, with a reported area under the curve (AUC) -- a measure of how well a test or classifier can distinguish between cases and controls -- of 0.79. Perfect test discrimination would yield an AUC of 1.0. Jakobsdottir and colleagues also reported that the positive predictive value (PPV) of the same test is affected by different values of disease prevalence reflective of age. Seddon and colleagues [42] evaluated six AMD risk-associated variants in CFH, ARMS2/HTRA1, C2, CFB and C3 with the goal of developing a predictive risk test for late-stage AMD. After controlling for smoking, body mass index (BMI) and vitamin intake, they demonstrated a strong association between these six risk variants and the prevalence of late-stage AMD, as well as progression to late-stage disease in early AMD patients. The progression test described by Seddon et al.,[42] which included genetic, environmental and treatment variables, achieved a performance of 83 per cent sensitivity and 68 per cent specificity, with a reported AUC of 0.82. McKay and co-workers [43] extended this test further, proposing a ten-SNP panel plus smoking history to predict the risk of late-stage AMD. Their inclusion of six CFH SNPs was designed to capture the haplotype structure of the locus, to improve classification performance. Zanke and colleagues [44] have presented risk scores by selecting marker-specific odds ratios from disparate sources and multiplying them together. As the latter approach does not benefit from a joint assessment of the markers (as they perform in combination), it may overestimate an individual's risk of disease.

In this study, we assessed the accuracy of a panel of 13 SNPs without consideration of environmental risk factors such as smoking or BMI, to predict the risk of developing CNV in Caucasian individuals 60 years of age and older. Test model development and validation were designed to evaluate these variants in eight AMD-associated genes (CFH, complement factor H-related 4 (CFHR4), complement factor H-related 5 (CFHR5) and coagulation factor XIII B subunit (F13B) located within the regulators of complement activation (RCA) region on Chr 1, C2 and CFB on Chr 6, C3 on Chr 19 and ARMS2 on Chr 10. The panel of 13 SNPs was tested in well-established case-control and sibling pair cohorts from five academic centres (University of Iowa, University of Utah, Columbia University, Harvard University and Melbourne University) to validate the accuracy of the predictive test and to estimate an individual's genetic risk for developing late-stage CNV. Most of the disease-associated genetic variants in CFH, ARMS2, C2, CFB and C3 were selected based on prior replication in multiple studies and performance in resolving the most frequent CFH haplotype combinations. Additional SNPs detecting variants in CFHR4 (rs1409153), CFHR5 (rs10922153 and rs1750311) and F13B (rs698859 and rs2990510) tagged novel extended haplotypes spanning the CFH-to-F13B region and were included to maximise the resolution of clinically relevant subtypes suspected to have high association with disease [45]. The additional SNPs were selected to distinguish the novel haplotypes from the more prevalent haplotypes reported previously (H1, H2, H3, H4) [13]. The performance metrics obtained during the clinical validation of the 13-SNP panel were used as a benchmark to compare with other published AMD-predictive tests directed at estimating an individual's risk of developing late-stage disease. Since the inclusion of several established non-genetic factors (eg smoking) was highly variable across the published tests, the focus of this investigation was to isolate the contribution conferred by genetic variation alone, in order to determine whether the more comprehensive collection of SNPs could further improve prediction accuracy. The methodology used in the clinical validation of the 13-SNP test panel was subsequently applied to two panels of markers [32, 42] that had been assessed previously and contained variants that overlapped with the markers contained within our 13 SNP panel. Both test panels were evaluated in the large collective cohort by using a validation step absent in prior publications. Testing the two panels in a large collection of subjects from different centres assembled from several independent collections was designed to minimise the introduction of selection bias inherent in a single cohort study. Additionally, the use of an independent validation sample was intended to aggressively challenge the 13-SNP panel, to anticipate performance metrics in a broader clinical setting more accurately. Running the three test panels (three SNPs, six SNPs and 13 SNPs) on the same samples allowed for the comparison of performance metrics based exclusively on genetic variants.

The average ages (± standard deviation [SD]) of cases and controls among all cohorts were 76.4 (± 7.3) and 76.5 (± 7.1) years, respectively, and the differences were not significant (p = 0.86). Age matching was applied during cohort ascertainment. The chi-square test was used to assess homogeneity of allele frequency across cohorts. Frequencies of markers rs10490924, rs403846, rs1409153, rs698859, rs403846 and rs10922153 were significantly different (P < 0.001) across cohorts. The frequencies of four markers -- rs10490924, (ARMS2) rs403846, (CFH) rs1409153 (CFHR4) and rs10922153 (CFHR5) -- in the control population and two markers -- rs698859 (F13B) and rs403846 (CFH) -- in the CNV population were unbalanced (Table 3). Removal of the Columbia University cohort eliminated four of the five deviations, leaving only one SNP (rs10490924) outstanding in the Boston control population. The Boston controls and Columbia cases and controls were excluded from the main analyses based on these observations. The remaining study population contained 1,132 CNV cases and 822 controls. For the purposes of the current analysis, investigations into the differences were not pursued but could be evaluated in the future by performing structure analysis to identify potential causes for the observed differences.

The sensitivity and specificity of predictions were calculated in an independent dataset using the test panels in Table 5. The ROC curve is shown in Figure 1. The probability of the risk of CNV was plotted as histograms for controls and cases in the independent dataset in Figure 2. It shows good separation between the two groups, with cases having a substantially higher probability of CNV, although some overlap is present.

Although the incorporation of non-static and self-reported variables is important in elucidating the modifiable risk factors that contribute to disease, their inclusion can degrade test performance in mainstream genetic testing. Ideally, a robust test panel, subject to rigorous validation, which captures the maximal genetic component should improve classification performance and accuracy of reporting. In line with these criteria, which are much stricter than in a discovery cohort, the Boston cohort controls and the Columbia cohort cases and controls were not considered for the calculation of the model. Possible explanations for the allele frequency deviations in these cohorts include admixture, cryptic population stratification, subtle differences in grading criteria, cohort age range, concomitant illnesses or medications, and should be explored further.

In order to compare performance across tests, a ROC curve was generated for each prediction panel to evaluate the AUC. By evaluating each test across the large collective cohort using the same validation procedure, we compared the power of the genetic variants to evaluate classification performance. The performance of the three-SNP panel described by Jakobsdottir and colleagues [41] revealed an AUC value of 0.77, compared with a value of 0.79 observed in the original study of 642 late-stage AMD cases and 142 controls. The differences in AUC values obtained between the original and the current study are likely to reflect the impact of testing across a large collection of independently collected cohorts compared with a single study that is potentially more sensitive to subject selection bias. The performance of the six-SNP test panel reported by Seddon and colleagues [42] as part of a joint gene-environment model exhibited a drop in AUC from 0.81 to 0.79 from training to validation in our data (significant at P < 0.05), similar to most of the tests evaluated. This decrease in AUC reveals the value of the inclusion of an independent validation set to challenge test performance and estimate metrics achievable in the broader clinical setting more accurately. We have emphasised the importance of both study design features to report performance more accurately and to anticipate utility in the more diverse clinical testing market more closely. Finally, modest gains in our 13-SNP panel were demonstrated with the highest AUC value obtained among all models evaluated (0.80). The additional variants that contributed to the performance of the predictive test located in CFHR5 and F13B highlight the complexity of the genetic structure of the RCA region and influence AMD disease biology.

In summary, the 13-SNP panel had a clinical sensitivity of 82 per cent and a specificity of 63 per cent, achieving clinical performance metrics comparable with models with fewer SNPs that include self-reported and/or non-static risk factors. The PPV of the panel was evaluated at different levels of prevalence, reflecting ranges covering estimates of late-stage disease in individuals > 40, > 65 and > 80 years of age in the general population. More favourable estimates of PPV were observed as the prevalence of disease increases with age. The values obtained revealed 11 per cent PPV at 5.5 per cent prevalence, 20 per cent PPV at 10 per cent prevalence and 28 per cent PPV at 15 per cent prevalence in the general population [41]. The prevalence figures reflect conservative estimates of late-stage disease in the general population and would be further enhanced and more clinically applicable in a setting of diseased patients, as in the study conducted by Seddon and colleagues [42]. The longitudinal study design of the Age-Related Eye Disease Study (AREDS) cohort used in Seddon's study was ideal for evaluating incident AMD by distinguishing between 'progressors' and 'non-progressors' but, more importantly, it established that the same set of variants were effective at distinguishing non-disease controls from patients with late-stage disease. Not surprisingly, the same core panel of SNPs covering the major genes associated with disease used in Seddon and co-workers' test panel was also utilised in the study conducted by Jakobsdottir and colleagues,[41] as well as in our current study.

The present confirmatory findings reflect the utility of these variants to predict the development of CNV in non-diseased subjects in our study, as well as the progression to late-stage disease in patients diagnosed with early forms of AMD [42]. PPVs improve significantly when applied to the population of patients diagnosed with early stages of disease. The utility of AMD genetic testing will advance if the result of a predictive test translates into actionable information for the physician. This study highlights the need to continue to explore the biology of CNV, to improve our understanding of the genetics associated with disease and extend these findings in future studies to evaluate clinical performance metrics in the more acute clinical population diagnosed with early-stage disease. A genetic test identifying individuals at high risk of developing CNV holds the promise for earlier detection through risk-based surveillance protocols and improved outcomes arising from more timely intervention.