Skip to main content

Determining the utility of diagnostic genomics: a conceptual framework

Abstract

Background

Diagnostic efficacy is now well established for diagnostic genomic testing in rare disease. Assessment of overall utility is emerging as a key next step, however ambiguity in the conceptualisation and measurement of utility has impeded its assessment in a comprehensive manner. We propose a conceptual framework to approach determining the broader utility of diagnostic genomics encompassing patients, families, clinicians, health services and health systems to assist future evidence generation and funding decisions.

Body

Building upon previous work, our framework posits that utility of diagnostic genomics consists of three dimensions: the domain or type and extent of utility (what), the relationship and perspective of utility (who), and the time horizon of utility (when). Across the description, assessment, and summation of these three proposed dimensions of utility, one could potentially triangulate a singular point of utility axes of type, relationship, and time. Collectively, the multiple different points of individual utility might be inferred to relate to a concept of aggregate utility.

Conclusion

This ontological framework requires retrospective and prospective application to enable refinement and validation. Moving forward our framework, and others which have preceded it, promote a better characterisation and description of genomic utility to inform decision-making and optimise the benefits of genomic diagnostic testing.

Background

Diagnostic efficacy is now well established for the application of diagnostic genomic testing in rare disease [1]. The impacts of this revolution in diagnostic care have been substantial both for individual patients and families, but also across health systems [2,3,4]. Achieving effective implementation within health systems can be complex and requires understanding health service utilisation patterns [5] in addition to contemporary assessments of broader utility. Given that the overwhelming number of studies reported to date of diagnostic genomics are short-term observational cohort studies rather than long-term randomised trials, the ability to assess utility in a comprehensive manner is impeded by the lack of control comparators as well as a focus on immediate outcomes such as diagnostic yield. We propose a conceptual framework to approach determining utility of diagnostic genomics to assist with further evidence generation and the development of more comprehensive models of utility with relevance across patients, families, clinicians, health services and health systems.

Main text

Previous work and frameworks

The need to define, systematically measure and value the utility of genomic testing has been recognised since the start of the transition of technology from research to clinical setting [6,7,8,9]. Valuations of genomic utility [10,11,12,13] and inclusion in economic assessments [14,15,16] have supported the understanding and delivery of patient-centred value-based implementation [17] and conceptualisation of the perceived benefits in clinical practice [18]. Attempts to standardise value measurement utilising existing literature with stakeholder refinement [19] has resulted in the development [20], and validation [21, 22], of a Clinician-reported Genetic testing Utility InDEx (C-GUIDE). The C-GUIDE seeks to capture the clinician perspective towards genetic testing utility relating to (1) understanding diagnosis and prognosis, (2) informing medical management, (3) awareness and actionability of reproductive and health risks for patients and family members, and (4) overall patient and family psychosocial well-being. Whilst clearly valuable, and a critical step towards robust and comparable measurement of genomic test utility, there is a need to embrace the multitude of contexts amongst both adult and paediatric testing scenarios [23], include patient reported outcomes [24, 25], and the diversity of clinical specialty fields in which genomic implementation is occurring [26]. Further, system-level decision-making, including but not limited to health technology assessments, necessitates other priority dimensions, such as cost-effectiveness, equity, budget impact [27], job creation and loss, and the value of genomic data for research and discovery. Importantly, we acknowledge that these do not all fall within a singular methodological approach to assessment [27]. Thereby, we build upon previous work to propose a broader conceptual framework to the potential measurement, assessment and valuation of genomic utility across multiple dimensions of relevance to patients, families, clinicians, health systems and economies.

Genomic utility ontology

Our conceptual framework consists of three core utility dimensions, the domain or type and extent of utility (what), the relationship and perspective of utility (who), and time horizon of utility (when). Within each dimension a possible classification system is described which could be used to determine cumulative genomic utility.

Domain of impact (what)

The first dimension is the domain of utility which comprises of the type of utility and extent of impact (Table 1). The type comes in many potential forms that would include diagnostic, reproductive, therapeutic, prognostic, investigative, psychosocial and discovery/research (new knowledge generation) domains. Further exploring these domains, psychosocial impact is likely to be related to clinical dimensions but brings an additional “value of knowing”, highlighted as being valuable to individuals and families, even in the absence of associated changes in clinical management [10,11,12]. Previous studies [28] have described feelings of suffering having been legitimised, a sense of closure, and feelings of altruism, experienced against the backdrop of substantial prior uncertainty [29] even in the absence of associated changes in clinical management or a “remarkable” finding [30]. Within the types of utility, there might be a variety of relative measures across a continuum from a minimal identifiable level of impact to maximal impact. Given the need to apply a standardised framework that enables reproducible measurement, qualitative levels or statements may assist in the definition of measures. For example, the extent of impact on the ‘diagnostic’ domain could be defined from (+) clarified an existing clinical diagnosis, to (++) changed an existing clinical diagnosis, or (+++) provided a new genomic diagnosis that could not have been reached by other investigations. We expect through application and revision of the framework that future iterations will include refinement of domains and measures.

Table 1 Domain of impact (what)

Relationship and perspective of utility (who)

The second dimension in which all the domains of utility can be considered, is the frame of personal, family and community (Table 2) context. For instance, a specific utility might relate to one’s self, an immediate family member, a more distant family member, the community of that individual and/or family, or indeed society as a whole. Where a diagnostic genomic result might have utility for one’s self diagnostically, this same result may be of demonstrable benefit to an immediate family member through cascade testing, or for a more distant family member in terms of whether they might be a living related kidney donor, and to the community more broadly as it might result in new knowledge that results in a novel treatment or clinical trial. Within a broader conceptualisation of “community” would also be additional third-party stakeholders such as clinicians, health payers and researchers whose perspectives would be critical for policy purposes.

Table 2 Relationship and perspective of utility (who) and time horizon of utility (when)

Time horizon of utility (when)

The experience, observation or perception of utility can also be described in terms of its timing (Table 2). The timing of benefit impacts the level of valuation, forming an interplay with people's inherent discounting of future gains. For instance, in Australia future costs and benefits are discounted by an annual rate of 5% in reflecting that a benefit in the future has less value than the same benefit today [31]. Potential examples of time horizons of utility might represent immediate avoidance of alternate invasive non-genomic investigation, short term access to directed or targeted treatment, and medium-long term utility from reproductive planning.

Additionally, evidentiary uncertainty across timespans may impact priorities and values. Where some forms of utility might be experienced in the immediate or short-term, others might be more distant or very long term. Whilst this might take the form of describing when such utility occurs or how long it occurs for, in the first instance we propose to seek inclusion of the former with potential to include the latter as our conceptual framework evolves and is tested. Though there may be some differences in relative surety or precision of utility across diverse incident timepoints and for different items being examined, this should not preclude inclusion given so long as the item’s existence is considered to be actual or potential beyond reasonable doubt. As such, we propose a third dimension of utility measurement across differing time horizons to ensure that this is captured.

Cumulative and preference-based valuation of utility

Across these three proposed dimensions of utility of what, who and when, one could potentially triangulate a singular point of utility axes of type, relationship, and time (Fig. 1). Collectively, multiple different points of individual utility might be inferred to relate to a concept of aggregate utility. Whether and the extent to which such a cloud effect is quantitatively or qualitatively approached should be the source of subsequent observation, experimentation, and analysis. Nevertheless, an understanding of the shape, size, and characterisation of a collective utility would be useful for understanding current and future interventions. For policymakers and those influencing complex systems, such as health systems, this would enable a more level playing field upon which to compare different potential interventions that might generate utility, and facilitate equitable implementation followed by informed audit & quality assurance evaluation. At present, there is a very real risk of biasing comparisons or value interpretations within a system that has limited resource boundaries, with resultant threats to generalisable community benefit as new interventions are evaluated for implementation upon an unknowingly asymmetric landscape. An attempt to describe aggregate genomic utility for such interventions that are being considered for implementation may make this endeavour more transparent and equitable whilst having positive effects on public policy and economy.

Fig. 1
figure 1

Proposed framework for description, assessment and summation of genomic utility

Future approaches to refine, value, and implement

Initial steps to begin understanding such a multidimensional and aggregate description of genomic utility might incorporate retrospective alignment to existing interventions under evaluation, such as diagnostic genomic testing in different rare diseases. Apart from attempting to validate such an approach, this would also seek to undertake two major activities that require real-world framing. Firstly, does the population contain as full as possible representation of utility types and diversity to accompany the range of potential individuals that utility aligns to and the time horizons it might be experienced in. Secondly, a discussion should ensue as to potential quantification of utility. One such framework might see different aspects of a point utility accruing different criteria which can be added together combinatorically to align to tiers of utility, while reflecting the priorities of the key stakeholders involved (Fig. 1). This framework would be somewhat similar to the current approach to genetic variant classification using the accumulation of various different American College of Medical Genetics and Genomics variant criteria or extend to multi-attribute utility theory [32]. In that way, different of interventions can be meaningfully compared to inform health care prioritisation.

Whichever approach might emerge from initial retrospective application and refinement, we propose that the genomic utility ontology might then begin to be incorporated into prospective studies of interventions, including cohort studies and randomised clinical trials. Pilot work might look at policy implementation within existing frameworks for evaluation and implementation of public funding or licensing of health interventions such as the broader implementation of clinical genomics in health systems, especially those underpinned by universal health care.

Major challenges and limitations

There are naturally many challenges to approaching a broadly applicable ontology to measuring utility of genomic testing. One is that any approach informed by existing literature is naturally limited by that literature in terms of not appreciating aspects of utility that are not measured, described or presented. Utility might also have different perspectives according to different values, sociocultural norms and health system structures. Our proposed multidimensional conceptualisation of utility description might also prove either too granular or too opaque for different groups of patients, clinicians or policymakers. Further, identifying an appropriately broad and descriptively reported series of past research cohorts upon which to build and validate this ontology is likely to generate substantial activity requirements. Nevertheless, and in spite of these challenges, the likely opportunity cost of not undertaking efforts to move iteratively towards a comprehensive genomic utility framework would be far greater than would need to be absorbed by undertaking such activities, ensuring the equitable and sustainable translation of genomics.

Conclusions

In summary, we propose a conceptual framework for the potential measurement, assessment, and valuation of genomic utility that builds from previous work whilst evolving a multidimensional construct that contextualises utility both now and into the future. This ontological framework requires application both in retrospective cohorts as well as in prospective studies to enable its refinement and potential validation. The relative values and preferences of different stakeholders across a continuum of genomic utility scenarios also requires further study. Moving forward this framework, and others which have preceded it, will continue to promote a better characterisation and description of genomic utility such that personal, local and system decision-making can be best informed to derive optimal benefit from genomic diagnostic testing.

Availability of data and materials

Not applicable.

Abbreviations

C-Guide:

Clinician-reported Genetic testing Utility InDEx

References

  1. Clark MM, Stark Z, Farnaes L, Tan TY, White SM, Dimmock D, Kingsmore SF. Meta-analysis of the diagnostic and clinical utility of genome and exome sequencing and chromosomal microarray in children with suspected genetic diseases. NPJ Genom Med. 2018;3:16.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Stark Z, Boughtwood T, Haas M, Braithwaite J, Gaff CL, Goranitis I, Spurdle AB, Hansen DP, Hofmann O, Laing N, et al. Australian genomics: outcomes of a 5-year national program to accelerate the integration of genomics in healthcare. Am J Hum Genet. 2023;110(3):419–26.

    Article  CAS  PubMed  Google Scholar 

  3. Stranneheim H, Lagerstedt-Robinson K, Magnusson M, Kvarnung M, Nilsson D, Lesko N, Engvall M, Anderlid BM, Arnell H, Johansson CB, et al. Integration of whole genome sequencing into a healthcare setting: high diagnostic rates across multiple clinical entities in 3219 rare disease patients. Genome Med. 2021;13(1):40.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Investigators GPP, Smedley D, Smith KR, Martin A, Thomas EA, McDonagh EM, Cipriani V, Ellingford JM, Arno G, Tucci A, et al. 100,000 Genomes pilot on rare-disease diagnosis in health care—preliminary report. N Engl J Med. 2021;385(20):1868–80.

    Article  Google Scholar 

  5. Mordaunt DA, Dalziel K, Goranitis I, Stark Z. Uptake of funded genomic testing for syndromic and non-syndromic intellectual disability in Australia. Eur J Hum Genet. 2023. https://doi.org/10.1038/s41431-023-01417-6.

  6. Lerner B, Marshall N, Oishi S, Lanto A, Lee M, Hamilton AB, Yano EM, Scheuner MT. The value of genetic testing: beyond clinical utility. Genet Med. 2017;19(7):763–71.

    Article  PubMed  Google Scholar 

  7. Grosse SD, Khoury MJ. What is the clinical utility of genetic testing? Genet Med. 2006;8(7):448–50.

    Article  PubMed  Google Scholar 

  8. Grosse SD, Wordsworth S, Payne K. Economic methods for valuing the outcomes of genetic testing: beyond cost-effectiveness analysis. Genet Med. 2008;10(9):648–54.

    Article  PubMed  Google Scholar 

  9. Grosse SD, McBride CM, Evans JP, Khoury MJ. Personal utility and genomic information: look before you leap. Genet Med. 2009;11(8):575–6.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Goranitis I, Best S, Christodoulou J, Stark Z, Boughtwood T. The personal utility and uptake of genomic sequencing in pediatric and adult conditions: eliciting societal preferences with three discrete choice experiments. Genet Med. 2020;22(8):1311–9.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Goranitis I, Best S, Stark Z, Boughtwood T, Christodoulou J. The value of genomic sequencing in complex pediatric neurological disorders: a discrete choice experiment. Genet Med. 2021;23(1):155–62.

    Article  PubMed  Google Scholar 

  12. Goranitis I, Best S, Christodoulou J, Boughtwood T, Stark Z. Preferences and values for rapid genomic testing in critically ill infants and children: a discrete choice experiment. Eur J Hum Genet. 2021;29(11):1645–53.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Jayasinghe K, Stark Z, Kerr PG, Gaff C, Martyn M, Whitlam J, Creighton B, Donaldson E, Hunter M, Jarmolowicz A, et al. Clinical impact of genomic testing in patients with suspected monogenic kidney disease. Genet Med. 2021;23(1):183–91.

    Article  PubMed  Google Scholar 

  14. Goranitis I, Wu Y, Lunke S, White SM, Tan TY, Yeung A, Hunter MF, Martyn M, Gaff C, Stark Z. Is faster better? An economic evaluation of rapid and ultra-rapid genomic testing in critically ill infants and children. Genet Med. 2022;24(5):1037–44.

    Article  CAS  PubMed  Google Scholar 

  15. Wu Y, Balasubramaniam S, Rius R, Thorburn DR, Christodoulou J, Goranitis I. Genomic sequencing for the diagnosis of childhood mitochondrial disorders: a health economic evaluation. Eur J Hum Genet. 2022;30(5):577–86.

    Article  PubMed  Google Scholar 

  16. Wu Y, Jayasinghe K, Stark Z, Quinlan C, Patel C, McCarthy H, Mallawaarachchi AC, Kerr PG, Alexander S, Mallett AJ et al. Genomic testing for suspected monogenic kidney disease in children and adults: a health economic evaluation. Genet Med. 2023:100942. https://doi.org/10.1016/j.gim.2023.100942.

  17. Regier DA, Weymann D, Buchanan J, Marshall DA, Wordsworth S. Valuation of health and nonhealth outcomes from next-generation sequencing: approaches, challenges, and solutions. Value Health. 2018;21(9):1043–7.

    Article  PubMed  Google Scholar 

  18. Buchanan J, Blair E, Thomson KL, Ormondroyd E, Watkins H, Taylor JC, Wordsworth S. Do health professionals value genomic testing? A discrete choice experiment in inherited cardiovascular disease. Eur J Hum Genet. 2019;27(11):1639–48.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Hayeems RZ, Luca S, Pullenayegum E, Meyn MS, Ungar WJ. Genome diagnostics: novel strategies for measuring value. J Manag Care Spec Pharm. 2019;25(10):1096–101.

    PubMed  Google Scholar 

  20. Hayeems RZ, Luca S, Ungar WJ, Bhatt A, Chad L, Pullenayegum E, Meyn MS. The development of the Clinician-reported Genetic testing Utility InDEx (C-GUIDE): a novel strategy for measuring the clinical utility of genetic testing. Genet Med. 2020;22(1):95–101.

    Article  CAS  PubMed  Google Scholar 

  21. Hayeems RZ, Luca S, Ungar WJ, Venkataramanan V, Tsiplova K, Bashir NS, Costain G, Inglese C, McNiven V, Quercia N, et al. The Clinician-reported Genetic testing Utility InDEx (C-GUIDE): preliminary evidence of validity and reliability. Genet Med. 2022;24(2):430–8.

    Article  CAS  PubMed  Google Scholar 

  22. Hayeems RZ, Luca S, Hurst ACE, Cochran M, Owens C, Hossain A, Chad L, Meyn MS, Pullenayegum E, Ungar WJ, et al. Applying the Clinician-reported Genetic testing Utility InDEx (C-GUIDE) to genome sequencing: further evidence of validity. Eur J Hum Genet. 2022;30(12):1423–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Best S, Stark Z, Phillips P, Wu Y, Long JC, Taylor N, Braithwaite J, Christodoulou J, Goranitis I. Clinical genomic testing: what matters to key stakeholders? Eur J Hum Genet. 2020;28(7):866–73.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Smith HS. Genomic medicine’s critical outcome measure-utility. JAMA Netw Open. 2022;5(8):e2225988.

    Article  PubMed  Google Scholar 

  25. Smith HS, Morain SR, Robinson JO, Canfield I, Malek J, Rubanovich CK, Bloss CS, Ackerman SL, Biesecker B, Brothers KB, et al. Perceived utility of genomic sequencing: qualitative analysis and synthesis of a conceptual model to inform patient-centered instrument development. Patient. 2022;15(3):317–28.

    Article  PubMed  Google Scholar 

  26. Meng Y, Clarke PM, Goranitis I. The value of genomic testing: a contingent valuation across six child- and adult-onset genetic conditions. Pharmacoeconomics. 2022;40(2):215–23.

    Article  PubMed  Google Scholar 

  27. Grosse SD, Rasmussen SA. Exome sequencing: value is in the eye of the beholder. Genet Med. 2020;22(2):280–2.

    Article  PubMed  Google Scholar 

  28. Halverson CM, Clift KE, McCormick JB. Was it worth it? Patients’ perspectives on the perceived value of genomic-based individualized medicine. J Community Genet. 2016;7(2):145–52.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Hylind R, Smith M, Rasmussen-Torvik L, Aufox S. Great expectations: patient perspectives and anticipated utility of non-diagnostic genomic-sequencing results. J Community Genet. 2018;9(1):19–26.

    Article  PubMed  Google Scholar 

  30. Stevens Smith H, Russell HV, Lee BH, Morain SR, and the Value of Exome Sequencing Delphi P. Using the Delphi method to identify clinicians’ perceived importance of pediatric exome sequencing results. Genet Med. 2020;22(1):69–76.

    Article  PubMed  Google Scholar 

  31. Pharmaceutical Benefits Advisory Committee (PBAC). Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee. 5.0. Canberra: Australian Government Department of Health; 2016.

    Google Scholar 

  32. Richards S, Aziz N, Bale S, Bick D, Das S, Gastier-Foster J, Grody WW, Hegde M, Lyon E, Spector E, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med. 2015;17(5):405–24.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors acknowledge and thank Australian Genomics for their support.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

AJM, ZS, ZF, SB and IG conceptualised, drafted, edited and approved submission of this work.

Corresponding author

Correspondence to Andrew Mallett.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

AJM is a recipient of a Queensland Health Advancing Clinical Research Fellowship.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mallett, A., Stark, Z., Fehlberg, Z. et al. Determining the utility of diagnostic genomics: a conceptual framework. Hum Genomics 17, 75 (2023). https://doi.org/10.1186/s40246-023-00524-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40246-023-00524-1

Keywords

  • Diagnostic genomics
  • Utility
  • Framework
  • Implementation
  • Ontology