A Methodology for Quantitative Measurement of Quality and Comprehensiveness of a Research Data Repository (IDR Snapshot) Vojtech Huser MD PhD

Transcription

1 A Methodology for Quantitative Measurement of Quality and Comprehensiveness of a Research Data Repository (IDR Snapshot) Vojtech Huser MD PhD

2 Acknowledgement Wisconsin Institute for discovery Private branch: Morgridge Institute CTSA grant (NIH (NCRR) 1UL1RR025011) U of Wisconsin Madison + Marshfield Clinic Marshfield Clinic Research foundation marshfieldclinic.org/birc 2

3 Agenda Introduction Methods Design requirements Measure components Results Marshfield Clinic Discussion/Conclusion marshfieldclinic.org/birc 3

4 Background Clinical Informatics Subfield of Biomedical Informatics Informatics = Technology + Information + People Biomedical and Health Informatics Bioinformatics (cellular and molecular) Medical (Clinical) Informatics (person) Public Health Informatics (population) Research Informatics Translational Informatics Hersh BMC Medical Informatics and Decision Making (2009) 9:24 doi: / marshfieldclinic.org/birc 4

5 Background Clinical Informatics Subfield of Biomedical Informatics Informatics = Technology + Information + People Biomedical and Health Informatics Bioinformatics (cellular and molecular) Medical (Clinical) Informatics (person) Public Health Informatics (population) Research Informatics Translational Informatics Hersh BMC Medical Informatics and Decision Making (2009) 9:24 doi: / marshfieldclinic.org/birc 5

6 Introduction Advances Federated research data warehouse Future data growth Conduct of research RCT vs. cohort and case control studies IDR = Integrated Data Repository Choice of multiple institutions Quick evaluation of an IDR TRIAD Frueh FW. Back to the future: why randomized controlled trials cannot be the answer to pharmacogenomics and personalized medicine. Pharmacogenomics 2009;10(7): marshfieldclinic.org/birc 6

7 SHRINE: Query federation example Weber at al. (2009) Application of Information Technology: The Shared Health Research Information Network (SHRINE): A Prototype Federated Query Tool for Clinical Data Repositories JAMIA 2009;16: Fusaro et al. Electronic Medical Record Analysis Using Cloud Computing AMIA CRI summit, 2010 marshfieldclinic.org/birc 7

8 Motivation Motivation 10 institutions with IDRs; budget allows only 3; How do you choose? Tracking improvement within an organization Assumptions (limitations) Goal: lifetime and complete EHR (+genetics, +behavior, +environment) Improvement limits General measures (rather then project specific) Simple evaluation, pragmatic approach Dichotomous approach to data sources (not profiling them) Early stage (community interest solicitation) linkedin.com/in/vojtechhuser 8

9 Measure Measures Qualitative description Quantitative: Simple counts, Composite measure Composite scores: APGAR, FICO, Google PageRank Score: A short hand way to communicate about a complex concept Data warehouse quality Size (1M vs. 10M), quality (billing data vs. structured EHR data) Possible requirements Design issues Completeness, Concision (simple), Measurability (objective, reliable), Independence Our requirements Intuitive to interpret Facilitates improvement Fairness linkedin.com/in/vojtechhuser 9

10 Data Structures (VDW) linkedin.com/in/vojtechhuser 10

11 Data Structures (i2b2) linkedin.com/in/vojtechhuser 11

12 Data Structures (HealthFlow) healthcareworkflow.wordpress.com linkedin.com/in/vojtechhuser 12

13 Data structures (data schemas) VDW Several tables Data domain specific structures Optimized for queries, users i2b2, HealthFlow One event table (+ attributes) Generic data structures Flexibility, extensibility 13

14 Event table (IDR Snapshot v1) (Event-DOB) code.google.com/p/idrsnapshot 14

15 Results Events table (ev_measure01) Generation took 2.5 hours Size: 43 GB (includes additional info) code.google.com/p/idrsnapshot 15

16 Code code.google.com/p/idrsnapshot 16

17 Code code.google.com/p/idrsnapshot 17

18 Code code.google.com/p/idrsnapshot 18

19 Cross institution comparison code.google.com/p/idrsnapshot 19

20 Monitoring code.google.com/p/idrsnapshot 20

21 What s in it for me? Use it Download our SQL code Use it to evaluate your IDR Collaborate with us Paper in writing (3 institutions involved so far) IDR Snapshot version 2 IRB note: aggregate numbers code.google.com/p/idrsnapshot 21

22 Future work Learn from IDRs of other institutions IDR Snapshot version 2 Event table structure HL7 vmr (lite, SQL format) Delphi method to extend measures Qualitative measures (transfers, insurance, PHR, family history) Institutional context Quantitative measures (general level, more complex average patient pattern) Structured data Beyond claims data (e.g. nursing homes, decision support audit trails) Specialty snapshot Prioritizing CDS interventions CKD example (dialysis data) code.google.com/p/idrsnapshot 22

23 Thank you Slides: Questions? 23