Iterative Methodology and Usability Improvement for Clinical Decision Support

Transcription

1 Iterative Methodology and Usability Improvement for Clinical Decision Support Fei Yu January 14th, 2014

2 Outline MindsEye Project Phase I: Usability Testing Method & Results Phase II: Improved Testing Method & Results Phase I & II Comparison

3 Challenges Four Grand Challenges to CDS Human-computer interface Summarization of patient level information in the interface Combining recommendations with comorbidities of patients through leveraging of patient history Data organization and presentation by using data visualization techniques Functions to interactively sort, sift and re-organize data UI functions that support prioritization and filtering of CDS generated recommendations Three Critical Aspects of MindsEye Project Seamless integration of UI with the practice workflow

4 Research Data MindsEye UI Database MindLinc EHR System Collected data from 30 organizations; The sample de-identified MDD dataset from MindLinc Data type: diagnoses, medications, 33,536 patient encounters from 3,016 unique patients. adverse reactions, as well as outcomes Data type: prescribed medications, comorbid conditions, demographics (race, gender, and age), visit type (inpatient, outpatient, or emergency), and treatment outcomes.

5 UI Design Identified Clinical Workflow Clinical workflow UI features

6 MindsEye Initial User Interface

7 Comparison MindsEye User Interface Evaluation Hypothesis UI helps the clinician gain an overview of patients conditions in an effective and efficient way. UI design to match point-of-care workflow to support improved decision making process Methodology: 4 Step Approach Pre-test Questionnaire Demo Video User Interaction Session: time, steps taken, successful completion, errors Post-test Questionnaire: User feedback is captured on a scale from 1(strongly disagree) to 5 (strongly agree)

8 Who is my patient? Phase I Evaluation: Usability Testing Method How is s/he doing? Comparsion What to recommend?

9 Comparison Phase I Evaluation Results Medicine is Risk-Averse - Favors knowing more Challenge to UI design - One size doesn t fit all task

10 Phase II: MindsEye Improved UI

11

12 Comparison Phase II Evaluation: Usability Testing Method Workflow Survey Tasks Usability testing design maps to the re-identified clinical workflow Redesigned pre & post-test questionnaires to reflect UI changes Simplified questions Mounted both questionnaires online: Redesigned user tasks and exactly mapped the clinical workflow Simplified task procedures and created baseline for evaluation

13 Comparison

14 Who is my patient? Phase I Evaluation: Usability Testing Method How is s/he doing? Comparsion What to recommend?

15 Comparison Phase II Evaluation Results

16 Comparison Data Security vs. Testing Location Phase I UI and de-identified patient data Phase II and meeting higher security requirements approved for testing only at ONE location Modified IRB by submitting UI changes UI and de-identified patient data loaded to one designated and encrypted laptop Testing can occur at any physician s location

17 Comparison Participants Recruitment Phase I Phase II 6 participants 19 participants Physicians who researchers knew Psychiatrists or physicians who had experience treating MDD patients very well Convenience sampling Recruitment flyers and documents for distribution

18 Comparison Phase I and II Comparison: Tasks

19 Comparison Phase I and II Comparison: Performance Phase I Phase II

20 Comparison Phase I and II Comparison: Efficiency & Like-ability Phase I Phase II

21 Comparison Lesson Learned Timeline View is the most favored feature revealed in UI evaluation; however, this data view still needs to be further optimized. Revised UI and evaluation methodology improved user performance and their testing results. Ethical issues resulting from Rx recommendation and predictive view. Usability testing with physicians needs to be flexible and customized to their work style and environment.

22 Thank you! Questions?