Supercharge Your Improvement Efforts with Predictive Analytics

Transcription

1 Session #29 Supercharge Your Improvement Efforts with Predictive Analytics Chris DeRienzo, MD, MPP, FAAP Chief Quality Officer & Neonatologist, Mission Health Andrew O. Johnson, PhD Manager, Data Science Clinical & Business Analytics, Mission Health

2 Learning Objectives Explain the value proposition for building a robust internal data science team into an integrated continuous improvement analytics strategy. Identify the required elements of a Data Science Starter Kit that can upgrade your organization s analytics capabilities from reporting to predicting. Illustrate the role data scientists can play within a health system through the development of an all-cause, 30-day inpatient readmission model.

3 We are dead in the water without a culture of continuous improvement, grounded in analytics, that permeates everything we do, and all that we are.

4 The U.S. health system ranks last among 11 industrialized countries on measures of access, equity, quality, efficiency, and healthy lives. Wasteful spending in the health system has been calculated at up to $1.2 trillion. 70% of hospital strategic initiatives fail. Only 32% of healthcare IT projects meet their objectives, while 24% totally fail, and 44% have difficulties in meeting their goals.

5 About Mission Health Western North Carolina 18-County Area Population (2016): 882,581 Percent over 65: 22% Mission Health is westernnorth Carolina s only not-for-profit, independent community healthcare system. Mission s BIG(GER) Aim is to get every person to their desired outcome, first without harm, also without waste, and always with an exceptional experience for each person, family, and team member. Employing over 13,000 dedicated professionals, the system is comprised of seven hospitals including tertiary, critical access, and inpatient rehabilitation, 750 employed/aligned providers, and one of the largest Medicare Shared Savings ACOs in the nation.

6 The Need at Mission Health Major process improvement was facilitated through the centralized performance improvement team a highly skilled team of process engineering professionals. Created internal dependency on the centralized team for complex projects. Access to data only through IT and Informatics. Difficult to provide real-time operational data or management roll-up data. The Need: 1. Provide health systemwide data in a way that allows broad access for analysis as well as operational uses ranging from daily unit management to board-level discussion. 2. Create a culture of continuous improvement by outfitting key clinical and operational roles with skills, tools, and a sense of ownership for improving their own processes. 3. Create the capability to leverage both operational and clinical predictive analytics.

7 Poll Question #1 Where does your organization spend most of its time right now? 1) A 2) B 3) C 4) D 5) E 6) F 7) G What is the best course of action? How can I influence the future? What are likely future outcomes? Why did this occur? D E G F Forecasting Statistical Analysis Optimization Predictive What-if Analysis Analytics 8) Unsure or not applicable B C Alerts & Triggers Query Drill-down What action is needed? What exactly is the problem? A Ad-hoc Reports How many? How often? Where? Standard Reports What happened?

8 The Turning Point Process transformation after low hanging fruit. Early wins always lead to more difficult and complex follow-up projects. Requires higher level of leadership, particularly from physicians. Clinical program development. Simply not possible without strong physician leadership. Requires sensitive project facilitation. Better together. Clinical and operational leaders and data scientists can accomplish more together than they can separately.

9 Results Successfully kick-started the journey up the curve in clinical analytics. Significant reductions in sepsis and stroke mortality, length of stay for bowel surgery and renal patients, and population screening for breast and colorectal cancer through workflows built into 50+ care process models. Moved further up the curve toward predictive operational analytics. Large scale unit and flow simulations for building projects, regional transport optimization, and census prediction. Launched integrated team to drive to meaningful clinical predictive analytics (e.g., readmissions predictor).

10 How We Did It Created a vision and support for data-driven continuous improvement grounded in analytics across Mission Health. Began with the BIG(GER) Aim and asked what do we need to best deliver on this promise in a population health world? Built our analytics team with in-house data science expertise. Cultivated a strong organizational motivation, capability, and process to move to predictive analytics operationally and clinically (initially focused on readmissions).

11 Tempering Your Organizational Enthusiasm

12 Moving From Reporting to Prediction Things you must have to be successful: Senior leaders with vision, budgetary impact, and desire for change. A trusted data infrastructure. Well-defined problems with predictive solutions. Skilled and empowered data scientists. The rest of the analytics team. Processes to develop and sustain data science projects.

13 Value Proposition for an In-house Data Science Team Benefit from intense scrutiny over the validity of data in your enterprise data warehouse (EDW) Decide which analytic avenues to investigate. Multiply returns on prior investments in data assets unique to your organization. Tune prediction models to the home-field advantage. Enjoy having allies when evaluating analytics vendors products Rely on trusted in-house statistical advisors aligned with your goals.

14 Poll Question #2 Forecast ahead 18 months Where do you want your organization spending most of its time? 1) A 2) B 3) C 4) D 5) E 6) F 7) G 8) Unsure or not applicable A What is the best course of action? How can I influence the future? What are likely future outcomes? Why did this occur? C B D E G F Forecasting Statistical Analysis Optimization Predictive What-if Analysis Alerts & Triggers What action is needed? Query Drill-down What exactly is the problem? Ad-hoc Reports How many? How often? Where? Standard Reports What happened? Analytics

15 Required: A Trusted Data Infrastructure If you don t have a reliable data infrastructure or EDW, you are wasting your time trying to operationalize data science. 76% of data scientists view data preparation as the least enjoyable part of their work. Source: Press, G. (2016). Cleaning big data: Most time-consuming, least enjoyable data science task, survey says. Forbes Magazine. Retrieved from

16 Required: Well-Defined Problems with Predictive Solutions Scopes creep when people get excited! Trust your data scientists to know when to abandon (or refuse to intake) a project. Try to beat the performance of an existing model, or show up sooner than it can. Start with the essential quantities that the organization cares about. Proceed incrementally. Use existing data mart or reporting tables when available. There are still so many beautiful things to be said in C-Major. ~ Sergey Prokofiev There is still much good music that can be written in C-Major. ~ Arnold Schoenberg

17 Twelve Key Questions to Intake a Predictive Project 1. What is the business problem? 12. Are there any minimum required levels of performance? What needs to be predicted? 11. Are there any models to benchmark against? How much improvement is needed? 10. Are there predictions being made now? Who is the targeted population? What will you do with the predictions? How early in the process do you need it? 8. Have there been other projects that focused on this? 6. How often do you need updated predictions? 7. What time of day you need the predictions?

18 Required: Skilled Data Scientists Where Is Everyone? Of the approximately 6,000 data scientists in the U.S., only 180 are estimated to work in the hospital and health care field. Given that there are nearly 6,000 hospitals and just 400 academic medical centers in the U.S., that s stretching the available labor force a bit thin. Source: Huesch, M. D., & Mosher, T. J. (2017). Using it or losing it? The case for data scientists inside health care. NEJM Catalyst. Retrieved from

19 No Unicorns Need Apply!

20 The Ability to Learn and Adapt Is a Key Requirement Specialization is for insects. Robert A. Heinlein In terms of species diversity, total biomass, range of habitat, adaptations to adverse conditions, and intra-species pro-social behavior, insects are the most successful animals on Earth. Every biology textbook ever

21 On the Proper Care of Data Scientists Top 5 rules for my data science team: 1Always ask questions. 2Try to be comfortable with failure and uncertainty. 4 5 Before going 3Do not use down a technical Always be nice methods you do rabbit hole, ask to the data not understand. What s the value architects. to Mission? Require them to continue learning and to set aside work time to do this.

22 A Tale of Two Job Offers The other issue is that I was uneasy about working with the physicians I d met, as I felt their project vision and expectations exceeded their appreciation of the organizational, technical, and personnel requirements involved. When I asked them what they wanted from the person in the data scientist position, I heard: a statistical analyst, a project manager, a health economist, a strategy manager, and small amount of EMR build consulting. I can t do all of that myself, Andy, to be honest, we do expect all of those things from our data scientists.

23 A Tale of Two Job Offers Can you give me any info on how I would be evaluated, and what sort of expectations you have for this position? Hi Andy, Good to hear from you. Here are answers to your questions: The expectation would simply be to add value to our highly collaborative analytics program.

24 Required: The Rest of the Team Knowledge Engineers Data Architects BI Developers QA and Training Problem Definition Research Data Discovery Descriptive Analytics Predictive Analytics Implementation Who is the customer? What problem are we trying to solve? Is this the right problem? What is the value added? Literature review. Identify competing models. Identify data sources. Collect new data if needed. Merge, join, and augment as appropriate. Understand structure of the data set - what does the data mean? Identify errors, outliers, case-deficient groups, etc. Understand statistical properties of the data set. Methodology assessment. Train model. Test model. Automate. Integrate into application. Validate. Train.

25 Integration with Analytics Team Readmission Predictor v1 Data Scientist: Prototype input data frames. Create prediction model. Validate model performance. Create model task/timing/output structure. Knowledge Engineer: Evaluate initial project proposal. Set scope, deliverables, and feasibility. Steer stakeholder validation of existing data elements (LACE/Readmission Explorer). Monitor progress to milestones. Data Architect: Productionize subject area mart (SAM) models for input data. Acquire additional fields from other sources. Validate DS-created input data. Build or modify application using model output. BI Developer: Develop best practices for in-app visualization. Build or modify application using model output. Training and QA Analyst: QA check involved apps. Maintain release schedule. Communicate new releases. Train end users.

26 Required: Processes to Develop Data Science Projects

27 Readmission Prediction v1 at Mission Existing tool: LACE Problem: LACE is good, but not great, and we have unique patient populations and unusual data assets for building our own model. Goal: To construct and automate the calculation of a risk model for 30-day, all-cause inpatient readmission. Requirements: Performance must beat LACE in our patient population. Be available before 8 a.m. day after discharge. First version must use fields currently existing in EDW.

28 A Relatively Simple Implementation EDW source mart EDW SAM R on DS server Staging server Mission Analytics Portal External source data Predictive model input SAM R model R output to forecast model database Internal source data Predictive model output SAM Visualization application

29 Model Performance Comparison

30 Key Takeaways Provide systemwide data in a way that allows broad access for analysis as well as operational uses ranging from daily unit management to boardlevel discussion. Create a culture of continuous improvement by outfitting key clinical and operational roles with skills, tools, and a sense of ownership for improving their own processes. Create the capability to leverage both operational and clinical predictive analytics Appreciate the importance of a skilled data science team.

31 Thank You