using measures in health care

Transcription

1 Measures That Matter Selecting and effectively using measures in health care Dr. Kate Goonan and Joe Muzikowski

2 Three Measurement Challenges 1. Identifying relevant process measures and using them to impact the business 2. Effective analysis of process measures to proactively identify trends and issues that should be addressed 3. Effectively integrating a balanced set of measures, making the relationship of process measures to customer results understood 2

3 We ll Discuss Today: 1. Key measures Why you need to identify and use key measures 2. Identifying key measures Aligning with your strategy and mission Building a world-class l measurement system 3. Fostering alignment How high-performing organizations use their measures Effective dashboards and scorecards Deploying measures to create line of sight 4. Driving improvement through measures Performance reviews Analysis methods and evaluation factors Use of comparisons and benchmarks 5. Summing gup/ Where eemeasurement easue e is sgoingg 3

4 This Discussion Is Not A comprehensive overview of regulatory measures A primer on the Baldrige Criteria or award process A course in statistical analysis 4

5 Measures Provide A link between strategy and tactics A basis for a business case for improvement Assessment of performance against a baseline Feedback to guide change Focus for the enterprise on what is important: desired behaviors and outcomes 5

6 Measurement Answers the Question: How Do You Know? 6

7 Measures Are Your Foundation Organizational Profile: Your Strategic Fundamentals 2 - Strategic Planning 5 - Workforce Focus 1 - Leadership 7 - Results 3 - Customer Focus 6 - Process Management 4 - Measurement, Analysis, and Knowledge Management

8 Baldrige Core Values and Concepts Visionary Leadership Customer-Driven Excellence Organizational and Personal Learning Valuing Workforce Members and Partners Agility Focus on the Future Managing for Innovation Management by Fact Societal Responsibility Focus on Results and Creating Value Systems Perspective

9 Measures Are Your Foundation Organizational Profile: Your Strategic Fundamentals 2 - Strategic Planning 5 - Workforce Focus 1 - Leadership 7 - Results 3 - Customer Focus 6 - Process Management 4 - Measurement, Analysis, and Knowledge Management

10 Baldrige Category 7 - Results The Results Category examines your organization s performance and improvement in key areas health care outcomes, customer focused outcomes, financial and market outcomes, workforcefocused outcomes,,p process effectiveness outcomes, and leadership outcomes. Performance levels are examined relative to those of competitors and other organizations providing similar product offerings. 7.1 Health Care Outcomes 7.2 Customer-Focused Outcomes How do you perform in those areas of most importance to your customers? Continued on next slide

11 Baldrige Category 7 - Results 7.3 Financial and Market Outcomes 7.4 Workforce-Focused Outcomes 7.5 Process Effectiveness Outcomes 7.6 Leadership Outcomes How do you perform in those areas of most importance to your other stakeholders and to your organizational success?

12 How Results and Processes Align in Baldrige Baldrige Category Related Results Item 1 Leadership 7.6 Leadership Outcomes 2 Strategic Planning Financial and Market Outcomes 3 Customer Focus 7.2 Customer-Focused Outcomes 4 Measurement, Analysis and Knowledge Management 7.1 Health Care Outcomes 5 Workforce Focus 7.4 Workforce-Focused Outcomes 6 Process Management 7.5 Process Effectiveness Outcomes

13 Journey to Excellence: How Baldrige Health Care Leaders Succeed Five fundamental practices drive the achievement of organizational excellence: Leadership Assessment Sensemaking Execution Results

14 A Quick Exercise Identify the three most important measures in your personal life, for example: Financial (salary level, net worth, retirement income) Actualization (personal or job satisfaction) Relationships (number of friends, quality of relationships) Now, note how often you assess your performance against these measures and what you re doing about it What are your observations? Are your actions aligned with what you want to achieve? 14

15 Guidelines in Developing Measures Select or develop measures that t drive behavior Measure real work outputs and tangible accomplishments against goals Ensure usefulness and relevance Assign responsibility for each performance measure to a specific position or person Develop measurement collection and analysis tools that provide adequate warnings of negative change 15

16 Tool #1: Measures Identification Exercise 1. List your key customers or customer groups 2. As specifically as possible, list their requirements Generic requirements may be things like quality, cost and timeliness, but try to be more specific 3. Describe how you know you are meeting their requirements Indicate how you currently assess your performance If you do not have specific metrics, what do you use? These are your output measures, or outcomes 4. Identify what other measures you might use that support the key measure(s) These are in-process or predictive measures used to manage the process itself 16

17 Tool #1: Measures Identification Exercise Our Customers Their Requirements Our Key Measures Supporting Measures

18 Tool #2: Process Definition Using SIPOC Supplier Inputs Process Outputs Customer Activities Having a defined process allows you to: Manage the process effectively Share learning with others about the process Understand the value of data and measures Identify opportunities for improvement SIPOC is an excellent bottomup tool

19 Process Performance Measurement Feedback Feedback Supplier Inputs Process Outputs Customer Activities Input Measures Process Measures Output Measures Action Plans & Improvement Initiatives Entity and Corporate Goals and Measures

20 Sharp s Process Model SIPOC and COPIS Sharp HealthCare, a 2007 Baldrige Recipient

21 Results Link to Processes in Baldrige What are your organization s main product offerings? [P.1a(1)] What are (your customers ) key requirements? [P.1b2)] What are your key strategic objectives and your timetable for accomplishing them? What are your most important goals for these strategic objectives? [2.1b(1)] What are your key performance measures or indicators for tracking the achievement and effectiveness of your action plans? [2.2a(6)] How do you identify product offerings to meet the requirements of your customer groups?? [3.1a(1)] 21

22 Results Link to Processes in Baldrige How do you use product offering information to identify customer requirements? [3.2c(2)] What are your key organizational performance measures? [4.1a(1)] How do you determine key work process requirements incorporating input from customers? What are the key requirements? [6.1b(2)] How does your day-to-day operation of these processes ensure that they meet key process requirements? [6.2b(1)] 22

23 Tool #3: The Strategy Tree Diagram Mission/Vision/ Values of the Organization Overall Goals Financial Key Indicator # 1 Key Indicator # 2 Key Indicator # 3 Customers Key Indicator # 4 Operational Key Measure Owners Citi Critical lsuccess Factors: What must we do very well to achieve our goals? People Key Indicator #

24 Scorecard Development Using the Strategy t Tree Diagram Act Follow-up on and communicate performance. Review/improve the Indicators Develop the Strategy Alignment Tree Diagram Cascade the Top-Level Scorecard Plan Develop the Top-Level Scorecard Do The Strategy Alignment Tree is an excellent top- down tool. Study

25 Cascading Your Scorecard Vision & Mission What How Organization Objective Indicators Targets Initiatives Scorecard What How Cascade & Linkage Entity Level Indicators & Projects Objectives Indicators Targets Initiatives How Objective: Create vertical and horizontal line of sight Individual development needs What Team / Individual Performance Evaluation Objective (Specific) Indicators (Specific) Targets Performance Management

26 How Measures Fit In at Poudre Valley PVHS s GPS is driven by its VMV, translated into strategic objectives and plans, and deployed at all levels through its measurement scorecards, with review and improvement cycles built in. Poudre Valley Health System, a 2008 Baldrige Recipient

27 Measurement Should Not: Start with strategy Strive for balance Be totally logical Rely only on IT Provide just a score Foster complacency Should: Start with customers Strive for relevance Be a bit emotional Use IT as a tool Provide insight Foster accountability 27

28 Start With Customers: Mercy s Stakeholder Requirements Mercy Health System, a 2007 Baldrige Recipient

29 Start With Customers: PVHS s Patient t Requirements P.1b(2): What are your patients...requirements? Poudre Valley Health System, a 2008 Baldrige Recipient

30 PVHS s Selection Process for Their Balanced Scorecard Poudre Valley Health System, a 2008 Baldrige Recipient

31 A World-Class Measurement System Considers Three Phases of Measurement 1. Institutionalize Develop meaningful measures 2. Transform Change organizational culture to ensure effective use of measures 3. Optimize i Review and improve the measures to keep them relevant 31

32 Setting up a World-Class Measurement System Define your customer requirements and strategic drivers Identify the financial / non-financial data and measures you use (or should be using) to run the business Make sure these clearly relate to your customer requirements, key processes, strategy and action plans Assign executive champions for all key measures Ensure clear, consistent definition and interpretation Prevent coloration of the data Establish appropriate targets and external comparisons and keep them current 32

33 Setting up a World-Class Measurement System Deploy the measures and align the organization around their use Communicate the measures to all employees Ensure user availability and data integrity Ensure accountability Link evaluation, recognition and rewards to the measures Review / analyze performance and gaps; set improvement priorities and ensure action is taken Periodically review and improve the measurement system 33

34 Tool #4: Is Your Measurement System World Class? Score your organization on a scale of 1 to 10 (10 = world class) Score Our measures allow us to track the issues that matter to all of our stakeholders. Our measures allow us to track stakeholder engagement as well as satisfaction. An explicit it causal model underpins our performance measurement system. Our measurement system was designed to provoke specific behavioral responses. Our measurement system makes it explicit who is accountable for what. An appropriate IT infrastructure exists to support our measurement system. Our people are clearly engaged in helping us to translate our strategy into action. Our measurement system provides insights rather than scores. We use appropriate visualization to communicate our performance. We understand the full cost of our measurement system and manage it accordingly

35 Excellence Indicators for Performance Results Broad base of improvement trends (3+ years) Results consistent across major segments (customers, markets, employees) Clear evidence of industry leadership Results benchmarked to world-class leaders outside of the industry show positive comparative performance Results demonstrate the organization is successfully addressing its customer requirements and strategic challenges Evidence that the performance can be sustained 35

36 How Sharp Aligns the Organization Vert tical Line of Sight Sharp HealthCare, a 2007 Baldrige Recipient

37 Bronson s Alignment Requirements and related measures are clearly l defined d - Safe: Avoid injuries from care intended to help Timely: Reduce wait times and harmful delays Effective: Care is based upon scientific knowledge. Efficient: Avoid waste Equitable: Provide care that does not vary in quality due to personal characteristics Patient and Family Centered: Patients and families are partners in care Measures can be clearly defined for each of these elements Bronson Methodist Hospital, a 2005 Baldrige Recipient 2010

38 Mercy s Leadership Model Aligns Their Organization Alignment with MVV and Strategy Dashboards Provide Linkage Line of Sight Alignment with Pillars and Stakeholders Mercy Health System, a 2007 Baldrige Recipient

39 Mercy s Performance Dashboard 1 Key measures es relate to Mercy s Pillars 2 Mercy Health System, a 2007 Baldrige Award Recipient

40 Drilling Down Into Mercy's Dashboard

41 NMMC s Scorecard Key measures relate to NMMC s Critical Success Factors (CSFs) North Mississippi Medical Center, a 2006 Baldrige Award Recipient

42 Drilling Down Into NMMC s Scorecard

43 St. Luke s Leadership Model Has Scorecards at the Center Two Types of Scorecards Align Processes with Strategy St. Luke s Hospital, a 2003 Baldrige Recipient

44 St. Luke s System Scorecard 1 2 Saint Luke s Hospital, a 2003 Baldrige Award Recipient

45 Drilling Down Into St. Luke s Scorecard 1 2 Saint Luke s Hospital, a 2003 Baldrige Award Recipient

46 Mercy s Organizational Performance System Link to Strategyt Benchmarks Review and Analysis Line of Sight Reporting Mercy Health System, a 2007 Baldrige Recipient Link to Improvement Initiatives

47 Sharp s Organizational Performance Measurement System Link to Strategy Link to Improvement Initiatives Line of Sight Performance Reviews Link to Customers Employee Evaluation Benchmarks Data Integrity Review and Improvement Cycle Sharp HealthCare, a 2007 Baldrige Recipient

48 SSM s Measures Deployment Entity Level Inpatient Loyalty Drivers or KCRs Pain Management Response Department Meds delivered w/in X min. of order Pain assessed per pain scale Call light response Individual I will respond in X min. I will assess 1X/shift/pt I will respond in X minutes I will ask if there is anything else 100% of time SSM Health Care, a 2002 Baldrige Recipient

49 Tool #5: Five Measurement Questions 1. Baseline Where are we starting ti from? 2. Trending How has the process been working? 3. Control Is the process staying in control and within the desired predetermined boundaries? 4. Diagnostic Where are the problems 5. Planning What can we predict and plan for about the future? 49

50 The Baldrige Framework Organizational Profile: Environment, Relationships, and Challenges 2 - Strategic Planning 5 - Workforce Focus 1 - Leadership 7 - Results 3 - Customer Focus 6 - Process Management 4 - Measurement, Analysis and Knowledge Management

51 Results Evaluation Factors Le-T-C-I Levels Numerical data that place performance on a meaningful measurement scale Trends Numerical data that show the direction and rate of improvements Comparisons Numerical data that show the relationship of an organization s performance relative to others (or the performance of units within an organization to one another) Integration Connections to important customer, product/service, process, or action plan performance requirements and Segmentation as appropriate

52 A Word on Segmentation Segments can be defined by, among other things: Service / business line Location Patient population (DRG, age, etc.) Workforce group (nurses, physicians, admin, etc.) Size of the group in question Your key segments probably include: Health care services and markets Workforce profile Facilities / locations Patient and other customer groups Main types of suppliers and partners 52

53 Displaying Current Levels % Agree d or Stro ngly Agre eed Company Overall Employee survey question: I am satisfied with my job

54 Displaying Levels and Key Segments % Agree ed or Stro ongly Agre eed Good Company Overall HQ Location 1 Location 2 Location Employee survey question: I am satisfied with my job. Results also could be segmented by type of response (agree; strongly agree, etc.)

55 Trends: How to Define and Use Them Do you track measures over time? Is your performance relatively consistent? Do you have enough data to discern an actual trend? Do you have a statistical basis for your conclusions about your performance? Do you know why you ve improved? If there is no discernable trend, is your performance sustained at a high level? Do you understand what accounts for unfavorable trends and for specific negative occurrences?

56 Displaying Trends with Segmentation % Agreed or Stron ngly Agree ed Good Company Overall HQ Location 1 Location 2 Location 3 Employee survey question: I am satisfied with my job

57 Comparisons: How to Use Them Effectively Do your comparisons come from recognized sources? Are the comparisons presented for high performance levels or just averages? Are comparisons used for a few, some, or most measures? Are comparisons used for the most important results? Are the comparisons appropriate to your organization s needs? Are the comparisons consistent across related measures?

58 PVHS s Measures Selection Process Incorporates Benchmarks Poudre Valley Health System, a 2008 Baldrige Recipient

59 Displaying Comparisons or Strong gly Agree ed % Agreed Good National Benchmark Main Competitor Industry Trend Company Overall HQ Location 1 Location 2 Location Employee survey yquestion: I am satisfied with my yjob

60 Health Care Organizations Now Have a Wide Range of Available Comparisons Clinical Satisfaction People Financial / Other Centers for Medicare and Medicaid Services Consumer Assessment of Health Plans Survey Press-Ganey Moody s HealthGrades Press-Ganey Management Science Assoc. Standard & Poor s Thomson Avatar ASHHRA American Medical Group Association Vermont Oxford Network State Dep t. of Public Health databases National Council on Compensation Insurance National Trauma Data Bank State Health Associations American Medical Group Association National Healthcare Safety Network Ingenix National Nosocomial Infection Surveillance Health Plan Employer Data and Information Set Managed Care Benchmarking Study Agency for Healthcare Research and Quality Integrated Healthcare Association Joint Commission National Patient Safety Goals Bureau of National Affairs HR Solutions American Society for Training and Development Occupational Safety and Health Administration The Advisory Board Dartmouth Atlas The Governance Institute Verispan Mayo Clinic Health Solutions Information Week s Most Wired Hospital Quality Alliance State Hospital Association Local and state health regulatory agencies

61 Bronson s Readmission Rate Note the direction of goodness arrow and use of three comparisons: local, state and national Bronson Methodist Hospital, a 2005 Baldrige Recipient

62 NMMC s Health Care Outcomes Note the linkage between inprocess measures and the outcome measure and use of the top decile comparison North Mississippi Medical Center, a 2006 Baldrige Recipient

63 Mercy Health System Note the measures related to key patient requirements and use of the 90 th percentile comparison Mercy Health System, a 2007 Baldrige Award Recipient

64 Mercy s Trust Index Note the use of a benchmark from outside of health care: the 100 Best Places to Work index Mercy Health System, a 2007 Baldrige Award Recipient

65 NMMC s Top Box Scores Top box scores can be used as a surrogate indicator for loyalty and retention; note the comparison to the 90 th percentile North Mississippi Medical Center, a 2006 Baldrige Recipient

66 Sharp s Patient Satisfaction Results are tracked against key patient requirements and top quartile comparisons Sharp HealthCare, a 2007 Baldrige Award Recipient

67 Sharp s Health Care Outcomes Sharp demonstrates positive comparisons to the AHRQ indicators Sharp HealthCare, a 2007 Baldrige Award Recipient

68 Integration: Where Process and Results Intersect Approach (What s done in the organization) Deployment (How the approaches are used in the organization) Learning (How approaches are reviewed and improved) Levels (Current performance in key measures) Trends (Performance of key measures over time) Comparisons (Performance of key measures versus benchmarks) 2010 Integration (How processes drive results to achieve organizational goals) 68

69 Integration: Making Sure Your Measures Are Meaningful Do the measures and their results link to your organization s strategic fundamentals? To what extent do the results show positive performance in areas of importance? Do your results include appropriate segmentation to show variance in performance?

70 What Hinders Effective Performance Reviews? Absence of expected measures Beyond those mandated by regulatory bodies Unclear linkage Difficult to see how the measures relate to customer requirements or strategy No segmentation Impossible to see variance in performance across sectors Fear of showing poor results Colorization of results 70

71 What Hinders Effective Performance Reviews? Difficulty accessing the data Buried in the organizational rubble Difficulty with benchmarks Obtaining i them; making sure they re relevant; believing i them Use of averages rather than high-performing comparisons Lack of historical data The rolling 12 month approach Difficulty discerning real progress Interpretation issues Is this result due to special cause or common cause? 71

72 Results Descriptions Results may be presented in: Graphic format Tabular format Text A combination of the above % Positive Patient Satisfaction Hospital A Hospital B Hospital C Our financial performance has demonstrated continued improvement over the past three years Key Measures YTD Measure A Measure B Measure C

73 The Importance of Graphical Displays They re eye-catching They tell a story in a limited space They complement your organization s branding They appeal to the right brained person

74 Use IT as a Tool: It May be Impressive, But Is It Useful? Remember: Data Information Information Knowledge Knowledge Insight Insight Wisdom

75 Tool #6: The Measure Presentation Sheet Measure Title 2010 Target: XXX Benchmark: YYY Year Comments: (Remarks on the indicator s evolution, actions in progress, etc. ) Description: Definition of the indicator. Rationale: Why this measure is important. Formula: How the indicator is calculated. Data source: Where the data in the formula come from. Frequency of measurement: How often data are collected and results posted. Impact(s): Relationship(s) to other measures. Reporting method: How and where the measures is available for review. Target: The current level for the target established for this measure. Benchmark: Which external comparison has been chosen. Owner: Person in charge of tracking and follow- up of the indicator

76 What s Confusing Here?

77 What s Wrong Here? No 3D graphs EVER!!!

78 Measurement is a Management Process Which measures should we use? How do we manage the measurement system to maintain relevance as the business changes? Designing measurement systems Revising measurement systems Implementing measurement systems Managing with measurement systems How do we access data and overcome the social, political and cultural barriers? How do we analyze the results and ensure that action occurs?

79 Measurement Success Factors Measures are aligned with customer requirements and to organizational strategy Measures are actionable with assigned responsibility Use of key measures is embedded into daily operations Measurement systems are simple to use and understand Technology is a tool, not a driver for collection, analysis and reporting Measurement drives organizational cultural and behavioral changes Performance results are effectively communicated to all employees 79

80 Where Healthcare Measurement Is Going The old paradigm: Pick any two you want The new paradigm: You must deliver all three and be transparent! Quality Quality Access Cost Access Cost

81 PVHS s Core Measures Results High performance against core measures and national patient safety goals is to be expected. What will differentiate your organization in the future? Poudre Valley Health System, a 2008 Baldrige Recipient

82 PVHS s Cost Comparisons Stakeholders will expect better cost containment, as well as transparency to allow for cost comparisons across health care providers. Poudre Valley Health System, a 2008 Baldrige Recipient

83 PVHS s Top Box Patient Satisfaction Once quality is a given, patients (and payors) will consider who can provide the best overall health care experience. Poudre Valley Health System, a 2008 Baldrige Recipient

84 PVHS s Patient Wait Times Patients will expect to get the same high level of responsiveness from their health care providers as they get from other service providers. Poudre Valley Health System, a 2008 Baldrige Recipient

85 Measures That Matter You can contact us at: Dr. Kate Goonan and Joe Muzikowski