Developing Quality Measures The Indicator Tree Approach

Transcription

1 Developing Quality Measures The Indicator Tree Approach Institute of Medicine Committee on Quality Measures for the Leading Health Indicators Rohit Ramaswamy, PhD, MPH, Gr. Dipl. (Bios) Clinical Associate Professor Gillings School of Global Public Health University of North Carolina, Chapel Hill

2 Objectives Describe the link between LHIs, objectives and the PH Quality Aims Present the tree approach to indicator development Provide some sample metrics using the approach for one tobacco related LHI

3 The Nine PH Quality Aims Population-centered protecting and promoting healthy conditions and the health of the entire population Equitable working to achieve health equity Proactive formulating policies and sustainable practices in a timely manner while rapidly mobilizing to address new and emerging threats and vulnerabilities Health-promoting ensuring policies and strategies that advance safe practices by providers and the population and that increase the probability of positive health behaviors and outcomes Risk-reducing diminishing adverse environmental and social events by implementing policies and strategies to reduce the probability of preventable injuries and illness or other adverse outcomes Vigilant intensifying practices and enacting policies to support enhancements to surveillance activities (e.g., technology, standardization, systems thinking/modeling) Transparent ensuring openness in the delivery of services and practices with particular emphasis on valid, reliable, accessible, timely, and meaningful data that are readily available to stakeholders, including the public Effective justifying investments by using evidence, science, and best practices to achieve optimal results in areas of greatest need Efficient understanding costs and benefits of public health interventions and facilitating optimal use of resources to achieve desired outcomes

4 Linking LHI to PH Quality Aims Key Questions Aim/Objective/Indicator Leading Health Indicators Objectives PH Quality Aims Question Answered What measure assess the health of the population for key health topics? What intermediate outcomes need to be achieved in a particular health topic in order to make progress on the corresponding LHI? What characteristics are needed to ensure quality implementation of the evidence based clinical and community interventions to achieve the desired outcomes?

5 Linking LHI to PH Quality Aims - Concepts Objective 1 PH Quality Aims Evidence Based Clinical and Community Programs Objective 2 Objective 3 Leading Health Indicators Quality Measures: Are our activities on the right track? Objective 4 Intermediate Outcome Measures: Are our drivers moving in the right direction? Ultimate Outcome Measures: Are we achieving our goals?

6 Developing Quality Measures: Practical Aspects While conceptually simple, the practice of developing quality measures could be complicated because: 1. Measures could be program dependent there may not be a single set of quality metrics for each objective and LHI. 2. The quality aims are abstract (e.g. proactive or transparent) and need to be translated into concrete and understandable measures. 3. All aims may not apply for all objectives for a health topic careful thought will be required to identify the aims that are relevant to each objective. A good strategy for the committee would be to develop representative quality metrics for each objective/lhi and provide guidance to practitioners on how to develop their own local metrics.

7 Defining Quality Measures The Tree Approach An indicator tree is a systematic way of moving from the abstract to the concrete. The root of the tree is the aim. The branches define the aim in increasingly measurable terms. The last branch of the tree is the measure.

8 Tree Structure From Abstract to Concrete Attribute Definition Quality Measure Aim Attribute Definition Quality Measure Intermediate Outcome Measures (Relating to Objectives) Leading Health Indicator Attribute Definition Quality Measure

9 The Tree Elements What they Mean Tree Element Aim Attribute Definition Quality Measure Description What quality objective do we want our public health policy or program to achieve (e.g. equitable )? What are the different dimensions of quality for this aim? How do we define these dimensions? How can we measure these dimensions for specific policies and programs?

10 Aims, Attributes and Definitions Part 1 Aim Attributes Definitions Population Centered 1. Comprehensiveness 1. Program addresses both clinical and community interventions Equitable 1. Availability 2. Accessibility 3. Affordability 4. Usability 1. Program is available to everyone, 2. Program can be accessed by everyone 3. Everyone can afford the program 4. Program is easy to understand and follow Proactive 1. Timeliness 2. Adaptability 1. Program activities take place just when needed 2. Program adapts quickly to new needs Health Promoting 1. Relevance 2. Persuasiveness 3. Adoptability 1. Program messages are appropriate for the population 2. Program messages are compelling and motivating 3. Program messages are easy to adopt Risk Reducing 1. Specialization 2. Visibility 1. Program is customized for different risk segments 2. Risk reduction activities are invisible to population

11 Aims, Attributes and Definitions Part 2 Aim Attributes Definitions Vigilant 1. Completeness 2. Responsiveness 1. Surveillance program covers entire population 2. Program responds quickly to emerging threats Transparent 1. Accuracy 2. Usefulness 3. Openness 4. Currency 1. Performance data is accurate and error free 2. Performance data is useful for decision making 3. Performance data is available and accessible by all 4. Performance data is current Effectiveness 1. Effectiveness 2. Sustainability 1. Program achieves desired results 2. Program results are maintained over time Efficiency 1. Efficiency 1. Program seeks to minimize resources and costs

12 Setting Quality Measures - Example Healthy People 2020 Topic: Tobacco Leading Health Indicator: Proportion of population using tobacco Selected Objectives: (TU 1.1) Reduce tobacco use in adults (TU 4.1) Increase smoking cessation activities in adult smokers (TU5.1) Increase recent smoking cessation reductions by adult smokers

13 Overall and Intermediate Outcome Measures Indicator Type LHI/Objective Measures Ultimate outcome LHI Proportion of population using tobacco Intermediate outcome TU 1.1 % adults aged 18 and over who are current cigarette smokers Intermediate outcome TU 4.1 % smokers aged 18 and over in the past 12 months who attempt to stop smoking Intermediate outcome TU 5.1 % smokers aged 18 and over who last smoked in the past 6 months to one year

14 Intervention Example Workplace Smoking Cessation Program Clinical Intervention: Clinicians ask all employees about tobacco use during the annual check up and provide counseling and treatment. Community Interventions: Work place competitions combined with education, smoking cessation groups and self-help materials.

15 Aims, Attributes and Definitions Tobacco Example Part 1 Aim Attributes Definitions Population Centered 1. Comprehensiveness 1. Program uses both clinical and community interventions. Equitable 1. Availability 2. Accessibility 3. Affordability 4. Usability Proactive 1. Timeliness 2. Adaptability 1. All employees eligible to participate in the program 2. All employees can make use of the program 3. All employees can afford the program 4. Education and self help materials are easy to follow 1. Program identifies and treats new employee smokers 2. Not applicable Health Promoting 1. Relevance 2. Persuasiveness 3. Adoptability Risk Reducing 1. Specialization 2. Visibility 1. Education programs use relevant examples 2. Education and counseling use stories and real life examples 3. Education and counseling sessions promote easy to follow step-by-step approach 1. Not applicable 2. Not applicable

16 Aims, Attributes and Definitions Tobacco Example Part 2 Aim Attributes Definitions Vigilant 1. Completeness 2. Responsiveness 1. Not Applicable 2. Not Applicable Transparent 1. Accuracy 2. Usefulness 3. Openness 4. Currency Effectiveness 1. Effectiveness 2. Sustainability 1. Accurate monitoring data on participation and outcomes 2. Participation and outcome data segmented by gender, location to create targeted campaigns 3. Participation and outcome data posted for public access 4. Participation and outcome data updated regularly 1. Program achieves desired results 2. Program results are maintained over time Efficiency 1. Efficiency 1. Program seeks to minimize resources and costs

17 Putting it All Together Metrics for Employee Based Smoking Cessation Program Part 1 Aim Quality Metrics Intermediate Outcome Population Centered Uses clinical and community interventions (Y/N) Final Outcome Equitable 1. % employees eligible to participate 2. % smokers accessing the program 3. Average OPC to employee for participation 4. % participant reporting materials easy to follow Proactive 1. % new employees who smoke enrolled cessation programs within 90 days or joining 2. Adaptability Health Promoting 1. % participants reporting that educational programs are relevant to their needs 2. Average number of real life example s per educational module 3. % modules having checklists and other self help tools

18 Putting it All Together Metrics for Employee Based Smoking Cessation Program Part 2 Aim Quality Metrics Intermediate Outcome Transparent 1. % reports with errors and incomplete data 2. % performance report reviews leading to improvement projects in program design or implementation 3. % results posted for public view 4. Average time interval for refreshing posting data Final Outcome Effective 1. % adults aged 18 and over who are current cigarette smokers 2. % smokers aged 18 and over in the past 12 months who attempt to stop smoking 3. % smokers aged 18 and over who last smoked in the past 6 months to one year 1. % population using tobacco Efficient Average program cost per smoker.

19 Conclusions Quality metrics measure the overall quality of the implementation of a system of program. Together, quality metrics and short and long term outcome metrics provide the basis for public health quality improvement. The indicator tree is a useful approach to develop relevant and actionable quality metrics aligned with the PH quality aims.

20 Some Take Homes for Committee LHIs are long term outcomes Need a complementary set of measures that include intermediate outcomes and process measures Also consider balancing measures (e.g., costs) Improvement measures need to be good enough (vs. NQF process)