QUALITY IMPROVEMENT FORM NCQA Quality Improvement Activity Form (an electronic version is available on NCQA's Web site) Activity Name: Section I: Activity Selection and Methodology A. Rationale. Use objective information (data) to explain your rationale for why this activity is important to members or practitioners and why there is an opportunity for improvement. This task centers on identifying a completed or nearly completed quality improvement activity (QIA). Your role is to summarize and analyze a change that has already been implemented you will not be starting a project from scratch! You will not be collecting data and should not need to access patient records. In this section, you will be explaining why your project was important to the organization, the staff, and/or the population served. You might want to provide some background information on how the need for this improvement was identified as well as how the change was selected. In the PDSA model (see below), you are talking about the objective or aim of the project Plan, as well as describing the prediction made: The proposed change was predicted to help the organization meet its goal/benchmark for (process/outcome). If applicable, this is also where you would describe the global, national, statewide, or healthcare system initiatives that the project was based on. (Don t forget to provide the source with citation and reference!) B. Quantifiable Measures. List and define all quantifiable measures used in this activity. Include a goal or benchmark for each measure. If a goal was established, list it. If you list a benchmark, state the source. Add sections for additional quantifiable measures as needed. Quantifiable Measure #1: Numerator: Denominator: First measurement period dates: Baseline Benchmark: Source of benchmark: Baseline goal: Write a brief title to describe the first measure. Describe the numerator. This will likely be some portion of the denominator. No results or number should be listed here (see Section II for details). Describe the denominator. This will likely be the whole amount or entire quantity of the items being observed or measured. (No results should be listed here.) List the range of dates that data was obtained for the first measure. List the benchmark goal this will be related to a global, national, statewide, or healthcare system driven goal. If this does not apply to your project, just list n/a (for not applicable) so the evaluator will know that you considered this. List the source of the benchmark or list n/a. List the goal that your organization planned to meet. This might be a specific rate or a percentage of improvement. You can think of this as a Prediction being made about the results of the change.
QIA Form 2 Quantifiable Measure #2: Numerator: Denominator: First measurement period dates: Benchmark: Source of benchmark: Baseline goal: Quantifiable Measure #3: Numerator: Denominator: First measurement period dates: Benchmark: Source of benchmark: Baseline goal: If you have additional measures, continue to list them as discussed above. If no additional measures, list n/a. If you have additional measures, continue to list them as discussed above. If no additional measures, list n/a.
QIA Form 3 C. Baseline Methodology. In this section, you will briefly describe the plan for data collection: who performed the data collection, how did they obtain the data, and what did they do with this information? It may help to think about the PDSA (Plan-Do-Study-Act) model for improvement and discuss steps taken to Plan and Do the project. Here is the model again for your convenience: Aim/Objective IHI Quality Improvement Model Measurement/Target Change/EBP/Innovation PDSA
QIA Form 4 C.1 Data Sources. [ ] Medical/treatment records [ ] Administrative data: [ ] Claims/encounter data [ ] Complaints [ ] Appeals [ ] Telephone service data [ ] Appointment/access data [ ] Hybrid (medical/treatment records and administrative) [ ] Pharmacy data [ ] Survey data (attach the survey tool and the complete survey protocol) [ ] Other (list and describe): In C1 and C2, indicate with a check or x which source was used to obtain the data and how it was obtained from that source. C.2 Data Collection Methodology. Check all that apply and enter the measure number from Section B next to the appropriate methodology. If medical/treatment records, check below: [ ] Medical/treatment record abstraction If survey, check all that apply: [ ] Personal interview [ ] Mail [ ] Phone with CATI script [ ] Phone with IVR [ ] Internet [ ] Incentive provided [ ] Other (list and describe): If administrative, check all that apply: [ ] Programmed pull from claims/encounter files of all eligible members [ ] Programmed pull from claims/encounter files of a sample of members [ ] Complaint/appeal data by reason codes [ ] Pharmacy data [ ] Delegated entity data [ ] Vendor file [ ] Automated response time file from call center [ ] Appointment/access data [ ] Other (list and describe):
QIA Form 5 C.3 Sampling. If sampling was used, provide the following information. Measure Sample Size Population Method for Determining Size (describe) Sampling Method (describe) Title of Quantifiable Measure #1 This column is where you will bring down the title of each Quantifiable Measure you listed in Section I above. Sampling may or may not be used in your project. If not used, explain this in the form. Simply stated, sampling has occurred if a small portion of the total number of people or things (such as observations, occurrences) have been selected for measurement. By studying a sample or a portion of the total, we hope to learn something about the larger data set or population from which the sample was chosen. Alternatively, you can indicate that 100% of the Population (the number of applicable people or things being measured) were included in the sample. Title of Quantifiable Measure #2 This will be some portion of the number in the next column. (Or could be 100% of the Population in the next column.) This is the entire number (100%) of people or things being measured. The sample size used in a study is often determined by the expense of data collection. Collecting 100% of the possible observations in a particular project might cost too much in terms of time or money. With that said, larger samples tend to give us good assurance that our data is generalizable to the whole population. (This might provide a point that you will want to later discuss in your opportunities for improvement.) Some common ways to determine a sample size are: Use the sample size of a similar study. Use a published table. Use a formula. Remember: Your objective here is to ask your change leader to describe how the sample size was determined. Random Sampling is just one example of a sampling method. See the graphic below for some ideas about sampling. Each of these would have different considerations for determining the sample size. For more information, please see this site: http://www.socialresearchmethods.net/k b/sampling.php
QIA Form 6 C.4 Data Collection Cycle. Data Analysis Cycle. [ ] Once a year [ ] Twice a year [ ] Once a season [ ] Once a quarter [ ] Once a month [ ] Once a week [ ] Once a day [ ] Continuous [ ] Other (list and describe): Indicate with a check or x how often the data were collected. C.5 Other Pertinent Methodological Features. Complete only if needed. This section is optional. If not used, record n/a (not applicable). [ ] Once a year [ ] Once a season [ ] Once a quarter [ ] Once a month [ ] Continuous [ ] Other (list and describe): Indicate with a check or x how often the data were actually reviewed and analyzed. This cycle is often different that the data collection cycle. For example, if we are recording observations of hand hygiene compliance, the data might be collected daily. But the checklists on which the data are collected might not be reviewed and analyzed until the end of the month. D. Changes to Baseline Methodology. Describe any changes in methodology from measurement to measurement. Include, as appropriate: Measure and time period covered Type of change Rationale for change Changes in sampling methodology, including changes in sample size, method for determining size and sampling method Any introduction of bias that could affect the results In this section, you will record any changes in the data collection or the data analysis that occurred. For instance, in the hand hygiene example used in C4 above, we would record a change in methodology if (after the initial measurement period) the data were analyzed at the end of every week, instead of every month. If there were no changes in methodology, a brief statement to that affect should be indicated here.
QIA Instructions and Form 7 #1 Quantifiable Measure: Title of Quantifiable Measure #1 Section II: Data / Results Table Complete for each quantifiable measure; add additional sections as needed. Time Period Measurement Covers Measurement Numerator Denominator Rate or Results In this column, you ll be recording a timeframe. For example, you might record in here that the first (baseline) measurement occurred in January 2014 (if data were collected monthly) or January March 2014 (if data were collected quarterly). These will be subsequent timeframes, following the baseline measurement. Baseline: This will be the first measurement this should be data collected on or before the change was implemented. Remeasurement 1: Remeasurement 2: Remeasurement 3: Remeasurement 4: Remeasurement 5: #2 Quantifiable Measure: Time Period Number Number % or other calculation Comparison Benchmark Rate or Percentage from Source in Section I Comparison Goal Facility Goal Statistical Test and Significance* If you have this information, include it. But if this information is not available, place an n/a or none here. Comparison You could divide the Comparison time saved after the Statistical change by Test the baseline. Measurement Covers Measurement Numerator Denominator Rate or Results Benchmark Your change allowed you Example: An AMI patient is admitted. Did that patient receive aspirin Goal to save an average and Significance* of 2 hours. Your numerator is 2 hours; the denominator is 6 hours. In this example: Same as above for next Baseline: on arrival? Yes or No. 2/6 = 1/3 or a 33% improvement. 33% would be recorded in the Rate measure (if there are any or Results column. Note: You exceeded the goal of 25%! other quantifiable measures). Remeasurement 1: Remeasurement 2: Remeasurement 3: Remeasurement 4: "What is the difference between a percentage and a rate?" Both are often referred to as a rate and both are calculated using a numerator and a denominator. However, the relationship between the numerator and denominator is different. Percentages Indicators that use percentage are the most commonly used indicator type in healthcare. Most of the Joint Commission measures monitor percentages to measure compliance. A percentage measures the number of a certain set of events which have a particular outcome when there are two outcomes possible, either Yes or No. Guidelines for Percentages: *Numerator and denominator are the same unit of measure. *Both the numerator and denominator are positive whole numbers. *The numerator is never greater than the denominator. If we look at all the AMI patients admitted during a time period, some subset or portion of those patients receive aspirin on arrival. Let s say 100 AMI patients were admitted in the first timeframe measured. Of those, 60 patients received aspirin on arrival. To get the percentage, we divide the number of AMI patients admitted and who received aspirin (numerator) 60 patients by the total number of AMI patients admitted (denominator) 100 patients. The patients who received aspirin are a subset of the patients admitted. The percentage of patients who received aspirin on arrival is 60/100 or 60%. 60% is our compliance rate. We can also say that the percentage of patients who did not receive aspirin on arrival is 40%. 40% would be the failure rate. Here is another example using a time study: Baseline (before change) = 6 hour average time Re-measurement 1 (taken after the change) = 4 hour average time Time saved after change = 2 hour average time Note: You will not be able to calculate a percentage for the baseline measurement in a time study so the baseline measurement will have a number recorded in the Rate or Results column but n/a for the numerator and denominator columns. Let s say your Comparison Goal or baseline goal for this quantifiable measure was to reduce the average length of time by at least 25% (as an example): You could also do the reverse. Let s say your goal is to reduce the baseline timeframe by half or 50% (basically you want to get the 6 hour baseline average down to 3 hours or less). Divide the 4 hour average time (in Re-measurement 1) by the baseline of 6 hours. In this example: 4 hours/6 hours = 2/3 or 67% (67% would be placed in the Rate or Results column). While you did reduce (improve) the timeframe to 67% of the original baseline, you did not quite reach the Comparison Goal of 50%.
Remeasurement 5: #3 Quantifiable Measure: Time Period Rates: A rate-based measurement, on the other hand, does not have the same numerator-denominator relationship as a percentage. For example, if we look at the number of patient falls per patient days, the number of falls and the number of days are two different units of measure. So the numerator QIA is Instructions not a subset (not and a portion) Form of 8 the denominator. Since we can t count 100% of the opportunities for a patient to fall, we use a substitute measure that will give us an idea of the magnitude of the opportunity for improvement. Patient days fits well with several such measures. Using a substitute measure produces a standard goal or benchmark Comparison that we can use across Comparison different size hospitals Statistical with Test Measurement Covers Measurement different Numerator numbers of Denominator patients. Rate or Results Benchmark Goal and Significance* Baseline: We might measure the number of patient falls per 1000 patient days. The numerator is the number of falls that Remeasurement 1: occurred during the time period. The denominator is the number of patient days in that same time period divided by Remeasurement 2: 1000. We use 1000 patient days as a denominator to produce a number that will make more sense to the viewer. Remeasurement 3: Remeasurement 4: Four (4) falls per 1000 patient days makes more sense to the human mind than.04 falls per patient day. You would probably choose the denominator (1000 patient days) based on how you want to compare your measures to other Remeasurement 5: hospitals. If most of the hospitals you are comparing to are using 1000 patient days, then you would probably want * If used, specify the test, p value, and specific measurements to use (e.g., 1000 baseline patient to remeasurement days. #1, remeasurement #1 to remeasurement #2, etc., or baseline to final remeasurement) included in the calculations. NCQA does not require statistical testing. A. Time Period and Measures That Analysis Covers. Section III: Analysis Cycle Complete this section for EACH analysis cycle presented. For this section, you ll provide a description of how and by whom the data were analyzed, what the interpretation of the data consisted of (including overall trends), how frequently data were analyzed, and how this information was reported to leadership and stakeholders. Think again about the PDSA model this section is all about the Study part of the model. You ll be comparing the baseline with the remeasurement results and the goals and/or benchmarks, and then discussing what was learned from this comparison. You should also include discussion about any changes to the organizational goals or benchmarks that were listed in Section I (see part B below).
QIA Instructions and Form 9 B. Analysis and Identification of Opportunities for Improvement. Describe the analysis and include the points listed below. B.1 For the quantitative analysis, include the analysis of the following: Comparison with the goal/benchmark Reasons for changes to goals If benchmarks changed since baseline, list source and date of changes Comparison with previous measurements Trends, increases or decreases in performance or changes in statistical significance (if used) Impact of any methodological changes that could impact the results For a survey, include the overall response rate and the implications of the survey response rate In this area, you should discuss the results of the QIA address the points listed. Describe how the results of the change unfolded over time. Include discussion of any opportunities for improvement in the QI process itself. Think again about the steps of the PDSA model to help identify ways the leaders and stakeholders might perform the same study in a better way. B.2 For the qualitative analysis, describe any analysis that identifies causes for less than desired performance (barrier/causal analysis) and include the following: Techniques and data (if used) in the analysis Expertise (e.g., titles; knowledge of subject matter) of the work group or committees conducting the analysis Citations from literature identifying barriers (if any) Barriers/opportunities identified through the analysis Impact of interventions
Section IV: Interventions Table QIA Instructions and Form 10 Interventions Taken for Improvement as a Result of Analysis. List chronologically the interventions that have had the most impact on improving the measure. Describe only the interventions and provide quantitative details whenever possible (e.g., hired 4 UM nurses as opposed to hired UM nurses ). Do not include intervention planning activities. Date Implemented (MM / YY) Check if Ongoing Interventions Barriers That Interventions Address This will consist of all of the steps that went into implementing the change. Common interventions include education and training, new checklists, announcements, modifications in the electronic health record or other documentation systems, etc. Include all interventions that were implemented after the initial analysis and after the remeasurements. As an example, education and training interventions might address the barrier of a knowledge deficit in a particular area.
QIA Instructions and Form 11 Section V: Chart or Graph (Optional) Attach a chart or graph for any activity having more than two measurement periods that shows the relationship between the timing of the intervention (cause) and the result of the remeasurements (effect). Present one graph for each measure unless the measures are closely correlated, such as average speed of answer and call abandonment rate. Control charts are not required, but are helpful in demonstrating the stability of the measure over time or after the implementation. Chart and graphs are great tools that can help explain the data and illustrate trends. We encourage you to complete this section, but (as indicated) this is optional.