Welcome to the Orientation to the Office of Developmental Programs - ODP. This lesson is Part 2 of a two-part webcast focused on Using Information and Tools for Quality Management. The information presented here was developed and reviewed by ODP leadership for you to build your knowledge in your important role as a member of the ODP team. Future lessons in this series will be made available, providing an overview of information related to ODP that focuses on core philosophies, operational components, and business practices. Course No: 004-02-02 rev 4-15-16 1
Please adjust your computer volume so that it is at a comfortable listening level for you. While viewing this webcast, there is a pause and rewind button that can be used throughout the presentation. Course No: 004-02-02 rev 4-15-16 2
Hello, my name is Ann Ligi, a Quality Management Lead for ODP. I provide technical assistance and support for quality management practices implementation and provide a public health based approach for person centered risk management training and technical assistance. I will be presenting the first section of this webcast. Later in the presentation, John Sloyer will be narrating. John is the Planning, Evaluation and Development Director for Selinsgrove Center. His role consists of directing Selinsgrove Center s Quality Management Program and supervising the Center s Quality Management Division in the Bureau of Supports for People with Intellectual Disabilities. Course No: 004-02-02 rev 4-15-16 3
The objectives for Module 102 Part 2 include learning how to: Analyze data Display data, and Report data Let s talk first about analyzing the data. Course No: 004-02-02 rev 4-15-16 4
You will want to make sure to review and correct the data as needed before you begin your analysis. This will enable you to be confident in the results, as well as saving you time and effort if an error is discovered later in the process. But also make sure you leave enough time for the analysis itself. And, keep the analysis simple. Please note that you do not need to apply data analysis to all data. Some data simply need to be tracked so that it is available to provide to regulatory agencies and others upon request. Other data need to be monitored to inform decision making and to ensure good performance is sustained. In this way, the organization is able to quickly pick up and proactively address any undesirable variations. Course No: 004-02-02 rev 4-15-16 5
The goal of data analysis is to highlight useful information, reach conclusions, and support decision making. During this process, you must make important choices about which information should be emphasized, minimized, or even left out of the analysis. Information you have collected that contains numbers is quantitative data. If the information you collected consists of words, then it is qualitative data. It is important to remain focused on the questions you are trying to answer and the relevance of the information to these questions. Course No: 004-02-02 rev 4-15-16 6
There are a variety of quality management tools that can be used to analyze and display data. These tools include: Tables Pie Charts Bar Charts Pareto Charts, and Line Charts Course No: 004-02-02 rev 4-15-16 7
In this slide, we see an example of data being displayed in table form. The categories in this data table include types of incidents in the Consolidated Waiver for Fiscal Years 2010/2011 and 2011/2012. A data table helps you organize all your information in one place and is often used to create lists of items with categories. Course No: 004-02-02 rev 4-15-16 8
An example of a Pie Chart is shown on this slide. This Pie Chart displays a provider s percent of medication errors by omission for each quarter. Analysis shows a significantly higher percentage of medication errors in the 3 rd quarter. This enables the organization to focus on what might have been different in the 3 rd quarter and take action accordingly. Pie charts provide the advantage of functioning as a visual aid to help look at statistical information and examine and interpret the data. Using different colors, pie charts divide information into sections resembling pie slices. Each slice is accompanied by a percentage and the size of the slice changes accordingly. Pie charts use percentages to illustrate quantities. The effectiveness of pie charts for examining percentages lies in the viewer s immediately understanding what you intend to communicate. Course No: 004-02-02 rev 4-15-16 9
An example of a Bar Chart is shown on this slide. This Bar Chart displays the number of medication errors of all types for each month for the 123 Main Street Site. Analysis of this monthly data shows relatively little fluctuation; however, it would be helpful to examine the data more closely, looking at additional factors regarding specific types of medication errors, to better identify opportunities for improvement. Bar Charts show a record in column form so, over a period of time, comparisons of the recorded information can be clear to see. Bar Charts: Summarize a large data set in visual form Clarify trends better than do tables Estimate key values at a glance Can be easily understood, and Provide information for predicting patterns Course No: 004-02-02 rev 4-15-16 10
This slide shows an example of a Pareto Chart. This Pareto Chart displays types of abuse report allegations for ABC Provider in 2012. Analysis shows that physical abuse allegations were the most prevalent during this reporting period. Based on this information, the organization should focus its resources on further examination of circumstances surrounding these reported allegations of physical abuse. This chart reflects the proven Pareto principle: 20% of the sources cause 80% of the problem. Key applications of the Pareto Chart also include: Studying different problems or variables Arranging data into categories Ranking each category by importance, and Looking for the vital few causes, not the trivial many Course No: 004-02-02 rev 4-15-16 11
An example of a Line Chart is shown on this page. This chart displays the number of falls by month at Provider ABC for an entire year. Line Charts, also known as Line Graphs, display information as a series of data points connected by straight line segments. It is a basic type of chart common in many fields. A line chart is often used to visualize a trend in data over intervals of time. Thus, the line is often drawn chronologically, as in the case above. Course No: 004-02-02 rev 4-15-16 12
Regardless of the type of tool you choose to use, one of the important aspects of data analysis is looking for patterns and trends over a period of time. When analyzing data for trends and patterns, be sure you have enough data points to review. For instance, include at least 12 months of data whenever possible. In the above example, we see the results of tracking three Providers relative to Confirmed Neglect Reports each month. The overall trend shows a decrease over the course of the year. Trending data also helps us to understand variation in the process being measured. To understand what variation means, we need to first recognize that there is always variation in data. Take, for instance, something as simple as the daily temperature or as complex as the success of a surgical procedure. However, variation can be viewed in two different ways: Either as an indication that something has changed over time, as detected through trending of data, or As random variation that does not mean a change has occurred. Course No: 004-02-02 rev 4-15-16 13
That is why understanding the nature of the variation is paramount in decision making about improvement efforts. To further explain variation, let s discuss two types of variation in patterns: Common Cause Variation occurs in every process. A process with common cause variation is stable and predictable. An example of common cause variation could be noting the time it takes you to drive to work everyday. Your daily trip takes the same amount of time, give or take a few minutes. Special Cause Variation is not part of the usual process. The variation is due to special circumstances. This type of variation in the process is not stable and cannot be predicted. The variation can have a positive or negative effect on a process. Understanding which effect the variation has on a process will help an organization determine the direction it needs to proceed. Using our same scenario, your typical daily trip to work is extended by 30 minutes one day because you came upon an accident. It s best to avoid a knee jerk reaction to the identification of a special cause. In our example, for instance, changing the route you drive to work because an accident occurred on one day is not warranted, and may result in an adverse outcome such as extending your drive to work time overall. A more in depth discussion of common and special cause variation will be addressed in a subsequent training module. Course No: 004-02-02 rev 4-15-16 14
We tend to spend most of our time developing a measurement, collecting, displaying, and analyzing data, but spend little time in disseminating the results of the evaluation to the appropriate audiences. Even the most rigorous data analysis can lose its impact if presented or reported in an ineffective manner. Let s turn this over to John Sloyer who will discuss ways to effectively present data. Course No: 004-02-02 rev 4-15-16 15
Thank you, Ann. Good data analysis reporting requires an appropriate format that uses effective visuals and communicates findings in clear, concise language. Data analysis results should be organized in such a way that the audience is able to find the most important information easily. This means presenting findings first and fleshing out details in the body of the report. During the data analysis process, you may conclude that progress has been made in achieving an objective or that you are not on track because of identified barriers. Based on this information, follow up activities should be described in the report about how you plan to sustain achievement and continue progress or what will be done to overcome the identified barriers. Course No: 004-02-02 rev 4-15-16 16
Let s look at an example of a data report template. In the example, you see that relevant information is listed in the top sections of the report. What we would like to focus on, however, are the three columns labeled Performance Measure and Data Source, Findings and Analysis, and Plan Do Check Act (or PDCA) Cycle Follow up. The performance measure and data source should be listed in the first column. The Findings and Analysis column should incorporate the data and data analysis. In this section, you would display available data using an appropriate tool such as a bar chart or line graph. Beneath the data display should be a written explanation about what the data mean. Are you on track to reach your objective? If not, why not? Have you identified barriers in achieving your objective? And finally, the PDCA column should describe follow up activities about how you plan to sustain achievement and continue progress or how barriers, if present, will be addressed. As you may recall from training Module 101, PDCA is the four step model used by many organizations to carry out change. Just as a circle has no end, the PDCA cycle should be repeated again and again for continuous improvement. A data report can be formatted in various ways but should always, at a minimum, include the components described above. Course No: 004-02-02 rev 4-15-16 17
On this slide is an actual quality management report developed by ODP s State Centers Employment Workgroup. The report has been consolidated and abbreviated for space considerations but, as you can see, it includes the important components of Performance Measure, Findings and Analysis, and PDCA Cycle Follow up. The Performance Measure section indicates that what we want to look at is the number of people who have paid work. This is a Quality Improvement (QI) Measure, and the objective is to increase the number of people who have paid work by 1%. In this case, the target for the workgroup is 305 people who have paid work. The Findings and Analysis section shows the data collected since the start of the Fiscal Year, including the 2 nd Quarter which is the focus of this report. The data are displayed in a table; however, it could also be displayed in a Line Chart, Bar Chart, etc., for different visual effects. Under the Table is an explanation of the data. In this case, the objective was not achieved and the average for the quarter was below the target. The reasons are provided. The PDCA Cycle Follow up section states what will be done to address the situation. In this case, vocational interests will be examined at 90 day intervals as well as annually. This format assists the reporter in organizing and displaying the data and results clearly and concisely, and provides the audience with a compact visual aid for understanding the contents. Course No: 004-02-02 rev 4-15-16 18
Another way an organization can communicate data results is through use of a Balanced Scorecard. ODP has adopted this method. A Balanced Scorecard is a Performance Measurement System, a Strategic Management System, and a Communication tool all wrapped up in one. Balanced Scorecards are used to help the organization achieve alignment that is, all components of the organization working at the same time on the goals and objectives, guided by the same Mission, Vision, Values, and organizational strategy. Course No: 004-02-02 rev 4-15-16 19
A balanced scorecard is a management system popularized by Dr. Robert Kaplan and Dr. David Norton in the 1990s. At the time, most management systems focused almost exclusively on financial measures. The balanced scorecard added additional non financial perspectives (learning and growth, business process, and customer satisfaction) in addition to the financial perspective, creating a more balanced view of a business, and stressing the need to monitor strategy by actual performance. Today, balanced scorecards are used for strategic planning and management by business, industry, government, and non profit entities. In addition to management, they facilitate strategic planning, improve communications internally and externally, and help align an organization s vision and goals to its strategies and performance. Course No: 004-02-02 rev 4-15-16 20
In addition to other ways of communicating data results, ODP has implemented use of Balanced Scorecards for system wide reporting and reporting at each of its Bureau levels. Course No: 004-02-02 rev 4-15-16 21
ODP s Bureau Directors are tasked with the responsibility to develop Bureau level Balanced Scorecards by determining how these scorecards will: Contribute to the development of and support ODP s Mission, Vision, and Values Contribute to the development of and support ODP s Strategy and Strategy Map Build ODP s Office wide Balanced Scorecard, and Cascade ODP s Office wide Balanced Scorecard into Bureau/Division level scorecards. Course No: 004-02-02 rev 4-15-16 22
The first step in building ODP s system wide Balanced Scorecard was to revisit and modify its mission and vision statements. Doing so provided ODP with a platform from which to springboard its strategic planning. A mission statement answers the questions: who are we and what do we do? A vision statement describes the future we hope for. Course No: 004-02-02 rev 4-15-16 23
Next, ODP accomplished the task of identifying its strategic themes. Strategic themes establish broad priorities and desired results. Themes are essential strategic elements that form the foundation for a balanced scorecard. They provide structure, support, and often boundaries for a strategic balanced scorecard. Course No: 004-02-02 rev 4-15-16 24
ODP has identified three strategic themes for its system wide Balanced Scorecard. They include: Deliver Quality Outcomes/Services Ensure Program Integrity and Accountability, and Foster Innovation Course No: 004-02-02 rev 4-15-16 25
Before determining high level, or strategic objectives that align with the identified strategic themes, an organization should review each of these perspective areas and ask themselves relevant questions. Course No: 004-02-02 rev 4-15-16 26
Strategic objectives that help an organization answer these relevant questions can then be formulated. Strategic objectives chosen should directly correlate with identified strategic themes. On this slide, for instance, we can see that ODP identified strategic objectives that directly correlate to the Strategic Theme of delivering quality outcomes and services. These strategic objectives include: Conduct accurate, person centered needs assessments Increase access to services for Pennsylvanians with developmental disabilities Build a qualified provider network, and Ensure safe, inclusive environments Course No: 004-02-02 rev 4-15-16 27
The next step involves development of strategic initiatives. Strategic initiatives support both strategic themes and strategic objectives and are the action projects that are needed to help the organization be successful with its overall strategy. Some of the strategic initiatives ODP has developed to support its strategic theme and strategic objectives related to delivery of quality outcomes and services include: Develop and deliver training to ensure quality outcomes and services Manage enrollment for individuals waiting for service, and Reduce restrictive interventions Course No: 004-02-02 rev 4-15-16 28
Once strategic themes, objectives, and initiatives have been identified, an organization continues to build its Balanced Scorecard by developing performance measures, keeping in mind each of the four perspective areas: Individual/Stakeholder Internal Processes Financial Learning & Growth While developing performance measures, it is necessary to determine targets for each measure. Targets should indicate a specific numerical value: For instance a percent, count, or dollar amount. Target setting will give everyone in the organization a clear sense of what the organization is aiming for. Without targets, an organization cannot know if it is reaching its goals. Course No: 004-02-02 rev 4-15-16 29
For instance, in order to measure and track its success in reducing the use of restrictive interventions, ODP has identified some of the following key performance measures: Number of restraints Number of individuals restrained Number of applications of physical, chemical, or mechanical restraint for behavior support/crisis management Number of crisis incident reports indicating use of restraint, including improper or unauthorized use of restraint ODP s Bureaus, including Community Services, State Centers, and Autism Services, support the elimination of restraints as a behavior modification practice. In order to achieve this goal, targets have been established as follows: Community Services targeted a 20% reduction in the number of restraints and the number of individuals restrained in FY 2011 12 from previous FY 2010 11 figures. FY 2011 12 year end data indicate a 12% decrease in the number of restraints and a 9% increase in the number of individuals restrained. Analysis suggests that more work needs to be done to achieve ODP s desired outcome. State Centers and Autism Services In FY 2011 12, there was a total of 8 restraints and 1 individual restraint reported. Both ODP Bureaus continue to work toward a target of zero restraints. Course No: 004-02-02 rev 4-15-16 30
The process just described would be followed to identify strategic objectives, strategic initiatives, and key performance measures within each of an organization s identified strategic theme areas. Course No: 004-02-02 rev 4-15-16 31
So far, we have talked about the various components of a Balanced Scorecard, including strategic themes, objectives, initiatives, and performance measures that reflect different perspective areas. To get a sense of how to build a Balanced Scorecard in which all of our identified components will fit, we have included on this slide an example of ODP s Office wide Balanced Scorecard. The information in this example was condensed for space purposes and is not all inclusive but it gives you a good idea about what a Balanced Scorecard actually looks like. From the top down, you see identified strategic themes (such as Delivering Quality Outcomes/Services) and perspective areas (such as Individual/Stakeholder). Information found in the oval shapes represent strategic objectives. Information in the rectangular shapes indicate strategic initiatives related to the perspective areas that support the strategic objectives and strategic themes. What we are essentially looking at is a map. Through use of arrows, the map highlights that delivering the right performance in one perspective area can only be achieved when performance in other perspective areas is also achieved. In other words, the map contains interlinked strategic themes, objectives, and initiatives in a balanced way. The color coding in the diagram represents status. So, for instance, information color coded in orange might indicate that an identified objective is active or has been implemented. Blue might mean that an initiative has been proposed but not yet implemented. Color coding will vary from organization to organization. Course No: 004-02-02 rev 4-15-16 32
In this slide we are demonstrating, in a very condensed version, the section of ODP s Officewide Balanced Scorecard that lists key performance measures, in each of the perspective areas. Looking at each key performance measure, it is easy to tell in which perspective area it belongs. For instance, the key performance measure that reads, percent expended to date vs. enacted budget, is a measure linked to the Financial perspective area. We spoke earlier about setting targets for performance. Information found in the third column of this diagram demonstrates some of ODP s targets, such as the target to ensure ISPs reflect individuals assessed needs 100% of the time. The remaining columns reflect available quarterly data for each performance measure. Course No: 004-02-02 rev 4-15-16 33
Ongoing monitoring of Balanced Scorecard information should occur to determine whether current performance in key identified areas meets expectations. Monitoring is a team effort in which leadership must: Work with staff to develop and submit local performance measures, targets, and data profiles Communicate with and educate staff about ODP s Office wide Scorecard and Bureau/Division level Scorecards initially and ongoing Analyze and report progress in meeting objectives routinely Work with and support staff to implement relevant initiatives, when performance deviates from expectations, to trigger improvement within the part of the organization they lead Course No: 004-02-02 rev 4-15-16 34
This concludes Part 2 of the ODP Orientation lesson Using Information and Tools for Quality Management. Both Parts 1 and 2 are second in our series of quality management modules for certification. The series is designed to serve as a resource for leaders in ODP and the service delivery system. The understanding and application of quality management for ODP and subsequently, Certified Quality Managers, are essential to comply with regulations, assure individual health and safety, and promote the use of positive practices so that choice and control enable individuals to lead the lives they desire. Now that you have viewed this presentation, we suggest that you meet with your supervisor to discuss in more detail the information provided in this webcast and your roles and responsibilities for quality management within your position. Course No: 004-02-02 rev 4-15-16 35