N EWSLETTER. Volume Six - Number Three March Risk Managing Dashboard Reports

Transcription

1 N Volume Six - Number Three March 2010 Risk Managing Dashboard Reports The healthcare field has fallen in love with dashboards. From the board room to human resources dashboards rule! Some are bland in appearance, even stodgy and formal. This type of dashboard is often portrayed as a scorecard and is used during performance appraisal season. Other dashboards are much more stylistic in design. Replete with colorful charts, graphics, and imagery, the intent is to wow the end user while providing essential information in a succinct format. The more colorful dashboards are often geared to the C Suite, providing systems-level data on performance, process, revenue and patient census. There is no doubt that in the harried healthcare field, dashboard reports can provide critical information in a sharp-looking, focused format. The content can help synthesize a variety of data needed for making difficult decisions. But is the old saying true, that a picture can be worth a thousand words? Can the stylish dashboard replace the need for reading spreadsheets, reports, and forecasts? There are other factors to consider in using dashboards that merit a risk management review. For example, is there a process in place to validate the accuracy of information used to populate the dashboard? Will the dashboard include data from outside sources for benchmarking? Is it a static or dynamic dashboard? Will the dashboard be homegrown or will the healthcare organization contract with a third party to slice and dice data to generate the information? What will be the uses of the dashboards? To what extent are dashboards subject to evidentiary protection? Could the dashboards be used as evidence in litigation? ALL RIGHTS RESERVED 2010 PAGE 1

2 The answers to these questions demonstrate that the wow factor with dashboard reports must be balanced carefully, making certain that the message does not get lost in the medium. In other words, dashboard reports are a useful tool, but not a replacement for other documents and communications that go to the critical success of a healthcare organization. Key Risk Management Concerns in Dashboard Reporting. There are a number of practical considerations to contemplate prior to developing or using dashboard reports. Consider some key issues that have come to light about dashboard reports: Data Accuracy A common phrase in the information management and software world is Garbage In, Garbage Out. Many dashboards depend on data from spreadsheet or relational database software to populate the stylish graphs, bar charts, and performance gauges in the report. A fundamental question goes to the accuracy and validity of the data. How old is the information? Was it collected in a consistent manner? Is there inherent consistency among the data samples used in the dashboard reports? For example, if the algorithm requires that the data is risk adjusted, has anyone confirmed that this process has been completed? Has the source of the data for a display been confirmed it is possible that the numbers are coming from an incorrectly assigned database. In some situations, relying on templates without verifying their accuracy can be risky the facility data displayed may be from a sample provided with the software to demonstrate its capabilities. Ensuring the use of correct units is also important. Liters to Gallons, kilograms to pounds, fiscal year versus calendar year. The software may use calculations for one type of unit, and the displays and databases may use another set of units. That the conversions are executed correctly must be tested both when deploying the dashboard system and after any updates to the software are performed. From a risk management perspective these are important considerations. The C Suite is apt to rely upon the dashboard data in making important decisions. Performance reviews may turn on the inferences drawn from a balanced scorecard. One can anticipate situations in which decisions may be based on inaccurate or stale data. The same can be said of data pools that were too small ALL RIGHTS RESERVED 2010 PAGE 2

3 to determine statistical significance or use of raw data that has not been riskadjusted as is required by protocol. Benchmarking Data Many healthcare organization leaders what to avail themselves of benchmarking with other facilities or entities. How do we stack up with the other ambulatory surgery center in the region? Is our volume as good? Where are we in the region in terms of patient falls? Do we continue to dominate in the area in terms of low UTI scores? The answers to such questions may be part of a comparative benchmark chart in a performance dashboard report. In other instances the benchmarking may be within the confines of a healthcare organization. How are we doing in year-over-year readmissions within 48 hours of discharge? Have we maintained the benchmark for nursing on recertification for advance life support? Benchmarking data may be used to drive system change, set priorities for resource allocation and staffing. Absent safeguards to verify that comparable data is used for benchmarking purposes, the dashboard may present a flawed portrait of important information. The result may be imprudent choices by the healthcare facility board and senior management team. A Static or Dynamic Dashboard The timeliness of critical information plays a pivotal role in prudent choice-making. A multipurpose dashboard report may contain components of static data and dynamic information. But how dynamic is the information? Are there rules or criteria for eliminating inclusion of data that is six months old? Three months old? Two months old? What is the cut off date for calculating year-over-year benchmark comparisons in the dashboard report? The answers to these questions are important. The risk of stale or incorrect information being used in a dynamic dashboard chart can cast doubt of the accuracy of the entire report and more importantly, call into question decisions based on the information. Homegrown or Proprietary Dashboard Reports Those accustom to working with spreadsheets and relational databases may utilize templates packaged with the software to populate homegrown dashboard reports. Others may decide to outsource dashboard reporting, contracting with a firm to slice and dice the ALL RIGHTS RESERVED 2010 PAGE 3

4 data into visually appealing formats. The dashboards are updated with new data inputs. Senior management, the board, and performance improvement committee members may see the colorful dashboards displayed on a website or in a downloaded report. There are risk management issues that arise, especially with respect to proprietary dashboard reports. For example, what safeguards are in place to secure the data? What controls are in place on the outsource vendor modifying the algorithms used in converting data into graphics or charts? In other words, if the board was to see comparable reports over the course of three quarters, would they know whether the calculations were consistent? That a change in calculation format is indecipherable to the end user may have serious consequences, including making the wrong choices for the healthcare organization. Uses of Dashboard Information Smart-looking dashboard reports can have multiple uses. Not restricted to the C Suite, the dashboard style report can be used in patient education and marketing. When posted on a healthcare facility website, the dashboard style report may be used to reflect competitive data for healthcare organizations. This may include information on patient satisfaction, quality scores, and other details. But how accurate is the comparative data? Could the dashboard create misunderstanding or worst yet be seen as misrepresenting important information? Might there be a possibility of regulatory concerns, including complaints from state consumer authorities or the Federal Trade Commission? Unrestricted dashboard report activity is a key risk management concern. Vaunted expectations, claims, or reliance on outdated information for marketing purposes may delude the benefits of dashboard reporting, especially when such data is generated for the publicʼs use. Reliability and security is also a concern. If an outside dashboard vendor creates the benchmarks based on comparable facilities in the region, how are they acquiring that data? They may be using a combination of their other clientʼs data and public data to produce the benchmark. If a facility is not a client of the outside vendor, the data may be incomparable to that taken directly from their own clients dashboard systems. Furthermore, if there are a limited number of facilities in a region, or the services are specialized, it may be simple for the public or competitors to determine whose data is being used for benchmarking. ALL RIGHTS RESERVED 2010 PAGE 4

5 Therefore, any sort of agreement with an outside dashboard vendor to allow anonymous reporting should be considered carefully. Evidentiary Protection Dashboard reporting can be used to facilitate better outcomes, performance efficiencies and patient safety. When generated in accordance with the terms of applicable state laws for quality improvement or assurance, the dashboards may enjoy some level of evidentiary protection. However, when used for purposes not included under the aegis of evidentiary protection, the dashboards may be subject to discovery and used as evidence in litigation. Risk management has longstanding concerns about protected uses of a variety of quality improvement information. Dashboard reports are not any different in this regard. Depending upon the design and content, the information may prove challenging especially if the report is shared beyond the scope of applicable evidentiary protection. Litigation and Dashboard Reports Like other documentation in the healthcare field, dashboard reports may be powerful in litigation. For example, if the report demonstrated that a hospital was aware of a serious infection control problem and that the issue had persisted for several months, it could be argued that the facility knew it was not meeting acceptable standards. The report may go further, demonstrating that the only intervention was to monitor the problem. Such information portrayed on a dashboard report could be a potent piece of evidence in a negligence claim. In other instances, a dashboard report may provide fodder for a health plan that wants to terminate a provider contract for performance and quality issues. To the extent that quality indicators and outcomes reflect substandard performance, the dashboard may serve as evidence to substantiate that the facility was in breach of the terms of the contract. Risk management and claims management would like to see dashboards designed in a manner that the content fulfills the need of providing leadership with salient information for decision-making purposes. Formats that assist in litigation against the healthcare organization merit scrutiny as do the terms of use of the dashboard reports. If the documents put leadership on notice of systemic problems or clinical care issues, the information should serve as a trigger for intervention to address these concerns. ALL RIGHTS RESERVED 2010 PAGE 5

6 Dashboard Reports in Perspective. Recognizing the risk management concerns about dashboard reports, one may well ask, should healthcare facilities continue to use such documents? Stated differently, do the risks outweigh the benefits of dashboard reports? Dashboard reporting can serve as a practical communication vehicle for healthcare organizations. With the onslaught of data impacting healthcare facilities and increased responsibilities for reporting to government, health plans, and other key constituents, dashboard reporting represents a viable way to meet such requirements. Moreover, as healthcare organizations embrace enterprise risk management, the dashboard reporting format represents an attractive way to depict relevant information impacting all the domains of risk. Risk Management Strategies for Dashboard Reporting. Evaluating the benefits and risks of dashboard reporting, there is good reason to use such tools in the healthcare field. However, in recognizing the potential risks, it useful to consider practical strategies to address such concerns. Working together with key stakeholders as an enterprise risk management team, it is possible to develop dashboard strategies for the entire healthcare enterprise. These strategies include the following: 1. Develop Protocol for Accepted Use of Dashboard Reports. Identify the potential uses for dashboard reports for the organization. From the list, ask the ERM team to select approved uses for dashboard reports. 2. Develop Approved Framework for Dashboard Design. Think about the types of information to be displayed in the dashboard report. Recognize that it may be necessary to have more than one format or framework. What will display well on a website may not be as clear on a print-out. Dashboards for a public website may contain different types of graphics and information than those used within the healthcare organization. Considering approved uses for dashboard reports, make certain that approved designs accommodate data requirements. Take into consideration too, ergonomics, and scalable fonts and graphics for end users with visual impairments. Think about field-testing template designs to identify areas for improvement prior to final approved use of the dashboards. ALL RIGHTS RESERVED 2010 PAGE 6

7 3. Implement Dashboard Information Content Requirements. Incorporate into the approved dashboard design protocol specifications for data and textual information. Consider issues such as use of abbreviations, italics, and underscores as well as use of excerpted text. Should a dashboard report include data or other information from an outside source, make certain that there is written permission to reprint including in the context of a dashboard report. 4. Establish Parameters on Accuracy of Data. Use recognized methods for validating the accuracy of data that forms the basis for graphic displays in a dashboard report. Develop some practical guidelines on such issues as sample size and risk adjusting data for this purpose. Think about what criteria should be used for validating the accuracy of benchmarking information in the dashboard reporting, especially comparable or comparative data. Include in a footnote or endnote the assumption that the data is believed to be accurate and that the dashboard is based on generally accepted principles for collecting and analyzing information in the creation of graphic displays. 5. Establish Parameters on Timeliness of Data. Recognize that dated information may be misleading. As such, incorporate into the guidelines the shelf life of data for use in the dashboard report. Recognizing that the data may be subject to frequent change. Consider including a disclaimer that the dashboard includes dynamic information that is subject to change and that the end user should review up-to-date information when making business or clinical service decisions. 6. Establish Contractual Requirements for Use of Third Party Dashboard Resources. Develop written agreements with third parties who develop or design dashboard reports. Consider such items as data reliability, consistent use of agreed-upon algorithms and formulas to convert the data into approved graphics, and timeframes for use of such information. Think about re-use of data by the third party for purposes of their own marketing and advertising. Build into the agreement a requirement for written authorization on the release or use of the data to other groups that want to use it for benchmarking. Work with legal counsel on designing acceptable terms for the agreement. ALL RIGHTS RESERVED 2010 PAGE 7

8 7. Provide Education on How to Use Dashboard Reports. Think about practical orientation and in-service education on How to Use a Dashboard Report. Recognize that the program content may vary for the end-user group. 8. Work with Legal Counsel on Evidentiary Protection Requirements for Dashboard Reports. Discuss with legal counsel what types of evidentiary protections may be applicable to dashboard reports. Ask counsel about the steps to follow to maintain evidentiary protection and incorporate this information in the orientation and in-service education for personnel. 9. Close the Loop on Outlier Dashboard Report Information. Incorporate into the protocol for using dashboard reports measures for acting on outlier information. Measures used should follow a consistent process, including validation of the information, action plan to address outlier data, and reporting back on the outcomes of the corrective action. Build into the protocol requirements for documenting recognition of and action on outlier data. 10. Monitor Use of Dashboard Reports. Incorporate a method for ongoing monitoring and use of dashboard reports. Identify unacceptable practices such as sharing protected information in an unauthorized manner or re-use of information without written authorization. Follow established practices to address unacceptable use of dashboard information. Recognize that the response may be disciplinary action, reprimands, or in the case of a third party, termination of a contract. Conclusion. Dashboard reports are more than eye-catching graphic displays. In todayʼs busy healthcare setting, dashboard reports serve a useful purpose in data analytics. Dashboard reports may also help guide or inform important decisions in strategic planning, opening and closing service lines, and making critical decisions involving staffing. Dashboard reports merit careful development and use. Education for end users is as important as establishing criteria and protocols on incorporating external ALL RIGHTS RESERVED 2010 PAGE 8

9 data for benchmarking information in such report. Should dashboard report information help identify service sector or quality of care issues, the end user should understand what steps to follow to resolve such matters. Documentation of action steps is important when closing the loop on outlier data in a dashboard report. If you would like assistance with developing an enterprise risk management program, please contact us at (860) ALL RIGHTS RESERVED 2010 PAGE 9