ROOT CAUSE ANALYSIS AND THE AUDIT LIFE CYCLE

Transcription

1 1 ROOT CAUSE ANALYSIS AND THE AUDIT LIFE CYCLE DEREK BUTLER SENIOR MANAGER GRANT THORNTON LLP AHIA 35th Annual Conference September 11-14,

2 Agenda Define what is Root Cause Analysis and discuss some key features Identify common uses of Root Cause Analysis Identify tools and techniques for conducting Root Cause Analysis Highlight benefits and challenges associated with conducting Root Cause Analysis 2

3 What is Root Cause Analysis? Root Cause Analysis - Definitions and Terms Root Cause Analysis (RCA) is a: Systematic, formalized approach used for reviewing an adverse event and identifying root causes Tool designed to help identify not only what and how an event occurred, but also why it happened using a structured format A process for analyzing the possible causes to a problem and related effect(s) It not only aims to identify the true cause of a problem, but the actions necessary to eliminate it and prevent recurrence Root Cause Analysis is also referred to as: Ishikawa diagrams Fishbone diagrams Cause-and-effect diagrams 3

4 What is Root Cause Analysis? Root Cause Analysis - Point of View When done correctly, a RCA is a powerful tool that deserves a prominent place in your problem-solving toolkit as internal auditors. Provides fact-based statements that describe the deepest underlying cause, or causes, of operational or performance challenges Force us to focus on things that can be changed and need to change. Efforts are redirect from areas where you have no control over (e.g., weather) Provides a healthy forum for exploring multiple possibilities. It is a tool that allows multiple points of views and voices to be integrated into the conversation and our thoughts for consideration Requires that a consensus to be reached around the factors that lead to the problem so that effective correction action plans can be developed 4

5 What is Root Cause Analysis? Root Cause Analysis - Point of View What I ve observed and learned over the years: RCA is sometimes not used appropriately (or at the right time) Solution-centered mindsets renders the RCA irrelevant Potential root causes are not validated Not used as intended and ends up as merely interesting artwork in a presentation to management RCA are sometimes a rushed production versus a methodical and thoughtful approach to finding potential and ultimately the root cause. Lack a focused problem statement which produces a whale versus a simple fish 5

6 When is Root Cause Analysis not the best tool? Root Cause Analysis - Sample Problem Descriptions To use or not to use - that is the question... When a RCA is not the best tool: The company wants to know if the number of emergency room complaints have increased over the last two years. A hospital is receiving numerous complaints from patients that are being admitted and who are upset because the admission process was cumbersome and required patients to endure a lengthy process. To better understand what is going on, the hospital wants to investigate the time it takes to admit a patient and what opportunities exist to streamline the process. Given the size and nature of the operations for many healthcare systems and the degree of sensitivity that now exists around patient access and information, managing the physical access and providing the appropriate degree of security to ensure a safe environment for patients, visitors, employees, and volunteer staff continues to be a major issue. As a result, the company is interested in benchmarking its security protocols and standards that are currently in place against those that are deemed to be leading practices and more widely used in the healthcare industry today. No focused problem statement (yet). 6

7 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements A problem well defined is a problem half solved 7

8 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements Understanding the severity of the problem is a key component to conducting an effective RCA. There are several standards that must be utilized when developing and writing an appropriate problem description. The fundamental elements to defining a problem addresses: 3W: What was affected? When was the problem discovered? Why did the problem happen? 2H: How much was affected? How often has the problem occurred? 1C: What is the consequence? Less focus on "where the problem took place" and "who discovered it" 8

9 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements Developing a complete problem description statements requires that we focus on what, when, where, and the how aspects of a problem. Ask What, How, and Why? Determine Root Cause What Happened? How it Happened? Why It Happened? Drive & Anchor the Improvement Trace the problem to its origins Categorize the causes of events (causal factor, direct cause, contributing cause, root cause) 9

10 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements There are often several causes to a problem: Causal Factor a condition or event that results in an effect Direct Cause cause that directly resulted in the occurrence Contributing Cause a cause that contributed to the occurrence, but by itself would not have caused the occurrence Root Cause cause that if corrected, would prevent recurrence of a nonconformity and similar occurrences both within the department and more widely across the organization Understanding not only the problem, but the causes to the problem is key to ensuring you do not have a poor RCA. Don't assume the problem is known and understood well enough to determine the root cause without understanding the underlying causes. Examples of traditional root causes tend to focus on lack of a procedure, training of stakeholders in applicable policy/procedures, lack of an effective tool or system to perform a specific task, etc. 10

11 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements Common questions that can stimulate the thought process and get you on the track of identifying the various causes to a problem. Objective What are the risks? What could this problem impact? Determine Significance Implications / Impact So What? / Why Should I Care? Professional Judgment Getting to the Cause Questions to Consider Does results indicate a theme or pattern? Is there a larger business issue? How widespread is the issue? How material is the issue to financial results or to the organization future and strategic plans? How critical is the area? Is this mission critical or non-critical? Who is affected by this problem customers, employees, vendors, etc.? What is the reputational risk? Would this issue be embarrassing on the front page of the newspaper? On television news? Is there a regulatory or legal compliance concern? Could there be fraud or illegal act committed? Should This Observation Be Rated High, Moderate or Low Risk? Why? How does this compare with previous experience? 11

12 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements There are certain "Rules of the Road" to developing an appropriate problem statement that will allow for an effective RCA to be conducted. Write complete sentences: The problem description needs to be written clearly Stick to the facts. Don t make it a story. Be precise in defining the requirement and deficiency Avoid abbreviations, acronyms Problems / issues (Major, Minor) tend to violate some requirement or may result from the absence or insufficient performance of a process Include the violated requirement in the problem description Evaluate the sufficiency of problem description Identify What, Where, When, as appropriate There needs to be enough information from the problem statement to conduct RCA and identify corrective action 12

13 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements Seek to understand the severity of the problem Self-check the problem description Is the statement complete and self-sufficient? Ask yourself So What? 13

14 Root Cause Analysis - Types Process Capability Barrier Analysis 5 Why s Control Plans Pareto Charts Tree Diagrams Fishbone Process Maps Risk Tree Analysis Apollo Interrelationship Digraph Pie Charts Histograms Run Charts Change Analysis Process FMEA Flowcharts Brainstorming Affinity Diagrams Control Charts Design FMEA Events & Causal Analysis Scatter Diagram Force Field Analysis 14

15 Root Cause Analysis Overall Process Regardless as to the RCA model that is used, every option will require the same fundamental steps to be performed: Step 1 Problem identification Step 2 Data collection Step 3 Identify possible causal factors Step 4 Root Cause identification and validation Step 5 Corrective action recommendations and implementation Every problem is an opportunity. Every defect is a treasure, if the company can uncover its cause and work to prevent it across the corporation. Kilchiro Toyoda, founder of Toyota 15

16 Root Cause Analysis Data Collection 16

17 Root Cause Analysis - 5 Why's A more methodical approach to conducting a root cause analysis is to use the "Five Why" approach and diagramming One of the simplest tools Easy to perform without statistical analysis Determines the relationships between different root causes of a problem by asking 5 times the question Why? The tool allows one to get down to the underlying causes without being deceived by appearances Provides a test for logic when examining actions or problems Can be used in an informal setting to elicit the best responses from your audience 17

18 Root Cause Analysis - 5 Why's 18

19 Root Cause Analysis The Fishbone Diagram Fishbone diagram, also referred as Ishikawa diagram or cause-and-effect diagram, is a common tool used for a cause and effect analysis, where one tries to identify possible causes for a certain problem or event. 19

20 Root Cause Analysis The Fishbone Diagram The Supervisor is not performing a review of the information provided by the application owners. Employees Employees not following established recertification procedures Recertification procedures are not clear on what evidence is required Procedures are not communicated or easily accessible to the employees responsible for performing the recertification The system is not notifying the Supervisor that the review was completed. System The system is not saving the information submitted. The current training schedule for application owners does not include training on the application recertification review process A training covering the user re-certification process has not been developed. The re-certification process did not include supporting evidence for ten employees. Policy & Procedures Training 20

21 Root Cause Analysis Flowcharting Provides a tool for understanding and analyzing processes from start to finish Flowcharts illustrate the direct causes linked to their effects Breaks down process into individual events to display these in shorthand form showing the logical relationships Develops causal chains to aid in identifying root causes 21

22 Root Cause Analysis Flowcharting There are various forms of flowchart diagrams that can be used to conduct a RCA. Some of the more common and frequently used flowchart types are described below: Basic Flowchart is a simple diagram of a step-by-step execution of a process. This type of flowchart is very simple and provides a good visual for creating and understanding a process. Basic flowcharts are widely used in the different areas of knowledge or documenting processes. Business Process Modeling Diagram is a standardized graphical notation that allows the creation of a graphical view of the business processes in a workflow. This flowchart diagram was developed with the purpose of helping all business users to easily understand each other. It is a simple and visual tool. 22

23 Root Cause Analysis Flowcharting Cross Functional Flowchart is a one that is used to show the processes within a business model. This type of flowchart uses swim lanes. Cross functional flowcharts are very simple and useful, but demand a certain degree of detail. Data Flow Diagrams is one of the main tools for structural analysis and design of the information systems. Data flow diagram graphically represents how the information system work and develop, the interactions between the system and external data sources in terms of data flows, the data flows that the information system receives from the outside. 23

24 Root Cause Analysis Flowcharting Workflow Diagram is a graphical description of the information flows, the relationships between the processes of information processing and the objects that are the part of these processes. Workflow Diagram describes the stages of the business processes from the initial to the final stage that allows the organization to trace the progress of the processes quickly and clearly. Decision Tree Diagram is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and other aspects. Decision trees are commonly used in operations and research, specifically in decision analysis, to help identify a strategy most likely to reach a goal. 24

25 Root Cause Analysis and Components of Professional Skepticism Regardless of the RCA that is being used (5 Why's, Fishbone, or flowcharts), the relationship between the problem and possible causes must be clearly understood. Be prepared to use: Knowledge and reasoning Technical and auditing skills Industry knowledge and experience Objective and rational thinking Interpersonal skills Keep an open-mind Gain an understanding of how other people approach their jobs Communicate effectively with others, regardless of their position or power Be prepared to probe or question others Maintain composure if conflict arises Gut Instinct Sixth Sense Smell Test RCA helps you think beyond your obvious gut feelings to arrive at the true root cause(s). 25

26 Root Cause Analysis - Validating Root Causes Validating the root causes is essential to performing an effective RCA and to developing the appropriate correction actions. Fishbone Diagram Problem Statement: The IT re-certification process did not include supporting evidence for ten employees. Possible Root Causes Questions to Explore Data Sources Validation IT has implemented a new strategy for recertification purposes, but staff training has not occurred. What strategy is currently being used? When will staff receive training? Re-certification strategy/policy Training schedule Staff interviews New strategy effective 9/1/16 Staff training scheduled in 2017 Less time is being devoted to the recertification process due to staffing shortages. Are there open IT positions? How often does the recertification process occur? Staff interviews Listing of open job requisitions No evidence exists that less time is devoted to the recertification process. 26

27 What makes for a good Root Cause Analysis? Root Cause Analysis Things to Consider Ensuring that the appropriate data collection and analysis is performed Focusing on evidence-based risk factors and causes of the problem Involving business owners and subject matter experts who have an understanding of the problem Narrowing and prioritizing to eliminate explanations over which you have no control. Eliminate explanations that are: Not logical based on the data that has been gathered Opinion-based Not able to be tested or verified Incorporating the 5 Why's concept into the process in order to continuously drive to the root cause. Keep asking...why? at least 5 times!!!!! CONNECTING THE DOTS!! Determining whether the root cause fits the problem statement Validating the root cause 27

28 What are some benefits to conducting a Root Cause Analysis? Root Cause Analysis Benefits Saves time and money Provides internal audit departments with a logical approach to problem solving, using data that already exists It can be used as an approach to deal with any problem operational, compliance or financial Helps to identify underlying causes of problems, so that permanent solutions can be found Establishes a repeatable, step-by-step processes, in which one process can confirm the results of another Provides process owners with a long-term perspective for the improvement within the business and avoids encountering repeat findings Identifies systemic, rather than one-time solutions to problems Helps to ensure biases are minimized, assumptions are challenged, and evidence is fully evaluated before recommending corrective actions Helps develop a culture of collaboration across different departments or business units in order to find solutions from different perspectives 28

29 What are challenges to conducting a Root Cause Analysis? Root Cause Analysis Challenges Business Management Support Reluctant to support Internal Audit's role Chief audit executive and auditors may need to work with management to demonstrate the value Time and Cost Considerations Time and resource commitment may not be available Many focus on a short-term fix Long term fixes may not be readily identified if resources are not available Lack of Data/Subjectivity Determining root cause may be difficult even when significant data is available Need input from multiple data sources to get to the root cause (e.g., stakeholders, documentation, and system information) Internal Audit Specific Too many Internal Audit departments are often suspected of only recommend that management fix the issue and not the root cause 29

30 HAVE QUESTIONS? We've got nswers!

31 Save the Date August 27-30, th AHIA Annual Conference