Resource Brief. Consistent Decision Making by Regulators. Robert Chambers. Council on Licensure, Enforcement and Regulation

Transcription

1 Resource Brief Council on Licensure, Enforcement and Regulation Promoting Regulatory Excellence Consistent Decision Making by Regulators

2 Resource Brief CLEAR promotes regulatory excellence through conferences, educational programs, networking opportunities, publications, and research services for those involved with, or affected by, professional and occupational regulation. A neutral forum to encourage and provide for the sharing of best practices, CLEAR serves and supports the international regulatory community and its vital contribution to public protection. Published by Council on Licensure, Enforcement and Regulation (CLEAR) 403 Marquis Avenue, Suite 200 Lexington, KY Copyright 2009 CLEAR. Rights to copy and distribute this publication are hereby granted to members of the Council on Licensure, Enforcement and Regulation (CLEAR), providing credit is given to CLEAR and copies are not distributed for profit. Disclaimer: Each Resource Brief represents the opinions of the author(s) and is not to be considered an endorsement by CLEAR.

3 Consistent Decision Making by Regulators Regulatory organizations 1 are tasked with the responsibility of making decisions that impact not only the profession or industry they regulate but also the welfare of the public. As such, it is essential for a regulatory body to make consistent decisions based on a clear strategy and well-defined policy if it is to fully achieve its statutory objectives. Consistent decision making also has the advantages of greater effectiveness, improved efficiency and enhanced fairness to applicants, licensees and complainants. The goal of decision making processes is to guide and limit administrative discretion exercised by regulatory staff and committees and thereby focus the efforts of the regulator on preferred outcomes. This article is concerned with decisions where the use of judgment or discretion by the regulator is required. Not all decisions made by a regulatory body require judgment; many everyday decisions are routine such as automatically suspending a registration for failure to remit annual license fees. The purpose of this article is to inform management of regulatory bodies of the importance of consistent decision making and describe the techniques and tools available to achieve it. Causes of Inconsistency in Decision Making It is essential to understand the causes of inconsistency before designing processes to make improvements to the decision-making processes. Some of the major causes are: 1 Regulatory organization, regulatory body and regulator are used interchangeably in this article. 1

4 1) Insufficient focus on what is important. Many regulators manage to metrics that are commonly described in a business plan such as number of applications processed, number of complaints investigated, and number of citations issued. There is often too little attention paid to defining statutory objectives and determining what it takes to achieve these objectives. For example a health profession regulator may have as one of its statutory objectives improving the quality of health care for patients. The focus must then move to defining what specific improvements are required to achieve that objective and what changes may lead to the desired outcome. 2) Silos that exist within the regulator. Strategies are typically developed piecemeal by departments within a regulatory body without considering the big picture of how best to apply the resources of each department in order to control risks and share successes. This fragmented approach can result in the isolation of individual departments and disruption of the overall efforts of the regulatory organization in achieving its objectives. 3) Ineffective and out-of-date processes. Processes that evolved over many years often need to be rethought with the current end result in mind. Although decision making using these traditional processes can result in consistency between decisions, the decisions may no longer be consistent with the overall strategy of the organization. 2

5 4) Inadequate training. Effective training of staff and orientation of new committee members is critical to implementing and maintaining consistent results. 5) Overconfident licensee committees within self-regulatory organizations. A common feature of self-regulation is reliance on committees of licensees (the socalled self in self regulation) that perform functions such as assessing complaints and reviewing quality of service to the public. Too often appointees to these committees make decisions that are based on personal beliefs or are influenced by vocal members who can hijack the decision-making process. Licensee committees generally require the advice of professional staff advisors in order to keep to an unswerving course from case to case. In some professional regulators, staff may be required to exercise considerable diplomacy in guiding committee decisions lest members take offence. Indicators of Inconsistent Decision Making A manager of a regulatory organization must be alert to the warning signs of inconsistent decision making. The red flags include: a. Decisions based on similar fact sets differ from case to case; b. Decisions by a regulator cannot be predicted with confidence and often ignore precedent; c. Decisions are frequently appealed and there is a high rate of reversal; 3

6 d. Complaints of unfairness are made by licensees and complainants alike; e. Processes are not transparent to stakeholders; f. There is excessive criticism by the media; g. Tolerance is shown by committees (e.g., complaints, professional conduct and quality assurance committees) for some licensees who do not respect the rules but who are able to gain the sympathy or affinity of committee members. This phenomenon of the likeable rogue occurs within all professions; h. Staff are critical of licensee committee decisions; i. Every case seems to reinvent the wheel wasting time and resources; and j. Record keeping is poor or incomplete e.g., too few data fields to record information relevant to decision making. Barriers to Consistent Decision Making There can be systemic barriers to a regulatory organization implementing processes to improve consistency. Some of these barriers are: a. Governing statutes or documents that affect decision making processes i.e., a poor regulatory framework; b. Inadequate financial and human resources; c. Misapprehension of the risks associated with improving processes and transparency (i.e., it is safer to continue with existing processes ); and 4

7 d. Elected members of a regulator s governing body believing that they represent the business interests of a constituency of licensees instead of the public interest. Risk Based Regulation A regulator is given objectives by statute and decision making should support the achievement of these objectives. A useful planning approach is to define the risks to the regulator not achieving its objectives and then controlling the risks as part of strategy formulation. In this way the regulatory body is required to focus on what is important in order to maximize the likelihood of achieving its statutory objectives. A regulator is the arbiter of important risks and therefore will always be subject to second guessing by stakeholders and the media but it is important that the process for identifying and controlling risks be transparent in order to demonstrate the disciplined approach of the regulator. Detailed planning is crucial to regulatory success and it requires top-down commitment from senior management. Without this executive support the implementation of new processes aimed at improving decision consistency will inevitably encounter problems such as other day-to-day priorities taking precedence. Although the need for process change may be recognized by a regulator, the impetus for this change is unfortunately often a public disaster that forces management to take action. Planning to optimize decision making begins by articulating the statutory objectives of the regulator. This can be a difficult exercise particularly where the objective is to protect the public interest because few regulators have taken up the challenge of defining what public interest means. A thorough understanding of and ability to articulate the statutory 5

8 objectives of the regulator is a prerequisite to taking the next step of assessing the risks to the regulator not achieving these objectives. Risk assessment is often best performed in consultation with industry experts who can help enumerate risks and facilitate the process of prioritization. Approaches that consist of gathering groups to enumerate risks using voting technology can consume inordinate amounts of time and produce misleading results. Voting on risks is a haphazard process because the views of the majority of voters may be off the mark as a result of not being subjected to challenge and expert insight. Once the risks are identified and prioritized, the next step is to identify the best approach to controlling the key risks. A regulator has various tools at its disposal including education, discipline, compliance reviews, quality assurance reviews, registration, rules and communication, and the cost/benefit of each approach needs to be considered. Responsibility and accountability for control is then allocated to departments within the regulator. The objective of planning is to focus resources on those risks that are important to achieve the greatest benefit at the lowest cost to the regulatory body and the industry. It is at this stage that the tools and criteria to guide decision making are designed to ensure that intended outcomes are achieved. For example, complaints should be made subject to an assessment process that prioritizes those complaints that are worthy of extensive or expedited investigation. 6

9 Illustration Connecting Strategy and Decision Making In designing tools, the definition of intended outcomes requires thought and planning before processes are designed that guide decision making. The design should include methods to monitor outcomes to ensure that they are working as intended. Tools that aid in consistency fall into five categories as follows: 7

10 1) Tools to determine if decision making is consistent: Testing using hypothetical fact situations. Staff and committees are asked to make decisions based on given sets of facts and their decisions are then compared to each other and to the preferred outcomes; Objective outside observation of decision-making processes; and Review of historical decisions (e.g., sanctions in settlements) with similar fact situations. 2) Tools to assess and prioritize risks: Worksheets to list and prioritize risks and allocate responsibility for control to agencies and departments. 3) Tools to guide decision making: Case assessment worksheets that help establish the importance and priority of cases; Specific guidelines (or parameters) to be applied by staff and committees for licensee misconduct (e.g., impaired driving convictions, disciplinary action by another regulator, falsification of an application, practicing with a lapsed license); Decision trees that steer decision making based on factors that may include the characteristics of the licensee, the complainant s background and the surrounding circumstances. A decision tree is a decision support 8

11 tool that uses a chart or model of decisions and their possible consequences, and it is used to identify the strategy most likely to reach a goal; and Precedent databases (primarily used in disciplinary matters). 4) Tools to monitor and report on decision making: Case management systems that identify milestones for decision making; Dashboards that report on key performance indicators for management purposes; and Tracking of cases with similar fact sets from initiation to resolution to assess consistency. 5) Tools to encourage consistent decision making Rules that are based on principles; Sound, documented processes including planning that focuses on outcomes; Training of staff and committees; Performance evaluation criteria for staff and committees that include consistency; and Transparency of processes and outcomes. 9

12 Decision Trees Decision trees are a common form of modeling that is simple for people to understand and interpret with minimal training. They have value even with little hard data because important insights can be generated through internal or external experts describing situations (i.e., choices, benefits and costs) and giving their recommendations as to outcomes. The definitions of intended outcomes are given careful thought before processes are designed to channel decision making to avoid unintended consequences. An algorithm is a logical sequence of steps for solving a problem that is translated into a computer program to act as an automated decision tree. If sufficient data is available, it is possible to correlate facts and outcomes to design defensible algorithms that objectively guide decision making. For example, in a profession it might be shown that criminal behaviour such as driving under the influence of alcohol correlates with a high risk of professional misconduct which often leads to findings of incompetence resulting in failure of a licensee to protect the public interest. With this evidence in hand, a regulator could make a criminal conviction a reportable event that automatically triggers an assessment of competence. Regulators seldom have sufficient reliable data to correlate fact situations with outcomes. Washing of historical data to eliminate input and other errors is usually cost prohibitive and sometimes impossible if all of the pertinent information was not collected originally. The lack of data should not deter the use of algorithms, however, because algorithms based on expert opinion are still helpful in guiding decision making. 10

13 The use of algorithms and decision trees to assess whether an investigation or quality review will be performed is usually unpopular among professionals that are self regulated. Some of these groups see selection criteria for investigation or quality assurance reviews as a form of profiling that discriminates against members. These groups will likely demand that decision trees be based on hard data and statistics and even then there will likely be resistance to change which will require determination by regulators. Ongoing Monitoring Once processes that control consistency have been implemented or improved it is important to monitor internal compliance. This includes management oversight of decision making by capable professionals who have bought into the new or improved processes. It is also important to conduct periodic independent reviews (e.g., internal audit reviews) of processes and outcomes to ensure the desired results are being achieved. Conclusion Decisions that require the use of judgment or discretion are made at every level in regulatory organizations ranging from intake of complaints to assessing risks. The ability of a regulatory body to achieve its statutory objectives including protecting the public interest is closely linked to consistent decision making based on clear strategy and welldefined policies. It is the responsibility of management to ensure that decisions by regulatory staff and committees are a) consistent with the strategy of the regulatory body and b) consistent 11

14 with other decisions based on similar fact situations. Failure to recognize and meet this responsibility will greatly impair the value of regulation to society., LL.B., CPA, CFE, FCA President AssetRisk Advisory Inc. Toronto, Canada Direct Fax