Reliability Assurance Initiative (RAI) Benefits and Impact Draft 1. Initial Version: September 30, 2013

Transcription

1 Reliability Assurance Initiative (RAI) Benefits and Impact Draft 1 RAI Benefits and Impacts Page 1 of 11 Initial Version: September 30, 2013 and the Risk Based Reliability Compliance Working Group (RBRCWG) industry focus team, representing the Compliance and Certification Committee (CCC), collaboratively developed this Reliability Assurance Initiative (RAI) Benefits and Impacts document regarding the RAI s expected results. A key aspect of the RAI is that it is an evolving initiative. As the electric reliability organization (ERO) Enterprise proceeds with the various transitions described in the RAI concept papers, and the CCC will collectively evaluate and address how to reach the described end state. Accordingly, this document will be revised to reflect the current state of the initiative. The RBRCWG will remain engaged in the process, so that questions and comments may be factored into future revisions of this document. Stakeholders with additional questions should forward them to RAIComments@.net. RAI background information is included in a series of RAI concept papers, which are posted on s website. 1 1 See RAI white papers at

2 Page 2 of 11 Table of Contents A. Introduction B. Assumptions C. High-level, Expected Benefits from the RAI D. Effects of the RAI on Small Registered Entities E. Specific RAI Items and Associated Benefits and Impacts 1. Assessing Entity Risk 2. Assessing Registered Entity Internal Controls 3. Scoping Compliance Monitoring 4. Processing Violations in Accordance with Risk 5. Strengthening the Feedback Loop to Improve Reliability Standards

3 Page 3 of 11 A. Introduction The purpose of s RAI is to implement processes that will achieve a mature end state 2 for the Compliance Monitoring and Enforcement Program (CMEP). This document supports RAI implementation by discussing the various, possible benefits and impacts in a manner that considers the perspectives of stakeholders. As various focus groups generate new approaches for improved CMEP efficiency and effectiveness for stakeholders and the ERO Enterprise, RAI development and implementation will progress as well. B. Assumptions The following are a set of assumptions to frame the discussion on the benefits and impacts of the RAI. 1. Various parts of the RAI, referred to herein as RAI Items, will be implemented to reflect the policy goals described in the RAI concept papers. 2. The implementation of RAI Items will impact existing processes or result in the development of new processes. 3. The compliance enforcement authority (CEA) will develop and consistently apply standard methodologies for assessing a registered entity s inherent risk 3 and internal controls. 4. The Regions will use the results of the entity risk assessments to appropriately scope compliance monitoring for registered entities. 5. will ensure that the knowledge and information gained from the compliance monitoring and enforcement process is factored into future revisions of Reliability Standards. C. High-level, Expected Benefits from the RAI In the 2012 ERO Effectiveness and Stakeholder Perceptions Survey, stakeholders identified three CMEP themes. The development and implementation of the RAI should yield benefits in each of these areas: 1. Reliability benefit versus compliance effort 2. Consistency of CMEP application 2 See RAI Overview Q&A document, response to Question A.1. at 3 Inherent risk is a function of a registered entity s various registrations and other relevant factors like its system design, configuration, size, etc.

4 Page 4 of Efficiency of the enforcement process Expected RAI benefits include: 1. Improved Bulk Electric System (BES) reliability due to enhanced focus on high reliability risks and registered entities internal controls relative to those risks, providing a more efficient use of registered entity and CEA resources; 2. Higher level of registered entity compliance program maturity; 3. Lower demand on industry and ERO Enterprise resources; 4. Increased regional consistency in CMEPs; 5. Improved standards development and retirement as a result of the informational feedback loop; 6. Evolved enforcement processes such that lower risk issues will be processed in a manner that is most commensurate with their risk; 7. Reduced regional monitoring burden due to registered entities having more mature compliance programs, which provide enhanced monitoring capability and more effective self-reporting capabilities; and 8. Greater reliance on entities management practices as a result of revamped self-certification process. D. Effects of the RAI on Smaller Registered Entities Implementation of certain aspects of the RAI could have differing effects on smaller registered entities as compared to larger entities. Smaller entities generally have smaller staffs and therefore few resources dedicated to compliance. The four components of the RAI are: Assessing reliability risk, Scoping compliance monitoring, Processing violations in accordance with risk, and Strengthening the feedback loop to the standards development process. Of these components, scoping compliance monitoring could have the greatest impact on smaller entities. As and the Regional Entities develop the scope, method, and frequency of an entity s compliance monitoring, they will consider risk and the environment within which the risk exists, as well as the registered entity s internal controls. This approach directly eliminates the one size fits all periodic compliance monitoring that is currently in effect. The inherent and control risks

5 Page 5 of 11 associated with a registered entity will be the drivers for the Regional Entity determining the appropriate compliance monitoring scoping for the entity. 4 It is the responsibility of the CEA to obtain an understanding of a registered entities control environment within the context of the audit objectives. 5 In this regard, while the registered entity may decline to document or share their internal controls, the Regional Entity will have to appropriately adjust the nature, timing and extent of audit procedures to obtain reasonable assurance of compliance. This would mean a more traditional audit approach for the registered entity. The other three components of the RAI should not have significant impact on smaller entities. Assessment of an entity s reliability risk will be primarily based on the entity s functional registrations, with smaller entities generally having smaller risk profiles. The processing of violations by the CEAs in accordance with risk will benefit smaller and larger entities alike. Strengthening the feedback loop to the standards development process will also benefit both smaller and larger entities. The CEA will evaluate an entity s inherent risk (i.e., the risk posed by the entity because of intrinsic factors such as its registered functions) in fundamentally the same manner for smaller entities as for larger entities. An entity s control risk or its internal controls (i.e., the chance that a violation or risk to BES reliability could occur but may not be detected and corrected or prevented by the registered entity s compliance program) will be evaluated by the CEA in a manner that can impact smaller registered entities more significantly than larger entities. This is because smaller entities do not normally have compliance staff resources dedicated to their compliance program. As such, the involvement of a smaller entity s staff resources in an effort to assess the entity s internal controls would result in a greater impact than it would for a larger entity with dedicated compliance resources. However, it should be recognized that more times than not the internal controls a smaller entity would have are much less sophisticated due to the size and relevant responsibilities that the organization would have. While it is the obligation of the CEA to consider and understand internal controls within the context of an audit and a registered entity may choose to either not document or share their controls, the question for each entity is whether or not the potential benefits of voluntarily establishing internal controls designed to reduce control risk and having them assessed by the CEA enhances compliance monitoring efficiencies. A general concern expressed by small and large entities is that they will be required to document and develop an internal controls program that adds additional overhead to the entity s current compliance program. 4 See RAI Overview Q&A document, response to Question A.4. at 5 Generally Accepted Government Auditing Standards (GAGAS), Chapter 6, Internal Controls

6 Page 6 of 11 Overlooked in this conclusion is that most, if not all, entities currently have internal control programs and management activities in place that help them comply with Reliability Standards, whether or not they refer to them as internal controls. 6 Therefore, the questions for entities are twofold: 1. Are the entity s existing internal controls adequate to ensure compliance with the Reliability Standards? 2. Will the entity s resource expenditure to organize and present its internal controls for assessment be offset by a potential reduction in the resources required to support CEA s compliance monitoring of the entity, should the internal controls assessment conclude that the entity s internal controls are effective? The internal controls needed by a smaller entity to comply with the Reliability Standards should be fewer and less complicated than those needed by a larger entity. It follows that the CEA s assessment of a smaller entity s internal controls should be simpler and less involved than with a larger entity. While much depends on the internal controls assessment methodology currently being developed by the Regions via various pilots and learning exercises the resulting methodology to be applied consistently across the Regions should be scalable to reflect the differences in internal controls needed by smaller and larger entities. By scaling the internal controls assessment methodology, the amount of effort required for smaller entities to organize and present their internal controls to the CEA should be significantly less than that required for larger entities. The simple value proposition for smaller entities, as well as larger entities, is if the required resource expenditure for the internal controls assessment will be offset by a reduction in compliance monitoring. However, there is no upfront guarantee of reduced compliance monitoring prior to the understanding and evaluating of internal controls and the CEA s determination that they are effective. The alternative is for CEA to perform a traditional audit that would tend to be broader in terms of the nature, timing and extent of necessary testing to obtain reasonable assurance of compliance. E. Specific RAI Items and the Associated Benefits and Impacts 1. Assessing Entity Risk Impact: The CEA will implement a systematic process to assess a registered entity s risks. Although a defined repeatable process is not yet established, the risk assessments will be based on factors such as the entity s registration, configuration, size, etc. 6 See RAI Internal Controls Working Guide, V 1, P.9 at 0Guide%20Document.pdf

7 Page 7 of 11 A general benefit provided by the risk assessment is that the CEA will document the various aspects of the entity s risks that form the basis for its assessment of risk to the BES. Three scenarios may occur concerning this aspect of the RAI. a. The risk assessment indicates that the entity is a candidate for reduced compliance monitoring, and the CEA elects to reduce the scope or frequency of compliance monitoring for the entity. Potential Outcome: Both the CEA and the entity may accrue benefits. The CEA can deploy compliance monitoring resources to other areas of potentially greater need. The entity can spend less time preparing for compliance monitoring activities and redirect those resources to operating, maintaining, and protecting the BES. b. The risk assessment indicates that the entity is a candidate for reduced compliance monitoring, but the CEA elects to maintain the scope or frequency of entity monitoring at the established level based on a subjective determination. Potential Outcome: This may occur when the entity is doing many things well to manage its risks, as noted in the assessment, but the CEA is not comfortable reducing the entity s monitoring activities. A potential benefit could result from the ensuing dialog between the CEA and the entity. This could lead to the development of a clear set of steps for the entity to accomplish in order to address the CEA s concerns. These steps could include enhancement of the entity s internal controls. Once completed, the CEA could reassess the entity s risks and conclude that reduced compliance monitoring is appropriate. c. The risk assessment indicates that the entity is a candidate for increased compliance monitoring, and the CEA elects to expand the scope or frequency of entity monitoring. Potential Outcome: The increased scope or frequency of the compliance monitoring for the entity provides the CEA with increased assurance that the entity is in compliance with the Reliability Standards. This would put an additional burden on the entity but with the potential benefit of increased BES reliability. 2. Assessing Registered Entity Internal Controls Impact: In accordance with GAGAS practices, the CEA will implement a systematic process to understand and evaluate internal controls as they relate to compliance with the Reliability Standards. A general benefit is that the entity will receive compliance-related feedback from the CEA on its internal controls. This could lead to developments that strengthen entity controls, improving compliance and enhancing operations, thereby resulting in a more reliable BES. While the Rules of Procedure (ROP) clearly express the use of GAGAS and Institute of Internal Auditor guidelines with regards to conducting audit engagements which in turn require the evaluation of internal controls, an entity cannot be found in noncompliance based on any activity related to its

8 Page 8 of 11 internal controls. Additionally, entities with effective internal controls in place may be given credit when assessing civil penalties. 7 The additional work required to organize and present its internal controls to the CEA in support of the assessment may create a need for additional resource attention. Two scenarios can occur in the context of this RAI item. a. The entity declines to share its internal controls with the CEA. Potential Outcome: The nature, timing, and frequency of audit engagements will be adapted to appropriately address risk in the absence of the ability to effectively understand and evaluate controls. Further, the entity would not benefit from the CEA s experience gained from reviewing internal controls of other entities. b. The entity shares its internal controls with the CEA, and the CEA determines the controls to be effective. Potential Outcome: The CEA accrues the benefit of increased understanding of how the entity, through its internal controls, ensures it is in compliance with the Reliability Standards. The entity can benefit from this assessment in several ways. First, the entity receives the feedback that the CEA believes the entity s internal controls are effective. Second, the entity via the dialog with the CEA in the course of the assessment may learn about potential enhancements to its internal controls. Third, the entity may benefit from a reduction in compliance monitoring scope, testing, or frequency, as determined by the CEA. 3. Scoping Compliance Monitoring Impact: The CEA will appropriately scope the compliance monitoring for a registered entity based on a number of risk criteria and the integrity of the entity s internal controls. A general benefit to both the CEA and the entity is a more effective use of resources (i.e., directing resources at areas of greater need with respect to compliance with the Reliability Standards). If the entity s risk assessment and internal controls provide reasonable assurance of compliance with the Reliability Standards, the CEA may reduce monitoring activities. For the entity, resources traditionally applied to preparing for periodic generic audits can be reduced and partially redeployed to operate, maintain, and protect the systems important to reliable operation of the BES. The scenario that can occur in the context of this RAI item: a. The entity experiences a reduced audit scope, extended cycle between audits, a combination of scope reduction and extended cycle, or spot check in lieu of traditional audit. 7 See FERC Policy Statement on Compliance, October 16, 2008 at

9 Page 9 of 11 Potential Outcome: Both the CEA and the entity will more effectively use compliance resources, directing efforts at areas of greater need. Additionally, given reasonable assurance that an entity is complying with the Reliability Standards, reduction in monitoring activities could be pursued. 4. Processing Violations in Accordance with Risk Impact: The CEA will continue to enhance the enforcement process to enable disposition of lower risk issues with potentially less resource expenditure. This could include the disposition of lower risk issues via enforcement discretion to make decline to pursue determinations. This could be used for potential violations that are minimal or (perhaps) moderate risk, detected through internal controls, promptly corrected, self-reported, and addressed in a manner that is indicative of a strong compliance culture. Three scenarios can occur in the context of this aspect of the RAI program. a. The ERO Enterprise enhances the enforcement process to enable the disposition of lower risk issues with potentially less expenditure of resources by the CEA or the entity. Potential Outcome: The CEA would benefit from a streamlined enforcement workload by focusing less on processing lower risk issues. The entity would benefit from reduced resource expenditures, including internal and external legal resources that are expended on collecting, processing, reviewing information for regulatory submittals related to lower risk issues, and responding to CEA follow-up questions on these issues. b. The ERO Enterprise develops a methodology that permits the entity to monitor compliance activities that identifies, documents, accesses, and corrects lower risk issues within its corrective action program without entering the issues into the enforcement process. Potential Outcome: Both the CEA and entity would benefit in a greater way because there would be no submittals to the CEA for lower risk issues. Additionally, the CEA would benefit from the entity maintaining records of these lower risk issues within its corrective action program because the CEA would not have to expend resources on tracking these items. However, if the entity does not have a corrective action program in place, it may result in the additional burden of developing and maintaining one. The entity would also incur the burden of CEA review of its corrective action program during compliance monitoring activities. c. The CEA s decline to pursue determination would be based on the specific facts and circumstances of the potential violation and other relevant factors, including the registered entity s inherent and control risks. Potential Outcome: The determination would allow the CEA the flexibility to consider the specific facts and circumstances of the issue and other relevant factors, such as the entity s inherent and control risks. From an entity perspective, this flexibility could lead to disparate

10 Page 10 of 11 treatment for similar issues for different entities or for similar issues for the same entity registered in more than one Region. Accordingly, developing and implementing guidelines for the use of the decline to pursue disposition method would be necessary to promote more consistent application. 5. Strengthening the Feedback Loop to Improve Reliability Standards Impact: The ERO Enterprise will help ensure that knowledge and information gained through the implementation of the RAI is factored into future revisions of existing Reliability Standards to improve the content and clarity. That knowledge and information will also be included in the evaluation of patterns that may indicate potential reliability gaps or risks for use in determination of appropriate approaches to address reliability risks, including development of training or guidelines, completion of a reliability assessment, or other data-based analysis. Potential Outcome: will formalize the feedback loop to help ensure appropriate information flows from compliance monitoring and enforcement to other programs, including Standards, Reliability Assessment and Performance Analysis, and Event Analysis. This feedback will benefit both the CEA and registered entities as the content and clarity of standards will be improved through the knowledge and information developed during the course of the RAI implementation. The opportunities to analyze trends to identify and educate entities on reliability risks will facilitate the improved application of resources to better minimize reliability risks to the BES. Contributors to the RAI Benefits and Impacts Document Jim Armstrong, John Bee, Exelon Corp. Christina Bigelow, Midwest ISO, Inc. Terry Bilke, Midwest ISO, Inc. Keith Comeaux, NRG Energy, Inc. Jennifer Flandermeyer, Kansas City Power & Light Co. Greg Froehling, Rayburn Country Electric Coop, Inc. Michael Gildea, Matt Goldberg, ISO New England, Inc. Bill Graham, Bob Hoopes, PPL Corp. Patti Metro, National Rural Electric Coop Assoc. Sonia Mendonca, Stephanie Monzon, PJM Interconnection, LLC Matt Morais, ERCOT Helen Nalley, Southern Company John Rhea, OGE Earl Shockley, Bill Temple, Northeast Utilities Martyn Turner, Lower Colorado River Authority

11 Page 11 of 11 Ed Kichline, Jana Van Ness, Arizona Public Service Co.