Reliability Assurance Initiative Implementation Status

Transcription

1 MIDWEST RELIABILITY ORGANIZATION Risk-Based Compliance Monitoring and Enforcement Reliability Assurance Initiative Implementation Status MRO Board of Directors Meeting October 5, 2016 Improving RELIABILITY and mitigating RISKS to the Bulk Power System

2 Agenda Overview of RAI and anticipated results Actual results, so far Proportional responses with examples Rebalancing compliance oversight plans Internal controls Summary 2

3 What Problems did RAI Address? 1. Interconnected system was designed to be reliable, not perfect. The zero tolerance compliance and enforcement regulatory approach was outside the design criteria of an interconnected system; therefore, unreasonable and unsustainable. 2. In managing complex risk, the hallmark of reliability is not that errors won t happen they will happen it s to keep the small stuff small to prevent an uncontrolled, cascading event. We needed a space outside of enforcement to resolve the small stuff and to recognize entities that are effectively finding and fixing small problems. 3

4 Came Down to Two Things The need for proportional responses in oversight and enforcement based on risk and the entity s response to risk The need to reduce administrative burden 4

5 Implementing the RAI Planning Phase: Develop the end game and a viable pilot Pilot Phase: Pilot the approach Implementation: Identify, then file necessary rule changes Full roll-out across MRO and the other Regional Entities 5

6 Anticipated Results Self-reporting would increase Fewer, less severe violations would occur Fewer, less severe events would occur Regulatory responses to entity behaviors around reliability would be proportional in both oversight and enforcement 6

7 Anticipated Results Administrative burden rebalanced and reduction in total cost of compliance Industry operationalizes standards into its day-to-day procedures and processes MRO integrates entity s self-monitoring work into its oversight plans Improved standards Feedback mechanisms from risk assessments and events Filling gaps Eliminating duplicative, administrative and low-value standards 7

8 The Results Improved Self-Reporting Total Instances of Noncompliance by Discovery Method ( August 31, 2013) Spot Check 8% Total Instances of Noncompliance by Discovery Method (August 31, August 31, 2016) Self-Report 39% Compliance Audit 35% Self-Report 72% Compliance Audit 22% Self- Certification 16% Investigation, 2% Self-Certification 6% 8

9 The Results The Use of Compliance Exceptions All Processing Methods Percentages ( August 31, 2013) All Processing Methods Percentages (August 31, August 31, 2016) Settlement Agreement 23% Settlement Agreement 10% Notice of Confirmed Violation 29% Find, Fix, Track, and Report 48% Notice of Confirmed Violation 28% Find, Fix, Track, and Report 22% Compliance Exception 40% 9

10 The Results Less Severe Violations Total Compliance Trend Index 400 Minimal Moderate Serious Compliance Trend Index

11 The Results Less Severe Events Total BPS Event Trend Index BPS Impact Index

12 The Results Proportional Responses: Translated to Specific Entities - Entity 1 Total Compliance Trend Index Minimal Moderate Serious Total Instances of Noncompliance by Discovery Method 50 Compliance Trend Index Total BPS Event Trend Index Self-Report 88% Spot Check, 6% Compliance Audit, 6% Total Instances of Noncompliance by Processing Method Settlement Agreement 6% BPS Impact Index Find, Fix, Track, and Report 53% Compliance Exception 35% Notice of Confirmed Violation 6% 12

13 The Results Proportional Responses: Translated to Specific Entities - Entity 2 Compliance Trend Index Total Compliance Trend Index Minimal Moderate Serious Total BPS Event Trend Index Total Instances of Noncompliance by Discovery Method Self- Report 50% Spot Check 8% Compliance Audit 42% Total Instances of Noncompliance by Processing Method BPS Impact Index Compliance Exception 36% Notice of Confirmed Violation 28% Find, Fix, Track, and Report 36% 13

14 The Results Proportional Responses: Translated to Specific Entities - Entity 3 Compliance Trend Index Total Compliance Trend Index Minimal Moderate Serious Total BPS Event Trend Index Total Instances of Noncompliance by Discovery Method Spot Check, 3% Compliance Audit 29% Self-Report 36% Investigation 29% Self-Certification 3% Total Instances of Noncompliance by Processing Method BPS Impact Index Compliance Exception 22% Settlement Agreement 28% Notice of Confirmed Violation 39% Find, Fix, Track, and Report 11% 14

15 Entity 1 Entity 2 Entity 3 Total Instances of Noncompliance by Discovery Method (2007 to August 31, 2016) Total Instances of Noncompliance by Discovery Method (2007 to August 31, 2016) Total Instances of Noncompliance by Discovery Method (2007 to August 31, 2016) Self- Report 88% Spot Check, 6% Compliance Audit 6% Self- Report 50% Spot Check 8% Compliance Audit 42% Self- Report 36% Spot Check 3% Compliance Audit 29% Investigation 29% Self- Certification 3% Entity Performance Ability to Self-Monitor 15

16 Entity 1 Entity 2 Entity 3 Total Instances of Noncompliance by Processing Method (2007 to August 31, 2016) Total Instances of Noncompliance by Processing Method (2007 to August 31, 2016) Total Instances of Noncompliance by Processing Method (2007 to August 31, 2016) Find, Fix, Track, and Report 53% Settlement Agreement 6% Compliance Exception 35% Notice of Confirmed Violation 6% Compliance Exception 36% Notice of Confirmed Violation 28% Find, Fix, Track, and Report 36% Compliance Exception 22% Settlement Agreement 28% Notice of Confirmed Violation 39% Find, Fix, Track, and Report 11% The Results Proportional Responses: How it Translates to Specific Entities 16

17 Entity 1 Entity 2 Entity 3 Total Compliance Trend Index Total Compliance Trend Index Total Compliance Trend Index Minimal Moderate Serious Minimal Moderate Serious Minimal Moderate Serious Total BPS Event Trend Index Total BPS Event Trend Index Total BPS Event Trend Index The Results Proportional Responses: How it Translates to Specific Entities

18 Calibrating Oversight and Enforcement with Registered Entity Performance Entity 1 Compliance Integrated into Operations Fewer audits, more self-certifications Entity 3 Compliance Separate From Operations More frequent audits, more spot checks Reliance on entity management practices (internal controls) - integrated into MRO's oversight Self-logging No or little reliance on entity management practices (internal controls) - less integration into MRO's oversight No self-logging Streamlined processes for resolving instances of noncompliance Collaborative mitigation plan development Convergence in understanding of risk Exemplary cooperation Formal enforcement actions MRO staff driven mitigation plan development Divergence in understanding of risk; denial of consequences Minimal cooperation 18

19 The Results Proportional Responses: Oversight Develop a Compliance Oversight Plan (COP) based on unique characteristics and risks associated with each registered entity Move away from audit centric to an oversight approach that includes all CMEP tools and entity s internal practices to self-monitor Adjust compliance monitoring approach based on multiple considerations: Performance Areas Inherent Risk Internal controls Compliance monitoring history Past events, noncompliance, and mitigation activities 19

20 The Results Rebalancing the Administrative Burden Regional Risk Assessment significantly narrowed MRO s scope of monitoring for NERC Requirements (Includes Currently Enforceable Requirements) 221 Requirements (Identified for Monitoring) 299 Requirements (Excluded from Monitoring) Of the 221 Requirements: 56 only applicable to RCs, PCs, BAs, and RSGs 48 are system event driven 117 requirements for TO s, TOP s, GOs, GOPs, and TPs Performance Areas 20

21 Compliance Oversight Plans and Consistency Working toward consistent IRA results across the ERO ERO Agreed-Upon Risk Factors: BA Coordination CIP - Connectivity CIP - Control Center Influence Critical Transmission Largest Generator Facility Load Monitoring & Situational Awareness Tools Planned Facilities RAS/SPS System Restoration Total Generation Capacity Transmission Portfolio UFLS Equipment UFLS Development & Coordination UVLS Variable Generation Voltage Control Workforce Capability 21

22 Risk-Based Compliance Monitoring Framework v1.0 Feedback to NERC Standards Committee 1 1 Identify potential gaps in the Standards 2 Identify Standards that are never in scope of Oversight Plans Feedback to NERC Standards Committee 2 22

23 Internal Controls Our Obligation [CMEP staff] should assess whether internal control has been properly designed and implemented and should perform procedures designed to obtain sufficient, appropriate evidence to support their assessment about the effectiveness of those controls. 23

24 Internal Controls The Basics MRO staff will review internal controls designed to prevent, detect, or correct noncompliance How well are registered entities self-monitoring compliance? Focus on areas identified in IRA that represent more significant risk Most effective where there are high populations of items (i.e., records) or activities (i.e., access) 24

25 Internal Controls What s Changing? CMEP staff will consider internal controls across all CMEP activities to understand the entity and make informed oversight decisions Not limited to reviewing internal controls through a standalone Internal Control Evaluation or ICE Objectives for reviewing internal controls Understand how the entity complies with a standard and ensures future compliance Decisions based on internal control effectiveness More formalized input into shaping COPs Internal controls have always shaped enforcement, risk determinations, and self-logging 25

26 Internal Controls What Should You Be Doing? Internal discussions within your organization How do you know what risk you pose to the bulk power system? How are you mitigating those risks? How do you know you are compliant? How do you assure management that you know you are compliant? How would you know if you weren t compliant, and how would you fix it? 26

27 Continued Improvements NERC places stronger emphasis on consistency of risk-based methodologies across ERO Compliance exceptions Shift focus from voluntary ICE to internal control reviews for development of COPs Timing tied to compliance activities Statesmanlike standards development and feedback loop to the Reliability Standards MRO Standards Committee subgroup of industry SMEs submitted a Standard Authorization Request to NERC in 2015 to test this process Increased MRO involvement in standards development 27

28 Summary Successful translation of concepts to implementation basic processes and procedures are in place COP approach will be a transition for registered entities and MRO staff Consistency across Regional Entities already a challenge for NERC, but new tools may help! 28

29 29