Entity Risk Assessment

Transcription

1 MIDWEST RELIABILITY ORGANIZATION Entity Risk Assessment NERC Workshop Reliability Assurance Initiative Dan Skaar, President and CEO April 3, 2013 Improving RELIABILITY and mitigating RISKS to the Bulk Power System

2 Madden - Stabler 2

3 If Registered Entities proactively fix and self report violations, why fine them?. aren t they just doing the right thing? Hey Wait a Minute Moment 1 (2008) Result don t fine Registered Entities for doing the right things Zero Dollar Penalty, ACP/FFT, Compliance Exceptions (RAI)? Anybody in there? Think McFly, Think 3

4 Our Philosophy: Risk and Reliability The hallmark of reliability in complex, real-time, interdependent systems is not that errors won t occur... it s that errors and operating anomalies won t create an uncontrolled cascading event outside of the design criteria - High Reliability Organizational Theory. 4

5 Using Highly Reliable Organization Theory to Manage RISK Five characteristics of highly reliable organizations 1. Preoccupation with failure Attention on close calls and near misses ( being lucky vs. being good ); focus more on failures rather than successes 2. Reluctance to simplify interpretations Solid root cause analysis practices 3. Sensitivity to operations Situational awareness and carefully designed change management processes 4. Commitment to resilience Resources are continually devoted to corrective action plans and training 5. Deference to expertise Listen to your experts on the front lines (ex. authorities follows expertise) 5

6 BTW, Small Stuff Matters The small stuff matters. Small stuff should be detected, reported, corrected, and prevented under normal management practices its good utility practice. If Registered Entities are proactively taking care of the small stuff before it escalates, it s not prosecuted as a federal case. They are simply doing their job and what the public expects! 6

7 Region had a lot of violations with PRC-005/PRC-008 there seemed to be wide variations in the technical application of these standards. Hey Wait a Minute Moment 2 (2009/2010) it s costing everyone a lot of money let s have a dialogue with the industry they are the experts...they can solve problems Result Engaged industry producing Application Guides Houston, we have a problem 7

8 Number of Violations Positive Impacts from Engagement with Industry Improving Protection Systems Maintenance and Testing PRC 005/008 Violation Trends PRC Application Guide Roll Out Note: A violation includes multiple elements Increase in self reported violations resulted from strengthening protection programs Stabilizing trend 2007/ Average Violation Severity Level (VSL) Decline in VSLs due to stronger programs /

9 Hey Wait a Minute Moment 3 (2010/2011) appeared to MRO staff that Registered Entities were only assessing compliance with Reliability Standards prior to an audit point in time compliance are Registered Entities systematically managing risk with Reliability Standards? lets open a dialogue with industry on our concern Result Model Controls around Reliability Standards You get what you inspect, not what you expect. 9

10 Example of Internal Control Program Scheme Procedure/ Process Control Control Activity Control Type Program Documents (Procedures) Standard Work Order Checklist followed and completed, exceptions noted, follow up notes signed Primary Control Supervisory Review Review for completeness and accuracy, follow up actions closed or scheduled to be completed, signed Secondary Control Management Oversight Periodic sampling of work orders to determine program is being completed and properly reviewed Tertiary Control 10

11 Capability Maturity Model Reliability Standards Regulation Emergent Mature Ad Hoc Less Repeatable Defined Managed Internal Control Design Pareto Principle 80% of inherent risk lies with 20% of Registered Entities Optimized More Risk Relationship Between Size/Function and Internal Controls HIGH LOW Small $$$ $ OPTIMIZED Large AD HOC Internal Controls around Reliability Standards

12 Risk Relationship with Internal Controls Internal controls should be commensurate with the level of inherent risk. 12

13 What s the Risk to Reliability? Uncontrolled, cascading event outside the design criteria (Section 215 of FPA) Causes identified in blackout reports Patterns Series of smaller matters Documentation doesn t match performance Others? 13

14 Framework for Considering Risk Inherent Risks Entity Specific System Wide Size and interconnections Geography/Topology Technology Past history, events, other Assessments/Studies Past/Current Performance Emerging Threats Cyber (Connectivity) Applicable Standards Control Risks Quality and rigor of internal controls to address risks to BES reliability Evaluated in terms of industry adopted framework or criteria (ex. corrective action programs, elements of operational excellence, HRO principles, etc.) Two Dimensions Criteria or Principles Hard Soft Procedures, systems, training, etc. Governance, culture, etc. Procedures and Risk Controls around Applicable Standards Detection Risks Quality and rigor of oversight, depth of procedures, etc. Governance/Training Procedures/Checklists Systems/ Flags IDENTIFY UNDERSTAND/ADDRESS MITIGATE/PREVENT

15 Our Collective Job: Managing RISK Prevent Risk Identify Risk Strategic and Tactical Address Risk Understand Risk Successful organizations have learned that the higher the risk, the more necessary it is to engage everyone's commitment and intelligence. Margaret J. Wheatley 15

16 Thank you! [E]ven when fully developed, risk analyses cannot be used as a substitute for sound professional judgment of engineers, contractors, or review boards. FERC 16