ERO Enterprise and Corporate Metrics

Transcription

1 ERO Enterprise and Corporate Metrics Quarter 2 Status Mark Lauby, Senior Vice President and Chief Reliability Officer Corporate Governance and Human Resources Committee Meeting August 10, 2016

2 Leading at Q3 Metric 1: Reliability Results No Category 4 or 5 events Metric 3: Risk Mitigation Effectiveness 3.2-Extreme Physical Events 3.4-Protection System Misoperations 3.6-Model Building 3.7-Equipment Performance Metric 4: Program Execution Effectiveness 4B-Standards Guidance and Training or Outreach 2

3 Watching in Q3 Metric 2: Assurance Effectiveness One Category 3 event occurred; gap analysis needed Metric 3: Risk Mitigation Effectiveness 3.1-Changing Resource Mix: Long-term Reliability Assessment (LTRA) collaboration with Regional Entities to be completed 3.5-Extreme Conditions: Monitoring for intervention strategies Metric 4: Program Execution Effectiveness 4A-Reliability Standards and Cost-effectiveness: Standards grading to be incorporated in risk-based input 4C-Registration: Documented review to be completed 4E-Operating Model: Dependent on Registration oversight, review of Certification, and user management completion 4F-Effectiveness Survey: Action plan approval pending 4G-GridEx: Updating NERC Crisis Action Plan 3

4 Emerging Activities in Q3 Metric 3: Risk Mitigation Effectiveness 3.3-Cybersecurity Preparedness: User Communities pushed to 2017 Metric 4: Program Execution Effectiveness 4D-CMEP: Watching mitigations and IRA completions 4

5 ERO Enterprise Effectiveness Survey Results and Action Plans Kristin Iwanechko, Associate Director, Member Relations and MRC Secretary Corporate Governance and Human Resources Committee Meeting August 10, 2016

6 Background Sub-metric F Issued May 2016 Survey questions modified from January 2015 survey 72 survey questions across 12 topic areas Ratings on a 5-point scale 1 Strongly Disagree 2 Disagree 3 Neither Disagree nor Agree 4 Agree 5 Strongly Agree 2

7 Results and Analysis 265 survey respondents 235 represented 368 NCR numbers 30 did not represent an NCR number Positive results overall Average ratings from 2.94 to

8 Highest Rated Items The personnel certification program is valuable in promoting skilled, trained, and qualified BPS operators The audit process (including pre- and post-audit activities) for my entity s most recent audit was welldefined, organized, and followed established procedures The audit report from my most recent audit identified clear, definitive, and actionable items to address My organization trusts the E-ISAC The E-ISAC shares information on threats and incidents (e.g., bulletins, assessments, weekly/monthly reports, etc.) that is reliable

9 Lowest Rated Items Reliability Standards address risk to reliability in a costeffective manner Reliability Standards are practical to implement The Reliability Standard requirements are clearly stated There is timely and effective response during the Registry Appeals Process The ERO Enterprise ensures efficiencies and minimizes duplication and activities not affecting reliability outcomes

10 Most Favorable Items 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% The audit process (including pre- and post-audit activities) for my entity s most recent audit was welldefined, organized, and followed established The personnel certification program is valuable in promoting skilled, trained, and qualified BPS operators The E-ISAC shares information on threats and incidents (e.g., bulletins, assessments, weekly/monthly reports, etc.) that is informative The ERO Enterprise communicates information about Reliability Standards, their development, and opportunities for stakeholder participation in an The audit report from my most recent audit identified clear, definitive, and actionable items to address Favorable Neutral Unfavorable 6

11 Most Unfavorable Items Reliability Standards address risk to reliability in a costeffective manner The Reliability Standard requirements are clearly stated Reliability Standards are practical to implement The E-ISAC is my primary resource for threat information and analysis The organization registration process is well-defined and the definitions are clear % 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Favorable Neutral Unfavorable 7

12 Year-Over-Year Analysis Subset of 37 questions Statistically significant increase (3.42 to 3.77) The risk-based registration process ensures that appropriate entities are registered or deregistered commensurate with risk to the Bulk Electric System. Statistically significant decrease (3.93 to 3.61) The implementation of spot checks was well-defined, organized, and followed established procedures. 8

13 Analysis to Determine Areas for Focus Five lowest-rated items Five most unfavorable items Items with higher than 6% unfavorable ratings and also lower than 60% favorable ratings Items with statistically significant decrease in year-over-year comparison Themes from anonymized comments 9

14 Action Plans Regional consistency in compliance monitoring and enforcement Regional consistency in organization registration Cost-effectiveness, clarity, and practicality of Reliability Standards 10

15 Regional Consistency in Compliance Monitoring and Enforcement Compliance Monitoring Developing consistent templates Consistency in the multi-region registered entity (MRRE) program Regional Entities implementation of risk-based compliance monitoring Monitoring approaches Compliance Enforcement Process reviews Assessing the risk of noncompliance Settlements and penalty determinations Developed MRRE templates for enforcement activities 2016 ERO Enterprise and Corporate Metrics Sub-metric D: implementation of risk-based CMEP Sub-metric E: develop and implement oversight plan modifications 11

16 Regional Consistency in Organization Registration NERC-led panel Common registration form ERO Registration Procedure and Risk-Based Registration Implementation Guidance posted Risk-based approach to review registration documents used by Regional Entities Review JRO/CFR processes during Regional workshops 2016 ERO Enterprise and Corporate Metrics Sub-metric C: review structure and consistency of registration program Sub-metric E: develop and implement oversight plan modifications 12

17 Cost-effectiveness, Clarity, and Practicality of Reliability Standards Enhanced periodic reviews Common Reliability Standard Metric Cost-effectiveness method 2016 ERO Enterprise and Corporate Metrics Sub-metric A: develop and pilot method for determining cost effectiveness 13

18 Next Steps Request CGHRC acceptance of final report and action plans no later than November 2016 Complete actions as described in the action plans Next survey to be issued in

19 15