Reliability Assurance Initiative Sonia Mendonca, Associate General Counsel and Senior Director of Enforcement
Agenda Reliability Assurance Initiative (RAI) Overview 2015 ERO CMEP Implementation Plan Inherent Risk Assessment Overview Internal Control Evaluation Overview RAI Enforcement Overview 2
RAI Compliance Activities Overview Risk-based Compliance Oversight Framework 3
IP Overview Purpose Annual operating plan for NERC and Regional Entities (REs) Implementation of risk-based approach for CMEP activities NERC release on or about September 1 of preceding year REs submit Regional IPs on or about October 1 NERC reviews and posts revised IP in November to include Regional IPs Updates occur throughout implementation year, as needed 4
IP Content Areas NERC IP provides details on: ERO Enterprise s Risk-based Compliance Oversight Framework Prioritized list of Enterprise-wide risk focus areas o Map to associated Reliability Standards o Do not include all potential risks to BPS o REs consider local risks and circumstances within regional footprint Guidance on Regional Risk Assessments Enforcement activities o Compliance exceptions o Self-logging program 5
2015 Risk Focus Areas Nine areas of focus for 2015 consideration 1. Infrastructure maintenance 2. Uncoordinated protection systems 3. Protection systems misoperations 4. Workforce capability 5. Monitoring and situational awareness 6. Long-term planning and system analysis 7. Threats to cyber systems 8. Human error 9. Extreme physical events 6
IP Transformation IP tailored to risk-based approach to CMEP Replacement of a static, one-size-fits-all list of Reliability Standards, Actively Monitored List (AML) Risk focus areas AML Audit Scope Monitoring plan reflects risk focus areas and IRA and ICE processes Removal of six-year audit cycles Three-year cycles remain for BA, RC and TOP REs will determine compliance oversight plan for other entities o Use existing CMEP tools 7
Key Takeaways Regional IPs provide further detail on risk focus areas and compliance oversight plans REs tailor compliance oversight plans for registered entities REs are at varying stages of implementing IRA and ICE processes NERC oversight and continued training will help ensure consistency 8
Resources 2015 ERO CMEP IP located on NERC website under Compliance Resource Documents at: http://www.nerc.com/pa/comp/resources/pages/default.aspx Refer to NERC RAI website for updates and details 9
IRA Overview IRA Information Gathering Assessment Results Second step in the Framework IRA Guide contains processes used to assess inherent risks and determine areas of focus for compliance oversight plan of an entity 10
IRA Process Figure 2. IRA Module Flow Chart Information Gathering Assessment Results Gather Risk Elements Module Output Risk Factor and Standards and Requirements Applicability Review Results Documentation Determine Entity Specific Information Needs to Perform IRA Develop Targeted Information Request List Risk Factor Analysis Review of IRA Conclusions Draft Compliance Oversight Plan for Registered Entity 11
Information Gathering Use Risk Elements, both ERO and Regional, to identify information needs Determine entity-specific information needs to perform IRA Inventory information on hand (e.g., information from prior audits, Transmission Availability Data System (TADS) information, etc.) Review Information Attributes lists and determine gaps or verification needs Develop a targeted information list based on gaps or verification needs Key Questions in Information Gathering What risks, and associated Standards and Requirements, identified in the Risk Elements are applicable to the functional registration of the registered entity? What risk factors are in scope? What registered entity specific information do we need? Where do we get information from? Is the information appropriate and sufficient? 12
Assessment Confirm the applicability of Standards and Requirements to the registered entity Identify possible risk factors and criteria for consideration Key Questions for Decision Making Phase Based on Requirement and registered entity data, Which Standards and Requirements are not applicable? Which risk factors are not applicable? Which risk factors are used to assess the level of significance of Standards and Requirements in scope? What are the areas of focus? What level of effort should be assigned to each area of focus? What is our preliminary Compliance Oversight Plan? 13
Results Document conclusions and results Preliminary compliance oversight plan determined Communication of results with the registered entity Key Questions for IRA Outcomes Phase What was done to support the conclusion? What level of information should the compliance oversight plan include? How is supporting information documented and maintained? 14
Results IRA Results include: Inherent risks to the reliability of the BPS that are applicable to the registered entity List of the Standards and Requirements that could help prevent inherent risks List of identified risks that are not mitigated by any existing Standards and Requirements Details of relevant information including key assumptions used during the IRA decision making process, timing of the IRA, etc. Key individuals involved (preparer, reviewer, approver), and information used during the assessment Summaries of the IRA analysis performed and conclusions reached 15
Key Takeaways Regional Entities will tailor information requests after identifying information available from other resources or already on hand All stages of the IRA process will be documented and IRA conclusions supported IRA results to be discussed with the registered entity IRAs will be performed and revised periodically based on audit schedules and other factors that may impact IRA results. Sale or retirement of asset that affected IRA results 16
Next Steps For 2015, Regional Entities will perform IRAs for scheduled audits. Regional Entities may perform IRAs for other registered entities as determined necessary NERC and the Regional Entities will continue to identify tools and template needs in order to promote consistency in IRA implementation 17
ICE Overview Third step in the framework ICE guide contains processes used to evaluate internal controls Optional step for the entities 18
What is an internal control? Risk Controls Residual Risk 19
What is an internal control? Internal Controls Management Practices People Tools Processes Systems 20
Internal Control Evaluation Process Identify key controls related to risks Request controls information Test effectiveness of controls Identify how well controls address risks and provide compliance assurance 21
Key Controls Identification Determine applicable entity level and activity level controls Key controls might include those that Represent a single point of failure Interact between different departments and other controls May be complex, manual, or known to fail Evaluate the use of the work of others 22
Request Controls Information Regional Entities may already have some controls information Past audit engagements Violation reviews Mitigation plans Self-Certifications Past internal controls evaluation Ensure available information is Sufficient Credible Timely Request information required from the entity Spreadsheet Word document Other regional specific format 23
Test of Control Effectiveness Assess overall internal control design Controls could be Preventative, Detective, and Corrective Administrative, Technical, or Physical Use generally accepted methodologies to review control evidence A few questions to consider while testing the effectiveness: Is the control working as designed? Is the control a result of a thoughtful approach? How well is the control implemented? What is the likelihood the control will detect and prevent noncompliance or the associated risk? 24
Finalize ICE Conclusions Determine controls that address associated risks A single control may not address the risk but most likely a combination of many controls Determine the right combination of preventative, detective, and corrective controls Determine maturity of controls This is different from audit! Allows entity to see the roadmap to strengthen controls Allows Regional Entities to provide feedback 25
ICE Outcomes Compliance oversight for the registered entity: On-site or off-site Audit Spot Check Self-Certification Other? Customized CMEP tools: Scope and focus of audit Scope and focus of self-certification Spot checks and investigations Document and communicate results with the registered entity Feedback to registered entity Areas of strength Areas for improvement 26
Key Takeaways The ICE guide will help bring consistency to a new and evolving process ICE will take time in the initial phase. Efficiencies will come over time Short-term pain, but long-term gain Compliance monitoring reflects specific risks posed by the entity Focused compliance monitoring aka surgical audits Targeted self-certification Strengthen controls through feedback to industry 27
Next Steps NERC and the Regional Entities will continue to identify tools and template needs in order to promote consistency in ICE implementation NERC and the Regional Entities will provide outreach and training to industry and regional staff 28
What is Risk-based Enforcement? The end state for enforcement involves reserving the enforcement process for those issues that pose a serious and substantial risk to the reliability of the bulk power system and, as to other issues, allowing NERC and the Regional Entities to exercise appropriate discretion whether to initiate an enforcement action or address an issue outside of Enforcement. 29
Why Risk-based Enforcement? Focus resources on noncompliance posing the highest risk to the reliability of the BPS ERO Enterprise caseload primarily comprised of lesser risk noncompliance Leverage existing registered entity management practices associated with identification, assessment and correction of noncompliance and encourage dissemination of such practices Large percentage of self-identified noncompliance 30
Enforcement Process Flow Risk and Control Assessment Input Audit, Spot Check, Etc. Log, Self- Report, Self- Certification Triage Record Compliance Exception Enforce Feedback to Risk and Controls Assessment 31
Compliance Exceptions Basics What is a compliance exception? Noncompliance that is not pursued through an enforcement action (section 5.0 of the CMEP) Who is eligible for a compliance exception? Registered entities chosen by the Regional Entities Available to all registered entities in 2015 What can be treated as a compliance exception? Minimal risk noncompliance Any discovery method Mitigated within 12 months Who determines compliance exceptions? Regional Entity staff o Usually Risk and Mitigation or Enforcement teams When are compliance exceptions determined? After review by applicable Regional Entity staff 32
Roles of NERC and FERC Compliance exception program is not meant to eliminate or reduce oversight or visibility All noncompliance is tracked and recorded Spreadsheet of compliance exceptions is provided to NERC and FERC on a monthly basis through non-public means Data and analysis provided to Board of Trustees and stakeholders regularly 33
Benefits of Compliance Exceptions Program Preserves finite resources Promotes accurate differentiation of risk Encourages self-identification of noncompliance by registered entity Enables appropriate exercise of discretion by Regional Entity 34
Self-logging Basics Who can log? Registered entities with effective management practices for identifying, assessing and correcting noncompliance What is logged? Minimal risk noncompliance When is the log reviewed? At least every three or six months Who reviews the log? Regional Entity staff What does the log replace? Self-Reports of each minimal risk noncompliance 35
Participation in the Self-logging Program Voluntary but Requires Regional Entity approval Formal ICE is not required Regional Entity may inquire as to controls associated with self-monitoring, identification, assessment and correction of noncompliance Review of registered entity capabilities Initiative and recognition of compliance obligations History of self-reporting and mitigation (accuracy, promptness) Quality, comprehensiveness and execution of internal compliance program History of cooperation Performance in audits Scope of logging May include all or a subset of Reliability Standards Determined by Regional Entity 36
Self-logging Program Reporting and Review Registered entities track their noncompliance on a log that looks very similar to FFT spreadsheet Self-logging of minimal risk issues only Registered entities submit their spreadsheets (logs) at least every three or six months Regional Entities review all self-logged noncompliance following the conclusion of the self-logging cycle Properly-classified minimal risk issues will presumably be treated as compliance exceptions 37
Oversight and Visibility NERC will exercise oversight of the Regional Entities implementation of the program If logged items are disposed as compliance exceptions, NERC will track these items for trending purposes NERC reports all noncompliance to FERC, including logged items that are disposed as compliance exceptions 38
Benefits of Self-logging Program Reduced administrative burdens Presumption that self-logged noncompliance will be processed as a compliance exception Relies on and promotes a closer understanding by Regional Entities of registered entities management practices Allows scalability 39
Enforcement Resources RAI webpage on nerc.com ERO Self-Report User Guide o Description of noncompliance o Risk assessment ERO Mitigation Plan Guide o Effective mitigating activities o Prevention of recurrence o Identification of underlying cause of noncompliance NERC Enforcement and Mitigation webpage All Notice of Penalty violations All FFTs 40
Project Timeline 2 0 1 4 May July 2014 June July Aug Sep Oct Nov Dec Jan Feb Mar Published the IRA Guide for comment 2 0 1 5 Sept. 2014 Published the Risk Elements Methodology for the modified Implementation Plan (IP) and 2015 CMEP IP Oct. 2014 Q1 2015 Finalized and released enforcement program documents Finalized and released IRA and ICE guides Published the 2015 IP with Regional Entity appendices Continue delivering training to industry and Regional Entity staff Submitted FERC informational filing Continue outreach and identify additional training needs. 41
Continued Outreach RAI 101 webinar recording available at NERC.com Various Regional Entity workshops Industry workshops in November November 6 Atlanta November 20 Phoenix Industry webinar on RAI implementation and progress in Q1 2015 42
*NEW* RAI website (http://www.nerc.com/pa/comp/pages/reliability-assurance-initiative.aspx) Weekly Standards and Compliance Bulletins Program news announcements on RAI page Resources 43
44