Reliability Assurance Initiative (RAI) Update. June 19, 2014, 3 pm 5 pm EDT Industry Webinar

Transcription

1 Reliability Assurance Initiative (RAI) Update June 19, 2014, 3 pm 5 pm EDT Industry Webinar

2 Administrative Items NERC Antitrust Guidelines It is NERC s policy and practice to obey the antitrust laws and to avoid all conduct that unreasonably restrains competition. This policy requires the avoidance of any conduct that violates, or that might appear to violate, the antitrust laws. Among other things, the antitrust laws forbid any agreement between or among competitors regarding prices, availability of service, product design, terms of sale, division of markets, allocation of customers or any other activity that unreasonably restrains competition. Notice of Open Meeting Participants are reminded that this webinar is public. The access number was widely distributed. Speakers on the call should keep in mind that the listening audience may include members of the press and representatives of various governmental authorities, in addition to the expected participation by industry stakeholders. 2 RELIABILITY ACCOUNTABILITY

3 A Consolidated ERO Approach to Risk- Based Compliance Monitoring Reliability Assurance Initiative Updates June 19, 2014 Jerry Hedrick, Director Regional Entity Assurance and Oversight NERC Sarah E. Stevens, Compliance Oversight and Assurance Manager SERC Reliability Corporation Thomas P. Tierney, Director of Compliance Midwest Reliability Organization

4 Agenda RAI Status RAI Oversight Plan Framework Risk Elements Inherent Risk Assessment Internal Controls Evaluation Oversight Scoping RAI v. IAC RAI Misconceptions Next Steps

5 RAI Oversight Plan Framework RE Functions Characteristics - ERO / Regional Events RISC Risk Elements Applicable Standards Input I R A Inherent Risk Assessment Scope Input I C E Scope Controls Not Evaluated Internal Controls Evaluation CMEP Tools Oversight Scoping Compliance Oversight Plan

6 RAI Compliance Status Inherent Risk Assessment Complete the guide on approach Risk Elements Identify and determine application of data Incorporate into the AML and IP Internal Control Evaluation Document approach Develop guide Next Steps for RAI and CIPv5 Programmatic approach to risk and discretion 6

7 RAI: Harmonious IAC Effective Compliance Monitoring Entity identifies Internal Controls (RAI) Regional Entity notes Entity s ability to identify, assess, and correct reliability concerns Regional Entity may choose not to test certain standards/requirements as risk assessment shows less risk for certain standards/requirements for that Entity Entity s documented and verified Internal Controls provide reasonable assurance of compliance Focus of compliance monitoring and enforcement activities shifts from finding every incidence of non-compliance to identifying gaps that could result in gaps in Bulk Electric System reliability

8 Risk Elements Identify risks at ERO level NERC/RAPA RISC Regional Risk Groups Identify risks at Regional level Scope Review ICE Identify IRA Transform risks into reliability elements (functional application of risk)

9 Inherent Risk Assessment How susceptible is the entity to identified risk themes? Review Identify Scope IRA Events Entity Specific Data ICE Legal or Regulatory Factors Affecting Reliability Regional Factors Affecting Reliability Compliance History

10 Inherent Risk Assessment Example - CIP Risk Assessment for three entities with no Critical Cyber Assets (CCA) Inherent risk assessment captured unique aspects of each and allowed for tailored monitoring scope Entity #1 BA for many independent municipalities on shared EMS, MRRE Conducted on-site audit of registered entity as MRRE audit Entity #2 BA previously identified CCAs, positive compliance history/mitigation plans, alternative compliance monitoring activities in period Spot check CIP-002 only Entity #3 LCC monitored by RTO Monitor testing by RTO, re-test sub-set of standards/requirements to ensure appropriate and sufficient testing

11 Inherent Risk Assessment Example Risk assessments conducted for three wind farms with similar registrations (GO/GOP) 131 applicable Requirements 28 Requirements on 2014 AML Inherent risk assessment captured unique aspects of each and allowed for tailored monitoring scope Wind Farm #1 Vanilla example 20 requirements in scope for audit (only 5 on AML) Wind Farm #2 Also responsible for interconnection of nuclear facility 25 requirements in scope for audit (only 5 on AML) Wind Farm #3 Doesn t own collector bus 2 requirements in scope (neither on AML) Guided Self-Certifications instead of Compliance Audit

12 Internal Control Evaluation Identification of control activities Existing registered entity documentation Prior knowledge from past interactions Walkthroughs of individual processes Design evaluation Will the control prevent, detect, or correct non-compliance with reliability standards? Operational effectiveness Gather evidence that demonstrates control is implemented as designed Control is achieving desired objective If control is effective, Regional staff may, in some circumstances, rely on control testing in place of testing for strict compliance

13 Internal Control Evaluation Example Internal controls related to COM-002 Preventive control involves random review of operator communications, followed by feedback and corrective actions Registered Entity uses three-part communication for routine communications Detective control involves complete review of any situation in which a directive may be issued Conclusion was that Registered Entity will identify and address issues timely Based on results of internal control testing, Standard was not tested directly

14 Internal Control Evaluation Example Internal controls related to PRC family of Standards Preventive control involves use of work order management system to track testing activities and alert on upcoming or missing tests Detective control involves random monthly management review of 10% of maintenance and testing records Conclusion was that Registered Entity will meet all requirements related to maintenance and testing intervals Criteria for management review not clearly documented Resulted in reduced sampling for related Standards While intervals will likely be met, not sure whether management review is consistent with Region s application of standards

15 Internal Control Evaluation Example Entity has internal control for maintenance Maintenance internal control covers multiple NERC Reliability Standards PRC-005 PRC-008 PRC-011 FAC-003 EOP-005 CIP-003 R6 CIP-006 R8 Regional Entity tests design and effectiveness of one internal control to gain reasonable assurance of compliance with seven NERC Reliability Standards

16 Using Work of Others Entity conducts independent evaluations of individual standards/requirements (entitylevel control governance) Conclusion: Entity effectively implements and independently tests NERC reliability standards/requirements Based on results of internal control testing and Entity independent testing, audit significantly scaled. 43 potential requirements in AML for the audit 13 requirements use work of internal independent audit group 3 standards repeat work of internal independent audit group to confirm adequacy of work Based on results of internal control testing and Entity independent testing, audit significantly scaled. 34 potential requirements in AML for the audit 15 requirements - use work of independent auditor 4 standards - repeat work of independent auditor to confirm adequacy of work

17 Oversight Scoping Example Compliance monitoring activities may become more frequent, but less intrusive Shift from large, infrequent audits to continuous monitoring Focused scope for monitoring places emphasis on areas that present highest risk to reliability of the BES Regions to make better use of all the tools provided by the CMEP, not just audits

18 Oversight Scoping Example ERO-based Oversight Address Interconnection- or Region-wide concerns Guided Self-Certifications, scheduled as needed Functional Oversight Address unique considerations on a per-entity basis Guided Self-Certifications or Spot Checks Compliance Audits based on triggers that merit broader, more comprehensive reviews Event Oversight, as needed Guided Self-Certifications Spot Checks or Investigations as needed

19 Guided Self-Certification Example FAC R6 Self-Certification Focused on narrower scope based on region-wide concerns Emphasis was on How do you know you re compliant? Abandoned check-the-box approach Defined how to perform self-assessment and set expectations for how results were to be communicated to Regional staff Inquired about internal controls How are existing, erroneous facility/equipment ratings identified? How are new errors in facility/equipment ratings prevented?

20 RAI Misconceptions Misconception: When a requirement is removed from scope, the Registered Entity is no longer responsible for compliance with that requirement. Truth: Everything that was enforceable, remains enforceable. Misconception: A reduction in scope or scale will lead to a shorter and less rigorous compliance monitoring engagement. Truth: Removing lower-risk requirements provides Regional staff time to focus on higher-risk requirements. Misconception: There is an old and new way of monitoring compliance. Truth: Regional Entities are identifying common processes and procedures to add more clarity and consistency in the audit process.

21 How You Can Help Establish/identify/evaluate internal controls to effectively manage high frequency security obligations Start transition to CIPv5 Familiarize yourself with Compatibility Matrix Read Lessons Learned from CIPv5 pilots Volunteer within your regions to assist with transition issues

22 Our Next Steps Complete RAI Guidance Documents Risk Elements IRA ICE CMEP Tools Train Regional Entities on high level concepts to ensure consistent application of RAI across 8 regions Evaluate results of RAI and CIPv5 pilots, integrate RAI concepts into CIPv5 compliance monitoring strategies for the regions

23 RAI Enforcement Activities Overview and Examples

24 24 RELIABILITY ACCOUNTABILITY

25 End-State Vision Focus on noncompliance that poses a serious and substantial risk to the reliability of the bulk power system (BPS) exercise discretion whether to initiate an enforcement action for issues that do not pose a serious or substantial risk Encourage registered entities to continue to self-identify, mitigate, and record noncompliance Under oversight of NERC and Regional Entities NERC and the Regional Entities continue to maintain visibility regarding all noncompliance 25 RELIABILITY ACCOUNTABILITY

26 Milestones Reached FFT process used to resolve moderate risk noncompliance Triage process in place throughout the ERO enterprise Dissemination of information on risk assessment and mitigation through the user guides Aggregation and compliance exception programs test processes designed to implement the end-state vision 26 RELIABILITY ACCOUNTABILITY

27 Triage and Process Flow Risk and Control Assessment Input Audit, Spotcheck, etc. Log, Self- Report, Self-Cert. Triage Record Compliance Exception Enforce Feedback to Risk and Controls Assessment 27 RELIABILITY ACCOUNTABILITY

28 User Guides Common understanding of: what constitutes a good initial record how risk of noncompliance is assessed how mitigation is documented and evaluated Final versions posted in May 2014 Available at: 28 RELIABILITY ACCOUNTABILITY

29 Aggregation/Logging Program Who Can Participate? Entities that have been assessed for the capability of self-assessment, continuous monitoring, and mitigation Additional entities will be included throughout the program as they are evaluated by Regional Entities What Can be Aggregated/Logged? Minimal risk issues (see User Guide) What Happens to Items Logged? Presumption that they will be resolved as compliance exceptions 29 RELIABILITY ACCOUNTABILITY

30 What s in the Aggregation Log? Region Name of Entity NCR Std. Req. Issue Description Risk Assessment Mitigating Activity 30 RELIABILITY ACCOUNTABILITY

31 Issue Description Issue Description Entity should provide sufficient details to assist its Regional Entity in making a fair and informed assessment of the noncompliance. These details should include: method of discovery corrective actions taken cause, if determined, of the noncompliance duration of noncompliance the full details surrounding the noncompliance itself 31 RELIABILITY ACCOUNTABILITY

32 Risk Assessment Risk Assessment In assessing risk, Entity should consider all factors that mitigated the actual and potential risk, taking care to avoid after-the-fact determinations. Only minimal risk issues are eligible for aggregation. 32 RELIABILITY ACCOUNTABILITY

33 Mitigating Activity Mitigating Activity Entity must describe its efforts to mitigate the noncompliance. Mitigating activities should resolve the noncompliance and prevent recurrence. 33 RELIABILITY ACCOUNTABILITY

34 Compliance Exceptions What Can be Compliance Exceptions? Minimal risk issues (see User Guide) Who Can Participate? 2014: selected entities 2015: all registered entities Additional entities will be included through RELIABILITY ACCOUNTABILITY

35 Compliance Exceptions All minimal risk issues eligible to be compliance exceptions regardless of discovery method Rebuttable Presumption Decline to Enforce: A self-identified, minimal risk issue, for a registered entity with demonstrated internal controls, that has been allowed to aggregate minimal risk issues 35 RELIABILITY ACCOUNTABILITY

36 Rebutting the Presumption of Discretion Possible decision to enforce if related management practices appear to be failing: Failure to identify noncompliance in a timely manner Failure to properly assess the risk posed by the noncompliance o Risk not minimal, but moderate or serious/substantial Failure to mitigate properly o Possibly demonstrated by: - Same noncompliance in the recent past - Same noncompliance in the near future Other facts and circumstances indicate broader programmatic failures 36 RELIABILITY ACCOUNTABILITY

37 37 RELIABILITY ACCOUNTABILITY

38 CIP Scenario: As a result of performing a required vulnerability assessment (R3), an entity finds a Protected Cyber Asset with software or logical ports that is different from the baseline configuration it developed (CIP-010 R1.1). The entity has identified this issue as posing a minimal risk to reliability. 38 RELIABILITY ACCOUNTABILITY

39 CIP Aggregation Participant Entity logs noncompliance in tracking spreadsheet Logs action plan info (Part 3.4) Triage Process Outcomes: Compliance exception More information needed Enforce Compliance Exception Minimal risk confirmed Presumption of discretion 39 RELIABILITY ACCOUNTABILITY

40 CIP Entity Not Participating in Aggregation Entity self-reports noncompliance to Regional Entity Describes action plan info (Part 3.4) Triage Process Outcomes: Compliance exception More information needed Enforce Compliance Exception Minimal risk confirmed Self-identified issue Eligible for discretion 40 RELIABILITY ACCOUNTABILITY

41 CIP Found at Audit Regional Entity discovers noncompliance Auditors increase testing to determine extent Triage Process Outcomes: Compliance exception More information needed Enforce Enforce Minimal risk confirmed FFT, SNOP or FNOP depending on other issues found 41 RELIABILITY ACCOUNTABILITY

42 CIP Compliance Exception Enforce 42 RELIABILITY ACCOUNTABILITY

43 CIP Factors Supporting Compliance Exception: Internal controls Strong baseline configuration processes Limited number of noncompliance Isolated noncompliance event Factors Supporting Enforcement: After further review, several BES Cyber Systems were not appropriately baselined Numerous related process issues identified in the course of the audit Known vulnerabilities were identified that can affect the poorly baselined machine or other BES Cyber Systems 43 RELIABILITY ACCOUNTABILITY

44 CIP R2 Scenario: An entity discovers that an employee completed CIP cyber security training 15 months and two weeks after the date the employee previously completed the training (Parts 2.2, 2.3). The entity has identified this issue as posing a minimal risk to reliability. 44 RELIABILITY ACCOUNTABILITY

45 CIP R2 Issue will be eligible for discretion and recording as a compliance exception Aggregation o Presumption of discretion Self-report/Audit o No presumption of discretion Whether issue will be a compliance exception depends on: Risk level only minimal risk issues are eligible Other relevant facts and circumstances 45 RELIABILITY ACCOUNTABILITY

46 CIP R2 Examples of Factors Supporting Compliance Exception: Employee was on leave and did not receive automated reminder s Entity self-identified issue through regular training log reviews Limited number of employees completing training late Unforeseeable technical issue with reminder messages Issue with employee was addressed promptly Employee completed CIP training in previous years Employees are generally aware of CIP obligations 46 RELIABILITY ACCOUNTABILITY

47 CIP R2 Examples of Factors Supporting Enforcement: No effective control, practice, or system in place to ensure training is completed in a timely manner Employees generally not aware of CIP obligations Multiple employees completing training late (or not at all) Entity did not discover issue promptly Entity did not mitigate issue promptly Underlying was foreseeable and could easily happen again (poor internal controls) 47 RELIABILITY ACCOUNTABILITY

48 CIP R2 Scenario: Entity staff discovers that, after a group of five visitors leaves its facility, the security guard only noted four of the visitors in the visitor log. (Part 2.2) The entity has identified this issue as posing a minimal risk to reliability. 48 RELIABILITY ACCOUNTABILITY

49 CIP R2 Issue will be eligible for discretion and recording as a compliance exception Aggregation o Presumption of discretion Self-report/Audit o No presumption of discretion Whether issue will be a compliance exception depends on: Risk level only minimal risk issues are eligible Other relevant facts and circumstances 49 RELIABILITY ACCOUNTABILITY

50 CIP R2 Examples of Factors Supporting Compliance Exception: Issue was promptly discovered Logs are regularly reviewed according to a process or procedure Other protections were in place at the time of the issue and were used to detect the noncompliance: o Video monitoring o Continuous escort with authorized staff o Visitor access proximate card were used Issue promptly mitigated: o Technical error resolved and backup solution implemented o Second security guard added at entrance to assist at busy times 50 RELIABILITY ACCOUNTABILITY

51 CIP R2 Examples of Factors Supporting Enforcement: Security guard at entrance is regularly overwhelmed with visitors and badged employees and no guard is added to assist Missing visitors/logs not discovered promptly (or at all) Issue not mitigated promptly or properly Escort was responsible for ensuring all visitors were logged and failed to do so Other protections were not in place or failed: o Continuous escort not maintained o Access controls or video monitoring not present or non-operational 51 RELIABILITY ACCOUNTABILITY

52 NERC Webinar Reliability Assurance Initiative Update MRO Pilot Company Experience Doug Johnson American Transmission Company LLC June 19, 2014 atcllc.com

53 Scope of RAI Pilot Project Testing New Compliance and Enforcement Models Midwest Reliability Organization (MRO) engaged American Transmission Company LLC (ATC) as a RAI pilot company in early Working with the MRO executive team, ATC is piloting the following RAI concepts and models: Use of the compliance exception concept as an enforcement discretion tool Use of self-logging and aggregation for minimal risk potential violations as an enforcement discretion tool Entity risk assessment process for compliance oversight scoping Processes to evaluate the capabilities of an entity s internal controls / management practices for compliance oversight scoping atcllc.com 53

54 Overview of Reliability Assurance Initiative ATC s Perspective The NERC Reliability Assurance Initiative (RAI) will redefine how NERC and the Regional Entities will regulate the industry. Focus of RAI has been on making NERC and the Regional Entities more effective / efficient regulators. RAI is about how NERC and the Regional Entities will execute new compliance and enforcement models. RAI is not about substantially altering how companies assure reliability. Cost effective regulatory oversight which continues to aid companies in assuring the secure and reliable operations of the Bulk Electric System is the expected outcome. atcllc.com 54

55 Purpose of RAI ATC s Perspective Enforcement Discretion Elimination of zero tolerance model Not all violations need to be processed with the same rigor Minimal risk violations should be self-logged and administered internally via the company s Corrective Action Program Minimal risk violations should be processed as compliance exceptions Compliance Oversight Need to stop chasing compliance at the detriment to reliability Elimination of one size fits all approach resulting in right sized oversight programs Risk-based determination of compliance oversight scoping More effective utilization of industry and regulator resources on higher risk activities 3-year audits should not remain NERC s/ MRO s primary oversight tool atcllc.com 55

56 Piloting the Enforcement Discretion Models Compliance Exceptions Non-Material and minimal risk violations identified during ATC s November 2013 compliance audit have been categorized and are being processed as compliance exceptions. Self - Logging and Aggregation ATC has recently begun self-logging minimal risk violations pursuant to NERC s aggregation program. ATC believes these enforcement discretion concepts are the most important benefits from RAI atcllc.com 56

57 Self-Logging and Aggregation New Compliance and Enforcement Models atcllc.com 57

58 Compliance Exceptions New Compliance and Enforcement Models atcllc.com 58

59 Piloting the Risk Assessment & Oversight Scoping Models Entity Risk Assessment In early 2013, ATC conducted an internal company assessment to identify applicable higher risk Reliability Standards ATC responded to a pre-audit survey which provided information to allow MRO to test their Entity Risk Assessment process. The ATC internal and MRO risk assessments resulted in similar results - some 35 Reliability Standards were determined to be of higher risk. Evaluation of Management Practices / Internal Controls ATC assembled the management practices / internal controls for our identified higher risk Reliability Standards. MRO tested their internal controls assessment process on a sample of ATC s management practices. atcllc.com 59

60 Piloting the Risk Assessment & Oversight Scoping Models (Cont.) Audit Scope MRO right-sized ATC s November 2013 audit scope based upon their risk assessment and internal controls evaluation. Future Compliance Oversight Model Risk based oversight model should not be limited to simply adjusting audit scopes. ATC expects that the new risk based compliance oversight model will result in the more effective and targeted utilization of all of the compliance oversight tools available for use by the Regional Entities. atcllc.com 60

61 Compliance Oversight Programs New Compliance and Enforcement Models atcllc.com 61

62 ATC s Risk Based Oversight Scope Interconnection-Wide FAC-003 (Vegetation Management): 2003 blackout (tree contact). COM-002 (Communications and Coordination): 2003 blackout (emergency coordination, clear communications). PRC-001 (System Protection Coordination): Protection system schemes must be coordinated between multiple operators to ensure adequate performance. PRC-005 (Protection system, UFLS, UVLS and SPS maintenance): Identified by RISC as high priority, protection systems need to act in virtually every power system event, and are key to limiting cascading. CIP-005 (ESP), CIP-006 (PSP), CIP-007 (CCAs): For entities with CCAs, the protection of the networks,cyber assets, and physical perimeters of critical infrastructure is a high risk in today s environment. Region-Wide TPL-002 (Cat B Transmission Planning) MRO is limited both thermally and by stability, the EHV transmission infrastructure was originally built up in the 70s and is now expanding a lot of new transmission infrastructure is required so TPLs are important. FAC-008 (Facility Ratings) MRO has identified a trend in its region related to incorrect element ratings used for Facility Ratings. A focused self cert has already taken place to address. Candidate for future removal if the trend ceases based on these efforts. PRC-023 (Transmission Relay Loadability) The inclusion of this standard is linked to the trend seen on FAC-008, but is important as well to ensure that operators have a chance to remediate operating conditions before equipment trips. TOP-002 (Requirement 11, short-term operations planning) MRO wants to ensure that each TOP has the necessary tools to adequately perform analyses of real-time operations. atcllc.com 62

63 ATC s Risk Based Oversight Scope (Continued) ATC Risk Based Scope CIP standards (balance of CIP not listed above, if CCAs) COM-001 (Telecommunications) EOP-001 (Emergency Operations Planning) EOP-003 (Load Shedding Plans) EOP-004 (Event Reporting) EOP-005 (System Restoration) EOP-008 (Loss of Control Center) FAC-010 (System Operating Limits Methodology for Planning Horizon) FAC-013 (Assessment of Transfer Capability) FAC-014 (Establish and Communicate System Operating Limits) NUC-001 (Nuclear Plant Interface Coordination) PER-005 (System Personnel Training) PRC-004 (Analysis of Misoperations) PRC-006 (UFLS programs) PRC-017 (Special Protection System Maintenance and Testing) TOP-004 (Transmission Operations) TOP-006 (Monitoring System Conditions) TOP-008 (Response to Transmission Limit Violations) TPL-001 (System Performance Under Normal Conditions) TPL-003 (System Performance Following Loss of 2 or More BES Elements) Summary of ATC s Overall Baseline Audit Scope Approx. 35 Reliability Standards and 100 Requirements atcllc.com 63

64 ATC s Ongoing Compliance Program Enhancements ATC will continue to adapt to the anticipated new compliance and enforcement models ATC s compliance program will be further enhanced and structured based upon the COSO internal control integrated framework The COSO framework includes the following components: Control Environment Risk Assessment Control Activities Information and Communication Monitoring Activities atcllc.com 64

65 ATC s RAI Transition Focus ATC s short-term transition focus includes the following activities: Strengthen processes and programs which support the COSO fundamental concepts associated with the COSO components and principles. Continue to build-out and refine internal controls with focus on the Reliability Standard requirements (high-risk) which are expected to represent MRO s customized oversight program for ATC Better define and execute activities to monitor the execution and effectiveness of internal controls Bring more formality and structure to ATC s Corrective Action Program Strengthen our position of being audit ready atcllc.com 65

66 Summary New Compliance and Enforcement Models Pilot Results ATC believes the new compliance and enforcement models which are being piloted have been demonstrated to be successful. A paradigm shift to these new models is necessary to achieve more cost effective regulatory oversight while sustaining secure and reliable operations. Benefits of RAI ATC s Perspective ATC s ability to self-log non-material / low-risk potential violations without the need to make self-report submittals. The self-logging would be supported by much less burdensome mitigation plan development and review processes. The efficient processing of non-material / low-risk violations through a compliance exception process which results in no enforcement reviews or actions. Right-sized compliance oversight programs based upon risk and more effective and targeted use of oversight tools with the likely elimination of the 3-year cycle audits. Less intrusive oversight engagements by our regulators due to demonstrated ability by ATC to find, assess, correct, and preclude recurrence of issues. Ability of ATC and regulators to focus resources on issues of greater risk to the BES. Overall a more cost effective approach to regulating. atcllc.com 66

67 Reliability Assurance Initiative Update A Registered Entity s Experience Aggregation of Minimal Risk Issues and Compliance Exceptions June 19, 2014

68 Outline The Pilot Program Aggregation of Minimal Risk Issues Compliance Exceptions Implementation Aggregation Results Lessons Learned Next Steps 2

69 The Pilot Program Aggregation of Minimal Risk Issues Program Period: October 2013 April 2014 NPCC requested NYPA s participation Registered Entity: Maintain and submit a tracking spreadsheet to the Regional Entity at least once every six months Regional Entity: Determine if any of the issues require further mitigation or enforcement action If no concerns, the issues may be eligible to be compliance exceptions 3

70 Implementation NYPA and NPCC Monthly conference calls Notification process for a possible minimal risk issue Process to determine actual and potential risk Process to track and remediate the issue 4

71 Monthly Conference Calls NYPA and NPCC Reviewed actions from last meeting Reviewed the tracking spreadsheet Reviewed and discussed each issue Discussed implementation matters Reviewed RAI pilot program status 5

72 Notification Process NYPA and NPCC NYPA leveraged its internal controls Discovery and investigation of possible violations NYPA s investigations included: Description of the issue and fact-finding Identification of standard(s) and requirement(s) Mitigation actions taken or proposed Potential and actual risk assessments Previous violations related to the issue NPCC was notified of possible violations, once confirmed internally 6

73 Notification Process NYPA and NPCC NPCC affirmed inclusion of minimal risk issues in the pilot program based on: NYPA s internal investigation documents Concurrence with NYPA s risk assessment Upon affirmation by NPCC: Issue was added to tracking spreadsheet NYPA issued an internal legal hold Supporting materials were submitted to NPCC If not affirmed by NPCC, the issue was self-reported 7

74 Aggregation Results for NYPA and NPCC For October 2013 to April 2014: NYPA discovered seven (7) possible violations through application of its internal controls All were or are being investigated internally o Three (3) are currently being investigated o One (1) was closed not a violation o Three (3) were affirmed by NPCC as minimal risk issues and included in the pilot program Compliance Exceptions: NYPA expects compliance exceptions for the minimal risk issues in the aggregation pilot Disposition as compliance exceptions is pending 8

75 The Challenge: Lesson Learned Violation Risk Assessments Alignment of Registered Entity s and the Regional Entity s assessment of the risks Factors considered: o Facilities associated with the issue o System conditions during the period of non-compliance o Input from internal Subject Matter Experts NYPA developed an internal procedure / control: o A consistent assessment of the risks specific to a possible violation o Aligned with NPCC s assessment Illustrated importance of having a dialogue with the RE to establish an acceptable level of trust 9 Helpful References: NERC s Self-Report and Mitigation Plan User Guides and the Aggregation Pilot Program Document

76 Lesson Learned Issue Tracking NYPA used its internal controls to: Maintain all relevant records for review Populate the tracking spreadsheet from internal investigation documents The Pilot Program tracking spreadsheet: Find, Fix and Track reporting format No NPCC or NERC IDs were initially assigned Not practical for supporting documentation Version control 10

77 Importance of Registered Entity s internal controls for a self-policing culture Less frequent dialogue with RE as comfort with Entity's internal controls increases In NYPA s case: Lessons Learned Other Reduction in internal discovery and investigation activities may be possible There is a possible significant benefit, if confirmed minimal risk violations become compliance exceptions 11

78 Developed an appreciation of NPCC enforcement staff s challenges Final disposition of violations processed under the Pilot Program is uncertain Final Comment: Lessons Learned Other Aggregation of minimal risk issues can benefit a Registered Entity provided: o The Entity has internal controls that are aligned with the objectives o Minimal risk issues are processed as compliance exceptions 12

79 Next Steps NYPA and NPCC NPCC entered NYPA s Pilot Program violations into its compliance tracking application (CDAA) to generate NERC and NPCC Tracking IDs NPCC is developing modifications to CDAA to allow an entity to directly enter violations and provide NPCC with enforcement options NYPA agreed to extend its participation in the RAI enforcement pilots NPCC plans to expand the number of Registered Entity s in its RAI enforcement pilots 13

80 80 RELIABILITY ACCOUNTABILITY