OPERATIONAL EXCELLENCE ACROSS THE ERO ENTERPRISE: Adding Value to the Compliance Monitoring and Enforcement Program A Discussion Paper By the Midwest Reliability Organization I. INTRODUCTION This discussion paper builds on the February 2014 ERO whitepaper Improving Coordinated Operations across the Electric Reliability Organization (ERO) Enterprise (Improving Operations). 1 The purpose of Improving Operations is to describe a vision and path forward to achieving a highly effective and efficient ERO Enterprise. 2 To this end, specific actions are outlined to be taken, or at least considered. 3 This discussion paper focuses on one of those responsibilities compliance with and enforcement of mandatory Reliability Standards and addresses how the ERO Enterprise can improve operational effectiveness and achieve operational excellence in the execution of the Compliance Monitoring and Enforcement Program (CMEP), 4 and thereby support a reliable bulk power system (BPS) and add value for stakeholders. II. BACKGROUND NERC included Improving Operations as part of its five-year performance assessment filing for the ERO Enterprise submitted to the Federal Energy Regulatory Commission (FERC or Commission), in FERC Docket No. RR14-5-000. 5 The Commission accepted the assessment, finding that NERC and the Regional Entities continue to satisfy the applicable statutory and regulatory criteria. 6 Nevertheless, the Commission remained concerned that the ERO model, as implemented, had not yet achieved the consistency and uniformity needed for an effective and 1 See http://www.nerc.com/aboutnerc/keyplayers/documents/ero_enterprise_operating_model_feb2014.pdf. 2 Id. at page 1. 3 Id. at page 3. 4 See generally North American Electric Reliability Corp., et al., 119 FERC 61,060, order on reh g, 120 FERC 61,260 (2007). The CMEP is referenced in Section 400 and found in Appendix 4C of the NERC Rules of Procedure, http://www.nerc.com/aboutnerc/pages/rules-of-procedure.aspx. 5 In this filing, NERC and the Regional Entities described the progress they had made on improving coordination of their CMEP activities. For example, they developed a direct feedback loop for registered entities through a consistency hotline, a standardized auditor handbook with plans to expand the handbook to cover all CMEP activities, and a more formalized manner to handle registered entities in multiple Regional Entity geographies. They also tried to interact more meaningfully with each other, for example, through the efforts of the ERO-Executive Management Group and the Regional Entity Management Group, which proactively and persistently stress the importance of working collaboratively and sharing information between and among NERC and the Regional Entities. Further, the Regional Entity staffs regularly met with each other to promote consistency in the CMEP processes and actively participated in NERC-led committees, task forces, and working groups. 6 See North American Electric Reliability Corporation, Order on the Electric Reliability Organization s Five-Year Performance Assessment, 149 FERC 61,141.
efficient compliance monitoring and enforcement program. 7 Accordingly, the Commission directed NERC to submit an informational filing by November 20, 2015, that describes its progress in increasing consistency and promoting coordination across the ERO Enterprise, including the specific actions listed in Improving Operations. 8 In addition, earlier this year, the Commission gave its conditional approval to the ERO Enterprise s Reliability Assurance Initiative (RAI). 9 RAI implementation entails the exercise of substantial discretion in monitoring compliance with and enforcing the mandatory Reliability Standards, and, therefore, represents a significant change in how the Regional Entities approach their CMEP responsibilities. The Commission directed NERC to file an annual report on RAI by February 19, 2016. 10 The convergence of two FERC proceedings the Five-Year Assessment and the RAI Annual Report present an opportunity to examine ways to build from the successes of an emerging regulatory environment and take the next steps in maturation of the ERO Enterprise operations. Accordingly, this discussion paper is intended to contribute to the ongoing improvement of CMEP operations. III. OPERATIONAL EXCELLENCE ACROSS THE ERO ENTERPRISE: A SOUND ORGANIZATIONAL STRUCTURE WITH FEWER, BUT MORE MEANINGFUL METRICS Improving effectiveness and efficiencies to achieve operational excellence in the execution of the CMEP is a challenge not to be underestimated. As Improving Operations recognizes, the current nine-entity model is not without challenges and risks. 11 FERC Commissioner LaFleur captured the situation at a 2013 Reliability Technical Conference when RAI was on the horizon. She first questioned whether the structure was right, and suggested building in the consistency [at the front end] so you don t have to negotiate to it. 12 At the same conference, NERC President and CEO Gerry Cauley stressed, [w]e have to operate as one organization. 13 With these thoughts in mind, the next logical step in the ERO Enterprise s efforts to ensure consistency in CMEP implementation is to operate truly as a single organization. 14 To this end, 7 Id. at P 70 (footnote omitted). 8 Id. at P 71. 9 See North American Electric Reliability Corporation, Order on Electric Reliability Organization Reliability Assurance Initiative and Requiring Compliance Filing, 150 FERC 61,108 (2015). 10 Id. As directed, on July 6, 2015, after seeking comments, NERC submitted its proposed ROP changes to the Commission, which still has them under review. See Proposed NERC ROP Appendix 4C Section 3A.0 Enforcement Discretion, submitted by NERC on July 6, 2015, in Docket No. RR15-2-000. 11 Operational Whitepaper at page 7. There is a nine-entity CMEP model because, in addition to its CMEP oversight responsibilities, NERC performs a limited role as a Compliance Enforcement Authority under ROP Section 401.1. 12 Remarks of Commission Cheryl LaFleur, Transcript of Commissioner-Led FERC Reliability Technical Conference, Docket No. AD13-6-000, et al. (July 9, 2013) at page 115. 13 Remarks of Commission Gerry W. Cauley, Transcript of Commissioner-Led FERC Reliability Technical Conference, Docket No. AD13-6-000, et al. (July 9, 2013) at page 104. 14 As discussed later, this is not to suggest that the ERO Enterprise should become one organization. 2
the current structure with nine different, albeit acceptable, ways of regulating and nine sets of processes, procedures, and systems, needs to be simplified. Simplification requires defining how the work gets done through uniform systems, work procedures, and processes. A unified foundation allows performance standards to be set to execute consistently and effectively. This is especially important now because, NERC and the Regional Entities have greater enforcement and oversight discretion under the RAI. Therefore, greater uniformity is necessary to assure that the ERO Enterprise is administering the CMEP even-handedly and without discrimination across North America. A sound, simplified, and more focused organizational structure with fewer but more meaningful metrics will enable the ERO Enterprise to achieve operational excellence, deliver value for the public it serves, and reduce risk to the reliable operations of the BPS. Operational excellence requires consistency in execution to legitimize measures. Conversely, a high degree of variability in execution by the Regional Entities in the field hampers the establishment of metrics to drive performance, and more important, may increase the risk to the reliable operation of the BPS. For example, NERC and the Regional Entities have taken different approaches to applying the Compliance Registry Criteria in NERC ROP Appendix 5B. 15 As a result, entities have been registered for functions they do not actually perform, such as a Reliability Coordinator (RC) being registered as a Transmission Operator (TOP), where the RC does not actually operate transmission. Likewise, RCs registered in more than one region may not be registered in the same way because the Regional Entities apply the Compliance Registry Criteria differently. Additionally, the application and use of joint registration organizations and coordinated functional registration appear varied across the registry, and not all entities on the registry are even legal entities. Such varied application of the criteria undermines a critical foundational aspect of the reliability regulatory program, namely, who owns and actually operates the BPS, and self-evidently affects how and with respect to whom the CMEP is implemented. While the current structure needs to be addressed, creating one organization for the purpose of monitoring and enforcing mandatory Reliability Standards is neither practical nor desirable. The current construct recognizes strong Regional Entities with technical expertise and in-depth knowledge of the grid in their areas support reliable BPS operations. The system is a network of connections, first developed on a local, then regional, and then interconnection-wide basis, therefore local and regional system knowledge and experience are essential in maintaining a strong technical understanding of reliability. The Regional Entities, which exist in close proximity to the entities with operational responsibility for local systems, are able to promote broader stakeholder engagement to better respond to risks. The ERO model depends on a strong decentralized, regional presence. 16 15 See http://www.nerc.com/filingsorders/us/ruleofproceduredl/nerc_rop_effective_20150319.pdf. 16 Improving Operations supports the current structure: [T]he highly technical and intricate complexities of planning and operating the bulk power system and given that an interconnected system is a system of systems the concept of local, regional and wide-area views on risk is necessary and has been long accepted. Improving Operations at pages 4-5 (footnote omitted). 3
So, the question is, what organizational structure can be employed to pursue operational excellence, or as Gerry Cauley said in 2013, how can NERC and the Regional Entities operate as one entity? Clearly, other enterprises have been able to function successfully and productively in this way. Illustratively, the financial and banking industry developed regionally with the establishment of the Federal Reserve System, which is comprised of a central bank to set monetary policy and collect statistics, and regional banks with their own boards, governance, and staff to meet the needs at a local and regional level. While the Federal Reserve Bank system has separate governance and staff, all the procedures for data collection and analysis are the same. Likewise, organizations that deliver goods and services across a wide geographical area typically develop a corporate field operations management team to oversee and support the local operations groups. This becomes a governance control to assure field operations deliver according to performance standards, which can then be objectively measured because there are uniform processes, procedures, and systems in place. As noted earlier, the ERO Enterprise has continuously moved in this direction, that is, to make processes more uniform to prevent discriminatory treatment of registered entities. 17 The auditor checklist, for example, is a way of ensuring each Regional Entity properly plans, performs, and reports audits. This uniformity does not prevent judgement or variations in the technical assessments of the subject being audited and the specific facts and circumstances under review. Its application recognizes that, say, a transmission operator in the western frontier of the Eastern Interconnection is much different than a transmission operator along the eastern seaboard, and takes into account that with different risks, different facilities, and different geography, the scope and perhaps even the compliance techniques used, will be different. But, every Regional Entity should be expected to follow the same processes and procedures for the conduct of its CMEP work. Today, the model allows inefficiencies and variability because each Regional Entity develops its own processes, procedures, and systems; consistency is mainly achieved reactively rather than being driven proactively by deliberate design. For example, in its oversight function, NERC currently asks the Regional Entities for their procedures to conduct a CMEP activity, instead of developing and designing uniform procedures to be followed and continuously improved. While the current approach aims for uniform outcomes, there are simply too many variables to accomplish that. More to the point, the goal should not be uniform outcomes, but just and reasonable results based on uniform processes, procedures, and systems. Along the same lines, at present, the structure of the implementation of the CMEP at the regional level and the structure of oversight at the national level are not aligned. Today, NERC has designed itself around programs that mirror and tend to replicate Regional Entity work, not the overall operations of the Regional Entities. As described by FERC, NERC s organizational charts have continued to generally reflect the broad functional categories, or program areas, initially approved by the Commission 17 See supra note 5. 4
when it certified the ERO (i.e., operating divisions within NERC). As such, they do not clearly reflect the relationships that exist in the performance of particular tasks. NERC operates in a matrix (i.e., cross-functional) environment with staff participating in specific programs that sometimes span operating divisions, making it difficult to discern lines of responsibility and authority. [ 18 ] Under these circumstances, the first step to establishing a uniform approach to CMEP implementation is to realign the NERC staff to correspond to a NERC CMEP field operations group to lead and support the Regional Entities in conducting their CMEP work, unifying and focusing CMEP operations across the ERO Enterprise. Expertise in such a group would include significant management leadership experience in running field operations, which in turn would entail working in a centralized office with the local operating offices. The mandate for the group would include, in close coordination with the Regional Entities: Development of policies, procedures and practices for the field operations at the regional level, Identification and development of training, Development of techniques to pilot and roll-out new initiatives, and Identification of field operations systems and other automation to improve productivity and transparency of information while protecting confidential information. A realignment of NERC staff is consistent with the observation in the Improving Operations that [t]hese roles suggest different qualifications and activities for NERC staff than in the past. 19 The ERO should structure its core activities around centralized functions (like standards), not those that are decentralized (namely, the component parts of the day-to-day CMEP operations). For those activities delegated to the Regional Entities, the ERO should establish the proper structure to support and monitor decentralized operations, and emphasize the core competencies necessary to manage decentralized operations, such as training programs, policy and procedure development, and systems development. In addition, the NERC CMEP field operations group should have deep operational (including cyber security) expertise in the Reliability Standards. This group would cover Registration, Compliance, Risk Assessment, Mitigation and Enforcement. All ERO staff members should also develop the skills to work horizontally and vertically, but more importantly, both program and execution responsibility should be under one roof with the field operation group assuming operational responsibility for field execution, side by side with the Regional Entities. 20 The NERC CMEP field operations group would ultimately be responsible for establishing the procedures that govern 18 See North American Reliability Corporation, Final Audit Report, Docket No. FA11-21-000 (May 4, 2012), at pages 16-17. While FERC made this statement over three years ago, NERC continues to operate under the matrix structure around program areas. 19 Operational Whitepaper at page 8. 20 As discussed in the Operational Whitepaper, while NERC has overall responsibility for program design, its leadership role must be conducted in a consultative fashion, which recognizes the regional nature of the risks being managed. Id. at page 9. 5
the day-to-day operations of the CMEP work, and the Regional Entities would be responsible for following the procedures. At the same time, the Regional Entities should have the discretion to respond to local and regional risks, and be responsible for providing insight and input to NERC to continuously improve field operations. Among its first tasks after developing consistent operating procedures in collaboration with the Regional Entities and ensuring effective, excellent execution, the NERC CMEP field operations group would need to use more focused metrics to measure the performance of the Compliance Enforcement Authorities (CEA) (NERC under ROP Section 401.1 and the Regional Entities) and, accordingly, ascertain the effectiveness of the CMEP. Performance metrics depend on uniform processes, procedures and systems and must be focused. Currently, however, there are over 100 goals, objectives, deliverables and actions in the NERC Strategic Plan, Improving Operations, and existing metrics. This is simply too many requirements to effectively manage and reasonably hold NERC and Regional Entities accountable. Furthermore, many of the metrics are measures, not metrics. 21 For example, recording how many audits are conducted annually is a measure, whereas interpreting what level of risk the entity posed to reliable operations, and the quality of the findings in terms of potential or actual risk to reliable operations, is a metric. Today, the ERO Enterprise tracks the number of audits conducted without regard to the quality of those audits. Therefore, the NERC CMEP field operations group should aim to reduce and simplify the applicable metrics, aiming for relatively few headline metrics (such as zero cascading events ) to better advance the statutory mission of reliable operations. Finally, in conducting its work, the NERC field operations group should focus on the work of the Regional Entities as CEAs, and not have any stakeholder responsibilities such as outreach. 22 Instead, the group would act as an important link and feedback loop between field operations and program leadership concentrating on operational excellence. Therefore, a separate external stakeholder relations group focused on interacting with, as well as training and educating, external stakeholders should also be pursued and segregated from the CMEP governance and activities. IV. CONCLUSION Led by NERC, the ERO Enterprise has laid the groundwork to pursue the vision of operating as one cohesive organization and reduce variability and risk in field operations performance. Building on this groundwork, a dedicated NERC field operations group, collaborating with the Regional Entities, would be able to achieve operational excellence, effective execution, and consistency to deliver value for stakeholders and reduce the risk to the reliable operations of the BPS. 21 A measure is a raw data item that is directly measured, and that will be used to calculate a metric. A metric is an interpretation of a measure or a set of measures that are used to guide performance. 22 Today, many of the NERC program areas co-mingle stakeholder and CEA activities, which dilutes the focus on operational execution. 6
Appendix NERC s Eleven Action Items Clarifying and Refining Roles and Responsibilities Action item 1 NERC includes in its review of Regional Entity business plans adequacy of resources and alignment of the plans for achieving delegated function objectives and outcomes described in the three-year plan. Action item 2 NERC, in collaboration with Regional Entities: a) develops a comprehensive overarching design and set of controls, reporting requirements, and feedback mechanisms for each delegated statutory function, including the essential oversight elements listed above; b) reevaluates its organizational structure and leadership qualifications necessary to provide effective design and oversight of statutory activities; c) identifies functional qualifications for select delegated activities, such as auditors and investigators; and, d) adopts risk-based approaches to monitoring performance of delegated functions and providing effective feedback and coaching to continuously improve overall ERO Enterprise performance; Action item 3 Regional Entities: a) Work in a coordinated fashion to support NERC in the development of comprehensive functional program designs and controls; b) adapt existing regional programs for delegated functions to conform to emerging program designs provided by NERC; and c) ensure Regional Entity staffs meet qualification and training requirements. Coordinated Strategic and Business Planning Action item 4 NERC and Regional Entities develop and maintain a joint three-year strategic plan for the ERO Enterprise describing the goals and deliverables for statutory functions and this plan should guide the development of each Regional Entity s annual business plans. Additional goals and deliverables that are complimentary or supplemental to the strategic plan may be developed at the Regional Entity level. Action item 5 NERC and Regional Entities develop and transparently report results based on a common set of performance measures focused on bulk power system reliability outcomes and effectiveness of the statutory programs. These measures are considered in the performance management program at each entity. Coordinated Operational Decision-Making Action item 6 ERO Enterprise senior leadership (ERO-Executive Management Group or ERO- EMG ) continues maturing the collaborative decision-making process and setting the direction and policy for the enterprise as well as driving this collaboration throughout the ERO Enterprise organizations. 7
Action item 7 All nine ERO entities abide by the joint enterprise decisions of the ERO-EMG, and the NERC CEO should make final determinations if consensus is otherwise not achievable for an action that is required. Achieving Consistency Action item 8 With NERC leading, the ERO Enterprise develops a core set of methods, practices, procedures, and tools to support unified implementation of the major statutory functions of NERC. In doing so, the ERO Enterprise commits to put the best talent available throughout the collective organizations to achieve this goal. Sharing Tools and Infrastructure for Delegated Functions Action item 9 - The ERO EMG develops ERO Enterprise IT Applications, where appropriate, to support common processes, to enhance the efficiency and effectiveness of Regional Entities practices, to increase the consistency of the interface with registered entities, and to facilitate NERC s oversight function. Coordinating External and Cross-ERO Enterprise Communications Action item 10 All parties of the ERO Enterprise, including leadership and staffs, convey shared and consistent messages from the enterprise perspective and communicate together to minimize messages emphasizing self-importance or uniqueness. Action item 11 NERC and Regional Entities continue the joint board coordination to ensure oversight and accountability of all elements of the enterprise, and should continue to refine and expand coordinated outreach to government entities in the U.S. and Canada, stakeholders, and media. 8