Report on 2011 NPCC Culture of Compliance Survey Initiative

Report on 2011 NPCC Culture of Compliance Survey Initiative Development In September 2010, NPCC Staff began an initiative that would attempt to identify a registered entity s Culture of Compliance. NPCC defines a Culture of Compliance as an environment in which a registered entity demonstrates proper awareness and commitment to meeting it s obligations as they relate to the NERC Reliability Standards. A survey was used as the mechanism to document an entity s Culture of Compliance. The framework of the survey was based on the following FERC policy statements: October 2005 Policy Statement on Enforcement (a.k.a. 13 Questions ) May 2008 Revised Policy Statement on Enforcement October 2008 Policy Statement on Compliance September 2010 Revised Policy Statement of Penalty Guidelines The FERC statements highlight attributes and aspects that they believe a registered entity s compliance program should contain to ensure reliability and be considered effective. The survey was created by NPCC as a Tool For Improvement. The goals of undertaking the survey effort were: For NPCC to gain an overall perspective of the culture in the region For each registered entity to identify opportunities for growth or confirm that the proper processes were already in place For NPCC to identify and share, with all registered entities, Examples of Excellence and Best Practices In October 2010, a working group was formed from within the NPCC Compliance Committee to meet and discuss the specific content and basis of the survey. The working group created a set of questions to ensure that the entity would understand the context of the expected response. The working group presented the survey to the NPCC Compliance Committee for comment several times. In January 2011, the survey and corresponding introduction letter were approved by the NPCC Compliance Committee to be sent to an initial set of registered entities (Trial Test or Phase 1). From February through August 2011, the survey was sent to all NPCC registered entities over four phases (Phase 1, Phase 2, Phase 3, Phase 4). NPCC Compliance Committee and stakeholder feedback, received through conference calls, meetings, and webinars, was Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 1

considered in making refinements to the questions before the second phase was sent out. NPCC began assessing the received responses in May 2011 and completed all of the assessments in early December 2011. The activities and efforts undertaken by NPCC related to the Culture of Compliance Survey has been shared with various sectors of the industry which include the NPCC Board of Directors, the NERC Board of Trustees Compliance Committee, the Electric Reliability Organization Compliance and Enforcement Management Group (ECEMG), the North American Generator Forum (NAGF), the North American Transmission Forum (NATF), and the Large Public Power Council (LPPC). In addition, the survey is being considered as the model for the culture of compliance component of NERC s forthcoming Entity Assessment efforts. Coordination of Responses The survey was sent to 298 registered entities within NPCC over four different phases. Due to de-registration activity in this timeframe, there were actually 285 registered entities that NPCC was requesting a survey from. It became apparent early in the evaluation effort that here were a considerable number of instances where corporate ownership dictated that multiple registered entities could in fact be grouped into one survey response. NPCC coordinated with multiple Primary Compliance Contacts to refine the list to allow for more accurate tracking and metrics. As a result, the modified list shows that the 285 eligible entities would result in 164 actual survey responses being returned. In actuality, NPCC received 145 survey responses covering 263 entities. Surveys Sent Out #Sent #Entities Covered Surveys Received #Received #Entities Covered Surveys Feedback Calls Assessed #Assessed #Required #Completed #Entities Covered Surveys Not Received # Not Received #Entities Covered 164 285 145 263 145 145 127 241 19 22 *9 surveys covering 13 entities will not be returned due to de-registration activity Evaluation Criteria The positive attributes and aspects identified by FERC, and used as the basis of the survey, do not take the following characteristics of a particular registered entity into account: Functions Registered as with NERC Number/Type of Assets Number of Employees Number of Customers Load Density Served Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 2

Initially, this left NPCC with the task of comparing apples to oranges. However, NPCC determined that a more reasonable means to assess the responses would be to use the knowledge and experience of NPCC Staff in a subjective fashion to determine, based on generic categories of the functional model, whether the minimum requirements of a sufficient culture were met. This was done to give all NPCC registered entities the same opportunity for NPCC to determine that the entity had conveyed the existence of a sufficient culture of compliance on their survey response. The categories for relative comparison used by NPCC Staff were RC/BA, TO/TOP, GO/GOP, DP/LSE, and PSE. These categories became self evident and are characterized by the common content, common themes, and common depth of data provided by an entity in that particular category. NPCC Staff performed the subjective analysis of each survey response and placed each survey into one of three measurement buckets: Green The survey response indicates that the entity meets or exceeds all minimum characteristics of a favorable Culture of Compliance. Yellow The survey response indicates that the entity does not meet all minimum characteristics of a favorable Culture of Compliance. Red The survey response indicates that the entity meets few or no minimum characteristics of a favorable Culture of Compliance. The overwhelming majority of survey responses left no doubt in the mind of NPCC Staff that the entity had a Green culture and an inherent fundamental understanding of their compliance obligations. A minority of responses left some question on the mind of NPCC Staff as to whether a Yellow culture was being portrayed. In those cases, a feedback phone call was used to verify and pursue additional questions to reach a conclusion one way or the other as to what measurement bucket to assign the entity. There were no survey responses that conveyed a Red culture. Feedback Effort NPCC determined that entities would benefit further if verbal feedback to each survey response was offered. This resulted in a tremendous effort for NPCC Staff to contact the entity, schedule a mutually agreeable time for the feedback call, and track pending and completed feedback calls. NPCC also wished to use the feedback call to offer its appreciation to each and every entity for participating in this voluntary effort. Feedback phone calls began in August with Phase 1 entities. The number of phone calls increased in September, October, and November, while tailing off into December. NPCC completed 127 calls covering 241 entities. Phone calls for 18 surveys associated with 22 entities that are only registered as a PSE will have feedback calls scheduled in December 2011. NPCC found this effort to provide personalized feedback to be an extremely rewarding experience and win-win for both the entity and NPCC. The feedback served to increase Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 3

the communication and relationship between both sides. From an entity perspective, it also allowed the entity s compliance personnel to hear confirmation from NPCC that the entity is moving in the right direction with regard to their compliance activities. As closure to the entity, NPCC further provided documentation of the feedback discussion and culture measurement color to each entity via written letter which was sent at the conclusion of each call. Examples of Excellence NPCC reviewed the survey responses in an effort to identify Examples of Excellence to share with all registered entities. An Example of Excellence is a program, procedure, ideal, plan, idea being investigated for feasibility, or initiative related to compliance program activities that is unique to the response of one entity s survey and cutting edge in nature. An Example of Excellence may exist as words only or it may be a full-fledged program or policy. An Example of Excellence is not a benchmark. 1. An entity is beginning work on establishing a documented concept of regulatory margin. Their desire is to exceed what would be considered normal activities by building a margin into some aspects of their compliance activities. Experience has taught them, upon review of lessons learned, that performing activities to a minimum standard may lead to a false sense of security. 2. A small entity supplements its training resources by using its own employees to self teach. A unique standard is assigned to each employee twice per year by the manager which requires the employee to perform research and report back to the manager with the proper understanding of the standard. The employee must also understand how any ISO procedures are related as well. The employee is then required to present to the rest of the O&M personnel. The manager reviews the effort of the employee and includes this on the annual performance review. This process serves to keep the employees involved on a more regular fashion while tying it to their compensation. 3. A larger entity with a reliability steering committee has a documented action item for their Information Group to determine if voice recognition technology exists that can assist with three-part communication. They are investigating if there is software that can offer real time prompts if three part communication is not used as required in COM-002. Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 4

Best Practices NPCC reviewed the survey responses in an effort to identify Best Practices to share with all registered entities. A Best Practice is a program, procedure, ideal, plan, or initiative related to compliance program activities that NPCC Staff has determined could be used as a benchmark for compliance activities. A Best Practice may apply to all or certain registered entities for use, in whole or in part, as a measurement tool. The basic elements of a Best Practice are not usually unique. Municipals working together 1. All seven New York State Municipals (DP, LSE) registered with NPCC belong to the Municipal Electric Utility Association (MEUA). The municipal s all work together with the same third party firm and the MEUA to implement compliance programs and processes that benefit all of them. The three parties work together to monitor standards, update programs as necessary, and share best practices amongst each other. Senior Management at these municipal s have all met together to review the uniform compliance program that they have all agreed to implement. Owners with sites in multiple regions 2. Entities that have multiple-regions associated with their ownership or operations groups have regular (monthly, bi-weekly) involving all sites in the multi-regions to discuss common issues in advance to root out all problems before they become compliance related. This allows for sharing of information amongst the sites about what the different regional entities are doing and saying. This allows for a site manager in a different region to begin reviewing how his particular site may stack up against what another NERC region is advertising. Electronic intranet access 3. Many entities have Corporate Compliance Program Intranet/Sharepoint sites that have hyperlinks to all relevant NERC 693 and NERC CIP policies and procedures, corporate compliance scorecard, corporate self certifications, corporate self reports, corporate code of conduct, corporate ethics and compliance documents, FERC affiliate restriction statement, etc. This provides for one stop shopping at any time of the day for any type of employee and is a subtle reminder of management s expectations and commitment to compliance. Orientation to NERC Standards 4. Many entities have programs that require new employees, regardless of job responsibility, to take an orientation class/tour that includes discussion on how the entity acts or operates to meet NERC standards. This reinforces the commitment and expectation of management at an early juncture in the employee s career. Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 5

Internal risk-based assessment 5. Recognizing that the compliance program has a finite number of employees and time that can be dedicated to it, some entities use an internal risk-based compliance monitoring system. Based on the entity s registration, current issues in the industry, and recent NOP s, the entity identifies the areas or standards with the greatest exposure to violation, greatest risk to the system, and greatest financial implication if a violation was to occur. This allows the entity to further focus in on a successful implementation of distinct aspects of the program which hold a greater overall potential to ensure that the entity does not have a NERC standard violation or will minimize the harm inflicted (reputation, financial, additional oversight by NPCC/NERC) if a NERC violation occurred. Communicating the existence and expectations of the program 6. Communicating the existence of the program by various means better ensures that complacency is not encountered and continues to reinforce the commitment of management. Typical examples of steps to build a solid foundation for communicating the existence and expectations of the internal compliance program: a. Mandatory instructor led or computer based training b. Internal all-hands meetings at corporate offices or field locations c. Monthly meetings involving Senior Management and monthly and/or periodic meetings with the staff having responsibility for managing compliance d. Internal operation and standing committees composed of company SME s e. Incorporation of reliability standards compliance roles and responsibilities in job descriptions for positions that have related accountability f. Inclusion of goals/target for compliance in department and employee annual performance plan g. Regular communication that include grass roots employees (emails, posters, Town Hall meetings) to solidify corporate commitment and expectations Steps to take to combat complacency 7. Example of programs or actions that larger entities take to determine if they are becoming complacent in their compliance activities. These steps used in varying degree can be extremely useful in battling complacency or identifying practices that could be adopted to maintain or to improve the overall program. a. Establish the foundation for communication as described in Best Practice #6. b. Use of an Internal Audit Group to conduct periodic audits of selected elements of the program c. Use of the Compliance Group to conduct regular assessments of standards applicable to the entity s registration d. Use of outside consultants and legal counsel to review selected elements of the program to make recommendations for improvement e. Scheduling discussions with other similarly registered entities within the region to compare program specifics Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 6

f. Use of trade forums to compare program specifics g. Attend NERC, NPCC workshops Making employees aware of responsibility 8. Training for an employee key to the entity remaining compliant can generally be broken up into two areas. Each area has distinct key issues that the entity should continually review to ensure that the employee is kept up to date. a. Regulatory concepts, policies, and procedures i. Defining NERC, FERC, NPCC ii. Existence of Penalties for not adhering b. Internal Processes and Procedures i. The specific role of that employee to the entity remaining compliant ii. Have a means or process in place to review new procedures and share changes to existing policy iii. Training on communicating in a manner to ensure reliability iv. Training on how the employee or company will respond when a concern or deficiency is identified v. Reporting a violation/deficiency vi. Means to prevent a violation by investigating certain flags vii. Retention of documentation policies and procedures Documenting the scope of Management Committees 9. Larger entities with Management or Steering Committees have charter documentation. The descriptions should remain clear and concise. Such language serves to keep the committee focused on their role and purpose. Key aspects of the scope of a committee charter include: a. Purpose b. Key Functions c. Goals d. KPI s the group is responsible for e. Membership makeup f. Frequency of Committee Meetings g. History of Committee Methods used to assess status of program 10. In the survey, NPCC asked how the entity gauged their progress toward the overall annual compliance goal. Many entities responded that metrics to gauge progress toward a goal were not applicable. They described that their goal was to not accumulate any NERC violations and as long as that ultimate goal was met, there was nothing to gauge or to give an indication as to the status of their compliance activities. In some cases, it was described that audit results are used as a gauge. Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 7

The end goal of the entity is typically a Key Performance Indicator (KPI and Lagging Indicator ) identified by Senior Management to not sustain any compliance violations for the year. Leading indicators can provide valuable information to an entity before a compliance violation occurs. Leading indicators are a means for the entity to gauge the status of compliance activities and the progress of all of the under-lying activities related to the overall KPI. Of course, the degree of utility that can be realized by using leading indicators is based upon the amount of data available which has a direct correlation to the size of the entity and the registered functions. The leading indicator can be used as a flag by the entity to identify an issue before an actual compliance violation occurs. All of the activities associated with leading indicators don t necessarily need to be completed to meet the overall KPI goal. However, the further that they are completed will better serve to flag an issue before a compliance violation occurs. Examples of Leading Indicator Metrics Progress on planned internal assessments tracked by completion date Performance on responding to NERC Alert within a certain timeframe Performance on providing comments to NERC CAN s Performance in communicating the issuance of new CAN s to relevant employees within a certain timeframe Performance on entering data related to Self Certification into library in advance Performance on reviewing data related to Self Certification in advance Performance of Senior Manager to sign in advance of Self Certification due date Performance on submitting Self Certification to region on time Progress on planned review of compliance procedures tracked by due date Progress on capital work milestones for TFE s that are not legacy Performance in documenting near miss and lessons learned related to compliance within a certain timeframe Performance on implementing corrective actions within a certain timeframe Performance on disseminating near miss and lesson learned to relevant employees Progress of planned operator training tracked by due date Performance of getting new operators NERC certified within a certain timeframe Performance on determining whether a self report is required within a certain timeframe once a issue is identified Performance on completing mitigation plan milestones by due date Performance on reviewing and updating CA and CCA assets by due date Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 8

Performance in producing incident reports related to violations within a certain timeframe Performance in producing the critique related to violations within a certain timeframe Progress on planned RSAW review Performance in participating on NERC Standard Drafting Team Performance in reviewing documents and providing comments to NERC Standards Drafting Team Performance in collecting and reviewing manual logs by due date Performance on determining whether bulk power event is a misoperation within certain timeframe Performance on reporting a misoperation to region with certain timeframe Progress on reviewing assets that may be effected by new BES definition by a certain date Performance related to actions identified as red flags from previous year Progress on planned vegetation maintenance activities Establishing a commitment to compliance 11. Many larger entities have a documented Compliance Process Manual or a Compliance Process Document that list the steps that the entity intends to take to formally establish the company s commitment to compliance via the ICP. Communication and education serve as the cornerstone. Typical components of a documented ICP: a. Establishment of an Oversight or Steering Committee with defined functions b. Identify employees and Senior Management who are responsible and accountable for applicable portions of the program c. Conduct robust training to ensure respect for compliance with NERC Reliability Standards and ethical standards d. Institute meaningful, ongoing, and expeditious communication relating to compliance issues to resolve uncertainties e. Develop procedures for monitoring and preventing occurrences while also providing procedures for timely corrective actions. f. Develop reporting protocols to inform relevant management and agencies g. Conduct internal assessments and self evaluations h. Identify, document, and actively monitor regulatory requirements (Note: Several responses acknowledged that there is not an overall guiding document in existence related to the ICP. The entity stated in these cases that their ICP was actually a series of activities, processes, and procedures.) 12. Smaller entities without a formal process manual or process document take fewer, yet still sufficient for their size, steps to establish the commitment of the entity to compliance. The typical aspects are: a. A formal Code of Conduct and Business Ethics exists Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 9

b. Using a small group of management employees perform all of the NERC functions. Third party assistance is routinely used to supplement. c. The small group monitors NERC, NPCC activities and attends the related functions. d. The entity communicates the generic NERC compliance requirements to the majority of the remaining personnel on an annual basis. e. Senior Management meets regularly to discuss and formally approves the compliance activities that is being undertaken by the smaller group of management. Next Steps NPCC will present a recap of the Culture of Compliance Survey effort at the December 2011 Compliance and Registration Workshop in Boston. The Examples of Excellence and Best Practices will be shared at that time with all attendees. In addition, this report will be posted on the NPCC website. Finally, a webinar will be held in January 2012 to share the Examples of Excellence and Best Practices with those entities that are unable to attend the workshop in Boston. Report on NPCC Culture of Compliance Survey Initiative - 12/12/11 Page 10