Results for 2001 indicate achievement year-to-date and/or projected levels.

Transcription

1 What the Program Has Achieved: Since its inception, the NYISO incentive compensation program has proven highly effective at motivating employees toward achieving high priority objectives for the organization. A primary reason the program has been so successful results from its being developed and implemented at the employee level rather than being directed top-down from senior management. This has achieved high-level commitment and buy-in from the employees. In 2000, the first year of the program, there was one Reliability Gateway goal and nine individual goals. The gateway goal was fully achieved and of the individual goals, seven were achieved at superior level and two at target level. This resulted in an achievement of 139 % of the 2000 incentive goal program. A summary of the 2000 goals is provided as an attachment. For 2001, a second gateway goal, the Market Gateway, was added to the Reliability Gateway and there were ten individual goals. Year-to-date the gateway goals are being fully met, five individual goals are tracking at superior level, four at target level, and one failed to meet the threshold objective. The program is tracking toward an achievement of 105 %. The following table summarizes the incentive program results during the first two years: Incentive Goals Gateway goals achieved 1 of 1 2 of 2 Goals achieved at Superior level 7 5 Goals achieved at Target level 2 3 Goals achieved at Threshold level 0 1 Failed to achieve goals 0 1 Total achievement 139 % 110 % Results for 2001 indicate achievement year-to-date and/or projected levels. Approach for the 2002 Program: For the 2002 program, the planning process was expanded to include not only the incentive goals, but also, development and tracking of the Business Plan. This change was intended to drive the business planning process further down into the organization, thereby providing more visibility to and involvement of all employees. 1

2 Work on the 2002 program began in June 2001 with the formation of the Business Plan and Incentive Goals team. The team includes thirteen individuals - two representatives each from Market Services, Operations & Reliability, Information Technology, Finance & Compliance and Strategic Initiatives; one representative each from Government Affairs & Communication and General Counsel & Regulatory Affairs; and the team leader who is the Manager of Strategic Planning. The team comprises a balance of managers/supervisors and individual contributors a mix that was intended to broaden knowledge and understanding of the process and to provide for orderly succession. The Budget, Standards and Performance Subcommittee (BS&PS), which reports to the Management Committee, submitted a list of suggested 2002 goals in May The BS&PS participated in the incentive goals process in prior years as one of their charter roles. The Business Plan and Incentive Goals team accepted the BS&PS suggestions as a primary input for developing the incentive goals and the 2002 Business Plan. Most of the BS&PS suggestions were incorporated into the incentive goals or in some cases included as Group or Department-level goals. Some were deemed impossible to implement at the present time. The following pages present a summary of the proposed 2002 incentive goals, the existing 2001 incentive goals, and a detailed explanation of each of the proposed incentive goals for Incentive Goals Alignment with Strategic Objectives The NYISO strategic plan identified four key strategic objectives for the organization. The incentive goals have been aligned with the four strategic objectives as indicated in the descriptions below. This relationship will be highlighted when the incentive goals are introduced to employees and it will be emphasized as the goals are tracked and reported on during the year. 2

3 Detailed Explanation of Incentive Goals for 2002: 1. Reliability Gateway Goal - Maintain Service to New York Control Area Achieving No Interruption of Load (> 300 MW for > 15 minutes) Because of Improper Implementation of NYISO Operating Procedures. Definition: The Reliability Gateway is a go/no-go goal, which emphasizes the importance of maintaining system reliability. Both Gateway goals must be achieved for there to be any payment earned under the NYISO Organizational Incentive Plan even if all of the other goals are achieved. The successful achievement of this Gateway is based upon the NYISO staff operating the bulk power system in compliance with all internal operating procedures, as well as those of NPCC, NERC and the DOE. Should there be a loss of load or deliberate load shedding, successful achievement of this goal is predicated upon the NYISO staff demonstrating, to the satisfaction of the CEO and the NYISO Board of Directors that the appropriate procedures were followed (i.e., the loss of load or deliberate load shedding did not result from inappropriate actions of the NYISO staff). Appropriately documenting the circumstances associated with a loss of load or deliberate load shedding event is the responsibility of the NYISO staff. In the event of a loss of load, the Operating Committee will review the events and provide their comments to the NYISO. The CEO and Board of Directors will exercise their discretion in determining whether or not this goal was achieved, to establish that the NYISO is responsible (to the extent of its bonus pool) if NYISO procedures are not followed and, as a result, service to NYCA load is not maintained. Why this is a challenge: A Gateway Goal should send the proper signal to employees about mission-critical activities. At the same time, because achieving the Gateway is a minimal threshold, there should be a very high likelihood of attainment (99+ %) with the appropriate level of commitment and support from staff. How it is tracked: System Operators actions are logged, and in the event that a load shed order is given analysis of the recorded data would provide the amount of load shed and the duration. BS&PS input: The BS&PS suggested using this goal as it is stated. 3

4 Comparison to 2001: This Reliability Gateway is identical to the goals used in 2001 and Since the NYISO began operations in 1999, there have been no instances of load shedding. Alternatives: The target levels for this goal are chosen to coincide with a NERC reportable event (>300 MW for >15 minutes). Alignment with strategic objectives: This Gateway Goal is aligned with the NYISO strategic objective to be the Leader in Reliable Service and Environmental Quality. Responsibility: The Operations Department is responsible for achieving this goal with substantial support from the Information Technology Group. 4

5 2. Market Gateway Goal Successfully Post DAM Schedules/ Forward Contracts 99 % of the Time. Definition: The Market Gateway is a go/no-go goal, which emphasizes the importance of ensuring that the marketplace is operational. As a go/no-go goal, unless it is achieved, a bonus payment will not be earned under the Corporate Goals and Objectives Bonus Program even if all of the other goals are achieved. The successful achievement of this goal is based upon the NYISO staff ensuring that the DAM is posted successfully. The goal will be successfully achieved if no more than five times during the year the DAM schedules/ forward contracts are unable to be posted. Should there be an occasion where the DAM schedules/ forward contracts are not posted due to a Force Majeure or by uncontrollable actions of the marketplace, that instance will not be counted as one of the five allowed occurrences. As long as the posting occurs prior to the end of the day, it will considered to be successful. Why this is a challenge: A Gateway should send the proper signal to employees about mission critical activities. At the same time, because achieving the Gateway is a minimal threshold, there should be a very high likelihood of attainment (99+ %) with the appropriate level of commitment and support from staff. How it is tracked: The OASIS MIS web site displays the date and time of actual posting of each day s DAM LBMP prices. The posting time will be used to assess achievement of this goal. BS&PS input: The BS&PS suggested a goal of 100 % successful posting (based on the 11 AM tariff requirement) with an allowance for late postings provided it was # 10 cumulative hours during the year. NYISO has proposed to retain the Market Gateway Goal as previously stated, but also, to add a new individual incentive goal (see # 5) to address the 11 AM posting time. Comparison to 2001: This Gateway Goal is identical to that used for 2001 when it was first introduced. Since the NYISO began operations in 1999, the DAM schedules / forward contracts have been posted successfully every day, never being more than two hours after the scheduled posting time. Year-to-date through September 2001, DAM postings have occurred after the 11 AM requirement on 11 days for a cumulative total of 298 minutes. 5

6 Alternatives: There was considerable discussion and debate about the definition of this goal both among the NYISO staff and at the BS&PS meetings. NYISO staff are concerned that there are many uncertainties regarding the performance of the SCUC with the anticipated changes such as virtual transactions, fixed-block transactions, etc., that may impact our ability to post prices on time. This situation is further complicated by the fixed period of time from the market closing at 5:00 AM until the 11:00 AM posting time. Alignment with strategic objectives: This Gateway Goal is aligned with the NYISO strategic objective to be the Competitive Market Champion. Responsibility: The Market Operations Department is responsible for achieving this goal with substantial support from the Information Technology Group. 6

7 3. Prepare Accurate Day-ahead, Non-weather-adjusted Load Forecasts Definition: This goal is based upon the cumulative 365-day accuracy rate of the daily peak load, nonweather-adjusted, day-ahead forecast (i.e., the average accuracy of the forecast over the 365 day period). The 2002 load forecast goal will not be adjusted for weather conditions. The concept for this change is to provide incentive to the NYISO staff to continually evaluate and select the most accurate weather forecast services upon which to base the zonal load forecasts. Zones that will be tracked are: West, East, J and K Threshold Target Superior NYISO proposed goal 2.8 % 2.5 % 2.3% BS&PS suggested goal 3.0 % 2.5 % 1.5 % Why this is a challenge: The non-weather-adjusted average error YTD through September 2001 was 2.87 %. The proposed Threshold for next year is a lower error percentage than that currently being achieved. The Target level, represents a 13 % improvement in accuracy compared to the current year s experience. The Superior level represents a level of accuracy that was achieved in only three of the last twelve months and is more accurate than the level being achieved on a weather-adjusted basis so far this year. How it is tracked: The error, expressed as the percentage difference between the actual and predicted daily peak load divided by the actual peak load, is tracked individually for each of the four zones every day. The average error is the sum of the four individual zonal errors divided by four. (Each individual zonal error is expressed as an absolute value, so negative errors do not cancel positive errors.) For 2002, there will be 365 observations of the average zonal error. The annual average zonal error is the sum of these observation divided by 365. This is the figure that will be evaluated against the goal attainment levels. BS&PS input: The BS&PS suggested the goal be based on the non-weather adjusted forecast for the dayahead zonal load forecasts with values as shown above. 7

8 Comparison to 2001: The 2001 goal is based on weather-adjusted forecast data with the following targets: Threshold Target Superior 2001 weather-adjusted goal 3.2 % 3.0 % 2.75 % Additional model refinements have resulted in improved forecasting for both zonal and statewide load. Through September 2001 achievement for weather-adjusted forecasts is 2.59 %. By comparison, the 2000 year-end weather-adjusted load forecast accuracy was 3.26 % The non-weather adjusted forecast accuracy is 2.87 % through September 2001 and by comparison the 2000 year-end figure was 3.5 % Alternatives: This is deemed to be the most appropriate goal regarding day-ahead load forecasting and its impact on the NYISO markets. The move toward use of the non-weather-adjusted accuracy has placed even greater burden upon the NYISO staff, while correspondingly reducing market risk faced by the Market Participants. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Competitive Market Champion. Responsibility: The Analysis and Planning Dept s Load Forecasting Unit is responsible for performing the load forecast calculations, obtaining weather service data, and for maintaining the monitoring records. 8

9 4. Comply with NERC Standards CPS-1 and CPS-2 Definition: Control Performance Standards (CPS) are criteria defined by NERC to monitor and compare power system operation across North America. The two CPS parameters are derived from the theory that system reliability is impacted by the effectiveness of frequency control. CPS-1 measures the Area Control Error (ACE) impact on frequency over a 12-month horizon, while CPS-2 measures the magnitude of short-term ACE values. The NERC compliance threshold for CPS-1 is 100% (typical values range from %) and for CPS-2 is 90 % (with a maximum of 100%). The NYISO proposed goals are as follows: Threshold Target Superior a) CPS-1 100% 12 of of of 12 b) CPS-2 90 % 10 of of of 12 (number of months achieved) Why this is a challenge: CPS-1 and CPS-2 values are not monitored in real-time as they are normally calculated on the following day. The CPS values are a function of the Area Control Error (ACE) that is influenced by generators not following their base point signals and by deviations in schedules with adjacent control areas. System Operators do not have direct control over the generating units or the control areas with which we schedule transactions, however they do act to avoid or resolve high ACE values thereby helping to maintain acceptable CPS values. A further complication results from changes that have been recently implemented, such as the elimination of accumulating base points and uninstructed overgeneration, which may affect the CPS quantities in ways that cannot be fully anticipated. How it is tracked: The data required to calculate CPS-1 and CPS-2 are collected continuously, and a daily summary is issued the following day. Compliance with the two standards is determined on a monthly basis. BS&PS input: The BS&PS suggested goals were as follows: Threshold Target Superior a) CPS-1 100% 12 of of of 12 mo. b) CPS % % % 9

10 For CPS-2, the BS&PS proposed a different approach than exceeding the minimum compliance threshold. In their view, it would be preferable not to incent operators toward overcompliance (approaching 100%), on the theory that there is a cost to regulating more than is required. Therefore, the BS&PS suggested the target and superior level goals should have narrower ranges, % and % respectively. Comparison to 2001: Below is a graph that shows CPS-1 and CPS-2 performance over the past twelve months. CPS1 CPS2 NERC Control Performance NYISO Compliance Year 2000/ Sep-00 Oct-00 Nov-00 Dec-00 Jan-01 Feb-01 Mar-01 Apr-01 May-01 Jun-01 Jul-01 Aug-01 Sep-01 Month Alternatives: There are no known studies substantiating that a savings would result from restricting control to a narrower band, although it is likely that there are regulation costs associated with achieving higher percentages. However, changes being planned for BME and SCD software (such as uninstructed overgeneration, virtual bidding, etc.) may have deleterious effects on CPS-2 that cannot presently be quantified. Also, the dispatchers are not able to observe CPS-2 performance in real time since the index is not calculated until the following day. 10

11 The reserve sharing agreement that was reached with ISO-NE will allow NYISO to reduce the in-state reserve requirement and may at times have a negative impact on CPS- 2. As a result, some of the envisioned cost savings that the BS&PS was hoping to achieve may be accomplished through the new reserve sharing practices. The NYISO does have the option of not using reserve sharing if it causes a perceived impact on reliability and/or CPS-2. The NYISO has an existing defined task to study the value of reliability within the planned Center for Market Excellence (CME) initiative. To address the issue raised by the BS&PS regarding CPS-2, this scope of this task will be expanded to specifically include research that attempts to quantify the economics of lower values of CPS-2 compliance. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Leader in Reliable Service and Environmental Quality. Responsibility: The Operations Department is responsible for monitoring and achieving this goal with substantial support from the Information Technology Group. 11

12 5. Timely Posting of DAM Schedules/Forward Contracts Definition: This is a new goal, created to complement the Market Gateway goal, which provides an incentive to achieve the 11 AM posting time for the Day-ahead Market a high percentage of the time. The NYISO proposed goal is as follows: Threshold Target Superior Achieve DAM posting 85 % 90 % 95 % by 11 AM this percentage of days in the year Why this is a challenge: This goal is a challenge due to the increased computational complexity and the finite period of time that is available to accomplish the Security-Constrained Unit Commitment (SCUC) process. This process requires the use of a complex software package, a number of manual actions, and at times, prompt decisions based on unusual circumstances that may be encountered. There are many uncertainties that could impact the SCUC process as a result of future changes being planned for the day-ahead market, for example, virtual transactions, and fixed-block transactions. These changes, even when thoroughly tested before implementation, may have unanticipated consequences that impact timely posting of the SCUC results. The situation is further complicated by the fixed period of time from the market closing at 5:00 AM until the required 11:00 AM posting time. The collective amount of contingency time in this period continues to shrink as the process becomes more complex. How it is tracked: Performance toward this goal will be determined based upon the DAM LBMP posting time on the OASIS portion of the NYISO web site. If the posting time is later than 11:00 AM, by as little as one minute, it will be counted as one day missed for this goal. BS&PS input: The BS&PS suggested a Market Gateway goal of 100 % successful posting (based on the 11 AM tariff requirement) of the day-ahead schedules/forward contracts with an allowance for late postings provided it was # 10 cumulative hours during the year. Comparison to 2001: This is a new goal for YTD through September the DAM postings have been late on 11 days. 12

13 Alternatives: This goal was added to demonstrate responsiveness to the MP s concerns and as an alternative to changing the Market Gateway Goal. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Competitive Market Champion. Responsibility: Market Operations is responsible for achieving this goal with substantial support from the Information Technology Group. 13

14 6. Real-time Price Certainty and Accuracy Definition: There are two issues regarding prices in the real-time market: 1) the number of hours that are reserved for review thereby causing price uncertainty and 2) the number of prices that are actually corrected after review thus resulting in inaccuracy. A goal focusing only on corrections might encourage making fewer price reservations. The reservation and correction process is conducted primarily by LECG, an outside consulting firm. The NYISO proposes a two-part goal to properly address each issue: Threshold Target Superior a) Hours Reserved: 22 % 20 % 18 % b) Intervals Corrected: 0.6 % 0.5 % 0.4 % It should be noted that price reservations are tracked based on hours whereas the realtime market produces prices at five-minute intervals. The reservation of a single price for any one of the five-minute intervals results in that entire hour being counted as reserved. Corrections, on the other hand, are tracked based on the individual five-minute interval prices. During a calendar month the number of interval prices will typically exceed 14,000. Thus, a 1 % level of intervals corrected would mean that 140 of the five-minute prices were corrected during the month. Why this is a challenge: The number of price interval corrections has been reduced from nearly 7 % at the beginning of 2000 to the present level of 0.48 % on a cumulative year-to-date basis. The potential to further reduce the number of corrections is limited. The challenge now will be to maintain the current level of accuracy as changes are made to the real-time market and the BME/SCD process. It could be argued that since little further improvement is expected this should not be an incentive goal. However, it is important that MPs see that NYISO remains focused on real-time price accuracy by way of this incentive goal. Price reservations have also been drastically reduced from more than 35 % early in 2000 to the current level of % on a cumulative year-to-date basis. The level of price reservations can be improved further, although not at the same rate of decline already achieved. Improving price reservations is the primary focus of this two-part incentive goal. 14

15 A further complexity results because both hours reserved and intervals corrected are determined by LECG, an outside consulting firm. In order to achieve the two-part goal, the NYISO Market Monitoring and Performance (MMP) unit will need to ensure the goals are conveyed to LECG and that their work is effectively managed. MMP will also work with LECG to develop more precise screens to reduce the need for reservations. LECG reviews real-time prices on a continual basis seven days per week. The process involves scanning the prices with software algorithms as well as inspection by a qualified analyst. Hours are reserved when anomalies are identified through this initial screening. Price reservations are announced to Market Participants within 24 hours of the day s realtime price posting which completes shortly after midnight. Reserved hours are then studied in a more detail primarily through a manual process. Corrections, if required, are announced to Market Participants within five calendar days of the reservation announcement. This five-day period evolved based on ECAs that were issued under the TEPs during NYISO s early days and was intended to allow sufficient time for weekends, holidays, and staffing requirements. Efforts are on-going to reduce the elapsed time till corrections are posted and the process is being reviewed as part of the SAS70 audit. A review of price correction data from two recent months follows: August 2001 October 2001 Average # of calendar days till correction Average # of business days till correction The higher numbers in August result primarily from mitigation due to thunderstorm alerts requiring a more extensive and detailed review by the analysts. How it is tracked: LECG provides a daily report that includes the number of hours reserved. A separate report details the intervals corrected. The MMP data services section maintains an archive file with data from all the reports and it tracks both the hours reserved and intervals corrected. The monthly figures as well as the cumulative percentages are provided in the NYISO monthly reports. BS&PS input: The BS&PS suggested a goal only for interval price corrections which was similar to last year but with smaller percentages as shown below. NYISO believes the suggested Target and Superior levels to presently be unachievable. 15

16 Threshold Target Superior Intervals Corrected: 0.5 % 0.25 % 0.1 % Comparison to 2001: The 2001 goal is based on intervals corrected as follows: Threshold Target Superior Intervals Corrected: 3.0 % 2.0 % 1.5 % Through September 19, the actual percentage of intervals corrected is 0.48 %, which exceeds the superior level of performance. The actual percentage of hours reserved is %. The graphs below show the 2001 year-to-date cumulative values for real-time price reservations and corrections as well as the proposed 2002 goals. 30% 28% 26% 24% 22% 20% 18% 16% Real-time Hours Reserved Jan Feb Mar Apr May Jun Jul Aug Sep YTD 2001 Cum Threshold 2002 Target 2002 Superior % 0.7% 0.6% 0.5% 0.4% 0.3% 0.2% 0.1% 0.0% Real-time Intervals Corrected Jan Feb Mar Apr May Jun Jul Aug Sep YTD 2001 Cum Threshold 2002 Target 2002 Superior

17 Alternatives: An alternative to the current NYISO proposal is to measure only the intervals corrected, as was recommended by the BS&PS and is the format of the existing 2001 goal. However, to balance reservations/certainty with corrections/accuracy, it is necessary to have both of the countering goals. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Premiere Business Model. Responsibility: The Market Monitoring and Performance unit is responsible for tracking and reporting on this goal. LECG is responsible for the day-to-day execution of the price review process. 17

18 7. Improve the Billing and True-up Process How it is tracked: The Customer Settlements department maintains a schedule of invoices to be issued each month and it tracks achievement relative to this schedule. BS&PS input: The BS&PS suggested a goal to issue the true-up bills within three months of original invoices. On-going discussions between Customer Settlements and the Transmission Owners, however, have concluded that a 3-month true-up period cannot be achieved given the existing metering limitations. As an alternative, the true-up intervals were agreed to be 4, 8, 12 and 24 months. Alternatives: The issuance of timely and accurate bills is a high priority issue for the NYISO and its Customers. This incentive goal is deemed to be the most appropriate way to incent the necessary action. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Premiere Business Model. Responsibility: The Customer Settlements department is responsible for tracking and reporting on this goal. 18

19 8. Improve Customer Responsiveness with Timely Resolution of Customer Inquiries Definition: The NYISO proposes a goal that will track the timeliness of responses via a Customer response program. Each answer sent to a Customer will include a message asking if the response was provided in a timely manner. The Customer will be able to reply immediately to express an unsatisfactory turn-around time and to identify the time that would have been acceptable. For purposes of tracking this goal, receiving no response from a Customer will be counted as a satisfied result. The proposed incentive goal is as follows: Threshold Target Superior Customer-identified 80 % 85 % 95 % responses that are timely Why this is a challenge: The NYISO operates some of the most complex electricity markets and one of the most congested transmission systems in the country. Many Customer inquiries require extensive review and evaluation of data and on-line systems to explain the mechanisms that affect markets and settlements. For example, during the first nine months of 2001, almost 50 percent of the inquiries tracked were transaction scheduling or billing questions. The Security Constrained Unit Commitment and Billing and Accounting system experts, whose primary responsibilities are running these systems, are often the ones called upon to explain the systems operations in response to these types of Customer inquiries. The time-consuming investigations that are required to address these questions have to be prioritized with dayto-day responsibilities and other special projects. Improving timeliness of responses to Customer inquiries will require a combination of internal process improvements for sharing information, continued improvements in the operation of our markets and transmission system, and further development of expertise across the organization. How it is tracked: Measurement of this goal will be done monthly and reported cumulatively for the year. This goal will be measured on an exception basis, through a modification to the existing Customer Relations Issue Tracking and Resolution system. With every response sent to Customers, a reply button will allow Customers to indicate if they feel their response was not timely. 19

20 This goal will be conservatively calculated, because measurement will only include responses to Customers. Many inquiries are addressed immediately on the telephone by Customer Account Representatives. The Threshold, Target and Superior values currently shown are preliminary values. Benchmarking data will be collected during the remainder of 2001 and as a result these values may need to be adjusted. BS&PS input: The BS&PS suggested there be three defined categories for inquiries and that there be four timing intervals to track responses: <4 hours, <24 hours, <10 days and <20 days. The suggested categories and goals were as follows: Threshold Target Superior Oper./real time In-day 75% <4 hrs 80%<4hrs 90%<4hrs Oper./real time Next day 85% <24 hrs 90%<24hrs 95%<24hrs Billing/analysis 10% <24 hrs 15%<24hrs 20%<24hrs All inquiries 65% <10 days 70%<10days 75%<10days All inquiries 80% <20 days 85%<20days 100%<20days During subsequent discussion with the BS&PS, it was agreed that multiple categories described above would be difficult to implement. The proposed NYISO alternative was considered to be acceptable. Comparison to 2001: The goal for 2001 is as follows: Threshold Target Superior Respond within 10 days 65 % 70 % 75 % Respond within 20 days 80 % 85 % 90 % Through September we have closed 78% of Customer inquiries within ten days and 88% within 20 days. Alternatives: The approach suggested by the BS&PS is complicated and would require development and implementation of an intricate tracking and reporting system. In addition, each inquiry received from Market Participants would require a judgment by Customer Relations to determine which category it should be assigned to. The proposed method of measuring timeliness asking the Market Participants if responses are timely requires that judgment determination to be made by Market Participants. 20

21 Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Best-in-Class Customer Service. Responsibility: The Customer Relations Department is primarily responsible for tracking and reporting on this goal. Support and commitment is required from the entire organization, however, to prepare the responses that are sent back to Customers. 21

22 9. Improve Customer Satisfaction with NYISO Services Definition: Customer satisfaction will be measured twice in 2002, once each in the Spring and Fall, via web-based surveys that will include all Customers. In 2000 and 2001 only one survey was conducted each year in the Fall. The targeted levels for improvement in overall Customer satisfaction will be as follows: Threshold Target Superior Average score This goal will be evaluated based on the average score achieved on thirty-four individuals survey questions. The improvement in average score will be based on the Fall 2002 survey compared to the Fall 2001 survey. The questions, which are included in the appendices, will be identical in both surveys to ensure consistency. Why achieving a high Customer satisfaction level is a challenge: NYISO Customers have divergent objectives for their participation in the markets and their interactions with the NYISO. On any given issue, the NYISO s responsibility to maintain the viable markets may very likely conflict with one or more of our Customers objectives. Across multiple issues, the NYISO s independent intent and actions to establish a workable competitive marketplace for electricity is likely to dissatisfy many Customers who actively seek to achieve advantages in the marketplace. Also, even with a level playing field, many contracts between parties in the marketplace result in financial winners and losers especially when Locational Based Marginal Prices vary from average levels. Customers who are successful in the marketplace will tend to be satisfied, while those who are not successful will tend to be unsatisfied. Thus, Customer satisfaction levels will always be lower than in industries where Customers do not have opposing interests. In addition, dissatisfaction with determinations made by committees and general unease with frequent changes in the industry is often projected onto the NYISO. How it is tracked: The incentive goal will measure the average score improvement, based on a scale of 1-5, in Customer Satisfaction between the Fall 2002 survey and the 2001 survey. The Spring 2002 survey results will be used to identify areas needing improvement, so that action plans can be developed and implemented to improve Customer Satisfaction and the Fall survey results. 22

23 BS&PS input: The BS&PS suggested that a percentage improvement over the 2001 survey, similar to the previous year, be established once the results of the 2001 survey are known. Comparison to 2001: The first Customer survey conducted in 2000 resulted in an average satisfaction score of 2.82 on a scale of 1-5. The average score achieved on the 2001 survey completed in October was 3.29, which represents a 16.7 % improvement over the 2000 survey. The individual category scores used to determine the average are shown below. Category: 2000 score 2001 score Training Customer Support Web Site Billing Tariffs Communications Average % Increase % Although a 16.7 % improvement was considered by NYISO staff to be a notable accomplishment, this achieves only the threshold level for 2001 goal based on the defined percentage improvement levels as shown below. Threshold Target Superior Average score Percent improvement 10 % 20 % 30 % Alternatives: A number of alternatives were considered in lieu of the approach described above. One option was to develop a Customer Satisfaction Index that would be based on a number of parameters that could be tracked internally by the NYISO. Although this approach has merit, it was not selected for an incentive goal primarily because it is based on an internal NYISO perspective. The Customer Satisfaction Index will, however, be developed in 2002 as a department-level goal. Depending on its effectiveness, it may be considered as an incentive goal for Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Best-in-Class Customer Service. 23

24 Responsibility: The Market Services Group is responsible for achieving this goal with substantial support from the entire NYISO organization. 24

25 10. Improve Project Management Deliverables with On-time Completion of High Priority Projects Definition: At least monthly, beginning December 2001 and through at least June 2002, the Senior Management Steering Group (SMSG), comprised of the leaders of Market Services, Strategic Planning, Operations, Information Technology, Regulatory Affairs, Finance, and chaired by the Manager of Projects, will meet to select projects that are meaningful to enhance the markets and/or NYISO operations. After June 2002 meetings may be held at a less frequent rate because resources for the year will have been mostly committed. At each meeting, the SMSG will select projects from the NYISO Program Plan that have their detailed functional specifications completed (including the project s description, objectives, deliverable, and resource requirements), and that are scheduled to be completed by December 31, In addition, the SMSG will take into account the priority assigned by the Project Prioritization Team (PPT) on which Market Participants play a key role. Should a selected project no longer be of high priority to the Market or NYISO operations, the SMSG might remove that project from the list. The NYISO s CEO will review the selections for appropriateness. At the first meeting, the SMSG will establish criteria for selecting projects from the Program Plan. Subsequent to that first meeting, every month (i.e., during January, February, March, etc.), the SMSG will again meet to select additional projects that meet the established criteria. Projects may not be selected that are well into their life cycle (and as a result, their successful implementation would be a given ). The incentive goal will be to achieve timely completion of a certain percentage of the high priority projects: Threshold Target Superior Percentage of on-time 80 % 85 % 95 % completion of projects designated by the PPT Why this is a challenge: Achievement of this goal is a challenge first and foremost because of the finite nature of NYISO resources available to accomplish projects and the nearly inexhaustible list of competing needs. This goal recognizes that dichotomy and seeks to reward performance that successfully and predictably applies finite resources to the satisfaction of high priority needs. 25

26 How it is tracked: Progress on this goal will be tracked in accordance with project schedules developed by the respective Project Managers, approved by the Project Sponsor in most cases a NYISO VP or Director, and maintained by the Project Services department. Based on the completion of activities and the achievement of milestones, Project Services will provide reports to the SMSG on the progress of the selected projects in accordance with the projects schedules. The SMSG will be the arbiter to determine whether a project has achieved its stated objective in accordance with its planned schedule. The SMSG may authorize deviations from original deliverables and/or schedules based on overriding circumstances (regulatory, scope expansion, or priority redirection for example). BS&PS input: The BS&PS suggested that the 2001 project management goal be used with the same values for percent completion. They also suggested that a 30 % weighting be applied to this particular goal. The NYISO proposes to weight each of the eight individual incentive goals equally (i.e %). Comparison to 2001: The incentive goal for 2001 is the same as that proposed for Through October 2001 there have been 40 projects identified and 24 have been completed on time. One project has missed its target, and the remaining 15 projects appear on target for successful accomplishment. Alternatives: There were no other alternatives considered. Alignment with strategic objectives: This incentive goal is aligned with the NYISO strategic objective to be the Premiere Business Model. Responsibility: Responsibility for achieving this goal is spread across the entire NYISO as are all of the various projects that make up the NYISO Program Plan. Responsibility for monitoring the success of this goal rests with the Senior Management Steering Group. This concludes the description of individual incentive goals. 26

27 27