SAN FRANCISCO PUBLIC UTILITIES COMMISSION CITY AND COUNTY OF SAN FRANCISCO AGREED-UPON PROCEDURES REPORT FOR SETTLEMENT QUALITY METER DATA PROCESSING

Transcription

1 AGREED-UPON PROCEDURES REPORT FOR SETTLEMENT QUALITY METER DATA PROCESSING TRADE YEAR

2 Contents Independent Accountant s Report on Applying Agreed-Upon Procedures 3 Section I Management s Executive Summary 4 5 Section II Summary of Findings 6 Section III Independent Accountant s Qualifications Summary 7 Section IV Description of Controls at SFPUC, Procedures Performed, and Results of Procedures 8 20 Page

3 INDEPENDENT ACCOUNTANT'S REPORT ON APPLYING AGREED-UPON PROCEDURES Barbara Hale Assistant General Manager for Power San Francisco Public Utilities Commission City and County of San Francisco APX, Inc. San Jose, California California Independent System Operator Folsom, California We have performed the procedures enumerated in Section IV of this report, which were agreed to by the management of San Francisco Public Utilities Commission ( SFPUC ), APX, Inc. ( APX ), and the California Independent System Operator ( CAISO or ISO ) (the specified parties), solely to assist you in evaluating SFPUC s compliance with Settlement Quality Meter Data Processing in accordance with the CAISO Scheduling Coordinator Self Audit Requirements. The procedures applicable to the Settlement Meter Data Processing level are the Monthly Meter Read Validating, Editing, and Estimating ( VEE ) Requirements and the profiled usage data files for the period August 1, 2014 through July 31, 2015 ( Trade Year ). Management is responsible for SFPUC s compliance with those requirements. This agreed-upon procedures engagement was conducted in accordance with attestation standards established by the American Institute of Certified Public Accountants. The sufficiency of these procedures is solely the responsibility of those parties specified in the report. Consequently, we make no representation regarding the sufficiency of the procedures described below either for the purpose for which this report has been requested or for any other purpose. We performed the procedures that SFPUC selected and deemed applicable in order to meet the requirement for audit and testing by Scheduling Coordinators per ISO Tariff Section , and which applied to SFPUC during the period of August 1, 2014 through July 31, 2015, as enumerated in Section IV. We were not engaged to and did not conduct an audit, the objective of which would be the expression of an opinion on compliance. Accordingly, we do not express such an opinion. Had we performed additional procedures, other matters might have come to our attention that would have been reported to you. This report is intended solely for the information and use of managements of SFPUC, APX, and CAISO in connection with regulatory oversight responsibilities of SFPUC, and is not intended to be and should not be used by anyone other than these specified parties. San Jose, California October 13, 2015

4 Section I - Management s Executive Summary Background The San Francisco Public Utilities Commission ( SFPUC ) has engaged APX, Inc. ( APX ) to provide Scheduling Coordinator services. APX is responsible for obtaining, processing, and submitting generation and load meter data to the California Independent System Operator ( CAISO ) for settlement purposes. This data is comprised of interval generation and station load data, interval load data, usage load data, and calculated data for streetlights, traffic signals, and other non-metered load collected from various sources. The SFPUC, as a certified Meter Data Management Agent ( MDMA ) self-provides this service by obtaining and processing the meter data and providing it to APX for submission to CAISO. SFPUC s electric load is metered by a combination of interval load meters and monthly usage, Customer Information System ( CIS ), meters. Approximately 87% of the total SFPUC load is recorded by approximately 240 interval load meters, which are currently read and reported by SFPUC. The remaining 13% is measured by the CIS meters, which are read by Pacific Gas and Electric Company ( PG&E ). SFPUC has approximately 1,745 service points that are metered based on elapsed usage. The meters are owned and read by PG&E on a monthly cycle as part of their normal meter reading process. Of these accounts, approximately 1,592 have simple kilowatt hour ( kwh ) meters, and the other 153 have time-of-use ( TOU ) meters that report reads by PG&E designated periods (on, partial, and off peak). In addition to the metered service points, SFPUC has 60 non-metered service points, for which PG&E has established an average monthly use value, which SFPUC uses in load calculations as if the service points were metered and read monthly. The total load for non-metered service points is less than 0.05% of the total CIS load. All of the usage meters are maintained and read by PG&E, and SFPUC receives a daily data feed containing the latest read information available from the PG&E systems. Load Data Processing CIS meter read data provided by PG&E is processed by SFPUC's Meter Data Management System ( MDMS ), subjecting it to an automated Validating, Editing, and Estimating ( VEE ) process, followed by manual staff review of any reads that failed validation. Interval load meters are read and undergo the VEE process by SFPUC using the Itron MV90xi ( MV90 ) meter reading application. The MV90 interval read data is loaded into the SFPUC MDMS. The MDMS profiles the CIS metered usage and calculates interval load for streetlights, traffic signals and other non-metered load, aggregates both based on load distribution factors, applies loss factors (as required under the PG&E Interconnection Agreement ("IA") through June 30, 2015 and as specified in the PG&E Wholesale Distribution Tariff, FERC Electric Tariff Volume No. 4 after June 30, 2015), consolidates the metered load by scheduling point and creates 60 minute interval Operational Meter Analysis and Reporting ( OMAR ) formatted files, which are submitted to APX for reporting to the CAISO. The SFPUC MV90 system also reads and performs the VEE process on all interval meters that record generation and station load for the Hetch Hetchy Water and Power ( HHWP ) hydro generation facilities. This data is aggregated by the MDMS at 5-minute intervals and submitted to APX for reporting to the CAISO. The SFPUC has an exclusive power purchase agreement for the Sunset photovoltaic generation plant located on SFPUC property; however this is a CAISO metered facility and SFPUC is not required to submit generation data. 4

5 Section I - Management s Executive Summary (continued) Load Data Processing (continued) File transfer of metered data to APX is via SFPUC FTP servers. The files are loaded to two different servers, one in San Francisco and the other in Moccasin, California, to ensure that APX will always be able to retrieve the files when required. In addition, the metered data files are prepared and exported over 3 consecutive days for each required submission; this ensures that metered data will be available for reporting to the CAISO even if SFPUC systems should be disabled for up to 48 hours. Scope of Procedures The objective of the engagement was to perform an agreed-upon procedures engagement of SFPUC s procedures to comply with the CAISO requirements. This report includes the results of procedures performed by the independent accountant on the process controls at the Settlement Meter Data Processing Level during the period of August 1, 2014 through July 31, Summary of Results Our review of the Settlement Meter Data Processing Level controls at SFPUC indicated that the process controls are adequate and functioning as documented. 5

6 Section II - Summary of Findings There were no exceptions noted for the period of August 1, 2014 through July 31, (See Section IV). 6

7 Section III - Independent Accountant s Qualifications Summary Stephen P. Carter, CPA - Principal Mr. Carter is a principal of Abbott, Stringham & Lynch in the Audit and Accounting Group. He has supervised this agreed-upon procedures engagement for SFPUC. He has over 40 years of experience in the field of public accounting. His areas of expertise are in auditing, review and evaluations of internal controls, and accounting systems design and reviews. He also consults with businesses on a variety of business, accounting, and tax issues. Abbott, Stringham & Lynch provides assurance, attest, tax, consulting and management advisory services to businesses located in and throughout California. Abbott, Stringham & Lynch has the qualifications and experience required to perform engagements of this nature. 7

8 Section IV - Description of Controls at SFPUC, Procedures Performed, and Results of Procedures We randomly selected the following sample days from which to perform tests: October 11, 2014 November 15, 2014 November 27, 2014 February 7, 2015 June 28, 2015 July 8, 2015 July 14, 2015 Process Controls 1 Verify documented procedures for meter data processing are up to date and are being followed. SFPUC has the following formally documented procedures which detail out the following: No exceptions were noted. Monitoring Customer Information System ( CIS ) Meter Reads 1. Daily Checklist 2. Monthly Checklist 3. Retrieving Pacific Gas & Electricity ( PG&E ) Daily Meter Read Files 4. Meter Data Management System ( MDMS ) Process 5. Verifying Usage Meter VEE Reads Usage Meter VEE Estimating Guidelines o Research Reads o Calculate Estimates We read the documented meter processing procedures found in the Monitoring CIS Meter Reads document, SFPUC CIS Meter Read Processing document, Monthly Meter Read VEE Requirements document, MV90 Daily Procedures document, MV90 30-Day High-Low Check Procedures document and MV90 kvar Check document, and ascertained that these procedures are being followed by interviewing the Utility Specialists and manager of EDS (Energy Data Systems) group. All procedures appear to be followed as documented. All applicable processes appear to be documented. 8

9 Process Controls 1 (cont.) Monitoring Customer Information System ( CIS ) Meter Reads (continued) Verify documented procedures for meter data processing are up to date and are being followed. 6. Appendix A: Screen Reference Use Meter VEE Review Electric Meter Read Adjustment Screen Adjustment Screen SFPUC - CIS Meter Read Processing 1. Overview Background---CIS Meters 2. CIS Meter Reads and VEE Identification of New Meter Reads System VEE Processing SFPUC Staff VEE Review Monthly Meter Read VEE Requirements 1. Background 2. Physical Meter Validation Checks 3. Data Cleansing 4. Meter Read Validation 5. Meter Read Estimation 6. Demand Estimations 7. Staff Review 8. Audit Trail We observed that the procedures are being performed as documented. All processes that are taking place as part of the meter data processing are listed in these procedural documents. 9

10 Process Controls 1 (cont.) Verify documented procedures for meter data processing are up to date and are being followed. MV90 Daily Procedures 1. MV90 Daily Checklist 2. Clearing the Task List Viewing the Task List Clearing Interrogation (Red) Tasks Clearing Failed Validation (Blue) Tasks Clearing Export (Purple) Tasks o Clearing Failed Export Tasks 3. Editing and Estimating Adding Missing Intervals Editing Using Fixed / Variable Data Estimating Data Forcing MV90 to Flag Estimated / Edited Data Validating Your Work 4. Referenced Procedures Communications (Remote Interrogation) Logs Delete / Post / All Read Manual Accept Manual Export (Upload) to MDMS 5. Verifying File Upload to MDMS Checking for Missing Reads in the Exported Data Files Checking for Failed Exports Ensuring That Estimated Data Is Flagged 6. Appendix A MV90 Failure Codes for Call On Demand Tasks 7. Appendix B Referenced Documents 8. Appendix C Glossary 10

11 Process Controls 1 (cont.) Verify documented procedures for meter data processing are up to date and are being followed. MV90 Daily Procedures (continued) 9. Appendix D Screen Access Instructions MV90 30-Day High-Low Check Procedures 1. High/Low Usage Tolerance (Average Daily Usage) 2. What to do if Meters Fail/Verifying 3. Changes in Usage 4. How to Mark Review Categories Comments 5. Completing the Temporary Irregular Recorder List 6. Reviewing the Temporary Irregular Recorder List 7. Developing Permanent Irregular Usage Lists MV90 kvar Check 1. Screen Content 2. Links 3. What to do if Meters Fail kvar Check 4. How to Mark Review Categories 11

12 Process Controls 2 Verify load profiles, if applicable, are applied properly. Also verify and SFPUC CIS Meter Read Processing 1. Reporting CIS Metered Load We identified and documented the procedures used to apply a No exceptions were noted. document the process for Profile Factors profile to each meter s It appears that SFPUC handling missing or Profiling Logic KWH Meter Read reported usage as is applying load profiles incomplete load profiles. Profiling Logic TOU KWH Meter described in SFPUC - CIS properly. Read Meter Read Processing Aggregation of Profiled Meter Data document. 3 Verify Distribution Loss Factors ( DLF ) are applied properly. Also verify and document the process for handling missing or incomplete DLFs. For reporting CIS metered load to the Meter Data Management Agent ( MDMA ), SFPUC applies a profile to each meter s reported usage, allocating the metered monthly usage to hour intervals based on the profile of the aggregate interval load data. All interval reads, calculated loads, and profiled usage are associated with a PG&E grid code as follows: P Primary S Secondary T Transmission A Airport Prior to July 1, 2015, the grid code corresponds to a DLF specified in the PG&E Interconnection Agreement ( IA ). We ascertained that these procedures are being followed by interviewing the Utility Specialists and manager of EDS group, and observing that the procedures are being performed as documented. We identified and documented the procedures used to apply the DLF value to the metered load. We ascertained that these procedures are being followed by interviewing the Utility Specialists and manager of EDS group and observing that the procedures occur as documented. No exceptions were noted. It appears that SFPUC is applying DLFs properly. 12

13 Process Controls 3 (cont.) Verify Distribution Loss Factors ( DLF ) are applied properly. Also verify and document the process for handling missing or incomplete DLFs. On and after July 1, 2015, the loss factors are specified in the Wholesale Distribution Tariffs. Transmission-level facilities (including the Airport) have no distribution loss factors. During the data preparation process for submitting metered data, the DLF value is applied to the metered load. In other words, metered load multiplied by DLF results in reportable load. 13

14 Process Controls 4 Verify that estimation performed is controlled and documented. MV90 Daily Procedures 1. Editing and Estimating No exceptions were noted. Adding Missing Intervals o Creating a Data File o Adding Intervals Using the Repetitive Add Function Editing Using Fixed / Variable Data Estimating Data o Estimating Data Linear Interpolation o Estimating Using Average Plug o Estimating Using the Copy Data Function Forcing MV90 to Flag Estimated / Edited Data Validating Your Work 2. Referenced Procedures Communications (Remote Interrogation) Logs We identified and documented the procedures used to estimate meter data, as described in the MV90 Daily Procedures document, Monitoring CIS Meter Reads document, and Monthly Meter Read VEE document, and ascertained that these procedures are being followed by interviewing the Utility Specialists and manager of EDS group, and observing that the procedures are being performed as documented. The sample data was tested to ascertain that estimation performed is controlled and documented. It appears that SFPUC is performing estimation for missing meters correctly and that the estimation process is functioning as required. No exceptions were noted. 14

15 Process Controls 4 (cont.) Monitoring CIS Meter Reads Verify that estimation performed is controlled and documented. 1. Verifying CIS Meter VEE Read Estimating Guidelines o Research Reads o Calculate Estimates Monthly Meter Read VEE 1. Meter Read Estimation 2. Demand Estimation 3. Staff Review 4. Audit Trail The MDMS prepares estimated interval reads for all interval load meters based on a 3-day average interval value for 12-month prior reads for comparable days. For work days, the average uses the work day 365 days prior (shifts if the day happens to be a holiday), the year-ago prior work day, and the year-ago next work day. For Saturday, it uses the yearago Saturday, the year-ago prior Saturday, and the year-ago next Saturday. For Sunday or a holiday, it uses the year-ago Sunday or holiday, the year-ago prior Sunday, and the year-ago next Sunday. SFPUC does not perform the function of submitting meter data by the final settlement deadline. APX performs the function of submitting accurate meter data to CAISO by the final settlement deadline. SFPUC does not perform the function of submitting meter data by the final settlement deadline, and therefore it is not within the scope of the SFPUC agreed-upon procedures herein. See APX Self Audit Report for

16 Process Controls 4 (cont.) Monthly Meter Read VEE (continued) Verify that estimation performed is controlled and documented. Estimated reads are developed prior to the beginning of each month for the full month. Thereafter, as MV90 reported read data are processed, any missing intervals for unreported meters are populated with the estimated interval values for the meter, and identified as missing (VEE code 'M'). If no estimated interval reads are available for a meter, the interval values are set to zero and identified as missing. The MDMS interval estimated values are interim values pending final VEE by MV90. When MV90 reports out actual reads, or at the point they determine that the read data are unrecoverable and report estimated reads, the MV90 reads replace the MDMS entries. 5 Verify that any SQMD submitted after T+48B days is documented and controlled to assure accurate and correct meter data is submitted to APX prior to resubmittal deadline. SFPUC has no automatic re-submission process. In any case where SFPUC determines that previously reported read data was incorrect (for example, PG&E determines that they programmed a meter incorrectly, and it should have been reporting 10 times the load), after the corrected reads are posted to the MDMS, SFPUC would (1) add SC export schedule entries for the dates that need to be resubmitted, which would trigger new exports for the dates, and (2) notify APX that new file(s) had been generated for re-submission. We ascertained that these procedures are documented. However, per our inquiry of the Utility Specialists and manager of EDS group, there was no SQMD submitted after T+48B days during the trade year. No submission after T+48B days was noted. 16

17 Process Controls 6 Verify that contingency plans are in place to ensure that if systems go down, data can still be submitted SFPUC MRTU Logical Meter Procedures 1. Data Exchange Interfaces 2. Meter Data No exceptions were noted. to APX in a timely manner. Overview MV90xi Software and Hardware Support 1. Information Technology Services Support Backup Server Updates IT Staff 2. Itron Support 3. Software ISO Meter Data Submission Overview 1. Process for Metered Data Submittal to the ISO We interviewed the Utility Specialists and manager of EDS group responsible for system contingency plans, and they explained the various procedures in place to properly and routinely back-up and safeguard all data. We identified and documented the contingency procedures used to determine that the meter data processing systems can function at all times. SFPUC appears to have adequate contingency plans and data recovery procedures in place to ensure that data can still be submitted to APX in a timely manner. 17

18 Verification and Testing 1 When reporting SQMD during Daylight Savings Time ( DST ), verify the data submitted to APX is in Pacific Standard Time ( PST ). The MDMS does not store any interval read data as local time. All interval read data sourced into the MDMS is assigned interval IDs during load processing; the interval IDs are unique keys to time dimension tables, which contain the corresponding times for UTC, PST, and PDT. We identified and documented procedures that are in place to see that all data is submitted to APX in PST. No exceptions were noted. All reads from MV90 are exported in CMEP interval data format, where the end time for the first interval of each row is in UTC; the MDMS attaches the corresponding interval ID for each imported UTC time. CIS meter reads are in local time, but when profiled to intervals, the interval time is identified by the interval ID. The only point an error could be introduced would be if a meter was read during the duplicate hour in the 2 a.m. fallback period in November, since there is no indication in the data source from PG&E whether the reported date/time is PST or PDT. However, this does not happen since the PG&E reads are taken during normal business hours. Even if it did occur, the maximum error would be 1 hour over an approximately 30 day period. Since most systems use only read dates and not times, SFPUC still has a greater degree of accuracy. The sample data was tested to conclude that data submitted to APX is in PST. No exceptions were noted. Sample testing results indicate that all data submitted to APX is in PST. The use of time dimension interval IDs facilitates all data preparation and reporting, since data can be selected by trade date (based on local time) or metered data submission date (based on PST time). 18

19 Verification and Testing 2 Verify system testing is completed prior to implementation of any meter data processing system change or modification. Validate appropriate documentation supporting the change has been created and that a comparative review was conducted prior to and after the system changes occurred. For any meter data processing system change or modification, SFPUC is required to test the system before and after implementation to verify that MDMS behaves in the new environment the same manner as it did in the current production environment. Proper proof of testing is required by CAISO for all software changes that affect SQM data processing. We interviewed the Utility Specialists and manager of EDS group and noted that the procedures are in place and being followed for all software changes. We identified and documented the procedures and results of comparative data testing before implementation. On July 1, 2015, SFPUC switched over its load reporting process. This was due in part to meet requirements from the expiration of the PG&E Interconnection Agreement. These changes improved load reporting accuracy and efficiency. Prior to implementation, SFPUC performed testing in the following area: It appears that SFPUC has adequately tested the system before and after implementation to verify that MDMS behaves in the new environment the same as it did in the current production environment. Testing historic data that went into the aggregations Testing aggregation of historic data into reporting aggregations Testing of streetlights inventory Testing of SC-Export process 19

20 Root Cause Analysis No. Audit finding Identification of the finding Corrective actions 1 Document the audit finding, how it was identified, the cause for the finding, what effects the finding had on meter data submitted to APX, and the corrective actions taken to prevent recurrence. There were no findings noted for the period of August 1, 2014 through July 31, As there were no findings noted for the period of August 1, 2014 through July 31, 2015, no corrective actions are necessary. 20

21 i 525 Golden Gate Avenue, 7th Floor b3n I T9 PIC I SCO San Francisco, CA Water Power Sewer I Operator of the Hetch Hetchy Regional Water System October 13, 2015 Katherine Graham, Chief Operating Officer APX, Inc. 224 Airport Parkway, Suite 600 San Jose, CA Dear Ms. Graham: At the request of APX Power Market, Inc. ("APX"), San Francisco Public Utilities Commission ("SFPUC") engaged Abbott, Stringham & Lynch LLP to review the procedures used to prepare data at the Settlement Meter Data Processing Level, for the California Independent System Operator ("CAISO") trade year. The primary objective was to comply with the CASIO Scheduling Coordinator "Self Audit" requirements for the trade year. We have reviewed the Agreed-Upon Procedures Report for Settlement Quality Meter Data Processing for the prepared by Abbott, Stringham & Lynch LLP, and the report is satisfactory and in compliance with CAISO standards. If you have any questions, please do not hesitate to contact me. Sincerely, David Robinett Manager Retail Services Edwin M. Lee Mayor Ann Moller Caen President Francesca Vietor Vice President Vince Courtney Commissioner Anson Morau Commissioner Ike Kwon Commissioner Harlan L. Kelly. Jr. General Manager Services of the San Francisco Public Utilities Commission