Data Miner 2 Roadmap As of February 21, 2018 www.pjm.com
Background Data Miner 2 will become the central data repository and interface for PJM public data. Key Product Features - Unified / highly available interface for all PJM public data - APIs available for system to system integrations - Metadata available - Large Data Export feature available. Download limits are as follows: UI Up to One Million Rows. API Up to 50K Rows per fetch 2
2018 Roadmap for Data Miner 2 Feb 2018 Mar 2018 Apr 2018 Rest of Q2 2018 Q3 2018 Q4 2018 Data Miner 2 Release 18.02 5-min LMPs (March 06) -Available for testing 5 min LMP (April 01) Hourly RT and DA LMP flat files Release 18.03 no longer posted on pjm.com Release 18.04 Mock Shutdown on PJM.com (May 2018) Begin Phased Retirements Legend Start Date End Date Release Notes: http://www.pjm.com/markets-and-operations/etools/data-miner-2.aspx 3
Availability of 5 minute LMP data in Data Miner 2 Based on member requests, the following data will be available beginning March 6 for testing purposes for the operating day of March 5 Five Minute Real-Time LMPs Daily only Five Minute Ancillary Services Prices Daily only Data Miner 2 beta environment will provide UI and API interface to access this data These Real-Time LMPs will be considered test data and will not be used for any settlement calculations. Note 1: PJM is publishing the LMP data solely for member testing purposes only. These LMPs will be available in the Data Miner 2 train environment until April 1. After April 1, the verified LMP data will be available in Data Miner 2 production Note 2: This data will be available in Data Miner 2 only. The csv files posted on pjm.com will remain the hourly LMP values 4
Future Plans Hourly RT and DA LMP flat files will no longer be posted on pjm.com after April 1 2018 Operational Data Page will be available in April 2018 with 1 year parallel period Evaluating Mock Shutdown in May 2018. More details to follow Existing flat files will be retired approximately one year after Go Live. Targeting August 15, 2018 to begin retiring existing pjm.com postings edatafeed data is in Planning stage. More details to follow 5
Technical Features Similar look and feel to Data Miner Data Miner 2 hosted on Microsoft Azure Cloud Platform Filtering, Pagination and Sorting Data Dictionary Search RESTful web API services Supports multiple Data Formats JSON XML Versioning 6
Support Issues Issue Resolution Status Multiple User accounts with same email address Develop solution to handle multiple user accounts Scheduled for 3/22 DA LMP postings are delayed Develop a solution to sync posting times Scheduled for 4/1 Enhanced API Guide and API code samples Multiple channels of communication Inconsistency around DateTime Filters behavior Provide visibility to code changes and release notes PNode filter not working on IE browser API Portal Documentation Updated. Published a new version on API Guide with the same updated Retire issues tab on API Portal Use Tech Change Forum Community as primary discussion board Create specific email distribution list for Data Miner 2 communication Code fix Release Notes are being published via Tech Notify and pjm.com website Released on 2/15 Scheduled for 3/1 Released to Prod on 1/11 Completed Code fix Scheduled for 4/1 7
Data Miner 2 Support API Guide http://www.pjm.com/markets-and-operations/etools/data-miner- 2.aspx Data Miner 2 Support Please Contact: DataMiner2Support@pjm.com Subscribe to DATAMINER2@LISTSERV.PJM.COM for all Notifications of Releases, Enhancements, Data related Issue and Resolutions Please visit Tech Change Forum Community page for Knowledge base on Support issues 8
Feed List Feed Name Feed Short Name Current URL Fixed Weighted Average Aggregate Definitions agg_definitions Monthly Financial Transmission Rights Zonal LMPs mnt_ftr_zonal_lmps http://www.pjm.com/markets-and-operations/energy/lmp-modelinfo.aspx http://www.pjm.com/markets-and-operations/energy.aspx Day-Ahead Transmission Constraints da_transconstraints http://www.pjm.com/markets-and-operations/energy.aspx Real-Time Hourly LMPs rt_hrl_lmps http://www.pjm.com/markets-and-operations/energy.aspx Ancillary Service LMPs ancillary_services http://www.pjm.com/markets-and-operations/energy.aspx Day-Ahead Hourly LMPs da_hrl_lmps http://www.pjm.com/markets-and-operations/energy.aspx Hourly Day-Ahead Increment Offer & Decrement Bid Data hrl_da_incs_decs http://www.pjm.com/markets-and-operations/energy.aspx Daily Generation Capacity day_gen_capacity http://www.pjm.com/markets-and-operations/energy.aspx Hourly Day-Ahead Demand Bids hrl_da_demand_bids http://www.pjm.com/markets-and-operations/energy.aspx Nodal Reference Prices for Export Credit Screening nodal_ref_prices http://www.pjm.com/markets-and-operations/energy.aspx Day Ahead Marginal Value da_marginal_value http://www.pjm.com/markets-and-operations/energy.aspx 9
Feed List (cont d) Feed Name Feed Short Name Current URL Real Time Transmission Constraints rt_transn_constraints http://www.pjm.com/markets-and-operations/energy.aspx Forecasted Interface Prices frcstd_interface_prices http://www.pjm.com/markets-and-operations/energy.aspx Hourly Demand Bid Data hrl_dmd_bids http://www.pjm.com/markets-and-operations/energy.aspx Nodal Reference Prices for Increment Offer & Decrement Bid Screening nodal_refe_prices_incdec http://www.pjm.com/markets-and-operations/energy.aspx Up-to-congestion Bid Screening utc_bid_screening http://www.pjm.com/markets-and-operations/energy.aspx Day-Ahead Scheduling Reserve dasr_results http://www.pjm.com/markets-and-operations/billing-settlements-andcredit/preliminary-billing-reports.aspx Non-Synchronized Reserve non_sync_reserve http://www.pjm.com/markets-and-operations/billing-settlements-andcredit/preliminary-billing-reports.aspx PJM Regulation Zone Preliminary Billing Data reg_zone_prelim_bill http://www.pjm.com/markets-and-operations/billing-settlements-andcredit/preliminary-billing-reports.aspx Reconciliation Billing Determinants Preliminary Operating Reserves Rates prelim_or_rates http://www.pjm.com/markets-and-operations/billing-settlements-andcredit/preliminary-billing-reports.aspx http://www.pjm.com/markets-and-operations/billing-settlements-andcredit/preliminary-billing-reports.aspx RTO Synchronized Reserve Preliminary Billing Data sync_reserve_prelim_bill Seven-Day Load Forecast load_frcstd_7_day http://www.pjm.com/markets-and-operations/energy/real-time/7-dayload-forecast.aspx Ancillary Service Market Results reserve_market_results http://www.pjm.com/markets-and-operations/ancillary-services.aspx 10
Feed List (cont d) Feed Name Feed Short Name Current URL PJM Synchronized Reserve Events sync_reserve_events http://www.pjm.com/markets-and-operations/ancillary-services.aspx PJM Historical Regulation Market Data reg_market_results http://www.pjm.com/markets-and-operations/ancillary-services.aspx Actual/Schedule Summary Report act_sch_interchange http://www.pjm.com/markets-and-operations/ops-analysis.aspx Forecasted Generation Outage Postings frcstd_gen_outages http://www.pjm.com/markets-and-operations/ops-analysis.aspx Generation outage for Seven days by type gen_outages_by_type http://www.pjm.com/markets-and-operations/ops-analysis.aspx. Generation and Extra High Voltage Losses gen_ehv_losses http://www.pjm.com/markets-and-operations/ops-analysis.aspx Real-Time Scheduled Interchange rt_scheduled_interchange http://www.pjm.com/markets-and-operations/ops-analysis.aspx RTO Transfer Limit and Flows transfer_limits_and_flows http://www.pjm.com/markets-and-operations/ops-analysis.aspx Wind Generation Section: all years wind_gen http://www.pjm.com/markets-and-operations/ops-analysis.aspx Off-Cost Operations off_cost_ops http://www.pjm.com/markets-and-operations/ops-analysis.aspx Load Forecasts (Historical) load_frcstd_hist http://www.pjm.com/markets-and-operations/ops-analysis.aspx Hourly Load: Preliminary hrl_load_prelim http://www.pjm.com/markets-and-operations/ops-analysis.aspx Hourly Load: Estimated hrl_load_estimated http://www.pjm.com/markets-and-operations/ops-analysis.aspx Hourly Load: Metered hrl_load_metered http://www.pjm.com/markets-and-operations/ops-analysis.aspx Daily cleared INCs, DECs, and UTCs day_inc_dec_utc http://www.pjm.com/markets-and-operations/energy.aspx 11
Feed List (cont d) FTR Auction Bids - Annual Feed Name Feed Short Name Current URL ftr_bids_annual FTR Auction Bids - Long-Term ftr_bids_long_term http://www.pjm.com/markets-and-operations/ftr.aspx http://www.pjm.com/markets-and-operations/energy/realtime/historical-bid-data/bids-ftr-auction-annually.aspx FTR Auction Bids - Monthly ftr_bids_mnt http://www.pjm.com/markets-and-operations/ftr.aspx Day Ahead Interface Flows and Limits da_interface_flows_and_li mits http://www.pjm.com/markets-and-operations/etools/oasis/systeminformation.aspx http://pjm.com/markets-and-operations/billing-settlements-andcredit.aspx Monthly Load - Reconciliation Billing Determinants bill_deter_mnt_load http://pjm.com/markets-and-operations/billing-settlements-andcredit.aspx Load Reconciliation Billing Determinants - Hourly load_recon_bill_deter_hrly Real-Time Marginal Value rt_marginal_value http://www.pjm.com/markets-and-operations/energy.aspx Operations Summary - Projected Area Statistics at Peak ops_sum_frcst_peak_area http://www.pjm.com/markets-and-operations/ops-analysis.aspx Operations Summary - Projected RTO Statistics at Peak ops_sum_frcst_peak_rto http://www.pjm.com/markets-and-operations/ops-analysis.aspx Operations Summary - Projected Scheduled Tie Flow ops_sum_prjctd_tie_flow http://www.pjm.com/markets-and-operations/ops-analysis.aspx Operations Summary - Forecast Transfer Limits ops_sum_frcstd_tran_lim http://www.pjm.com/markets-and-operations/ops-analysis.aspx Operations Summary - Actual Operational Statistics ops_sum_prev_period http://www.pjm.com/markets-and-operations/ops-analysis.aspx 12
Feed List (cont d) Feed Name Feed Short Name Current URL Monthly Equivalent Forced Outage Rates Generation Fuel Mix Daily Uplift By Zone mnt_efor gen_by_fuel day_uplift http://www.pjm.com/markets-and-operations/energy/realtime/historical-bid-data/eford.aspx http://www.pjm.com/markets-and-operations.aspx - The pie chart that is present at this location http://www.pjm.com/markets-and-operations/energy/upliftdata.aspx Pricing Nodes pnode https://dataminer.pjm.com/dataminerui/pages/public/pnode.jsf http://pjm.com/markets-and-operations/billing-settlements-andcredit.aspx Load Reconciliation Billing Determinants - Daily load_recon_bill_deter_daily Settlements Verified LMPs rt_da_monthly_lmps http://www.pjm.com/markets-and-operations/energy.aspx Settlements Verified Ancillary LMPs ancillary_services_monthly http://www.pjm.com/markets-and-operations/energy.aspx 13