Innovation Roadmap: External Engagement Report. November 28, 2018

Transcription

1 Innovation Roadmap: External Engagement Report November 28, 2018

2 Purpose of this document The purpose of this document is to summarize the feedback provided by stakeholders and communities to the development of the IESO s innovation roadmap and to outline how this feedback was incorporated into the roadmap 2

3 Purpose of roadmap: cost-effectiveness and reliability of Ontario s electricity system The purpose of the roadmap is to understand potential change, challenge and opportunities facing IESO and the broader electricity sector through 2030 in order to ensure a cost-effective and reliable electricity system into the future per IESO s mandate Dialogue: facilitate an internal and external dialogue on the evolution of Ontario s electricity and broader energy sector as it relates to IESO s mandate Situational analysis: assess existing and emerging challenges and opportunities facing IESO and the broader electricity sector Prioritization: Identify and prioritize key areas of focus for learning, capability building and enabling the innovation of others to address the existing and emerging challenges Action: be supported by a multi-year work plan to focus and coordinate IESO and sector efforts 3

4 Context: why was the innovation roadmap developed now? Electricity systems all over the world are facing significant change Adoption of emerging technologies; increased digitization, new business models/solution providers, changing consumer preferences, decarbonization Innovation creates both challenges and opportunities for the IESO E.g. new products/services may improve efficiency of electricity system operations and increase competition OR create uncertainty if IESO /other sector participants are not prepared for change Roadmap establishes priorities for innovation activities learning, capability building, enabling the innovation of others (to compete in IESO-administered markets) for improved electricity system cost-effectiveness and reliability Stakeholders identify IESO as being uniquely positioned to work with others within the sector to prepare for and guide change in support of improved electricity system cost-effectiveness and reliability Roadmap sets a framework and focus for sector evolution that provides an opportunity for coordination of IESO efforts with regulatory and policy changes to support modernization/innovation for cost-effectiveness and reliability 4

5 What is innovation? Novel solutions to important challenges relevant to the work of the IESO, for which existing solutions are inadequate or inefficient Solutions such as emerging technologies, new business models, practices, regulatory and policy constructs and etc. Emerging technologies, business models and etc. may be both solutions to existing problems and create new challenges in and of themselves (e.g. distributed energy resource proliferation) Use of existing solutions in novel ways In practice: innovation is the process of developing and implementing new methods, ideas and/or products OR using existing methods, ideas, and/or products/services in new ways 5

6 Stakeholder engagement in roadmap development Participants: Over 100 organizations/200 individuals engaged (Aug-Nov. 2018) Three sessions with IESO Stakeholder Advisory Committee (SAC) Four focus group sessions Industry, Academia, Government, Associations, Energy Transformation Network of Ontario (ETNO) Corporate Partners Committee Energy Transformation Network of Ontario (ETNO) Public webinar Written submissions Purpose: Feedback from the external stakeholders was compiled into the results presented in this report. Feedback was also used to inform roadmap development in: Ensuring diverse advice, insights, recommendations, and input Prioritizing the Areas of Focus 6

7 Contributors to this report Contributor Group Energy Transformation Network of Ontario Corporate Partners Committee (ETNO CPC) Stakeholder Advisory Committee (SAC) Associations and Academics (A&A) Government Industry contributors Type of Engagement* Structured verbal comments Verbal comments Open verbal discussion Structured verbal comments Open verbal discussion Written comments Date Number of Participants Types of Organizations Represented Oct. 16, Toronto Hydro Corporation, Veridian Corporation, Hydro One Inc., Ontario Power Generation, Electrical Safety Authority, MaRS Discovery District, Hydro Ottawa Limited, London Hydro, University of Waterloo, Natural Resources Canada (NRCan), Alectra, Union Gas Limited, Essex Power, Tormont Cat, Ontario Ministry of Energy, Northern Development and Mines, Ontario Energy Board Oct. 17, SAC Members Oct. 21, Ontario Energy Association, Consumers Council of Canada, Canadian Manufacturers and Exporters, Ryerson University, York University, Ministry of Energy, University of Windsor, Energy Storage Canada, Electricity Distributors Association Oct. 21, Environment and Climate Change Canada Oct. 16, Ministry of Energy, Ontario Energy Board, Beworks, Flo, TransCanada Energy, Peak Power, Plug N Drive, EnelX, Nest/Sidewalk, Toronto Hydro Before Nov. 1, Amp, EnelX, Atmospheric Energy Systems, Advanced Energy Management Alliance, Ontario Power Generators, Energy Storage Canada 7

8 How feedback was collected and compiled Feedback was collected through: Structured verbal comments: respondents were asked to provide answers to an icebreaker and 8 topic questions shown on the next slide, addressing one question at a time. The ETNO CPC and Government focus group data was collected this way. Open verbal discussion: respondents were asked to provide an answer to the icebreaker question. For the rest of the session, respondents were shown the topic questions, but the discussion flowed freely. To organize the results, responses were assigned to relevant topics after the data was collected. The A&A and Industry focus group data was collected this way. Verbal comments: stakeholders were provided with a draft document and asked to make comments. They did not see the questions. To organize the results, responses were assigned to relevant topics after the data was collected. The SAC meeting data was collected this way. Written comments: stakeholders were allowed to submit written comments. Some organized them within the 8 topic questions; some did not. 8

9 Icebreaker and topic questions Icebreaker: Who should take the lead in modernizing Ontario s electricity system and why? The private market, utilities, government organizations, or someone else? Topics: 1.What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? 2.What innovation is happening/on the horizon for Ontario, including that from other sectors, which may impact the electricity sector? 3.What are the current barriers for innovation in the electric sector? 4.What are the external factors that IESO needs to consider as part of the roadmap? 5.What is IESO s role in removing barriers and enabling innovation in the electricity sector? 6.What innovation activities should IESO prioritize? Why? 7.With whom should IESO engage to implement innovation? 8.What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? 9

10 Structure of this report Feedback is compiled in three ways: Icebreaker responses: This question was put to three of the focus groups (ETNO CPC, A&A, and Industry). Each respondent s answer was tallied individually. In other words, a single focus group produced more than one response. Graphical summary of topic responses: This is a summary of the detailed feedback across all groups, to provide a higher-level overview. As stated previously, the responses from open discussion were assigned to topic areas after the data was collected. Each group s answer was tallied individually. In other words, a single focus group produced one response. See the appendix for more detail. Detailed feedback from topic responses: This is a listing of all of the information collected across all groups. As stated previously, the responses from open discussion were assigned to topic areas after the data was collected. Each group s answer was tallied individually. In other words, a single focus group produced one response. See the appendix for more detail. 10

11 Icebreaker: Who should take the lead in modernizing Ontario s electricity system and why? The private market, utilities, government organizations, or someone else? This question was posed to three focus groups: ETNO CPC, A&A, and Industry. Twenty-four responses were recorded. LDCs Other Private Market Everyone Everyone, no stated leader Government Everyone, led by Government Everyone, led by IESO Everyone, led by Private Market 11

12 Summary of identified disruptions Topic 2: What innovation is happening/on the horizon for Ontario, including that from other sectors, which may impact the electricity sector? Topic 4: What are the external factors that IESO needs to consider as part of the roadmap? Key take-aways: New technologies and changing consumer behaviour were most often identified as driving disruption in the electricity sector, followed by policy changes and new business models. DERs, storage, and decarbonized transportation were the mostidentified technology disruptors. Consumer adoption rates and changing demographics were the most-identified customer behaviour disruptors. Changes in political priorities, climate change, electrification, and misunderstanding of innovation were the most-identified policy change disruptors. Direct sales to customers and new third-party energy services were the most-identified business model disruptors. Changes in provincial priorities Climate change responses Electrification Misunderstanding need for innovation Geopolitical Federal regulations Carbon tax Natural gas expansion Perception of cost Generators/retailers sell direct to customers New third-party energy services Data-driven disruption LDCs take new roles Financial innovations to allow cost recovery Allow renewable co-ops to sell to IESO and custs Changing Demographics Policy changes New business models Expand transmission to new parts of province Adoption rates Indoor agriculture Changing customer behavior Grid defection Customer choice Interactions with: - Other disruptions - Other energy sources Online retail (drones) New technologies: understand them and how they can be modeled and used; Agnostic to them Boundaries blurring between distribution and transmission Smart cities Distributed energy resources (DERs) Storage Decarbonized transportation Autonomous vehicles Internet of things Artificial intelligence Legend: # of groups that provided the response

13 Summary of identified opportunities Topic 1: What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Key take-aways: Localized pricing/ demand response New grid operations were the most-identified opportunity resulting from electricity sector disruption. The identified opportunities included: Allow aggregated resources New grid operations See more resources on the grid Allowing aggregated resources on the grid Localized pricing and demand response The ability to see more resources on the grid Create a reliability standard Residential demand response Electric vehicle demand response Energy efficiency as a wholesale resource Legend: # of groups that provided the response

14 Summary of identified barriers Topic 1: What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? Topic 3: What are the current barriers for innovation in the electric sector? Key take-aways: Don t do innovation for sake of innovation Burden of grid costs if widespread defection Cost was identified as the biggest innovation barrier, followed by lack of communication among stakeholders. Within cost, the most-identified barriers were: Stranded assets Unnecessary innovation Grid maintenance in the face of widespread defection Global adjustment Private sector unwilling to take on risk Government unwilling to offset risk Cannot access IESO data Data/new computational literacy Data/computing literacy Stranded assets Risk Cost Global adjustment Lack of communication Revenue requirements for investment in DERs Share information and disseminate to others Pilot results are not scalable Make LDC technology uniform Inconsistent/no standards Set consistent standards, such as interconnection Update outdated standards Must adapt to disruption; new business as usual Reliability Small economy vs. other geopolitical drivers Legend: # of groups that provided the response

15 Summary of suggested roles for IESO Topic 5: What is IESO s role in removing barriers and enabling innovation in the electricity sector? Topic 7: With whom should IESO engage to implement innovation? Local Distribution Companies (LDCs) Key take-aways: Ontario Energy Board Creating standards Legend: # of groups that provided the response 6 3 Providing leadership and sharing information were the most-identified roles for IESO, followed by suggestions for making IESO better. Stakeholders suggested leadership across a wide range of potential partners, including LDCs, OEB, private market, customers, and more. Helping to create standards and determining and funding research were two more suggested leadership roles. Stakeholders most often identified IESO data as information to share, followed by coordination with LDCs, acting as a central repository, communicating market needs, and communicating the IESO mission and mandate. Stakeholders most often identified three areas of improvement for IESO, including adapting the bulk capacity model, becoming technology agnostic, and getting more value from existing resources. Private market Customers Ministry of Energy Existing resources DER providers Energy Storage Canada CanSIA EDA Market Renewal Team ETNO Energy Storage Advisory Group Ontario Power Generator Across stakeholders IESO data Coordinate with LDCs Central repository of information What market needs to provide/what IESO has IESO mission/mandate Determining and funding research Leadership Share information Adapt bulk capacity model Be agnostic to technologies Help OEB incentivize utilities to innovate Make IESO better Get more value from existing resources Defer to OEB/ government Follow work done elsewhere Improve forecasting Improve operational efficiency 5 4 Adapt dispatch model

16 Challenges not already articulated Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Key take-aways: Stakeholders identified six different challenges in delivering a reliable, costeffective electricity system. Each was identified by one group. Some suggested that there will be a struggle to finance new capacity based on a 1-year contract Climate change will threaten the resiliency of the grid Definitions of operational vs. capital expenses limit cost recovery and suppress innovation Without innovation, Ontario may run out of generation Incumbency limits new market entrants LDCs may go bankrupt in the face of widescale grid defection Legend: # of groups that provided the response

17 Topic 1: What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? Gaps expressed by three different groups: Better understand customer behavior: no one knows how customer demographics are going to change, or how customers will react to disruptions. How quickly will customers adopt new technologies? What will drive them to defect from the grid? Do we know what they want? Lack of communication: The industry is very siloed; communication needs to increase How to use DERs/other resources: How to unlock the value of resources to meet distribution needs, who is best positioned to aggregate smaller resources, need to be innovative to manage today s problems (i.e. connection), need to understand how to use DERs for reliability, need wholesale option for aggregators. Does IESO have sufficient visibility into DERs on the Dx grid? Don t understand new technologies: do we (utilities and end customers) truly understand the potential of battery and non-battery storage systems? What about other technologies capabilities, costs, use cases, benefits? Can we anticipate the effect of big data and artificial intelligence? 17

18 Topic 1: What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? Gaps expressed by two different groups: Policy understanding: How to assist the new government to understand the value of innovation? How to address policy uncertainties? What about geopolitical concerns? How to adapt to new business models: how to incorporate new energy services and providers demanded by consumers? How will data-driven companies disrupt the electricity industry? Look to others: What are other jurisdictions doing? What can the rest of the world teach Ontario about operating wind and solar? 18

19 Topic 1: What do you see as the biggest capability and knowledge gaps in preparing for change in the electricity sector? Gaps expressed by only one group: Need to adapt to disruption as the new business as usual Need to adapt grid to allow hyper-localized pricing structures Need to determine how to address stranded assets Private market needs to innovate after years of leaving it to government Need to understand what level of reliability is demanded by customers Need to understand how individual disruptions interact with each other Someone needs to look at the system as a whole: not just transmission or distribution or this LDC or that customer class. Boundaries between distribution and transmission are blurring 19

20 Topic 2: What innovation is happening/on the horizon for Ontario, including that from other sectors, which may impact the electricity sector? Innovation expressed by five different groups: Decarbonized transportation: electric vehicles, hydrogen/fuel cell vehicles, smart charging Innovation expressed by four different groups: Storage: utility storage, hydrogen storage, behind-the-meter storage, thermal storage batteries, distributed storage Innovation expressed by three different groups: Autonomous vehicles: autonomous vehicles, EVs as localized demand response DERs: energy prosumers on the distribution grid, DERs in wholesale markets Innovation expressed by two different groups: Artificial intelligence: to operate the grid and for optimizing the system Smaller supply resources: less than 1 MW participate in wholesale market New business models: peer-to-peer sales, generators direct to end user sales, LDCs take on new roles, financial innovations to allow new cost recovery 20

21 Topic 2: What innovation is happening/on the horizon for Ontario, including that from other sectors, which may impact the electricity sector? Innovation expressed by one group: EV demand response: two-way EV connections to grid More sophisticated price signals: smooth grid demand using multiple and localized price signals Changes in consumer behavior: adoption rates of new tech Digitized grid: asset management with data at grid nodes Electrification Residential demand response Energy efficiency as a wholesale resource Carbon tax Self powered appliances Online retail, delivery by drones Indoor agriculture Smart cities 21

22 Topic 3: What are the current barriers for innovation in the electric sector? Barrier expressed by four different groups: Ignorance: data is not shared throughout the industry, pilot project results are not scalable, stakeholders do not understand new technologies or their benefits, academics needs to understand what is happening in industry Barrier expressed by three different groups: No standardization: LDCs do their own thing but processes and standards need to be the same across the province, LDCs are driven to protect their natural monopoly, not all LDCs are adopting technology at the same rates (limiting IESO capabilities), industry is not always willing to standardize and lose a competitive advantage, there is no standard for how reliable is enough Risk: the public sector is unwilling to take on the risk of innovation, the province is unwilling to innovate because of the immediate cost, the federal government is not driving innovation 22

23 Topic 3: What are the current barriers for innovation in the electric sector? Barrier expressed by two different groups: Small suppliers: resources less than 1 MW are not allowed in the wholesale market, aggregated loads cannot be included, IESO must be able to model, monitor, and operate aggregated loads and DERs, the market should allow energy efficiency as a resource, the enrollment process for residential DR should be easier, Cost: how will society pay for stranded assets, what happens when resources are no longer cost effective, how is the grid maintained if consumers defect Restricted data: allow customers and their chosen providers to access data and allow aggregation, stakeholders do not have access to the necessary data to innovate Connection: storage and other DERs experience long and costly connection processes Technology agnosticism: IESO must have a technology agnostic system develop a system that does not predetermine a technology. Don t predetermine who can aggregate resources. 23

24 Topic 3: What are the current barriers for innovation in the electric sector? Barrier expressed by one group: Customer choice: customers should be allowed to chose their energy services provider for energy efficiency programs and other services New business models: allow renewable co-op generators to sell to the IESO and public Outdated standards: standards were developed for centralized generation and need to be updated 24

25 Topic 4: What are the external factors that IESO needs to consider as part of the roadmap? External factors expressed by four different groups: Policy understanding: changes in dominant party, geopolitical concerns, policy driving natural gas expansion, political perception of electricity cost, political understanding of the need for innovation, federal regulations, GHG requirements, changing goals External factors expressed by three different groups: Better understand customer behavior: changing demographics, understand behavior changes, what drives adoption, rates of grid defection Cost: global adjustment, demand for affordable energy, innovation to drive reduced costs, concerns about stranded assets, maintaining the grid in the face of defections, innovation that people want New technologies: artificial intelligence, internet of things, EVs as supply assets, global innovation, system balancing potential of DER New business models: increased transmission to remote areas, resource extraction in remote areas, third parties in energy management, How will datadriven companies disrupt electricity? 25

26 Topic 4: What are the external factors that IESO needs to consider as part of the roadmap? External factors expressed by two different groups: Climate change External factors expressed by one group: Small/open economy (other states, EU, other countries) NERC/FERC/RTOs price taking Need to adapt to disruption as the new business as usual Cyber security and cloud computing Indoor agriculture Revenue requirements for private investment in DERs Electrification Interactions between electricity and other energy streams 26

27 Topic 5: What is IESO s role in removing barriers and enabling innovation in the electricity sector? Roles expressed by four different groups: Leadership: leadership across stakeholders in the sectors, looking at the market as a whole, providing value to LDCs, creating standards, determining and funding research, thought leadership, create a model for others to follow Information sharing: coordinate and share with LDCs, innovation sharing, provide access to IESO data, become central repository of information Roles expressed by three different groups: Improving IESO: adapt bulk capacity model, improve forecasting, improve operational efficiency, be agnostic to all technologies, get more value from existing resources, monetize environmental attributes of existing resources, adapt dispatch scheduling and optimization model to distinguish between load and generation 27

28 Topic 5: What is IESO s role in removing barriers and enabling innovation in the electricity sector? Roles expressed by two different groups: Send clear information to market: clearly define what s necessary, publish the requirements, clearly define what s available, identify jurisdictional and leadership gaps, send market signals that allow others to address problems, remain technology agnostic Roles expressed by one group: Define IESO mission/mandate: respond to or lead innovation? Ensure that the focus is on the public good. Respond to others leadership: follow innovation developed elsewhere, defer to OEB/government Help OEB incentivize utilities to innovate 28

29 Topic 6: What innovation activities should IESO prioritize? % of Areas of Focus respondents* Unlock value of new and existing resources 18% New capability to collect, store, share, analyze and use data 15% Understanding of new distribution system operations and business models 13% Leadership with respect to policy and regulatory changes in regards to IESO s core mandate 11% Alternative approaches to provide system resiliency 10% New methods for operating a more dynamic electricity grid 8% Increase transparency and visibility or resources operating on the distribution system 7% Leadership with respect to emerging cyber security risks 5% Prepare for increase in customer-led and LDC-led DER deployment 5% Prepare for changing consumer choice 3% Other 3% Human resource readiness to meet innovation needs 2% *Twenty-four people responded across three focus groups Respondents were asked to identify their top priorities across the draft IESO areas of focus. Each respondent was given 5 or 6 votes to distribute. 29

30 Topic 7: With whom should IESO engage to implement innovation? Partners suggested by four different groups: Stakeholders: Share information and seek input through one roadmap, not several Partners suggested by three different groups: LDCs Partners suggested by two different groups: OEB Private market: EV manufacturers, battery makers and researchers, reliability and power system analysis companies, smart grid technology providers and researchers Customers 30

31 Topic 7: With whom should IESO engage to implement innovation? Partners suggested by one group: Existing resources: demand response and hydro storage Ministry of Energy Energy Storage Canada CanSIA EDA IESO Market Renewal team DER providers ETNO ESAG OPG 31

32 Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Challenges expressed by five different groups: Cost: stranded assets, maintaining the grid after widespread defection, cost of innovation research/implementation, cost recovery (capital vs. operational expenses) for utilities, expenses for regulated vs. non-regulated utilities Challenges expressed by two different groups: Regulations: from utility regulators and from the federal government 32

33 Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Challenges expressed by one group: Lack of standardization: LDCs not held to same standard; don t have a standard for reliability, challenge assumptions about operating reserves and costs Policy makers do not understand the need for innovation Public perception of the cost of energy Climate change / resiliency Financing new capacity based on a one-year contract Innovation must focus on what s best for customers The market does not understand what problems need to be addressed The current system is not technology agnostic Without innovation, Ontario may run out of generation Without change, the grid will not be able to handle innovative loads LDCs may go bankrupt if there is widespread grid defection No one is looking at the sector as a whole, only their own areas Incumbency limits new market entrants 33

34 Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Challenges that were not expressed in other topics: Current definitions of operational vs. capital expenses may limit innovation because of limited cost recovery Climate change affecting the resiliency of the grid Financing new capacity based on a one-year contract Without innovation, Ontario may run out of generation LDCs may go bankrupt if there is widespread grid defection Incumbency limits new market entrants 34

35 Topic 8: What other challenges or opportunities exist in delivering a reliable, cost-effective electricity system for Ontario? Opportunities expressed by one group: New technologies on the market Artificial intelligence Distributed energy resources A less centralized market Overall lower costs 35

36 External feedback (raised by at least two groups) outside of the draft areas of focus identified by IESO Understanding new technologies (three groups): the areas of focus talk about unlocking new resources or utilizing new resources, but they do not talk about researching or better understanding them. Risk (three groups): Innovation has been led by government; private sector is not ready/willing to innovate alone, government is not willing to offset the risk Send clear info to market (two groups): related to risk, stakeholders asked the IESO to tell them what is necessary so they could create the solution in the private market. Cost (three groups): While the roadmap talks about ensuring a costeffective electricity system into the future, many stakeholders talked about lowering costs today. Climate change (two groups): One could argue that climate change is the driver of disruption, but it s not explicitly mentioned in the AoF. 36

37 How stakeholder comments on the scenarios have been integrated into the roadmap Key stakeholder input Integration of Stakeholder/ engagement input Need to better reflect the future role of Transmission-connected (Tx resources) Considerable focus on evolution of customer needs, Dx sector, DERs Need for economic analysis (how likely is DER uptake based on cost forecasts? Benefits of DERs vs. Tx resources? Existing system costs) Need to understand economics of consumer behaviour/choice Updated to better articulate continued role of bulk system resources and make clear that enabling innovation of others includes enabling existing and new Tx-connected resources to compete to provide new products and services in Ontario markets Clarified purpose of scenarios is to stimulate dialogue; identify potential challenges and opportunities that the IESO and sector may face Scenarios and modelling work undertaken by IESO with input from Non-Emitting Resource Sub-Committee (NERSC) of Market Renewal outlines potential competitiveness of consumer-owned resources in future markets As part of the roadmap implementation plan, IESO will develop a white paper on the potential rate of consumer adoption of DERs and other price-responsive energy consuming technologies 37

38 How stakeholder comments on the areas of focus has been integrated into the roadmap Key stakeholder input Integration of Stakeholder/ engagement input Stakeholders were consistent in identifying unlocking value of new and existing resources as the highest priority (driven in part due to concern over stranded assets; fears over grid defection; need to show continual cost reduction going forward) Stakeholders continued to emphasize the role for DERs going forward, incorporating DERs into the IESO markets and operating resources cost effectively was a top priority across the board Prioritization of areas of focus within roadmap integrate engagement feedback (focus group participants ranked areas of focus; final prioritization generally aligns with the rankings provided) In particular, data access was moved up in the priorities based on input from stakeholders Many stakeholders emphasized need for better information sharing and data as a means to enable the innovation of others and level the playing field 38

39 How stakeholder comments on implementation of the roadmap have been considered Key stakeholder input Integration of Stakeholder/ engagement input Need to be cognizant of political realities and perceptions, especially when it comes to costs in the system Research is required with respect to consumer behavior (e.g., technology adoption, choices, etc.) and understanding new technologies that are on the horizon (e.g. big data, AI, etc.) Workplan needs to emphasize building capabilities of the IESO (e.g. dispatch model, forecasting, etc.) From IESO s perspective the purpose and intended outcome of any innovation initiative must be improved cost-effectiveness and reliability of Ontario s electricity system for the benefit of Ontario ratepayers As part of the implementation plan, IESO will set out initiatives (e.g. research, white papers) to understand and communicate the potential impact and timing of changes related to trends/changes in consumer behaviour, technology adoption and other factors that may influence electricity supply and demand As part of the implementation plan IESO will outline planned capital projects that will build IESO capability to enable the innovation of others 39

40 APPENDIX 40 40

41 Definition of Group The data in this report is compiled to show the number of groups that submitted the same response. A group is defined as a single discussion group (focus group or SAC). The written responses are also treated as a single group. We did not compile data by the number of independent organizations that made a particular response, because: Within a given focus group, a participant may agree with another participant s statement without explicitly saying so. If we defined group as individual organization, we would likely underrepresent the actual count. We wanted to identify the number of independently-identified responses to avoid group think. 41