INTEGRATED DEMAND AND SUPPLY SIDE MANAGEMENT AND SMART PRICING FOR ELECTRICITY MARKET

Transcription

1 INTEGRATED DEMAND AND SUPPLY SIDE MANAGEMENT AND SMART PRICING FOR ELECTRICITY MARKET A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Science and Engineering 2017 By Zixu Liu School of Computer Science

2 Contents List of Tables... 5 List of Figures... 6 List of Symbols... 8 Abstract Declaration Copyright Publications Acknowledgement Introduction Context and Motivation Research Problems and Research Objectives Contributions Thesis Organization Publications Background and Related Work Introduction Smart Grid Demand Response in the Retail Market Strategic Bidding in the Wholesale Market The Electricity Pool of England and Wales Related Work in the Electricity Market- A Literature Review Smart Pricing Design for the Retailer Optimal Bidding Problem for the Generator

3 2.2.3 Integration of Demand and Supply Side for the ISO Critical Analysis Coordination Mechanism for the ISO Optimal Pricing Model for the Retailer Optimal Bidding Strategies for the Generator Chapter Summary Integrated Demand and Supply Side Management and Smart Pricing for Electricity Market- A Simulation Tool Introduction The Statement of Simulation Tool The Detail of Designed Simulation Tool Generation Side Retail Side Balance Mechanism Numerical results Summary A New Market Mechanism for Integrated Demand and Supply Sides Management in Electricity Market Introduction Problem Statement Proposed Market Model Calculating the MCP function Retail Side Integrating Supply Side to Demand Model Numerical Results Summary An Analytical Optimization Method for Implementing the New Market Mechanism

4 5.1 Introduction Problem Statement and the Proposed Algorithm Proof Process Numerical Results Chapter Summary Demand Based Bidding Strategies under Interval Demand for Integrated Demand and Supply Management Introduction Context and Motivation Strategic Bidding in Wholesale Market Problem Statement and the Proposed Approach Optimal Bidding Strategies for the Generator The Forecasting Model Numerical Results Chapter Summary Conclusions and Future Work Summary of Objectives and Contributions Future Work Bibliography Word Count:

5 List of Tables Table 2.1: A brief comparison between the traditional grid and the SG [15] Table 2.2: Roles and actors in the SG Table 3.1: Retailers parameters Table 3.2: Ten generators production information Table 3.3: The input demand of each hour Table 3.4: The simulation result of balance mechanism Table 4.1: The comparison between new mechanism and the current mechanism Table 4.2: The MCP table and corresponding supply segments Table 4.3: Aggregated bidding curve/mcp function Table 4.4: The fitness value of each hour Table 5.1: Aggregated bidding curve/mcp function Table 5.2: The fitness value of each hour of new method Table 6.1: The Statistical analysis of the forecasting error Table 6.2: Comparative MAPE results between two ANNs Table 6.3: Test Results of 90% and 95% confidence interval

6 List of Figures Figure 1.1: Last accepted bid dispatch model Figure 2.1: The market model in the SG Figure 2.2: An example of the RTP program Figure 2.3: Typical demand curve Figure 2.4: The structure of the Stackelberg game model Figure 2.5: The structure of forecasting model Figure 2.6: Integration of the elasticities with the price computation Figure 3.1: The structure of the electricity market Figure 3.2: The working process of proposed simulation tool Figure 3.3: An example of step function Figure 3.4: An example of getting aggregated bidding curve Figure 3.5: The process of retailer s pricing model Figure 3.6: The comparison between the test demand and result demand Figure 3.7: The demand-difference in each hour Figure 3.8: Retailer 1 s brought power and sold power Figure 3.9: The MCP (red) and retailers sales price (1: blue, 2 green) Figure 4.1: An example of the balance point Figure 4.2: The running process of the new mechanism Figure 4.3: An example of the MCP function Figure 4.4: An example of the match equilibrium Figure 4.5: Running result of the Genetic algorithm Figure 4.6: Running result of GA under new MCP function Figure 4.7: The customers response demand of each hour and z h Figure 4.8: The result MCP vector and the retail price

7 Figure 5.1: Example of the new method Figure 5.2: The solution space of three scenarios for problem Figure 5.3: Customers response demands Figure 5.4: The resulting MCP vector and retailer s retail price Figure 6.1: The Regression function (solid line) and its 95% confidence band (the region bounded by dashed lines) Figure 6.2: The process of proposed strategic bidding model Figure 6.3: An example of a production cost curve for a generator Figure 6.4: An example of a bid curve for a generator Figure 6.5: Structure of the neural network Figure 6.6: The predicted MCP and real MCP of each hour in week Figure 6.7: The predicted MCP, real MCP, upper and lower bound of 90% confidence interval of each hour in week

8 List of Symbols Parameter h i j m i p i,mi q i,mi q i,mi MCP h s h S h,k D h,k MCP h,k p g β h,c β h,h max p h min p h j C N PRJ j N j p h C b Description h-th hour i-th generator j-th retailer The number of segments in this generator i s bid curve Bid price of m i -th segment in generator i s bid curve Lower bound of m i -th segment in generator i s bid curve Upper bound of m i -th segment in generator i s bid curve Market clear price in hour h The number of total segments in all generators bidding curves (MCP function) of hour h, where s h = i m i The k-th segment of the proposed step function (aggregated bidding curve) in hour h The corresponding quantity of S h,k The corresponding unit price of S h,k The pre-negotiated profit level of all generators The cross-price elasticity The self-elasticity The maximum price that the retailer can offer to its customers The minimum price that the retailer can offer to its customers The total bill constraint of retailer j s customers The retailer j s profit of day N The retailer j s optimal retail price in hour h The increased part all the customers bills when the 8

9 D b D j,h MCP h f P h j f s h,k MCP h,k D h,k D h,k z h Z ps g Er h Er MCP v MCP h s h s D h,sh s D h,sh generation cost is increased The total demand of all customers The total response demand of retailer j s customers in hour h The MCP after using balance mechanism The retail price in hour h after using balance mechanism The k-th segment of the MCP function in hour h MCP of k-th supply segment in hour h s MCP function The lower bound of corresponding supply demand for MCP h,k The upper bound of corresponding supply demand for MCP h,k Fitness value of a MCP vector in hour h (GA-based approach) Fitness value of a MCP vector (GA-based approach) Population size (GA-based approach) The max number of generations (GA-based approach) Fitness value of a MCP vector in hour h (Analytical optimization Method) Fitness value of a MCP vector (Analytical optimization Method) The MCP vector which makes the electricity market reaches the match equilibrium Market clearing price for sh -th segment in hour h under the MCP table (Table 4.2) The lower bound of corresponding supply demand for MCP h s h The upper bound of corresponding bidding supply s demand for MCP h h h All hours in (1,..,H) except for hour h r,j D h Retailer j s demand in hour h. 9

10 PR j Retailer j s profit in period (1,,H). P Peak hour RC j Revenue constraint for retailer j. o Off-peak hour D s h (MCP h ) Lower bound of MCP h s corresponding supply in Table 4.2 s D h(mcph ) Upper bound of MCP h s corresponding supply in Table 4.2 S max The biggest number in the vector (s 1, s 2 s h s H ) P peak The value of peak point in x axis for quadratic function. I The set of all generation units belong to generator i K The set of all generation units k k-th generation units S The set of generator i s generation units whose production cost c k satisfies the equation (6.2) MLP k The minimum marginal profit that generation unit k can accept q k The bidding quantity of generation unit k p k Bidding price of generation unit k [d, d] Declared interval demand by the ISO in a hour MCP(d) L The lower bound of the 95% confidence interval for the real MCP of demand d MCP(d) L The upper bound of the 95% confidence interval for the real MCP of demand d c k,i Production cost of generation unit k ms i Market share of generator i 10

11 Abstract INTEGRATED DEMAND AND SUPPLY SIDE MANAGEMENT AND SMART PRICING FOR ELECTRICITY MARKET Zixu Liu A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy, 2017 On the one hand, the demand response management and dynamical pricing supported by the smart grid had started to lead to fundamentally different energy consumption behaviours; On the other hand, energy supply has gone through a dramatic new pattern due to the emergence and development of renewable energy resources. Facing these changes, this thesis investigates one of the resulting challenges, which is how to integrate the wholesale market and the retail market into one framework in order to achieve optimal balancing between demand and supply. Firstly, based on the existing mechanisms of the wholesale and retail electricity markets, a simulation tool is proposed and developed. This enables the ISO to find the best balance between supply and demand, by taking into account the different objectives of the generators, retailers and customers. Secondly, a new market mechanism based on the interval demand is proposed in order to address the challenges of the unpredictable demand due to the demand response management programs. Under the proposed new market mechanism, the corresponding approaches are investigated in order to support the retailers to find their profit-optimal pricing strategies, the generators to develop their best bidding strategies, and the ISO to identify the market clearing price function in order to best balance supply and demand. In particular: 1) For the ISO, our designed mechanism could effectively handle unpredictable demand under the dynamic retail pricing. It also enables the realisation of the goals of dynamic pricing by utilising smart meters; 2) In the retail market, we extend the smart pricing model in the current research in order to enable the retailers to find the most-profitable pricing scheme under the proposed new mechanism with the demand-based piecewise cost (i.e., market clearing price) function; 3) For the wholesale market, we developed a pricing forecasting model in order to forecast a market clearing price. Based on this model, we analysed the optimal bidding strategies for a generator under an interval demand from the ISO. Simulation results are provided in order to verify the effectiveness of the proposed approaches. 11

12 Declaration No portion of the work referred to in this thesis has been submitted in support of an application for another degree or qualification of this or any other university or other institute of learning. 12

13 Copyright i. The author of this thesis (including any appendices and/or schedules to this thesis) owns certain copyright or related rights in it (the Copyright ) and s/he has given The University of Manchester certain rights to use such Copyright, including for administrative purposes. ii. Copies of this thesis, either in full or in extracts and whether in hard or electronic copy, may be made only in accordance with the Copyright, Designs and Patents Act 1988 (as amended) and regulations issued under it or, where appropriate, in accordance with licensing agreements which the University has from time to time. This page must form part of any such copies made. iii. The ownership of certain copyright, patents, designs, trade marks and other intellectual property (the Intellectual Property ) and any reproductions of copyright works in the thesis, for example graphs and tables ( Reproductions ), which may be described in this thesis, may not be owned by the author and may be owned by third parties. Such Intellectual Property and Reproductions cannot and must not be made available for use without the prior written permission of the owner(s) of the relevant Intellectual Property and/or Reproductions. iv. Further information on the conditions under which disclosure, publication and commercialisation of this thesis, the Copyright and any Intellectual Property and/or Reproductions described in it may take place is available in the University IP Policy (see ter.ac.uk/docuinfo.aspx?docid=487), in any relevant Thesis restriction declarations deposited in the University Library, The University Library's regulations (see /regulations) and in The University's policy on presentation of Theses. 13

14 Publications The following papers have been produced during the research of this project, which are closed related to each thesis chapter. Liu, Z., Zeng, X. J., & Ma, Q. "Integrating demand response into electricity market." Evolutionary Computation (CEC), 2016 IEEE Congress on. IEEE, Liu, Z., & Zeng, X. J. "Integrated Demand and Supply Side Pricing Optimization Schemes for Electricity Market." Advances in Computational Intelligence Systems. Springer International Publishing, Liu, Z., & Zeng, X. J. Demand Based Bidding Strategies under Interval demand for Integrated Demand and Supply Management. Evolutionary Computation (CEC), 2018 IEEE Congress on. IEEE, 2018 (in Review Process). Liu, Z., & Zeng, X. J. Integration of Demand and Supply Side Mechanism Design for Electricity market. Energy, (in Review Process). 14

15 Acknowledgement Firstly, I would like to express my deep gratitude to my supervisor Dr. Xiaojun Zeng. I was extremely fortunate to receive his guidance, his inspiration and his endless support throughout all these years. He taught me how to do research and introduced me to the world of academia. He is a great example to me. I would like to thank Ms. Rose Goodier for providing me a lot of help in English academic writing. Her help was invaluable to me. Also, I am grateful to Dr. Fanlin Meng for his support throughout these years. My sincere thanks to all members of staff at the University of Manchester for providing me with a perfect working environment during all these years. Last but not least, I would like to acknowledge from the bottom of my heart the endless love and support of my family, especially: my parents, Chao Liu and Shulin Wang; and my wife, Xilan Wu. They have always stood by me and kept me going, and without them I would not have been capable of finishing this PhD. 15

16 CHAPTER 1. INTRODUCTION Chapter 1. Introduction 1.1 Context and Motivation In 1886, the first alternating current power grid system was installed in Great Barrington, Massachusetts [1]. The original grid was a centralized unidirectional system consisting of electricity transmission, electricity distribution and demanddriven control. By the middle of the 20 th century, the electricity grid had developed into a very large, mature and highly interconnected system. In the electricity grids, the central generation power stations delivered electricity to major load centres via high capacity power lines. Then electricity was branched and divided in order to supply it to small industrial and domestic users over the entire supply area. The location of the power stations strongly influenced the structure of the power grid, since power stations are located closely to fossil fuel reserves, hydro-electric dams are located in mountainous areas and nuclear power plants are sited in places which considered the availability of cooling water. By the late 1960s, the electricity grid had reached the overwhelming majority of the population in developed countries. A traditional electricity meter installed in the end-user s house was used to measure the levels of consumption. In general, there was only limited communication between the electricity suppliers and the customers due to the technological limitations which existed at that time, such as limited data collection and the processing capability of electricity grids. At that time, the communication infrastructure did not allow interactions between electricity 16

17 CHAPTER 1. INTRODUCTION suppliers and their customers. The fixed-tariff arrangement commonly charged in the grid as a result which totally ignores the important role of customers in energy management. Because, under the fixed-tariff pricing scheme, the customer had no incentive to shift their electricity loads from peak-demand to off-peak demand periods. Throughout the 1970s to 1990s, growing demand led to increasing numbers of stations. In some areas, the supply of electricity, especially at peak times, could not fulfil the demand from the demand side, and this resulted in poor power quality, including blackouts, power cuts and brownouts. Towards the end of the 20 th century, the use of domestic heating and air-conditioning usually led to daily peaks in demand. In order to meet that requirement of demand, a number of peaking power generators were turned on for this short period. The relatively low utilisation of these peaking generators and the necessary redundancy in the electricity grid resulted in high costs to the electricity companies, which were then obviously transferred in the form of increased tariffs. Since the early 21 st century, improvements in electronic communication technology provided the opportunity to resolve the limitations and the cost of the electricity grids. With the new smart meter, the peak-hour electricity price was no longer forced onto all customers. In parallel, renewable energy, such as wind energy, was deemed desirable to use to replace part of traditional fossilfired power stations due to the environmental damage concerns. But the dominant forms of renewable energy such as wind power and solar power are highly variable, so a more sophisticated control system was urgently required. Power from photovoltaic cells and wind turbines had called into question the imperative for large and centralised power stations. The change of the electricity grid from a centralised topology to a highly distributed one was inevitable. 17

18 CHAPTER 1. INTRODUCTION Furthermore, the centralised power stations were perceived to be potential terrorist attack targets [2]. Due to the increasing concern over terrorist attacks in some countries, a more robust electricity grid, which is less dependent on a centralised power station, had become more desirable. Based on the above analysis, a more intelligent and reliable grid with twoway communication is urgently required. Over past decades, the electricity grid has been under reconstruction in many countries. The smart grid is built to replace the traditional electricity market, such as the New England electricity market in US and the England and Wales s electricity market in the UK. Some developing countries, such as China, India and Brazil, are also seen as pioneers of smart grid deployment [3]. The smart grid (SG) is also known as an electricity power system with advanced, intelligent and automated communication and control techniques in its generation, transmission, distribution and consumption process [4]. The smart grid has the following features: The smart grid uses technologies such as state estimation [5] which improve fault detection and enables the self-healing of the network. This ensures that the supply of electricity is more reliable. The next-generation transmission and distribution infrastructure is able to handle bi-direction energy flows, which allows for distributed generation. The local sub-network generators, such as photovoltaic panels, wind turbines and batteries of electric cars could be added to the network and be managed by the smart grid. This makes the network topology more flexible [86] [87]. The traditional electricity grid does not allow rapid fluctuations in distributed generation which may be caused by renewable energy in some scenarios such as cloudy or gusty weather. In order to ensure 18

19 CHAPTER 1. INTRODUCTION stable power levels, it only permits the more controllable generators on the grid, such as gas turbines and hydroelectric generators. But the smart grid with advanced control and monitoring algorithms, advanced energy storage, and the next-generation transmission and distribution infrastructure reduces the variability associated with renewable energy, enabling large amounts of renewable energy electricity (such as solar power and wind power to be available) on the grid, thus reducing emissions [110]. The deployment of smart grid technologies overall improves the efficiency of the energy infrastructure. In particular this includes the demand-side management, for example, using the washing machine during off-peak periods. The overall effect lead to less redundancy in transmission grids and greater utilization of generators, leading to lower electricity prices. The two-way communication infrastructure enables smart meters in the home and business to arrange the usage of the demand based on the electricity price. Overall, this will reduce total demand during the high cost peak usage periods which prevent system overloads. The smart grid allows for systematic communication between retailers and customers which permits both of them to be more flexible and sophisticated in their operational strategies. The retailers will be able to sell electricity in order to maximize their profit and the customers will be able to minimise electricity costs by adapting demand to the lower electricity price period. Only the better infrastructure cannot make the grid smart. The one core technology that makes the grid smart is the demand-side management (DSM). Demand-side management programs are implemented in the retail market (a 19

20 CHAPTER 1. INTRODUCTION retail market exists when the retailer sells electricity to its customers at the retail price) of the smart grid. This offers a class of solutions designed to control the demand of the electricity on the customer s side in order to reduce consumption in peak demand periods or shift consumption from peak demand periods to offpeak demand periods [6]. Among various DSM programs, the demand response (DR) in the retail market of the smart grid attracts much attention due to its ability to incentivise customers to change their electricity consumption patterns. According to a report of the U.S. Department of Energy, demand response is defined as follows [7]: Demand response is a tariff or program established to motivate changes in electric use by end-use customers in response to changes in the price of electricity over time, or to give incentive payments designed to induce lower electricity use at times of high market prices or when grid reliability is jeopardized." In the retail market, there exist two main DR programs. These are: direct load control (DLC) and smart pricing [91]. DLC is an approach for residential load management in which the retailer can remotely control operations of certain appliances in the household based on an agreement with customers [8]. The retailer needs to know the usage information of its customers in order to implement DLC, and this will cause privacy problems for customers. Instead of DLC, smart pricing programs (dynamic pricing) give time-varying electricity prices which reflect the value and the cost of generation in different time periods. Customers are encouraged to individually and voluntarily manage their demand by reducing consumption in peak demand periods or shift consumption from peak demand periods to off-peak demand periods. With the help of a smart meter in the house, customers can optimally respond to the retail price in smart pricing programs [89] [90]. However, there is one main question in order to fully utilize the smart pricing programs in the retail market: that is, how to design 20

21 CHAPTER 1. INTRODUCTION efficient smart pricing strategies by taking into account customers potential response for the retailer? This question is also an important part of DSM. Apart from the retail market, another important part in the smart grid (electricity market) is the wholesale market. A wholesale electricity market exists when competing generators offer their electricity output to retailers at wholesale prices. In the smart grid, with the dual concerns of climate change and energy security, renewable energy resources have become an increasingly attractive proposition and are now beginning to achieve significant levels of penetration. Compared with the past, supply of the wholesale market has become more unpredictable due to fluctuations in distributed generation. Such fluctuations could be caused by cloudy or gusty weather. The costs of supply may also vary according to different times and days. This is why dynamic pricing is useful in the retail market, as it provides time-varying electricity prices which reflect the value and the cost of generation throughout different time periods. In the wholesale market, day-ahead electricity trading (auction) is operated in order to dispatch the electricity in many de-regulated environments. The main method that is used to dispatch supply and demand economically is the last accept bid method. In this method, generators are required to submit blocks of generations along with the associated prices. All the bids are then aggregated and sorted by price in ascending order. This aggregate curve is called the supply curve. Similarly, the aggregate demand curve can be created in the same way based on retailers bids. The aggregate demand and supply curves are then plotted against one another, and the intersection point is defined as the market-clearing price (MCP). All bids to the left of the intersection point are accepted and all generators are paid based on the MCP. Figure 1.1 shows an example of the last accept bid method. 21

22 CHAPTER 1. INTRODUCTION Figure 1.1: Last accepted bid dispatch model The selection of the bids is extremely important for all generators, since their profits are based on the result of dispatch of their units and on the market clearing price. So, how to select optimal bidding strategies in order to maximize profit is the main problem faced by the generator. This kind of question can be referred as the optimal bidding strategy problem [9]. The solution approaches for this problem have been presented in many papers. Although the problem description differs in the detail of each paper, they all focus on the object of maximizing the profit for a generator subject to some common constraints. In summary, the models for solving the optimal bidding strategy problem can be broadly classified in two types. The first type of model selects the best strategies for a generator s profit by estimating the bidding behaviour of other generators (competitors). The second type of model focusses on predicting the MCP for the next day, based on the prediction to decide the best bids for a generator. Unlike other products, electricity is hard to keep in stock, ration or have customers queue for. It has to be always available for demand. Therefore, a controlling agency, the Independent System Operator (ISO) [10], is needed to 22

23 CHAPTER 1. INTRODUCTION coordinate the dispatch of generation units in order to meet the expected demands of the system across the transmission grid [11]. The previously mentioned last accepted dispatch is one of the coordination schemes operated by the ISO. Currently, with the development of demand response in the retail market, some research introduces demand response in the retail market to the wholesale market. As we have said before, with the help of the demand response programs, customers will reduce their cost by shifting their demand from the period of higher price to the period of lower price. This shifting decreases the total demand in peak-hours which obviously reduces the total cost of generation and prevents the overload for generators. But this shifting also makes the generation scheme hard to schedule for the ISO in the wholesale market. Furthermore, every group in the smart grid has a different objective. For example, retailers and generators aim to maximize their own profit. In order to achieve that goal, generators and retailers would reduce their cost and price the electricity higher for the retailers and customers respectively. By contrast, customers would like their electricity bills to be as low in price as possible. Due to the different objectives, it is hard or impossible for all the groups in the market to achieve their objectives at the same time. But the performance of a market is measured by social welfare [12]. Social welfare is a combination of the cost of a commodity (in this thesis, electricity) and the benefit if the commodity as measured by society s willingness to pay for it. To achieve such a social welfare goal, ISO thus has to perform an optimization coordination scheme to select the optimal production scheme to ensure the supply meet the demand and balanced the conflict goals among all groups in the electricity market. Therefore, designing a market-coordination mechanism which can effectively and best balance the 23

24 CHAPTER 1. INTRODUCTION demand and supply by taking into account customers reaction is the main problem faced by the ISO. 1.2 Research Problems and Research Objectives As stated before, the penetration of renewable resources in the wholesale market and the DR programs in the retail market cause demand and supply to become more unpredictable. The conventional system balancing methodologies are invalid under this scenario. So, one of the important motivations of our research is to integrate the demand and supply sides into one framework to balance the supply and demand in the smart grid. Therefore, the research problem that we are looking at in this thesis mainly focuses on investigating and developing a coordination mechanism to support the ISO s balancing of the demand and supply in the electricity market in an optimal or most effective way. To achieve this, designing the integrated smart pricing schemes for generators (optimal bidding strategies) and retailers in order to maximize their profits under the integrated framework are also important problems that we are dealing with in this thesis. As stated before, every group in the electricity market has a different objective. Furthermore, under the demand response programs such as real-time pricing, customers have the incentive to reduce their electricity usage at the peak period or to shift their usage to the off-peak period, which means the demand from the retail market is unpredictable. In that scenario, the existing mechanisms in wholesale market no longer work. Therefore, a market-coordination mechanism which can effectively balance demand and supply by taking into account the customers reaction is needed for a market. In order to solve this problem, one of our objectives is to overcome the weakness of existing 24

25 CHAPTER 1. INTRODUCTION mechanisms and develop a new coordination mechanism for the ISO to coordinate conflict goals for all groups in the electricity market. In order to support the ISO in the implementation of our new coordination mechanism, we designed two methods to find the best balance between demand and supply. These methods are: the Genetic Algorithm-based method and the Analytical Optimization algorithm. For the first method, how to use Genetic algorithm to solve the combination explosion problem in our balancing problem in order to implement the new coordination mechanism was the main challenge that we faced. Normally, the calculation time of the Genetic algorithm is very long. Therefore, the question of how to design a new Analytical Optimization algorithm to efficiently balance the demand and supply in our new coordination mechanism is another challenge that we wish to solve. For the Analytical Optimization algorithm, the technical challenge is to prove the feasibility and correctness of this newly-designed algorithm. With regard to most of the retailer s pricing models in the retail market, they all assume that the unit electricity cost and the total cost of the retailer are based on its own demand. But, in reality and in our integrated framework, the unit cost (MCP) is based on the demand of all retailers, which means that the change of one retailer s demand could change all retailers unit costs. Therefore current pricing mechanisms cannot be worked when we integrate the demand and supply sides into one framework. How to design an efficient smart pricing model for the retailer in the integrated framework is another problem. Besides, we need to solve the optimal bidding problem for the generator in the wholesale market. With introducing the demand response to the wholesale market, the bidding problem becomes more complicated. So another objective is to propose a new, more robust and reliable wholesale pricing and demand pricing mechanism for the generator and the retailer respectively. 25

26 CHAPTER 1. INTRODUCTION In summary, the objectives of our research are: 1. We aim to overcome the weakness of the existing mechanism in order to enable the ISO to find the best balance between supply and demand. We do this by taking into account the different objectives of the generators, retailers and customers. 2. We also aim to develop a new coordination mechanism for the ISO to coordinate conflict goals for all groups and best balance the unpredictable levels of supply and demand in the electricity market. 3. Under the proposed coordination mechanism, we also aim to propose a new, more robust and reliable wholesale pricing and demand pricing mechanism for the generator and for the retailer respectively. 1.3 Contributions The section outlines contributions that have been made in this thesis. Firstly, based on the existing mechanisms of the wholesale and retail electricity markets, a simulation tool is proposed and developed. This enables the ISO to find the best balance between supply and demand, by taking into account the different objectives of the generators, retailers and customers. Secondly, a new market mechanism based on the interval demand is proposed in order to address the challenges of unpredictable demand due to dynamic pricing and demand response management programs. The contribution of our new market mechanism can be further discussed from the ISO-level perspective (i.e. the coordination mechanism in the integrate framework), the retailer-level perspective (smart pricing design) and generator-level perspective (optimal bidding problem). 26

27 CHAPTER 1. INTRODUCTION At the ISO level, we design a new mechanism for the electricity market to integrate supply and demand response. Our mechanism could effectively handle unpredictable demand under the dynamic retail pricing. It also enables to realise the goals of dynamic pricing by utilising smart meters such as effectively using renewable energy resources, reduce CO2 emission, minimise the bill for the customers and maximize the profit for both retailers and generators. In this new mechanism, the ISO provides the hourly interval demand to generators. Such interval demand is more robust and they enable the ISO to consider the demand variates under different prices from retailers. After receiving the bids from generators based on the interval demand, the ISO generates a MCP curve or function defined on each interval demand. Then the ISO provides the information about the obtained MCP curve to the retailers. Each retailer optimizes its retail price of each hour under each MCP vector (normally 24 hours) and estimates its customers hourly response demand (the customers of this retailer normally response to the retail price of each hour with a demand) to each retail price vector. Based on these estimated response demands, retailer can obtain the demand curve under each MCP vector and send it to the ISO. Finally, the ISO finds the match equilibrium between the demand and supply curve. Another important contribution with the proposed new mechanism in the ISO level is that we propose an effective analytical algorithm to find the match equilibrium between the demand and supply side in the new mechanism. The GA-based method is capable of solving the combination explosion challenge when implementing our mechanism, as well as enabling the ISO to best balance the unpredictable levels of demand and supply in the electricity market. However the calculation time of the GA-based method is very long. That is because that the genetic method could efficiently solve the combination explosion challenge with a small solution domain. When dealing with the large 27

28 CHAPTER 1. INTRODUCTION scale large-scale global optimization problem, the GA is time consuming. The analytical optimization method is designed to improve the efficiency of the GAbased method which is able to find the match equilibrium in a short time. In order to ensure the correctness and feasibility of this new method, we also prove that the monotonicity and convergence properties are held in the algorithm. Based on the learned demand model, Ma [13] built a pricing model for the retailer to generate an optimized pricing decision in order to achieve maximum profit in the retail market. The demand model is to estimate customer s consumption behaviour in relation to electricity prices and learned from past data. In Ma s pricing optimization model, the unit electricity cost and the total cost of the retailer are based on its own demand. But in our framework, the ISO declares an interval demand to all generators and then generates a MCP curve for all retailers according to the generators bids and the declared interval demand. As a result, the cost function for each retailer becomes a piecewise function which means that Ma s smart pricing model cannot be used in our framework directly. So we extended Ma s smart pricing model to enable it to integrate into our framework. The modified model can generate optimized pricing decisions to support each retailer to achieve maximum profit under the demand-based piecewise cost function. In the wholesale market, the ISO declares an hourly interval demand to the generators under our new mechanism. We built a forecasting model for predicting a MCP function under the declared interval demand. This is the main difference with current work in the wholesale market. Normally the current research only predicts a single value for the MCP. But this single predicted value cannot be 100% precisely, meaning that the predicted MCP always has a prediction error. In some scenarios, this part of the error could influence the result of the bidding auction. Therefore, we developed a feed-forward neural 28

29 CHAPTER 1. INTRODUCTION network and introduced the notion of the confidence interval to it. In contrast to the single prediction, the confidence interval can predict the interval of the maximum bound of the next day s MCP. This method is able to ensure that there is a high probability that real MCP will occur within this interval. Based on prediction of the proposed forecasting model, we also analyse the best bidding strategies for a generator under an interval demand from the ISO. 1.4 Thesis Organization The rest of this thesis is organized as follows. In Chapter 2, we firstly introduce the background of the smart grid, the demand response programs and the strategic bidding program in the wholesale market. Secondly, we investigate the state-of-the-art of research work on the smart grid in three perspectives: The smart pricing in the demand side, the optimal bidding in the generation side and the coordination mechanisms for the ISO. Thirdly, a critical analysis of the related work is provided. In Chapter 3, we develop an integrated framework and a method of pricing optimization which integrates and enhances the existing demand modelling and the retailers pricing optimization methods. As mentioned previously, current market mechanisms have limitations which cannot be implemented in the integrated electricity market directly, so we modify the current market mechanisms and integrate them into our framework. Based on this integrated framework and the modified methods, a computing simulation tool for the ISO is developed to support the ISO in finding the most acceptable and negotiable scheme for coordinating and balancing conflict goals among the generators, the retailers and the customers. This ensures the fair distribution of cost and benefit among all groups in the market. This chapter can be viewed as our first attempt 29

30 CHAPTER 1. INTRODUCTION to solve the integrated demand and supply side pricing optimization problems for the electricity market. In Chapter 4, in order to conquer the weakness of the proposed simulation tool in Chapter 3, we propose a mechanism which can find the match equilibrium between the demand and supply side when integrating demand response into the electricity market. A genetic algorithm-based approach is used in order to implement this mechanism. Simulation results confirm that the proposed mechanism can generate accurate predictions of electricity demand, maximize the retailers profit under the minimum production cost in supply side, and achieve the requirements of the ISO for balancing the demand and supply and Economic Dispatch. In Chapter 5, we proposed an effective analytical algorithm to replace the genetic algorithm-based approach for implementing the new mechanism in Chapter 4. Although the simulation results of genetic algorithm-based approach are valuable, it is still time consuming to use this approach to run the new mechanism. The new analytical algorithm could overcome this weakness. In order to ensure the correctness and feasibility of this new algorithm, a proving process is also provided in this chapter. In Chapter 6, we mainly devoted to solving the generator s optimal bidding problem under our new mechanism. In previous two chapters, we propose a mechanism which can find the match equilibrium between the demand and supply side when integrating the demand response into the electricity market. But how to select the optimal bidding strategies to maximize its profit is still an unsolved problem for each generator. Therefore, in the wholesale market, we developed a feed-forward neural network to forecast the market clearing price and introduced the notion of the confidence interval to the forecasting model. The confidence interval can predict the exact range of the next day s MCP. 30

31 CHAPTER 1. INTRODUCTION Furthermore, we analyse the optimal bidding strategies for the generator under an interval demand from the ISO. The thesis is concluded, and details of future work are provided in Chapter Publications The work presented in this thesis has resulted in two publications, with two further journal papers under review: 1) Liu, Z., Zeng, X. J., & Ma, Q. "Integrating demand response into electricity market." Evolutionary Computation (CEC), 2016 IEEE Congress on. IEEE, ) Liu, Z., & Zeng, X. J. "Integrated Demand and Supply Side Pricing Optimization Schemes for Electricity Market." Advances in Computational Intelligence Systems. Springer International Publishing, ) Liu, Z., & Zeng, X. J. Demand Based Bidding Strategies under Interval demand for Integrated Demand and Supply Management. Evolutionary Computation (CEC), 2018 IEEE Congress on. IEEE, 2018 (in Review Process). 4) Liu, Z., & Zeng, X. J. Integration of Demand and Supply Side Mechanism Design for Electricity market. Energy, (in Review Process). Chapter 3 is an expanded version of 2) Liu, Z., & Zeng, X. J. Chapter 4 is an updated version of 1) Liu, Z., Zeng, X. J., & Ma, Q., while Chapter 5 was presented in 4) Liu, Z., & Zeng, X. J. and Chapter 6 was presented in 3) Liu, Z., & Zeng, X. J.. 31

32 CHAPTER 2. BACKGROUND AND RELATED WORK Chapter 2. Background and Related Work In this Chapter, the background and related work is described in order to put this thesis into context. Firstly, the related background of the smart grid, the demand response and strategic bidding in the wholesale market used in this thesis are presented in Section 2.1. Secondly, a description of related work on the electricity market is provided in Section 2.2. More specifically, the smart pricing design for the retailer in the retail market (demand side) is provided in Subsection and the optimal bidding problem for the generators in the wholesale market (supply side) is provided in Subsection In Subsection 2.2.3, we review some current research work relating to demand and supply side integration in the electricity market. Thirdly, a critical analysis of the existing related work is provided in Section 2.3. The chapter is concluded in Section Introduction In this section, we firstly introduce some background concepts of the smart grid. Then the concepts of demand response and strategy bidding are examined. After that, an example of electricity market in practice is illustrated in detail Smart Grid In contrast to the traditional electricity grid, which is used to deliver electricity from a few central generators to a large number of customers, the SG uses two-way flows of electricity and information in order to create an 32

33 CHAPTER 2. BACKGROUND AND RELATED WORK Table 2.1: A brief comparison between the traditional grid and the SG [15] Existing Grid Smart Grid Electromechanical Digital One-way communication Two-way communication Centralized generation Distributed generation Few sensors Sensors throughout Manual Monitoring Self-monitoring Manual restoration Self-healing Failures and blackouts Adaptive and islanding Limited control Pervasive control Few customer choices Many customer choices automated and distributed advanced electricity delivery network [14]. By utilizing modern information technologies, the SG is capable of delivering electricity in more efficient ways, responding to wide ranging conditions and events such as electricity generation, distribution and transmission, and adopting the corresponding strategies. For instance, since lowering peak-demand and smoothing demand reduces the total generation costs and avoids overload in the grid, the electricity utility (retailer) can use demand response programs such as real-time pricing in order to convince some customers to reduce their peak hour s demand, or to shift the peak-hour s demand to a non-peak hour. Table 2.1 gives a comparison between the current grid and the SG [15]. More specifically, the SG can be regarded as an electric system that uses information, two-way, cyber-secure communication technologies, and computational intelligence in an integrated fashion across electricity generation, transmission, substations, distribution and consumption in order to achieve a system that is clean, safe, secure, reliable, resilient, efficient, and sustainable. This description covers the entire spectrum of the electricity system from 33

34 CHAPTER 2. BACKGROUND AND RELATED WORK generation to the end point of consumption [16]. According to the report from NIST [17], the basic requirements and benefits of the SG can be summarized in the following points: Enhancing the efficiency of the existing electricity grid; Optimizing facility utilization and averting the construction of peak load electricity plants; Enabling predictive maintenance and self-healing responses to system disturbances; Accommodating distributed electricity sources; Reducing greenhouses gas emissions by enabling renewable energy sources; Increasing the consumer s participation in the electricity market and providing more choice for the customer. In a technical perspective, the SG can be divided into two major systems: the Smart Infrastructure System and the Smart Management System. The Smart Infrastructure System is the energy, information and communication infrastructure underlying the SG. It supports two-way communications. The smart meter plays an important role in the smart grid and is indispensable to the two-way communication infrastructure. It can record electricity energy consumption in intervals of an hour or less and can communicate that information, at least on a daily basis, back to the utility company for monitoring and billing purposes [8]. In the UK, all households will be installed with smart meters by 2020 as part of the smart grid plans spanning the whole country [18]. The Smart Management System is the subsystem that provides advanced management and control services and functionalities. The key reason behind the grid becoming smarter is the development of management application and 34

35 CHAPTER 2. BACKGROUND AND RELATED WORK Figure 2.1: The market model in the SG Table 2.2: Roles and actors in the SG Role Customer Retailer Generator ISO Actors in the SG The end users of electricity. May also generate, store and manage the use of electricity. The organization providing services to electrical customers. The generators of electricity in the market. Selling electricity to the retailers. Market operator, balancing the supply and the demand. programs such as demand response. Based on the smart infrastructure, the purpose of the smart management system is to achieve the following objectives: energy efficiency improvement, supply and demand balance, emission control, operation cost reduction, and utility maximization [88]. The traditional electricity market can be divided into two parts: the wholesale market and the retailer market. In order to achieve such goals as supply and demand balance, generation cost reduction and utility maximization, the SG integrates these two markets into one. Here provides the model of the 35

36 CHAPTER 2. BACKGROUND AND RELATED WORK electricity market in the SG which is shown in Figure 2.1. This model consists of 4 parts. Each part plays a different role in the SG. The core role among them is the ISO, this being the controlling agency, or system operator, in the SG. The responsibility of the ISO in the SG is to coordinate the dispatch of generation units in order to meet the expected demands of the system across the transmission grid. As well as fulfilling the demands of the system, the ISO s dispatch should also achieve the requirements of Economic Dispatch (ED) and Unit Commitment (UC). The ED means that the dispatch should minimize overall production costs by optimally allocating projected demand to generating units that are online. In addition to determining the amount of electricity which each generating unit should be producing when it is online, the ISO must also determine when each generating unit should start up and shut down. This function is known as UC. Brief descriptions and actors of all other parts are given in Table Demand Response in the Retail Market There are many well-developed electricity management programs in the smart management system of the SG. These include fuel substitution problems, conservation and energy efficiency programs and demand response (DR) programs. Among all of these programs, DR is one of the most promising strategies for helping to balance the load and supply, reduce the peak demand and increase grid reliability [19]. We have already introduced the basic notions of DR in Chapter 1. In this section, we will illustrate the benefits of DR and different DR programs in detail. DR can be defined as a set of activities of reducing or shifting the electricity usage by end-user customers from their normal consumption patterns in response to the changes in the price of electricity over time [20]. Further, DR can 36

37 CHAPTER 2. BACKGROUND AND RELATED WORK be also defined as the incentive payments designed to induce lower electricity use at the times of high prices or when system reliability is jeopardized [7]. The use of the DR programs in the SG could bring many benefits. These benefits can be classified in four main categories: Participants; Market wide; Reliability and Market performance benefits. They are listed in detail as follows [21]: The customers who participate in DR programs can reduce the electricity bills by shifting their usage of peak hours to off-peak hours or by reducing their usage of peak hours. The benefits of DR are not limited to the customers who shift their demand in response to the prices of electricity only. Nevertheless some of these benefits are market-wide. Since the shifting of the demand will flatten the aggregated demand profile and reduce the over-cost of producing electricity, consumers who do not participate in DR programs could therefore also benefit if this reduction in production cost translates into a reduction in prices. A flattened demand profile could avoid overload or outage in the SG and hence increase the reliability of the SG. As with the well-designed DR program, participants are given the opportunity to help to increase the reliability of the SG. The reliability benefits can also be considered as one of the market-wide benefits since it can affect all market participants. The final category of DR programs benefits is improving the market performance. Under DR program, consumers have more choices in the market, even when retail market competition is not available. Consumers can manage their demand since they have the opportunity to affect the market. 37

38 CHAPTER 2. BACKGROUND AND RELATED WORK In general, DR can be categorized into two different types: incentive-based DR (IDR) and price-based pricing DR (PDR) [21]. IDR refers to customers getting a financial reward for non-dr periods by reducing their electricity usage during periods of system need or stress. IDR programs include the Direct Load Control program, Interruptible/Curtailable programs, Demand Bidding, Emergency Demand Response Program, Capacity Market programs and Ancillary Services. PDR programs are based on dynamic pricing rates in which electricity tariffs are not flat, so the rates fluctuate according to the real time cost of electricity. The ultimate objective of PDR is to flatten the demand profile by offering high prices during peak hours and lower prices during off-peak hours [92]. These rate methods include Time of Use [93], Critical Peak Pricing [94], Real Time Pricing and Day Ahead Pricing. In the following section, we introduce several of the most popular DR programs. Incentive-based DR Direct Load Control is a demand response activity by which the electricity retailer remotely shuts down or cycles a customer s electrical equipment (e.g. air conditioner, water heater) at short notice. Interruptible/Curtailable programs: This refers to the programs in which the customer receive tariffs or contracts that provide a rate discount or bill credit for agreeing to change electricity consumption, such as reducing the load during system contingencies [8]. Price-based DR Real Time Pricing (RTP) programs refer to the rate method in which customers are charged hourly at fluctuating prices which reflect the real cost of electricity in the wholesale market. RTP programs can effectively incentivise 38

39 CHAPTER 2. BACKGROUND AND RELATED WORK Figure 2.2: An example of the RTP program customers to lower their demand usage during peak-hours or to shift their demand usage from high demand hours to low demand hours. Many economists are convinced that RTP programs are the most efficient and that direct DR programs are suitable for the competitive electricity market and should be considered by the policymakers in the SG [22]. Figure 2.2 shows an example of the RTP program. Day Ahead Pricing (DAP) programs are based on RTP. Under DAP, customers will receive the prices for the next 24 hours before the delivery time, and these prices will be fixed in the day of consumption. The advantage of DAP over RTP lies in the fact that the customers will have 24 hours to plan their electricity usage based on the informed prices [23]. As stated before, PDR programs are based on the dynamic pricing rate. Dynamic pricing is a sub-discipline of pricing in Revenue management. In the following paragraph, the concept of revenue management is introduced. 39

40 CHAPTER 2. BACKGROUND AND RELATED WORK Revenue management is the application of disciplined analytics which predict consumer behaviour at the micro-market level and optimize product availability and price in order to maximize revenue growth. The primary aim of Revenue management is to sell the right product to the right customer at the right time for the right price and with the right pack. The essence of this discipline lies in understanding customers' perception of product value and accurately aligning product prices, placement and availability with each customer segment [115]. The normal process of revenue management consists of data collection, segmentation, forecasting, optimization and dynamic re-evaluation. Revenue management has incredible benefits and has been widely used in many industries including financial services, the car rental industry and hotel and hospitality services. Revenue management encompasses a wide range of opportunities to increase revenue. A company can utilize these different categories like a series of levers in the sense that all are usually available, but only one or two may drive revenue in a given situation. The primary levers are: Pricing, inventory, marketing and channels [116]. (Here, we simply introduce the pricing, as PDR is in many ways related to it.) This category of revenue management involves redefining pricing strategy and developing disciplined pricing tactics. The key objective of a pricing strategy is to anticipate the value created for customers and then to set specific prices in order to capture that value. A company may decide to price against their competitors or even against their own products, but the most value comes from pricing strategies that closely follow market conditions and demand, especially at a segment level. Once a pricing strategy dictates what a company wants to do, pricing tactics determine how a company actually captures the value. Tactics involve creating pricing tools that change dynamically, in order to react to changes and continually capture value and gain revenue. How to 40

41 CHAPTER 2. BACKGROUND AND RELATED WORK determine the optimal price in order to maximize the revenue or profit is referred to as price optimization. This involves constantly optimizing multiple variables such as price sensitivity, price ratios, and inventory. A successful pricing strategy, supported by analytically-based pricing tactics, can drastically improve a firm's profitability [117] Strategic Bidding in the Wholesale Market The bidding problem in the wholesale market is related to a pool trading in which the sealed auction is widely employed. Electricity generators, and sometimes retailers too, are required to offer price and quantity bids to the ISO [24]. Then the ISO determines the winning bid and a uniform market clearing price (MCP) using a simple merit order dispatch procedure. Since competition mainly exists at the generation side, and the transmission system is still a monopoly, the bidding problem in the wholesale market is concerned mainly with electricity generators. Therefore, the most important objective of investigating strategic bidding behaviour is to help generators in the competitive market to maximize their own profits. On the other hand, since the performance of a market is measured by the economic notion termed social welfare, investigating strategic bidding behaviour could also help the ISO to identify the potential market electricity abuse and to limit such abuse by introducing appropriate market management rules. An auction is an economically efficient mechanism which is used to allocate demand to generators, and that is why the formation of wholesale electricity markets in many countries is based on auctions. Bidding is an issue connected to the auction. So, auction rules and bidding protocols are the main factors in influencing the development of bidding strategies. 41

42 CHAPTER 2. BACKGROUND AND RELATED WORK For the first factor auction rules, almost all operating electricity markets in the word employ the sealed bid auction with a uniform market price (MCP). In the sealed auction, all bidders simultaneously submit the sealed bids, so that no bidder knows the bid of any other participant. The bidding protocol is another important factor relating to bidding strategies. Depending on different market designs, the electricity bid can be classified as single-part bid which includes a single price component and multipart bid which has several prices components [25]. Both types of bids should include several electricity price segments depending on the amount of energy supply. A multipart bid, also called a complex bid, may include separate prices for ramps, start-up costs, shut-down costs, no-load operation, and energy, which can reflect the cost structure and technical constraints of generation units. Usually optimization methods are used in the market clearing procedure. The reason for this is that the optimization methods need to take into account not only the bid prices, but also the technical constraints and related economic information when determining the winning bids and the MCP. This approach leads to a centralization of the unit commitment decisions at the ISO s level: All generators are required to send all the relevant information and the ISO makes the optimal decisions. This approach can guarantee the technical feasibility of the resulting schedule. A typical use of the multipart bid is the England Wales electricity market. All generators or generation units are required to submit a combined bid consisting of many items for the next 48 hours [26]. In the single-part bid scheme, generators only submit their independent prices for each hour. The ISO will use a simple market clearing process to determine the winning bids and schedules for each hour. The clearing process is based on the intersection of the demand and supply bid curves. The last 42

43 CHAPTER 2. BACKGROUND AND RELATED WORK accepted bid dispatch model which has been introduced in Chapter 1 is the most common clearing process used in the single-part bid. In contrast to the multipart bid, the single-part bid scheme is intrinsically decentralized: the ISO does not make unit commitment decisions. Therefore the production and technical constraints should be considered by each of the generators themselves. They need to internalize all these constraints into their bid since the bidding structure does not explicitly account for the recovery of these costs. The single-part bid scheme has been implemented in several electricity markets including California, Australia and Spain. After introducing the basic bidding environment in the wholesale market, we now address the methods used to solve the optimal bidding problem. Broadly speaking, there are two main methods for developing optimal strategies for the generators. The first method is based on the prediction of the MCP in the next trading period. Based on the estimation of the MCP, it is quite straightforward for a generator to determine its best bidding strategy by simply bidding a price which is cheaper than the MCP. The most common way to predict the MCP is the regression. However, predicting the MCP is mainly based on the analysis of historical data. There is very little historical data in traditional electricity markets, so this makes the prediction difficult to implement. With the development of the SG, a two-way communication infrastructure is built in the grid. It can help the generators to collect the needed information quickly and accurately. The historical data is no longer a problem for the generator. Another problem with this method is an implicit assumption that the bid from one generator will not influence the MCP. Compared with the past, the wholesale market becomes more deregulated and competitive: more and more generators gain entry to the market. It is hard or impossible for a generator to manipulate the market. Therefore, with solving these two problems, the method of predicting the MCP 43

44 CHAPTER 2. BACKGROUND AND RELATED WORK gains more focus in current research. The second method of developing optimal bidding strategies is to utilize the estimation of bidding behaviour of the rival generators. The techniques most used in this method are probability analysis and fuzzy sets. However, this method has its drawbacks: Firstly, the bidding information of other generators is not available to any single generator. Secondly, the increasing number of generators in the market makes the estimation hard or impossible for a single generator. A description of publications of these two methods will be given in Section The Electricity Pool of England and Wales Based on differences in power system infrastructure and political configuration from the government, the electricity market in different countries have employed different market rules or bidding mechanisms. In this section, we focus on the main features of the Electricity Pool of England and Wales as an example of the electricity market in practice. The Electricity Pool of England and Wales (EPEW) is a centralised model that controls the scheduling and dispatch of generation in order to meet the forecasted system load. The EPEW operated the spot market at one day ahead of physical delivery and the market was cleared on a half-hourly basis. The main features of this market are listed in detail as follows [119]: Compulsory: All the electricity energy produced and consumed in England and Wales had to be physically traded through the pool. Passive demand role: Only the generators submitted bids. The demand is assumed to be inelastic in the EPEW. A fixed value is set, determined by a demand forecast, which is based mainly on historical data and weather forecasts. Demand response to prices in the EPEW is ignored. 44

45 CHAPTER 2. BACKGROUND AND RELATED WORK Complex bids: The bid submitted by generators does not consist of simple price-quantity pairs, but of a medley of parameters designed to represent the cost and technical constraints associated with generating electrical energy by a specific unit. The parameters include incremental offers, start-up costs, no-load costs and the operating limits of the units such as: generation limits, minimum up and minimum down times. The generators in the EPEW are required to submit only one bidding function throughout the trading day. The bidding function contains up to a maximum of three segments. Each of these corresponds to an incremental price that is non-decreasing in the subsequent segment [118]. Centralized scheduling: The generators bidding data were submitted to the Generation Ordering and Loading programme by the ISO (National Grid Company) in order to produce the generation schedule for the trading day. An optimal generation schedule is determined centrally for the day-ahead on the basis of these bids and the load forecast. However, the scheduling program did not take account of the transmission constraints. Centralized pricing: at each half-hour the adjusted marginal price of the marginal unit in this schedule sets the System Marginal Price. All of the electrical energy generated is purchased at this price for that half-hour. This electricity market is designed to minimize risks for the generation companies. Two reasons support this view. Firstly, if a unit bid its operating cost, it would never have to generate at a loss in the bidding. Secondly, since all the dispatched generation is paid based on the system marginal price, generators who have lower costs do not have to predict the system marginal price. They can simply bid low (or even zero) in order to ensure that they are dispatched for generation and leave the task of setting the system marginal price to a truly marginal generator. If efficient generators had to predict the system marginal 45

46 CHAPTER 2. BACKGROUND AND RELATED WORK price, they would probably raise their bidding price to compensate for the loss they would incur when predicting incorrectly [119]. But there are two main drawbacks to the EPEW. Firstly, this market does not consider the effect of demand response from the demand since the demand is assumed to be inelastic. Furthermore, the generators in the EPEW are not allowed to change their offering prices at different periods. This is due to the operation limitation of the thermal plant which needs a long time of cool start (around 6 hours). This means that the generation side can t respond to the hourly demand changes. As stated in section 2.1.2, the demand response programs could directly reflect the production cost of each hour on the generation side, flatten the demand profile in trading and reduce the total generation cost. Therefore, ignoring the demand response from the demand side is incorrect in concept and inefficient in practice. The second drawback of the EPEW is that there is not enough competition in the market. The British Government floated only two generating companies: Powergen and National Power. While there were initially four more participants in the market, all of these companies were only interested in providing base generation. Therefore, only National Power and Powergen own the mid-merit plants that normally set the system marginal price. Under these conditions, the incentive to compete for setting the system marginal price is much weaker than the temptation to adopt a bidding strategy that will increase the system marginal price. If there is enough competition, it can be shown that the generators optimal bidding strategy is to bid their marginal cost. As stated before, the EPEW is designed to reduce the risk of generators, and this should lead to lower prices. In practice, however, there wasn t enough competition. Moreover, electricity prices did not drop as much as they should have done when the price of fuel dropped. 46

47 CHAPTER 2. BACKGROUND AND RELATED WORK Recently the Virtual Power Plant (VPP) has been developed in many countries such as the United States, Europe and Australia. The VPP could help to solve the weaknesses of the EPEW, as mentioned before. A VPP is a cloud-based distributed power plant that aggregates the capacities of heterogeneous Distributed Energy Resources for the purposes of enhancing power generation, as well as trading or selling power on the open market. In the field of power generation, the VPP integrates several types of power sources to give a reliable overall power supply [4]. The source is often a cluster of different types of dispatchable and non-dispatchable, controllable or flexible load distributed generation systems. The VPP has the ability to deliver peak load electricity or load-following power generation at short notice. This overcomes the weakness of traditional thermal plants which were unable to respond to the hourly demand changes. For example, the VPP in United States not only deals with the supply side, but also helps to manage demand. It ensures the reliability of grid function through demand response and other load-shifting approaches, in real time. In the field of energy trading, a VPP acts as the intermediary between Distributed Energy Resources and the wholesale electricity market and trades energy on behalf of Distributed Energy Resource owners who by themselves are unable to participate in that market [121]. The utilization of VPP in the market could increase the competition among generators. 2.2 Related Work in the Electricity Market- A Literature Review The research problems we are looking at in this thesis are the integrated demand and supply sides management program and the smart pricing problems 47

48 CHAPTER 2. BACKGROUND AND RELATED WORK in the electricity market. These problems always involve interactions among the generation side, the demand side and the ISO. Thus, in the following sections, we are going to investigate and explore the related work from the retailer side perspective, the generator side perspective and the ISO perspective respectively. On the retailer side, the state-of the-art of customers demand modelling methods which are response to the time-differentiated prices and the retailer s smart pricing methods by taking into account the customers response is discussed in subsection On the generator side, the related works on optimal bidding problems for the generator are explored in subsection At the ISO level, we will discuss the current work of the coordination mechanisms for the ISO when integrating the demand side into the supply side in subsection Smart Pricing Design for the Retailer Under time-differentiated pricing programs, customers might respond to the price of electricity by shifting their demand from peak-hours to off-peak hours, or by curtailing their electricity demand during peak-hours. How to exactly model customers consumption behaviour when faced with time-differentiated price signals is the key to solving the retailer s smart pricing problems. In the following sections, we will firstly explore the existing customer demand modelling methods. Then the research on smart pricing design for the retailer will thereafter be discussed. In an electricity market, different customers need different levels of electricity, and they might experience different levels of satisfaction with the same price and amount of consumed electricity [27]. The customers preference and their electricity consumption patterns can be represented in the form of utility functions, which are based on a concept taken from microeconomics [28], 48

49 CHAPTER 2. BACKGROUND AND RELATED WORK [29]. They represent the corresponding utility functions as: U(x, ω), where x is the electricity consumption level of the customer and ω is a parameter which might vary among customers and at different times of the day. The utility functions represent the customer s levels of satisfaction with electricity consumption. The properties of the utility functions are introduced in [29]: Property 1: Utility functions are non-decreasing. This implies that the marginal benefit is non-negative, therefore we have: U(x,ω) 0. Property 2: The marginal benefit of users is a non-increasing function. That is, 2 U(x,ω) x 2 concave. 0, this property with the property 1 indicates that utility functions are Property 3: For a fixed consumption level x, a larger ω gives a larger U(x, ω), which can be expressed as U(x,ω) ω 0. Property 4: When the consumption level is zero, U(0, ω) = 0, ω > 0. Through modelling the customers electricity consumption patterns in the form of utility functions, Samadi also proposed a single-level optimization based on a smart pricing model for the retailer to set the electricity price and maximize the welfare function in [29]. In this model, Samadi models the energy cost of distributing electricity by the retailer at each hour h as C h (L h ), where L h is the amount of energy offered by the energy provider. A quadratic function is used to model the cost function. They also assume that the cost function is increasing and convex. So the retailer s welfare maximization problem is formulated using the utility function minus the cost function, which is shown below: max h H n N U(x h n, ω h n ) C h (L h ) (2.1) s. t. n N x h n < L h, h H x 49

50 CHAPTER 2. BACKGROUND AND RELATED WORK Where U(x h n, ω h n ) h represents the utility function of customer n at hour h, x n h is the amount of energy consumed by customer n at hour h and ω n denotes the ω parameter of the utility function of customer n at hour h. The above optimization problem can be solved using convex programming techniques in a centralised manner. There are also some similar works, such as [30-32] [36] [95] and [96]. In [30], the authors model and analyse the interactions between the retailer and its customers as a four-stage Stackelberg game, in which the customers preference and electricity consumption patterns are modelled via various utility functions. In [32], the authors propose a RTP-based electricity scheduling scheme as demand response for the residential electricity demand using a Stackelberg game model. The difference between this work and [30] is the utility function used to model customers behaviour. In [31], the authors propose a decision scheme based on a Stackelberg game between the retailer and the customers, in which household appliances are divided into 3 different levels. In contrast with other work, the objective of the retailer in this work is to maximize its profit which is most commonly used in practice. In all the above examples of research, they commonly define a type of utility function to model customers consumption behaviour and assume that all the customers will maximize their utility function. Furthermore, they all assume that the retailer knows the exact utility functions of its customer. However, there does not exist a unique utility function which can model all customers preference in reality. Even if such a function existed, how could the retailer know about this function? Therefore, without knowing the exact utility function from customers, modelling customers response to timevarying price is the key to solving such problems. 50

51 CHAPTER 2. BACKGROUND AND RELATED WORK Figure 2.3: Typical demand curve The Elasticity of the Demand In 1989, Schweppe and his co-workers developed the concept of spot pricing of electricity [33]. They investigated a system wherein the spot prices changed in real time and the customers could adjust their demand depending on the spot price. In [34], the authors indicate that the customer will try to shift their consumption to other period when assuming customers demand is curtailable. Therefore the concept of the price-elasticity of demand is important for demand modelling. The conception of elasticity originates from the discipline of microeconomics. Generally it is safe to say that the demand for most commodities decreases when the price of the commodities increase as show in Figure 2.3. It is not possible to use a precise function to describe the curve, so economists often linearize this curve around a given point. Then they define the price elasticity of demand as the relative slope of this demand curve. ε = q p (2.2) q 0 p 0 51

52 CHAPTER 2. BACKGROUND AND RELATED WORK This elasticity coefficient ε indicates the relative change in demand for a commodity that would result from a change in the price of this commodity. In [35] it is assumed that the price and quantities have been normalized with respect to a given equilibrium point (q 0, p 0 ). Equation (2.2) can be rewritten as (2.3). ε = q p (2.3) Self-elasticity is used to describe a change in the price of commodity q which will affect the demand of p. Self-elasticity will always be negative. At the same time, this change in price for a will also affect the demand of other commodities. We use a positive Cross-elasticity to describe this effect. These two effects are described as equation (2.4). { qa = ε aa p a ; ε aa 0 q a = ε ab p b ; ε ab 0 (2.4) We have introduced the concept of elasticity in the view of microeconomics before. From these basic notions, we know the customers reaction to the fluctuating prices of electricity based on the time frame. In [35], the authors consider the short-term response. This short-term response is defined as the time that elapses between the publication of the prices for the next 24-hour interval and the actual demand periods. A self-elasticity coefficient means the relationship between the demand in an hour and the price of that hour. If the customer decreases the demand in some hours and increases the demand in other hours, we have called this action: the rescheduling of demand. In contrast with the self-elasticity, the cross-elasticity coefficients relate the demand in one hour to the prices during other hours. Due to the difference of the claiming price by the retailer or wholesale market in each hour and the preference price of the customers, the customer may shift the demand in one hour to an hour with a lower price. Equation (2.5) expresses this action. 52

53 CHAPTER 2. BACKGROUND AND RELATED WORK 24 q i = j=1 ε ij p j (2.5) Another assumption in [35] is that the change of the demand does not extend beyond the 24-hour scheduling period. Therefore all the self-elasticity and cross-elasticity coefficients in this time interval can be arranged in a 24 by 24 matrix: E, which is shown in equation (2.6). Q is the matrix of the quantity in each hours and P is the matrix of the price within each hour. Q = E P (2.6) The self-elasticities are located in the diagonal elements of the matrix E. The rest of the matrix corresponds to the cross-elasticities. If the prices of the electricity in hour i changed, the demand throughout all the period will be affected by this change. This impact is shown in column i of Matrix E. The most aggressive of all customers responses to the actual change of the prices from their preference prices will determine the structure of the elasticity matrix and the value of its elements. But in [35], only the properties of the elasticity matrix have been discussed. The process of how to model the matrix for a set of customers is lacking. Therefore, the next section will review a new method of modelling the elasticity matrix. Learning Based Demand Modelling Current studies on learning customers' energy consumption behaviours mainly focus on understanding the aggregated energy usage patterns of customers, i.e. they either learn the aggregated response of a pool of customers to the price signals or they learn the aggregated response of each individual customer to the price signal. In [13], the authors model the interaction between the electricity retailer and customers as a one-leader and multiple-followers Stackelberg game with imperfect information. The day-ahead real-time pricing is 53

54 CHAPTER 2. BACKGROUND AND RELATED WORK Figure 2.4: The structure of the Stackelberg game model applied in the market. Based on the past usage data of the customer, they model the customers consumption behaviours as an elasticity matrix by using the adaptive least square method. In this Section, we will review the process of the estimating elasticity matrix. This structure of the market in [13] is shown in Figure 2.4. The retailer plays the lead role in the market and the customers acted as followers. As we discussed in the last Chapter, the demand of the customer at each hour is not only affected by the price of the current hour but also affected by the price of other hours. If the price of electricity at hour i is much higher than the other hours prices, the customers will react to shift the usage of electricity to another hour or hours. So [13] considers the utility function from all customers at each hour as equation (2.7). y h = R h (u L ) = R h (p 1, p 2,, p H ) (2.7) In which y h is the amount of electricity demand by all customers at hour h (MKw), h H, p h is the price in hour h. As aforementioned, it is not impossible 54

55 CHAPTER 2. BACKGROUND AND RELATED WORK for the retailer to know all y h. So there is a need to find an estimated reaction function R h(p 1, p 2,, p H ) which is as close to R h (p 1, p 2,, p H ) as possible. This property can be expressed as equation (2.8). Mnimize (R h R h ) 2 h H (2.8) They select a linear function as the estimated reaction function. Generally speaking, a demand model is a nonlinear function. So in [35], the authors have assumed that the prices and quantities have been normalized with respect to a given equilibrium point (q 0, p 0 ). In the small range around this point, the demand curve is assumed as linear. Another support point for linear function is that the price of electricity changes slowly with time. Due to these reasons, many nonlinear functions can be approximated well by a linear function locally. The learning method used in [13] is the adaptive least square method which updates its parameters when any new data are available. The reason is that the adaptive learning method can effectively catch the slow variation of a system. This has already been proved in [37]. The form of the estimated reaction function for each hour h H can be represented as equation (2.9). R h(p 1, p 2,, p H ) = α h + β h,1 p β h,h p H (2.9) The β h,c in equation (2.9) is the cross-elasticity as described before. As mentioned before, the exact utility function of every customer is impossible for the retailer to know in reality. However, the retailer finds it easy to get the information of the price of electricity and the consumption of electricity from all customers throughout each hour h H in the last N days. Therefore they assume that such information in previous N days is available, the problem formulation for the demand modelling can be rewritten from (2.8) as: Min λ N n (α h + β h,1 p 1 (n) + + β h,h p H (n) d h (n)) 2 n ℵ h H (2.10) 55

56 CHAPTER 2. BACKGROUND AND RELATED WORK s.t β h,c > 0 if h c β h,c < 0 if h = c β h,h β c,h 0 h=1 c=1 h c In which 0 < λ 1 is the forgetting factor which exponentially reduces the influence of old data. Through using the Recursive Quadratic Programming method, problem (2.10) can be solved. For a 24 hour scheduling period, the result of problem (2.10) can be arranged in a 24 by 24 matrix as below: β 1,1 β 1,2 β 1,H β 2,1 β 2,2 β 2,H [ β H,1 β H,2 β H,H] (2.11) There are also some similar studies on learning based demand modelling [38] [39] [40] [97] and [98]. In [38], the authors utilize linear regression models in order to learn the price-elasticity of demand. This gives the aggregated response of the customers to price signals in one distributed power system. In [39], they propose an agent-based model to study customers' price elasticity of demand and the economic effects on electricity markets. This shows that reductions in price spikes, customers' bills and emissions of greenhouse gases and other pollutants can be achieved when customers respond to the price signals. Different with [13], the work of [39] only considered the self-elasticities. In [40], they propose a daily load curve forecasting model for residential customers based on a time series and stochastic regression framework where the customers are assumed to respond to the price signals. The daily load curve is represented in the terms of a set of periodic smoothing-spine basis functions. In contrast with previous works which only consider the price-elasticities, the 56

57 CHAPTER 2. BACKGROUND AND RELATED WORK evolving of the basis function coefficients also considers the day of the week and holidays adjustments, as well as weather effects. Smart pricing Design for the Retailer Smart pricing schemes which influence customers' energy usage decisions via price signals are a very promising option for demand response management. After introducing the demand modelling problem, the next question is how to design an efficient smart pricing scheme for the retailers by taking into account the customers' responses in order to benefit both the retailer and its customers. The smart pricing design methods in current work can be divided into two parts: single-level optimization based approaches [29] [41] [103] and two-level game theory based approaches. The work of [29] is a typical example of single-level optimization based smart pricing design and is concerned with how the retailers set electricity prices in order to maximize the welfare function, where they model the customer's energy consumption patterns in the form of theoretic utility functions. The details of [29] have been illustrated at the start of Section The two-level game theory based smart pricing design deals with how the retailers determine the electricity prices based on the expected responses of customers where the interactions between the retailer and its customers are represented as a Stackelberg game [99] [102]. There are some works on the topic of two-level game theory-based smart pricing approaches, including [30-32] [42] [43] [44] [100] and [101]. For example, [32] also models the interaction between the retailer and its customers as a 1-leader, N-follower Stackelberg game. More specifically, a detailed customer-side consumption model is provided. Given the price vector P = {p 1, p 2,, p H } for the time horizon, the optimal scheduled start time s for the appliance a of the customer is obtained by solving the following 57

58 CHAPTER 2. BACKGROUND AND RELATED WORK optimization problem (2.12), where t 0 is the time slot requested to turn on at the outset and each time slot of delay for appliance incurs a cost of φ a dollars. T a denotes the non-interruptible operations duration of appliance a and the power consumption for this duration of appliance a is E a KW. s+t a Min(s t 0 ) φ a + h=s p h E a (2.12) They then model the retailer-side problem as a profit-maximization problem. The objective function of the retailer is defined as the gross profit, GP, which is equal to the revenue, T t t=1 π t p n,a subtracting the cost of energy usage to the t provider where π t is the retail price, p n,a is the energy consumption at time slot t of appliance a for customer n and T is the appliance scheduling horizon. There are two parts in the energy cost: the cost of buying the consumed energy of this appliance from the wholesale market, C e = T t=1 t t p n,a, where t is the wholesale price; the other cost C m, caused by the mismatch between the actual load and planned supply by this appliance. As a result, the profit maximization problem for the retailer is shown as follows: Max T (π t t ) t t=1 p n,a C m (2.13) Finally, the above proposed 1-leader N-follower Stackelberg game is solved by distributed algorithms executed at the retailer-side and the customer-side respectively. In the work of [29] [30] [32] and [36], it defines a utility function for the customer and assumes that the retailer knows the exact customers utility functions. As we said before, there does not exist a unique utility function which can model all customers preferences in reality. Even if such a function existed, it would be difficult for the retailer to know this function. The work of [13] overcomes this weakness. It provides a solution for the retailer s smart pricing problem which is to achieve the maximized profit based on a demand responsive 58

59 CHAPTER 2. BACKGROUND AND RELATED WORK model. The demand responsive model is learned from history data which has been introduced before. More detail of [13] is shown below. In [13], except for modelling the customers consumption behaviour as the elasticity matrix by learning the past usage data, they also propose an optimal pricing model for the retailer. For each hour h H, they define the minimum and maximum price that the min max retailer can offer to its customers, where p h and p h are usually set based on several factors, such as the cost of electricity (the wholesale price), customers average income and affordability, and the constraints which are enforced by government policy. This constraint for the retailer can avoid the high price offered to the customers. p h min p h p h max (2.14) They also set a maximum supply capacity for each hour h, which is denoted as E h max. Therefore, a constraint is shown as: R h(p 1, p 2,, p H ) E h max, h H (2.15) Furthermore, similar to the constraints (2.14), a constraint on the total revenue should exist due to the pressure from the ISO and from customers acceptability. p h (α h + β h,1 p β h,h p H ) h H R max (2.16) A flattened demand profile could lower the total cost of the demand and avoid the overload in the grid. Therefore keeping the Peak-and-Average ratio (PAR) at a low level is another problem that the retailer is faced with. If we define the daily peak and the average load as L peak and L avg, then the PAR of the demand in that day can be represented as: PRA = L peak L avg (2.17) 59

60 CHAPTER 2. BACKGROUND AND RELATED WORK Finally, the optimal pricing problem for a retailer can be expressed as: Max ((p h c h ) R h(p 1, p 2,, p H )) h H (2.18) s.t R h(p 1, p 2,, p H ) E h max, h H p h min p h p h max p h (α h + β h,1 p β h,h p H ) R max h H PRA PRA max In [13], the problem (2.18) is transferred to a quadratic programming problem which can be solved by OPTI TOOLBOX [73] Optimal Bidding Problem for the Generator A market can be classified as a monopoly, an oligopoly or a perfect competition according to the level of competition. The level of competitiveness in the perfect competition market is the highest among these three kinds of market. In contrast, the competition level in the oligopoly market is lower and there rarely exists any competition in the monopoly market. The main criteria used to distinguish the different kinds of the market are the number of suppliers and consumers, and the amount of influence that individual actions can have on the market price. For example, in a perfect competition market, no supplier has the power to manipulate the market by influencing the market prices [107] [108] [109]. The following paragraphs review some papers of optimal bidding problems in two aspects: imperfect and perfect competition. Imperfect Competition A market is said to be imperfect if a supplier is able to manipulate the market by means of withholding output or by raising the offer price beyond its marginal 60

61 CHAPTER 2. BACKGROUND AND RELATED WORK costs in order to increase the market price. Profit is increased if the price rise is sufficient enough to compensate for a possible loss in the sales quantity. Electricity supply is a capital-intensive industry with a high barrier of entry for new generators. This is seen particularly in the traditional electricity market, with the restriction of a centralized generation structure. It is hard to increase the number of generators. Coupled with the fact that electricity cannot be stored economically in bulk, electricity markets can easily be manipulated by either the monopoly generator or several oligopoly generators. In the current research, some papers focus on the subject of optimal bidding in the competitive market. The objective of these papers is mainly to devise optimal bidding strategies that maximise the profits of generators. As we said before, there are two main methods for designing optimal bidding strategies for the generator: Modelling rival generators bidding behaviour and predicting the MCP of the market. Due to the properties of the imperfect competition market, the first method is widely used in these papers. For example, in [45], the authors model the bidding strategies of generators in such a way that each generator adjusts its bidding function subject to the expectation of the rivals bidding action. The details of this work are given as follows. In [45], generator i submits linear non-decreasing offer curve of the form (t) (t) (t) (t) (t) B (i) (P(i) ) = αi + βi P(i) for each hour of in the day ahead market where (t) (t) (t) (t) α i and βi are bidding coefficients for producer i, B(i) is the revenue and P(i) is the generation output. Market dispatch model: after reviving the bids from the generators, the market administrator computes the MCP R t and solves the following economic dispatch problem to compute the generation output for each generator. R t (t) (t) (t) = α i + βi P(i) t = 1, 2, 3 24 (2.19) 61

62 CHAPTER 2. BACKGROUND AND RELATED WORK (t) P jmin n (t) (t) P j Pjmax j=1 = Q t t = 1, 2, 3 24 (2.20) P j (t) j = 1,2, n, t = 1, 2, 3 24 (2.21) In (2.19), the market administrator dispatches the generation to each generator so that bid prices coincide with the MCP for each hour. Equation (2.20) ensures that the total amount of generation fulfils the demand Q t. Equation (2.21) ensures that all generators are assigned the generation in each hour. The amount of allocated generation is between their lower and upper bounds. Generator s strategies: the strategy of a generator in this framework is to determine offer curves for each hour with the aim of maximizing hourly benefit or providing minimum stable output. These two strategies ensure that enough generation is allocated so that the offer curves are profitable. The generators consider an optimization model for each of these problems then formulates a unit commitment model that integrates the strategies form the previous two models. The hourly benefit model: this model finds the generation offers which can maximize hourly profit with given data about estimated demands and estimated bidding behaviour of other bidders. The model is shown as follows: Max φ t (α (t) i, β (t) i ) = R t P (t) (i) C i (P (t) (i) ) (2.22) Subject to 2.19 to 2.21 Where C i (P (t) (i) ) is the cost of the generation for generator i which is a function of the generation. In (2.22), φ t (α i (t), βi (t) ) is the hourly benefit objective function. The constraint of (2.19) requires the bidding coefficient of other generators. But a single generator does not have such information, so the authors in [25] use a joint probability distribution for estimating all other 62

63 CHAPTER 2. BACKGROUND AND RELATED WORK producers. The hourly benefit model then becomes a stochastic optimization model. Minimum stable output bidding strategy: this model aims to ensure that average output from a generator achieves near minimum generation level. This model is shown as follows: Min δ t (t) (t) (t) (t) (α, β) = P i Pmin + γ(p i Pmin) 2 (2.23) Subject to equation (2.21) Where γ is a positive penalty parameter. Overall generator bidding model: this model integrates the strategies from the previous models in order to help the generator to determine when the it should be on in 24 hours. The overall bidding strategy model is shown as follows: Max Ω(μ t ) = M + 24 [μ t φ t (t) (t) t=1 (α i, βi ) S(τ)μt (1 μ t 1 )] (2.24) Subject to 24 t=1(μ t μ t 1 ) 2 N (2.25) Where the objective function (2.24) is constructed to be positive (M is a sufficiently large number) in all cases, since this model will be solved by a genetic algorithm representing the fitness of the bidding strategies. In model, μ t is a binary variable that is 1 if a unit is up for hour t, otherwise 0. S(τ) is the start-up cost of the generator. Constraint (2.25) ensures that a unit has the maximum number of start-ups and shut downs in a day. There are also some similar works, such as: [46] [47] [106]. In [46], they present the problem of strategic bidding under uncertainly in a wholesale market. In those works they defined a set of scenarios for other generators behaviours and maximized the profit of this generator throughout all scenarios. An algorithm that allows a generator to maximise its welfare by trading in electricity markets is presented in [47]. The nature of the market equilibrium is investigated by solving 63

64 CHAPTER 2. BACKGROUND AND RELATED WORK the algorithm iteratively until all participants cease to modify their bids. The paper highlighted the fact that the equilibrium does not always exist and that, if it does, there may be more than one solution. Perfect Competition Ideally, an electricity market should be sufficiently well-designed to ensure vigorous competition participants and should leave no scope for gaming. According to the economists definition of efficiency, perfect competition would lead to an allocation of resources that was completely efficient [48]. In reality, however, the majority of existing electricity markets are more akin to oligopoly than to perfect competition and are always operating at a lower efficiency. With the development of the SG, this situation has been changed. The advanced infrastructure in the SG allows distributed generation which enables more and more generators to participate in the market. Even customers can sell electricity to the market. Compared with the past, the electricity market has become increasingly deregulated and competitive. Therefore, numerous papers are concerned with optimizing bidding policies under the perfect competition market. In the perfect competition market, using the first method to solve the optimal bidding problem for the generator is unrealistic. Since more and more generators have entered the market, it is hard, or impossible, for a generator to estimate all competitors bidding behaviours, especially when there are such a huge number of generators in the wholesale market. Selecting the optimal bidding strategies based on the predicting MCP is the way to solve this problem. Normally generator s bidding decisions must consider the anticipated MCP, generation award and costs, and competitors decisions. As we said before, the information of all other generators is not available to any single generator at the time of its bid. Therefore, forecasting the MCP in the wholesale electricity market 64

65 CHAPTER 2. BACKGROUND AND RELATED WORK is the most essential task and basis for any decision making for the generator. It is quite straightforward for a generator to determine its best bidding strategy by simply bidding at a cheap price than the MCP. The most common way to predict the market behaviours is regression. The basic idea is to use the historical MCPs and other information such as the demand of the retail market, fuel price, etc. in order to predict MCPs. In other words, we use the history and other available information from tomorrow to fit the MCP of tomorrow. A well-established nonlinear regression method is the artificial neural network (ANN). ANN method has been widely used for short-term electricity price forecasting [54] [55] [56]. The applications of ANN to the electricity price forecasting were first discussed in [49] [50] and [51]. Based on the neural network and time series, a price forecast model was built for the UK electricity market in [50]. In [51], the price forecasting problem in the Victorian electricity market of Australia was discussed. A threelayered back propagation neural network model for forecasting MCP and the corresponding load of the daily market was investigated in [49]. The model only needs publicly-available information as the input data for predicting the MCP and the load of the electricity market. The forecasting results showed that this method is more accurate than the conventional method. Recently researches have also made efforts to apply the neural network for MCP forecasting. For example, [52] proposed a neural network to forecast next-week prices in the electricity market of mainland Spain. The Levenberg-Marquardt algorithm is used to train a three-layered feedforward neural network. Compared with previous models such as the time series model, the proposed neural network is easy to implement and shows good performance as well as being less time consuming. In [53], authors examined several artificial neural network based models which have different inputs for day-head price forecasting. The exam results provided 65

66 CHAPTER 2. BACKGROUND AND RELATED WORK evidence that ANN models can be considered as useful and robust forecasting tools for market participants. Here, we mainly review the work in [49] in order to know the general process of using ANN to predict the MCP in the market. BP neural network: the multi-layer feed-forward neural network is used in the model of [49]. It has been proved that almost any finite-dimensional vector function defined on a compact set can be approximated to any specified accuracy by a BP network if there are enough data, number of hidden units and enough computational resources. The one hidden layer BP network with linear output is: Y = W 2 σ(w 1 X + β 1 ) + β 2 (2.26) Where, X is an n-dimension network input vector, Y is a dimension network output vector, W 1 and W 2 are weight matrices for the input layer and hidden layer respectively, β 1 and β 2 are bias vectors for the input layer and hidden layer respectively. The transfer function of a hidden layer is a hyperbolic tangent sigmoid transfer function defined as: σ(x) = 1 e x 1+e x (2.27) Model structure: as we said before, there are many factors which may influence the MCP in the wholesale market; Selection of input variables is extremely important for the neural network forecasting model. Based on the market analysis and experimental results, the following factors are considered in [49] as input variables in the neural network: 1) Historical MCP; 2) Hourly forecasting demands; 3) Fuel prices; 66

67 CHAPTER 2. BACKGROUND AND RELATED WORK ) Weather; 5) Hour indices. With the above consideration, the structure of forecasting is shown in Figure Training the BP neural network: Network training is used to select the network parameters to minimize the fitting error for a sampling set. For the given training data set {x p, t p i = 1,2,..., P}, the objective function is defined as: E = 1 p t 2 p Y p 2 p=1 (2.28) Where X p is the input vector of p-th sample in the training set, Y p is the network output vector of p-th sample, t p is the target output of p-th sample. Usually the global minimum cannot be obtained, but only a set of nearly optimal weighting coefficients {W 1, W 2, β 1, β 2 } can be found. The training algorithm used in [49] is an error back-propagation training algorithm. Following the training process, this model can be used to predict the MCP of the next day by means of inputting the required data. Figure 2.5: The structure of forecasting model 67

68 CHAPTER 2. BACKGROUND AND RELATED WORK Integration of Demand and Supply Side for the ISO Most of the existing research deals with the wholesale market and the retail market separately. In all the above mentioned papers of the last two subsections, the focus is only on a part of the entire electricity market: either the wholesale market or the retail market. For instance, [13] [31] and [36] do not consider that the change in demand will influence the generation cost and the market clearing price in the wholesale market, which thus undoubtedly changes the retailers pricing strategy. Most of the research on strategic bidding behaviours in the wholesale market has only considered the demand side as given demand curves such as [57] [58] [59] [60] [61] and [62]. For example, [57] considers the demand model as a fixed set of price-quantity values, which is a step function. Both of [58] [60] and [61] use the linear function to represent demand model and the coefficients in the function are given. [61] considers the demand from the demand side in each hour as a constant. However, in above papers, they only totally neglect the effects of the demand side, e.g., the influences of the demand elasticity on the bidding behaviour in the wholesale market. As we said before, the demand response program could not only help customers to reduce their electricity costs but it could also help to reduce the overall production costs. Besides, through using the demand response program to predict customers consumption behaviours, retailers can decide how much power they should buy, or the ISO can schedule the production scheme for generators. Therefore, in order to overcome these aforementioned weaknesses and to utilize the advantages of demand response programs, some of the research integrates the demand side and the supply side into one framework, e.g. [12] [35] [63] and [65]. In [35], the authors factor in the elasticity matrix of demand in electricity prices for customers. This matrix reflects the customer s 68

69 CHAPTER 2. BACKGROUND AND RELATED WORK reaction to electricity prices. With this model, the integrated electricity market can price the electricity in the retail market and improve the effectiveness for the wholesale market in order to avoid the waste of surplus electricity. In [12], the authors quantify the effects of the demand response on the electricity markets. This proves that, with the participation of customers, their demand shifting can significantly reduce the operating costs on the generation side. Some papers also consider the influences of demand elasticity on gaming behaviour in the wholesale electricity market, and they focus mainly on the effects of strategically changing the consumers load profile in different trading time intervals. This is characterized by the cross-price elasticity of the demand side [35] [66] [67]. Here, we review the work in [35] and [12] in order to prove the benefits of introducing demand response programs into the wholesale market. Quantifying the Effect of Demand Response on Electricity Markets The objective of the ISO in [12] is also to maximize social welfare. Social welfare in [12] is the difference between the value that users attach to the power that they use and the production cost for this energy, which is shown below: max T t=1 (GS t OC t ) (2.29) Where GS t and OC t are the consumer gross surplus and the system operating cost at period t respectively. These two parts will be illustrated follows in detail. T is the period in the optimization horizon. Generators offers: The generators submit a complex bid that embodies not only their operational cost data but also their operational constraints. The operating cost includes the no-load cost, the running cost and the start-up cost. To make the solution of this unit commitment problem possible using a mixedinteger linear programming package, piecewise linear cost curves are used in [12]: 69

70 CHAPTER 2. BACKGROUND AND RELATED WORK I B i,b,t OC t = i=1 (u i,t N i + b=1 MC i,b P Sg + S i,t ) (2.30) Where i is number of generating units; B is number of segments in the generator s offer curve; u i,t is the status of the generating unit i at period t (0: on, 1: off); N i is no-load cost of generating unit i; MC i,b is marginal production cost of the generating unit i on segment b of its piecewise linear cost curve; P i,b,t Sg is the output of generating unit i on segment b of its piecewise linear cost curve during period t; S i,t is the start-up cost of the generating unit i at period t. Demand-side bids: The authors assumed that some customers would have the ability to reschedule their demand according to their demand curve. They will decrease the hourly demand when the price of this hour is too high. This kind of customer is called the price-responsive customer. The rest of the customer population will not make the change in demand when any of the prices are claimed. The equation (2.31) shows that the consumer gross surplus is calculated based on the accepted demand-side bids and the marginal value that consumers attach to these bids. GS t = K J MB k,j,t k,j,t k=1 Sg D Sg (2.31) Where MB k,j,t Sg is the marginal benefit of segment j of the bid of demand-side bidder k; J is number of segments of the bid for bidder k and K is total number of demand-side bidders. This model allows a consumer to purchase a certain amount of electricity (D T z,t, where z is the index of price taking bidders) regardless of the MCP. The consumers who have the ability to adjust their demand should submit bids for electricity that are sensitive to electricity prices. This kind of bid is quite flexible due to the sensitivity property for electricity prices, which should include the following characteristics: (1) A conventional price-volume bid at a specific period; j=1 70

71 CHAPTER 2. BACKGROUND AND RELATED WORK (2) The minimum energy consumption at any period; (3) The maximum energy consumption at any period; (4) The total energy consumption over the scheduling horizon; (5) The price-taking bid for meeting an inflexible demand. These specifications can be translated into certain constraints on the demand at each period and on the total demand over the optimization horizon. Details can be found at [12]. System Constraints: The ISO wants to match the power produced by generators with the demand of the price-taking and price-sensitive users at each period. So getting the generation and demand schedule by solving the optimization program is the responsibility of the ISO. Equation (2.32) summarizes the constraint set by [12]: I i=1 P i,t K k=1 D k,t Z z,t z=1 D T = 0 (2.32) Where K is the number of price-responsive demand-side bidders; Z is the number of price-taking demand-side bidders; P i,t is the production of the generating unit at period; D k,t is the consumption of the price-responsive bidder at period; D z,t T is the consumption of the price-taking bidder at period. Through the simulation and test for the designed mechanism, [12] obtained evidence that the change in operational cost is equal to the change in social Figure 2.6: Integration of the elasticities with the price computation 71

72 CHAPTER 2. BACKGROUND AND RELATED WORK benefit for all participants, meaning that demand shifting spreads the reduction in the operating cost of the generators among all the market participants. Integrate the Elasticity Matrix with a Scheduling Program As well as introducing the elasticity matrix, the authors of [35] also proposed a scheduling program integrating with the elasticity matrix as mentioned before. Figure 2.6 illustrates how the price of electricity and the generation on an hourly based for a 48 hour period are set by the ISO with the consideration of the elasticity of the demand for electricity. The authors also suggest that the loops must be implemented many times in order to finally cause the convergence between expect demand and changed demand after knowing the prices of the next period. 2.3 Critical Analysis In this section, we will discuss the limitations of the existing research and highlight our research motivations. We are going to conduct the critical analysis by looking at three aspects: The coordination mechanism for the ISO perspective; the optimal pricing model for the retailer and strategic bidding for the generator Coordination Mechanism for the ISO As we have already stated, some of the current research deals with the wholesale market and the retail market separately. But introducing the demand response to the wholesale market could bring many benefits which have been discussed previously. Therefore, many papers integrate the demand side and supply side, and provide the valuable coordinate mechanisms and simulation results [35] [63] [12] [65] [66] [67]. However, there are still notable weaknesses in these publications: 72

73 CHAPTER 2. BACKGROUND AND RELATED WORK Firstly, there is a mismatch between the predicted demand from the ISO and the demand resulted from the retailers pricing in the existing wholesale pricing mechanism. Normally the ISO forecasts the demand of customers for the next day and informs the generators. After receiving the bids from the generators, the ISO calculates the MCP for each hour and sends these to the retailers. After that, the retailers price the electricity for their customers. The problem is that the demand is price-dependent, meaning that the customers may react to the retail price with a new demand which is different from the one predicted by the ISO. As a consequence, this leads to a mismatch between the predicted demand from the ISO and the demand resulting from the retailers pricing. Secondly there is a missing link between the market clearing price (MCP) and the demand caused by the pricing response in the existing wholesale pricing mechanism. As stated above, the change of demand means that the ISO has to recalculate the MCP, which causes a change to the MCP, as the MCP is the unit cost for the retailers. Therefore we can state that there is a missing link and a mismatch between the original MCP and the demand. Thirdly, the retailers pricing methods in the current research of demand response are very likely to fail due to the mismatch between the ISO s estimated demand and actual demand under the retailers pricing schemes. In the existing research, each retailer provides the elasticity matrix to the ISO or the ISO estimates the customers demand preference based on past data. Then the ISO based on this matrix or the estimated customers demand model in order to estimate the demand and calculate the MCP. But the retailers pricing methods are obviously different from the ISO, which makes the ISO s predictions inaccurate. That is, after having received the MCP from the ISO, the retailers decide their demand based on their own pricing considerations and methods. Due to the reason of pricing response mentioned before, there is a mismatch 73

74 CHAPTER 2. BACKGROUND AND RELATED WORK between the ISO s estimated demand and the retailers demand, which means that the MCP will be changed under the actual demand. But the retailer s pricing models are normally based on its unit cost. Therefore, we have stated that the retailers pricing methods will fail due to the mismatch between the ISO s estimated demand and the actual demand. Having noticed these shortcomings, in order to enhance the existing demand modelling and the retailers pricing optimization methods, the first motivation of our research is to propose an integrated framework for the ISO to coordinate the demand and the supply. We can then develop a method of pricing optimization under the existing mechanism of the retail and wholesale markets. Based on this integrated framework and the modified existing methods, a computing simulation tool for the ISO is developed to support it in finding the most acceptable and negotiable scheme for coordinating and balancing conflict goals among the generators, the retailers and the customers. The details can be found in Chapter 3. The second motivation of our research is to investigate and develop a new coordination mechanism to support the ISO s balancing of demand and supply in the electricity market. In this new mechanism, the ISO gives the hourly interval demand to generators. Such interval demand is more robust and they enable the ISO to consider the demand variates under different prices from retailers. This can overcome the previously mentioned weakness. The detail can be found in Chapter 4 and Optimal Pricing Model for the Retailer In the research papers of [29-32] and [36], the authors commonly define a type of utility function to model customers consumption behaviour and assume that all the customers will maximize their utility function. Furthermore, they all 74

75 CHAPTER 2. BACKGROUND AND RELATED WORK assume that the retailer knows the exact utility functions of its customer. However, a unique utility function which can model all customers preferences in reality does not exist. Even if such a function existed, how could the retailer know about it? Therefore, without knowing the exact utility function from customers, modelling customers response to time-varying price is the key to solving such problems. This is also why we select the elasticity matrix to model the customers consumption preference in our research. Most of the retailer s pricing models in the retail market, such as [13] [30] [32] and [42], assume that the unit electricity cost of the retailer is a simple cost value and that the total cost is based on its own demand. But, in reality, the unit cost (MCP) is based on the demand of all retailers, which means that the change of one retailer s demand could change all retailers unit costs. Therefore current pricing mechanisms cannot be worked out when we integrate the demand and supply sides into one framework. Based on the above analysis, our second objective is to propose a new, more robust and reliable demand pricing mechanism for the retailer under our integrated framework. The detail of our pricing model for the retailer under the integrated framework can be found in Chapter 3, 4 and Optimal Bidding Strategies for the Generator As we said before, there are two main methods for designing optimal bidding strategies for the generator. These are: Modelling rival generators bidding behaviour and predicting the MCP of the market. The first method is widely used in the imperfect competition market [45] [46] [47]. For example, in [45], the authors model the bidding strategies of generators in such a way that each generator adjusts its bidding function subject to the expectation of the rivals bidding action (More details see Section 2.2.2). In the modern electricity market, 75

76 CHAPTER 2. BACKGROUND AND RELATED WORK using the first method to solve the optimal bidding problem for the generator is unrealistic. Two reasons are given in support of this view. Firstly, the information of other generators (such as bidding history, capacity, etc.) is not available to any single company, as most auctions in the wholesale market are uniform-price auctions which require all bidders to simultaneously submit sealed bids to the ISO. Secondly, due to the fact that until recently most markets were structured as monopolies or duopolies, the wholesale market has become more competitive. More and more generators have entry the market. It is hard, or impossible, for a single generator to estimate all the competitors bidding behaviours, especially when the number of generators is so huge within the wholesale market. This is why we stated that selecting the optimal bidding strategies based on the predicting MCP method is the best way to solve the optimal bidding problem for the generator. The papers using the second type of method to solve the optimal bidding program also have drawbacks [49] [50] [51] and [52]. A single predicted value of MCP cannot provide enough help for a generator to select its best bidding strategy. Normally the predicted value cannot be precisely 100%, which means there is always an error in its prediction. In some scenarios, this kind of error could influence the result of bidding. For example, if the predicted MCP is higher than the real MCP, the generator still may lose the bidding when it sets the bidding price slightly smaller than the predicted MCP. In order to overcome the this weakness of the current research, we introduce the notion of the confidence interval to the ANN forecasting model and use it to help the generator to select the best bidding strategy. In contrast to a single forecasted number before, we calculate the confidence interval of next day s hourly MCP to provide what the exact range of the real MCP could be. This 76

77 CHAPTER 2. BACKGROUND AND RELATED WORK method could ensure that there is a high probability (90%, 95% or higher) of the real MCP occurring within the interval. Furthermore, both of the previously mentioned papers have another common drawback: they all neglect the effect of demand response in the market. Normally the ISO forecasts a demand of customers for the next day and tells it to the generators. Then the ISO produces a MCP for each hour to the retailers after receiving the optimal bids from generators under this amount of demand. After that, the retailers price the electricity for their customers. The problem is that the demand is price-dependent, which means that customers may react to retail price with a new demand which is different from the one predicted by the ISO. Furthermore, this change of demand also leads to a change in optimal bidding strategy for each generator. Therefore we state that the existing wholesale pricing mechanisms are very liable to fail in real market. That weakness means the current wholesale mechanism can t handle the unpredictable demand under dynamic retail pricing. Therefore another motivation of our research is to solve the generator s optimal bidding problem under our integrated framework by considering the influence of demand elasticity on the demand side. Our method to overcome this weakness is that the ISO declares an interval demand to the wholesale market rather than declares a single demand. Such the interval demand is more robust than a single demand figure and enables the ISO to consider the demand variates under different prices and to handle unpredictable demand under dynamic retail pricing. Based on the confidence interval of hourly MCP, how to select the best bidding strategies for a generator under an interval demand is another important problem that we have solved in this thesis. The details of this section can be found in Chapter 6. 77

78 CHAPTER 2. BACKGROUND AND RELATED WORK 2.4 Chapter Summary This chapter has provided a detailed background and a thorough review of the existing literature. This is necessary for understanding the rest of this thesis. We have, in particular, looked at the smart grid and related concepts and techniques, the background of demand response programs and the strategic bidding problem in the wholesale market. Following this, a detailed search has been carried out on the related work on the smart grid. This has been looked at in three perspectives: Smart pricing on the demand side, optimal bidding on the generation side and coordination mechanisms for the ISO. Finally, a critical analysis of the existing literature has been provided. 78

79 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Chapter 3. Integrated Demand and Supply Side Management and Smart Pricing for Electricity Market- A Simulation Tool 3.1 Introduction In this chapter, under the existing mechanism of the retail and wholesale markets, we propose an integrated framework for the ISO to coordinate the demand and the supply and to develop a method of pricing optimization which enhances the existing demand modelling and the retailers pricing optimization methods. As we discussed before, the existing market mechanisms have limitations that cannot be implemented in the integrated electricity market directly, so we integrate the existing market mechanisms into our framework. Based on this integrated framework and the modified methods, a computing simulation tool for ISO is developed to support the ISO in finding the most acceptable and negotiable scheme for coordinating and balancing conflict goals among the generators, the retailers and the customers. This chapter, which is adapted from our published work: [68], can be viewed as our first attempt to solve the integrated demand and supply side management problems for the electricity market. The rest of this chapter is organized as follows: Firstly, the simulation tool statement is given in section 3.2. Secondly, the proposed simulation tool and related algorithms are presented in section 3.3. Thirdly, a balance mechanism for supporting the ISO is given in section 3.4. This mechanism could coordinate the 79

80 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE conflict goals among all groups in the market when the production cost in generation side is increased or decreased. Fourthly, numerical results are given in section 3.5 showing that the simulation tool improves the efficiency of the electricity market. This chapter is summarized in section The Statement of Simulation Tool As in papers [63] and [64], we consider a smart power system with several generators, retailers and many customers as part of the general electricity market which is shown in Figure 3.1. The retailers buy electricity from the wholesale market and sell it to their customers. The smart meter in each customer s home interacts with the retailer through an underlying two-way communication network (e.g., the smart metering infrastructure) [4]. With the smart meter, the retailer is able to record the usage information of each customer. In the proposed model, an Independent Operation System (ISO) exists in the wholesale market as a market coordinator. The ISO schedules the production scheme and sets the Market Clearing Price (MCP) for the retail market while minimizing the total generation cost. As shown in Figure 3.1, there are four parts to the market: the generators (power producers), which produce the required electricity for the retail market; the ISO, who acts as the coordinator to the market, scheduling the generation scheme for the next day and determining the MCP; the retailers (power provider) who buy the electricity from the wholesale market based on MCP and sell it to its customers after re-pricing the electricity for each hour; Customers, who decide on the quantity of electricity that they brought from their retailer after receiving an hourly electricity price. 80

81 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.1: The structure of the electricity market Figure 3.2: The working process of proposed simulation tool Unlike other market models which requires detailed information on operation constraint from the generators to run an economic dispatch [69] [12], our proposed mechanism requires each generator to submit a bid curve (bid price vs. bid quantity). All other parameters such as ramp rate, star-up cost, noload cost, etc. are internalized in the bid curve through the self-unit commitment simulator. A bid is defined as a pairing of price and quantity (p i, q i ) of energy submissions by generator i at hour h. A bid curve or supply function of generator 81

82 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE i is a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,2,, M i ) sorted by price in increasing order by the bid price, which can be represented as a piecewise constant curve. The format of the piecewise step function is required by the California Power Exchange [69] [70] [104]. In mathematics, a function on the real numbers is called a step function if it can be written as a finite linear combination of indicator functions of intervals [71]. This interval is called a segment in the bid curve. Figure 3.3 provides an example of the step function. Every group in the electricity market has different objectives. For example, retailers and generators aim to maximize their own profit. In order to achieve this goal, generators and retailers would reduce their cost and price the electricity higher for retailers and customers respectively. In contrast, the customers would like their electricity bills to be as low as possible. Furthermore, unlike other products, electricity is hard to keep in stock. It is also hard to ration or have customers queue for it. It has to be always available for demand. But there is always a mismatch between the predicted demand from the ISO and the demand resulting from the retailers pricing in the existing wholesale pricing mechanism. Normally the ISO forecasts the demand of customers for the next day and informs the generators. After receiving the bids from the generators, the ISO calculates the MCP for each hour and sends these to the retailers. After that, the retailers price the electricity for their customers. The problem is that the demand is price-dependent, meaning that the customers may react to the retail price with a new demand which is different from the one predicted by the ISO. In consequence, this leads to a mismatch between the predicted demand from the ISO and the demand resulting from the retailers pricing. So, the question of how to support the ISO in order to effectively balance the demand and supply by taking into account different objectives of the generators, retailers and 82

83 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.3: An example of step function customers is the main challenge that we face. In order to solve this challenge, a simulation tool has been designed for the electricity market. The whole process of our simulation tool, which is shown in figure 3.2, runs as follows: Firstly, the ISO estimate all customers hourly demand for the next day and send it to all generators; these estimated data are called the vector of expect demand. Then each generator submits the hourly bid curve to the ISO. For each hour, the ISO aggregates all bid curves to a supply curve. The hourly expected demand and hourly supply curve are then plotted against one another, and the intersection point is the MCP of this hour. All the bids to the left of the intersection point are accepted and dispatched. According to the results of the dispatch, each generator gets to know its scheduled generation for every hour of the next day. After receiving the hourly MCP, each retailer sets prices for its customers to maximize its own profit according to the customers demand model. The customers demand model is built by the retailer which can model customers consumption behaviours. The customers new expected demand vector under the retail price is then estimated by the retailer and sent to the ISO. After that, 83

84 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE the ISO compares the new expected demand with the supply (old expected demand). If the supply and demand of each hour is not balanced, then they start again with a new loop: the new expected demand vector has to send to all generators again Otherwise the process will be ended if the difference between the old expected and new expected demand is converged to near zero or pre-set threshold value. Based on the last bidding result, the ISO will schedule the production scheme for the next day. As state in Chapter 1, the penetration of renewable resources in the wholesale market and the DR programs in the retail market cause the demand and supply to become more unpredictable. The existing market mechanisms have limitations that cannot be implemented in the integrated wholesale and retail electricity market directly. So, a simulation tool which is based on the existing mechanisms of the wholesale and retail electricity markets is proposed and developed in order to overcome this weakness. It enables the ISO to find the best balance between the supply and demand by taking into account the different objectives of the generators, retailers and customers. 3.3 The Detail of Designed Simulation Tool In this section, we will introduce the detail of working process (Figure 3.2) of our designed simulation tool. Firstly, the bidding process of the generators and the method of how the ISO calculates the MCP vector are present in subsection Then we describe the retailer s optimal pricing model in subsection Generation Side At the first stage of our proposed simulation tool, the ISO estimates all customers hourly demand for the next day and sends it to all generators, as 84

85 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE shown in Figure 3.2. These estimated data are called the vector of expected demand. Then in the second stage, each generator submits its bid curve a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,2,, M i ), sorted by the bid price in increasing order, to the ISO for each hour. The m i is the number of segments in this generator i s bid curve. Then at the third stage of Figure 3.2, the ISO aggregates all bidding curves to a supply curve of hour h. Based on the supply curve and the response demand of all customers for each hour, the ISO can obtain an MCP vector: MCP=(MCP 1, MCP h, MCP 24 ). In order to reach the requirement of economic dispatch, the production cost on the generation side should be minimized and the demand from the retailer should be fulfilled. In the following section, we will show the method of obtaining the MCP h in the third stage in Figure 3.2. Firstly, we use Algorithm 1 to obtain the aggregate bidding curve and the production scheme of hour h. In Algorithm 1, i is the number of generators; m i is the number of segments in generator i s bidding curves; s h is the number of total segments in all generators bidding curves, where s h = i m i ; S h,k is the k-th segment of the proposed piece function (aggregated bidding curve) in hour h; MCP h,k is the corresponding unit price of S h,k ; D h,k is the corresponding quantity of S h,k ; D h is the expected demand in hour h. Algorithm 1 Calculate the MCP of hour h Input: Each generator s bid curve of hour h: generator i s bid curve can be represented as a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,2,, m i ) sorted by price in increasing order by the bid price; estimated all customers demand D h. Output: Market clearing price of hour h: MCP h ; the production scheme of hour h. 1: For each generator i, get the unit price p i,mi and the corresponding quantity bids q i,mi ( q i,mi = q i,mi q i,mi ) of each segment in the generator i s bidding curve of hour h. 85

86 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE 2: Let p 1 p 2 p Sh be the ordered unit prices p i,mi of all segments in generators bidding curves and q 1, q 2,, q Sh be the corresponding quantity bids q i,mi, where s h is the number of total segments in all generators bidding curves. 3: Assigns the value of p 1 and q 1 to MCP h,1 and D h,1 respectively, assigns 0 and q 1 to the lower and upper bound of S h,1 respectively. 4: For k=2 to s h do 5: Assigns the value of p k and q k to MCP h,k and D h,k respectively; assign D h,k 1 + q k to the upper bound of S h,k and assign the value of upper bound of D h,k 1 to the lower bound of S h,k. 6: End for 7: A new set of segments (MCP h,k, (D h,k, D h,k ], k = 1,2,, s h ) is obtained, which can be plotted as a step function in the coordinate system. This new piecewise constant function is the aggregated bidding curve of all generators in hour h. 8: The aggregate bidding curve and the expected demand of hour h: D h are then plotted against one another. The intersection point in the curve: MCP h, is the market clearing price of hour h. All the segments to the left of the intersection point are accepted and dispatched. The dispatched generators along with the corresponded allocation be the scheduled production scheme of hour h. In Algorithm 1, the ISO sorts all bids segments of all the generators bidding curves in increasing order by unit price and then start from the smaller one in order to put them to the aggregated bidding curve. The segments in the aggregated bidding curve are also ordered in the increasing order, which means the amount of electricity which has the lowest unit price is always dispatched first. Therefore, through using Algorithm 1, the ISO can obtain the production scheme for the next day under the minimum production cost. According to this scheme, the ISO dispatches the generators. Figure 3.4 shows an example to illustrate how Algorithm 1 works. From the figure, we can see that generator 1 s bid segments are: (0.5, (0, 100]; 2, (100, 150]), generator 2 s bid segments are: (1, (0, 150]; 1.5, (100, 150]). Then the Algorithm 1 aggregates these two sets of 86

87 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.4: An example of getting aggregated bidding curve segments as a new one: (0.5, (0, 100]; 1, (100, 250]; 1.5, (250, 450]; 2, (450, 600]). This new set of segments is the aggregated bidding curve which is also plotted as a step function in Figure 3.4. Here s h is equal to 4. If the expected demand in hour h: D h equals to 450. Then the intersection point of the aggregate bidding curve and the expected demand of hour h in the coordinate system is (450, 1.5). All the segments to the left of the intersection point are accepted and dispatched. The Algorithm 1 will output the market clearing price MCP h : 1.5, and the scheduled production scheme of hour h: (generator 1: 100; generator 2: 350). As the market administrator, the ISO has the responsibility of keeping the electricity market stable. When all factors in the market stay the same, the ISO should achieve the following objects: 1) All customers bills (all retailers revenue) will be no more than yesterday; 2) Each retailer s profit is kept at a certain level; 3) All generators profit is kept at a certain level. Every participant in the market wants to maximize its own profit. A conflict situation is obviously induced. It is hard, or impossible, for all participants in the 87

88 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE market to maximize their profit at the same time. For example, if the retailer sets flat and high prices in all periods in order to maximize its profit, there is no time period that customers can shift their demands to, thus the benefit to customers cannot be ensured. That is why we set three objectives for the ISO to coordinate the conflict goals among all participants. The first two objectives can be achieved by setting the constraints in the retailer s pricing model. The details of this part will be illustrated in section Here we talk about how we achieve the third objective in the market. If all generators profits are smaller than the pre-negotiated profit level, the ISO will adjust the hourly MCP of the next day obtained by Algorithm 1. The MCP is slightly increased until all the generators profit for the next day is no less than the pre-negotiated profit level p g. The object of the ISO is to increase the hourly MCP as little as possible. Therefore, this problem can be formulated as the following optimization problem: Min h MCP h for h = 1, 24. (3.1) Subject to (MCP h D h C h ) h p g MCP h MCP h for h = 1, H; Where MCP h is the original market clearing price of hour h obtained by Algorithm 1; MCP h is the resulted market clearing price of hour h which will be sent to all retailers. All generators will be paid based on this value and their dispatched generations in hour h; C h is the total generation cost of hour h which can be calculated based on aggregated bidding curve obtained by Algorithm 1. This linear programming problem (3.1) can be solved by the OPTI toolbox in MATLAB [72] [73]. After ensuring the profit of all generators, the ISO will send the MCP vector to all retailers. Now we have completed the illustration of how the ISO determines the MCP vector at the third stage in Figure

89 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Retail Side At the fourth stage in Figure 3.2, each retailer sets optimal prices for its customers in order to maximize its own profit after receiving the MCP vector. Normally the retailers in the market are always competitive in-between. But, as we are discussing the pricing for the next day and the competition which exists within a day can be ignored. This is due to the fact that the customers switching within one day is very limited, and thus can be ignored. Therefore retailers are pricing independently and there is no cross-impact between the prices of different retailers in this scenario, meaning that each retailer has its own customers. That is, we do not consider the competition among retailers with 24 hours in our simulation tool. If needed, a retailer can set price differential bounds with its competitors to take the competitive impact of prices into consideration. Throughout the thesis, we assume that the price and demand information in the retail market for the last N days is available. By using Ma s method in [13], we get the customers estimated reaction function for each hour from the demand modelling by learning the historical data. As we discussed in Section 2.2.1, the demand of the customer at each hour is not only affected by the price of the current hour but is also affected by the price of other hours. For example, if the price of electricity at hour i is much higher than the other hours prices, the customers will react in order to shift the usage of electricity to another hour or hours. So in [13], the authors consider the utility function from all customers at each hour to describe these kinds of consumption behaviours as equation (2.7) (Section 2.2.1). There is a need to find an estimated function R h(p 1, p 2,, p H ) which is as close to the utility function as possible. This estimated function is called an estimated reaction function for all customers. The form of estimated reaction function for each hour h (1, H) can be represented as: 89

90 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE RF h (p 1, p 2,, p H ) = α h + β h,1 p β h,h p H (3.2) In the equation (3.2), β h,c is the cross-price elasticity which measures the responsiveness of the customers demand for the electricity at hour h (1,, H) to the change in price of electricity at some other hour c (1,, H), which is always greater than 0. When h equal to c, β h,h is defined as selfelasticity, which is always less than 0 [34]. The properties of the cross-price elasticity and self-elasticity have been introduced in Section Ma uses the adaptive least square method to update parameters in the equation (3.2) for each hour h (1,, H) when any new data are available. After estimating the elasticity function for each hour, the next step is to solve the retailer s pricing problem in order to achieve the maximum profit under certain constraints. In this part, the retailer s optimal pricing model will be discussed. For each hour h (1,, H), we define the minimum and maximum price that the retailer can offer to its customers. p h min p h p h max (3.3) Where p min h and p max h are usually set based on several factors, such as: the cost of electricity (wholesale price), the customers average income and affordability, and the constraints imposed by government policy. For instance, in order to avoid a loss, the retail price of electricity should be higher than the wholesale price. Besides, the upper boundary of the retail price of electricity is often from the competitors, but in our market model, we had not considered competition between retailers, meaning that customers do not have the option to choose the retailer. Moreover electricity is a life necessity, which means that, no matter how much the price of electricity changes, the total demand of electricity obviously does not change. Therefore a constraint must be set on the 90

91 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE price which is possible by means of government policy or by the customers acceptability. As with the constraints (3.3), a constraint on the total revenue should exist due to the first objective of the ISO and the pressure from the customers acceptability. Thus, we have Constraint (3.4): p h RF h (p 1, p 2,, p H ) h H C N (3.4) Where C N is the customers bill of yesterday (day N). Finally, the pricing optimization problem for retailer j can be expressed as follows: Max PRJ N+1 j = ((p j h MCP h ) RF h (p j 1, p j 2,, p j h H H )) (3.5) Subject to p j h RF h (p j 1, p j 2,, p j j h H H ) C N p min h p h p max h, h (1,, H) Where C N j is the total bill constraint of retailer j s customers; PRJ j N+1 is the total profit of retailer j within the given period (1,, H), which is usually set as 24 hours. Problem (3.5) is a quadratic programming problem which can be solved by the SCIP solver [74]. Retailer j can price the electricity of day N+1 for its customers under wholesale price MCP h in order to maximize its profit by solving the optimization problem (3.5). Under the constraint (3.4), the ISO ensures that the customers bill is no more than yesterday s. But the retailer also needs to ensure its profit is maintained at a certain level, as we showed in equation (3.6), where PRJ j N is retailer j s profit for day N. In some cases the resulting profit from problem (3.5) may not satisfy equation (3.6), for example the increase of the wholesale electricity price would cause the decrease of the retailer s profit. In such a scenario, the retailer should adjust the retail price in order to ensure that its profit satisfies the equation (3.6). 91

92 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.5: The process of retailer s pricing model PRJ N j δ PRJ N+1 j PRJ N j + δ (3.6) For the retailer j s re-pricing process, the constraint (3.4) is modified as constraint (3.7) at first, k is equal to 1 at this time. After that, the retailer solves the modified optimization problem (3.5) again. If the profit of retailer j still does not satisfy the equation (3.5), then add 1 to k and start a new loop. p j h RF h (p j 1, p j 2,, p j H ) C j N + (3.7) = 0 + (k 1)ε Where k is the number of iteration times; is the increased revenue restriction. The re-pricing process can be formulated as problem (3.8): Min (3.8) Subject to PRJ N j δ J(p j ) PRJ N j + δ 92

93 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE In order to easily understand the pricing model in the retail market, the retailer s whole pricing process is shown in Figure 3.5. Now we completed the illustration of how the retailer sets optimal prices for its customers in order to maximize its own profit after receiving the MCP vector at the fourth stage in Figure 3.2. In next stage, all customers new expected demand vector under the retail price is then estimated by the retailer and sent to the ISO. After that, the ISO compares the new expected demand with the supply (old expected demand). If the supply and demand of each hour is not balanced, then they start again with a new loop: the new expected demand vector has to send to all generators again Otherwise the process will be ended if the difference between the old expected and new expected demand is converged to near zero. Based on the last bidding result, the ISO will schedule the production scheme for the next day which is the end of the simulation tool. 3.4 Balance Mechanism In the last section, we introduced the designed simulation tool and related algorithms for the electricity market. This simulation tool relies on a stable generation cost. However, the price of generation is continually fluctuating and the generation cost would be certainly affected by this fluctuation. In our proposed simulation tool, the profits of the retailers and generators are controlled within a certain range. Within the same parameters in the market, the ISO always wants the bill of retailer j s customers to be no higher than the day before. But this cannot be ensured when the production cost are increased in the wholesale market. This is because the customers play a far more limited role than do the generators and retailers in the current electricity market. They merely accept the retail price passively regardless of any ensured welfare. When 93

94 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE the production cost is increased, the market clearing price and the retail price will be increased by the wholesale market and the retailer in order to ensure their profits. In such a scenario, the increased generation cost and the change of all generators and retailers profit will transfer in its entirety to the customers bill. As a non-discriminatory third-party, as well as being the administrator in the market, the ISO desires all groups to take responsibility for this increased generation cost. Therefore, we need to design a balance mechanism for the ISO. The balance mechanism is based on the result of the simulation tool which was introduced in Section 3.3 and Figure 3.2. For the customers part, the estimated bill of all retailer j s customers in the running result of our simulation tool is formulated as equation (3.9). j C N+1 = p h j RF h (p 1 j, p 2 j,, p H j ) h H (3.9) When the generation cost is increased, we use C b to present the increased part all the customers bills. j C b = j C N+1 j j C N+1 (3.10) j In equation (3.10), C N+1 is the estimated bill for all retailer j s customers when the generation cost is increased. We Use D b to describe the total demand of all customers. In the balance mechanism, every group in the market should afford C b proportionally when the generation cost is increased. Here we use a, b and c to present the proportion of the increased customers bill that the generators, the retailers and the customers should be afforded respectively, where the sum of a, b and c is 1 (the value of a, b and c should be negotiated by the ISO, the retailers and the generators). In order to achieve that goal, the balance mechanism will adjust the market clear price MCP h and the retailer j s optimal retail price P h j. 94

95 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Both MCP h and P h j are the running result of the simulation tool when the generation cost is increased. We use MCP h f and P h j f to describe the balanced MCP in the wholesale market and retailer j s balanced retail price in hour h respectively. MCP h f = MCP h C b a D b, h (1,, H) (3.11) P h j f = P h j C b b D b C b a D b, h (1,, H) (3.12) Based on the results of equations (3.11) and (3.12), the retailer and the customer buy the electricity. We use D j,h to describe the total estimated demand of retailer j s customers in hour h. Then the total bill of all the customers is shown as (3.13). P j f j h h D j,h j = j C N+1 C b (a + b), h (1,, H) (3.13) From the equation (3.13), we can see that, after using the result of the balance mechanism ( MCP h f and P h j f ), the customers only need to afford part of the C b when the generation cost is increased. This means that The ISO s goal of every group in the market affording C b proportionally is achieved. With this result, the customers do not need to passively take all the responsibility for the increased production cost. This is an advanced property of our proposed simulation tool. Therefore in this paper, customers will positively participate in the electricity market with the help of this balance mechanism. 95

96 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Table 3.1: Retailers parameters Parameters Retailer 1 Retailer 2 Revenue restriction (cents) Maximum profit (cents) Maximum retail price (cents/kw) Table 3.2: Ten generators production information No. Segment output(kw) Marginal cost(cents/kw) Numerical results In this section, we will illustrate some simulation results of the proposed simulation tool. The parameters in the simulation are showed at the beginning. All the data used here are derived from the PJM [75]. We will then analyse the simulation results. 96

97 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Table 3.3: The input demand of each hour Hour R 1 R 2 Hour R 1 R 2 Hour R 1 R Figure 3.6: The comparison between the test demand and result demand Parameter setting Based on the market model shown in Figure 3.1 and 3.2, we set ten generators and two retailers in the market. Each retailer has its own customers, which means there is no competition between retailers. The parameters of maximum retail price, revenue and profit restrictions are set differently for these two retailers. In the equation (3.5), the δ is set as 5000 cents, so their profit will 97

98 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.7: The demand-difference in each hour Figure 3.8: Retailer 1 s brought power and sold power N be no more or less than PRJ j about 5000 cents. The reason why we set the different parameters is to show the ISO s controllable ability in the market. Every group in the market is monitored by the ISO. The difference between the two retailers is shown in Table 3.1. For the generation part, we set ten generators which have different scales of output. The minimum total profit of ten generators set as e+07 cents. The ten generators generation information 98

99 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE is shown in Table 3.2. The balance threshold value (the difference between the supply and demand) of the simulation tool is set as 2% of the input demand. Simulation results This part shows the experimental results of our designed simulation. Since the requirements of minimum generation cost and all generators total profit in the wholesale part are set as the constraints in our model, which can be ensured in the experimental result. Therefore we mainly analyse the results of the retail market in this section. The test demand in the simulation is shown in Table 3.3. Each retailer has its own demand. After 9 loops, the simulation tool stops. The result of retailer 1 s and 2 s profits are e+06 cents and e+06 cents respectively; the corresponding revenues are e+07 and e+07 cents. All of these values are satisfied the requirements in Table 3.1, which means that the customers bill no more than yesterday, the retailers and generators profits are kept in certain level. These results show that our simulation tool could support the ISO to achieve the objectives which were shown in subsection Figure 3.6 compares the original demand (Table 3.1) and the final estimated demand of all customers. The peak-and-average ratio (which is defined in equation 2.17, Section 2.2.1) is decreased from to through running our program. As mentioned before, a flattened demand profile could decrease the total customers bill and generation cost, and increase the reliability of the grid. Figure 3.6 proves that our simulation tool could achieve these objectives. In Figure 3.2, we can see that the difference between the new estimated customers hourly demand and the estimated customers hourly demand in the previous loop (this demand is also the supply in this loop) is the main factor for determining whether the program should continue to run or not. 99

100 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE Figure 3.9: The MCP (red) and retailers sales price (1: blue, 2 green) Table 3.4: The simulation result of balance mechanism Simulation result After balancing Retailer1 s revenue Retailer2 s revenue Customers bill In this section, this difference is called the demand-difference. Figure 3.7 shows the value of the demand-difference of two retailers in 24 hours. The figure shows that the demand-difference converges to 100 kw in most hours. Even for the biggest demand-difference of hour 5, kw is a very small number compared to the demand of retailer 1 s customers in that hour. This is also proved in Figure 3.8. Figure 3.8 compares the electricity of retailer 1 brought from the wholesale market and the estimated customers response demand. From this figure, we can see that the demand-difference in each hour is a very small number compared to the usage of electricity in that hour. All of these values mean that our simulation tool is able to support the ISO to dispatch the supply in order to fulfil the demand. The vector of the market clearing price and the retailer s price in the simulation result are shown in Figure 3.9. The figure shows that retailers increase 100

101 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE the electricity price in hours 14, 16, 17, 18, 20 and 21. All of these hours are within the peak-period. This means that our optimal pricing model could change price sensitive customers behaviour and help them to save money by switching to cheaper times. Furthermore, our pricing model could also enable retailers to improve their profit without increasing the revenue. Now we present the simulation result of our balance mechanism. We still use Table 3.3 as our input demand for the simulation tool. For each generator s production detail, we modify Table 3.2 in which we add 1 to the price of each segment. Then we run our simulation again. The result revenues of retailer 1 and 2 are e+07 and e+07 cents respectively. The total customers bill is e+07 cents. In order to ensure retailers and generators profit, the total customers bills are increased in comparison to before. This means that all the increased production costs are transferred to the customers bill. Therefore, it is clear that we should run the balance mechanism. The increased customers bill is approximately cents. Here we set the parameter a, b and c equal to 0.25, 0.25 and 0.5 respectively in our balance mechanism. This means that customers will only pay half of the increasing fees caused by the increase in production cost. The simulation results are shown in Table 3.4. As we previously summarised, the retailers and the generators could afford the extra fee by decreasing the MCP and the retail price. This proves that our balance mechanism can support the ISO in order to coordinate the conflicting goals of different groups in the market. 3.6 Summary The main work presented in this chapter has been to develop an integrating framework and a method of pricing optimization which integrates and enhances 101

102 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE the existing demand modelling, the retailers pricing optimization and the generators cost minimization methods. Based on this integrating framework and these methods, a computing simulation tool for the ISO has been developed in order to support the ISO and to find the best acceptable and negotiable scheme for coordinating and balancing conflicts goals among the generators, retailers and customers. This ensures the fair distribution of cost and benefit among all groups in the market. The simulation results presented in Section 3.5 also show that the simulation tool improves the production efficiency of the day-ahead market as the gap between retailers bought and sold electricity tends to be zero. It has been observed that the balanced mechanism shown in Section 3.4 is useful for managing the risk of increased production costs. Under the administration of the ISO, retailers and generators afford part of the customers increased bill by reducing the MCP and the retail price. However, our simulation tool has the following limitations: Firstly, the demand from the retail market is not predictable due to the effect of the customers demand response. In order to find the best balance between supply and demand, the ISO runs many loops from Stage 2 to Stage 6 when implementing our simulation tool (Figure 3.2). In each loop, the generators need to submit their bids. The ISO also needs to schedule a generation scheme and the retailers need to set a price for their customers. When deploying our simulation tool in the real market, it requires a lot of information exchange among all groups in the market. This means that our simulation tool is not sufficiently efficient. Secondly, the ISO does not know the exact details of each retailer s pricing method. It is hard, or impossible, for the ISO to set the profit constraint for each retailer. Thirdly, the simulation tool relies on a stable generation cost. When the generation cost is fluctuating, the balance mechanism is needed to handle the conflicting goals among all groups in the market. A constitution is 102

103 CHAPTER 3. INTEGRATED DEMAND AND SUPPLY SIDE drawn up in the balanced mechanism. Is the constitution fair enough to enforce a genuine market s participants so that their overall objectives may be maximally achieved? These limitations are improved and overcome in following chapters. 103

104 CHAPTER 4. A NEW MARKET MECHANISM Chapter 4. A New Market Mechanism for Integrated Demand and Supply Sides Management in Electricity Market 4.1 Introduction In Chapter 3, we have developed an integrating framework and a method of pricing optimization which integrates and enhances the existing demand modelling, the retailers pricing optimization and the generators cost minimization methods. Based on this integrating framework and methods, a computing simulation tool for the ISO was developed in order to support the ISO in finding the best acceptable and negotiable scheme for coordinating and balancing conflict goals among the generators, retailers and customers. However, our simulation tool has the following drawbacks: Firstly, the demand from the retail market is not predictable due to the effect of the customers demand response. In order to find the best balance between supply and demand, the ISO runs many loops from Stage 2 to Stage 6 when implementing our simulation tool (Figure 3.2). In each loop, the generators need to submit their bids. The ISO also needs to schedule a generation scheme and the retailers need to set a price for their customers. This means that our simulation tool is not sufficiently efficient. Secondly, the ISO does not know the exact detail of each retailer s pricing method. It is hard, or impossible, for the ISO to set the profit constraint for each retailer. Thirdly, the simulation tool relies on a stable generation cost. When the generation cost is fluctuating, the balance mechanism is needed to handle the 104

105 CHAPTER 4. A NEW MARKET MECHANISM conflicting goals among all groups in the market. A constitution is drawn up in the balanced mechanism. Is the constitution fair enough to enforce a genuine market s participants so that their overall objectives may be maximally achieved? Furthermore, as stated in Section 1.2, the penetration of the renewable resources in the wholesale market and the DR programs in the retail market cause the demand and supply to become more unpredictable. In such a scenario, the existing mechanisms in the wholesale market and the retail market no longer work when integrating the demand and supply side into one framework. In order to overcome the weaknesses of our simulation tool and to solve the aforementioned research problems in Section 1.2, we have designed a new market mechanism for the integrated demand and supply sides management in the electricity market. This mechanism can efficiently find the match equilibrium between supply and demand. This Chapter is adapted from our published paper: [76]. The rest of this Chapter is organized as follows: Firstly, the problem statement is given in Section 4.2. Secondly, the proposed mechanism and related algorithms are presented in Section 4.3. Thirdly, numerical results are given in Section 4.4 showing that the new mechanism improves the efficiency of our simulation tool. This chapter is summarized in section Problem Statement As in Chapter 3, in this Chapter we also consider a smart power system with several generators, retailers and many customers as part of the general electricity market. This is shown in Figure 3.1 (Section 3.2). In our proposed new mechanism, the ISO firstly declares an interval demand to the wholesale market for each hour. The interval demand is more robust than 105

106 CHAPTER 4. A NEW MARKET MECHANISM a single demand figure, thus enabling the ISO to consider the demand variates under different prices. In contrast to the simulation tool and the existing mechanism, in which the generator only bids for a single figure of demand, each generator bids for an interval demand under our new mechanism. They submit its bid curve a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,2,, m i ) sorted by price in increasing order by the bid price to the ISO for each hour. The s is the number of segments in this generator s bid curve. Then the ISO calculates the aggregated bids curve (step function) for each hour after receiving the bid curve from each generator. This hourly-aggregated bids curve is the calculated hourly MCP. The most important feature of this MCP is a function which is dependent on demand. This is different from the current work in which their calculated MCP is a single number and not dependent on the demand (as the demand is just a one number in that case). Each MCP function consist a set of s segments (MCP h s h, (D h,sh s segments in this MCP function, D h,sh s, D h,sh ], s h = 1,2,, s h ), where s h is the number of s and D h,sh are the lower and upper bound of corresponding supply demand for MCP h s h. After that, the ISO sends hourly MCP functions to all retailers. So there are two main problems on the supply side: Firstly, for the generator, how to select a bid curve under the declared interval demand from the ISO in order to maximize its profit? This problem will be solved in Chapter 6. Secondly, given the bid curves from all generators, how to generate a MCP function for each hour. In order to solve this problem, a method is presented in Section 4.3. For the retail market, we consider that each electricity retailer has its own customers in the market as shown in Figure 3.1 and the day-ahead real-time pricing is applied. The customers consume the next day s electricity depending on the announced price, and their preference is to maximize their utility (such as 106

107 CHAPTER 4. A NEW MARKET MECHANISM minimizing their bills, or maximising the convenience, or combining both). In our new mechanism, the ISO sends hourly MCP functions to all retailers. Since each MCP function is a step function which consist of a set of segments, each segment is a price with a lower and upper bound of corresponding supply quantity. Then the retailer can obtain a set of MCP vectors for 24 hours: MCP vector = (MCP 1,, MCP h,, MCP H ), in which for any hour h (1,, H), MCP h (MCP 1 h, MCP 2 s h,, MCP h h ). By using the estimated customers reaction function learned from historical data and basing it on each MCP vector for the next day, the retailer determines the 24 hour prices for the next day in order to maximize its profit. Each MCP vector is then matched with a customers response vector. Then customers response demand function which is dependent on the MCP vector can be obtained by the retailer and sent to the ISO. This respond demand is a function of the MCP vector. This is different from current mechanisms which only calculate a single vector of 24 figures for the response demand. The main problem for the demand side is the retailer s pricing problem. That is, how to price optimally in order to maximize the profit under each MCP vector for the retailer. This problem is solved in Section 4.3. After receiving the customers response demand function from each retailer, the ISO aggregates these functions to a demand function. Based on obtained MCP functions and demand function, the next step of the proposed mechanism consists of the ISO finding the match equilibrium between demand and supply. The balance points in each hour of the next day are the proposed result of this mechanism. Due to the effect of demand response, the customers always react the retail price with a response demand which is the demand they buy from their retailer. The balance point in hour i means that the customers response demand to the retail price (the amount of electricity that the retailer sells to its customers) and the supply from the wholesale market in this hour (the amount of electricity 107

108 CHAPTER 4. A NEW MARKET MECHANISM Figure 4.1: An example of the balance point that the retailer buys from the wholesale market) are located in the same segment in the MCP function. Figure 4.1 shows an example of the balance point in hour h. The customers response demand and supplied demand are located in the same segment of MCP function, which mean that they have the same unit cost (in the Figure 4.1: MCP 3 h ) for the retailer. Due to the unchanged unit cost (equal to MCP 3 h ), the retailer s optimal pricing model is successful after integrating the demand side with the supply side. This differs from the current research in demand response. If the unit price (equals to MCP) is changed, the optimal prices proposed by the retailer s pricing optimization model before, no longer optimal and the customers demand also becomes different. The generation side will produce more or less electricity in comparison with the customers demand, because this quantity of production is the ISO s required production. The ISO s aim of balancing supply and demand will also fail. That is why finding the match equilibrium is so important for the ISO when integrating the demand and supply side into one framework. Besides, the customers demand in hour i will not only respond to the price of this hour, but it will also respond to the prices of other hours, making integration more difficult and 108

109 CHAPTER 4. A NEW MARKET MECHANISM Figure 4.2: The running process of the new mechanism complicated. So, the challenge of finding a proper integrating algorithm for the ISO to obtain the match equilibrium in each hour of the next day is the main problem that we have solved in this chapter. The working process of the proposed mechanism is presented as follows: firstly, the ISO declares an interval demand to the wholesale market for each hour. Then, based on the informed hourly interval demand, each generator submits its bid curve to the ISO for each hour. A bid curve or supply function of a generator is a set of bid segments sorted by price in increasing order by the bid price, which can be represented as a piecewise constant curve. After receiving the bid curve of each hour from each generator, the ISO generates the aggregated MCP curve (MCP function) for each hour and sends these to the retailers. Based on hourly MCP functions, the retailer can get a set of MCP vectors for 24 hours. Then each retailer optimizes its retail price under each MCP vector of the next 24 hours. Each MCP vector is then matched with a customers response vector. Then the customers response demand function, which is dependent on the MCP vector, can be obtained by the retailer and sends it to the 109

110 CHAPTER 4. A NEW MARKET MECHANISM Table 4.1: The comparison between new mechanism and the current mechanism Player Current mechanism Our new mechanism ISO Predicts a single demand for each hour: D Generator Submit quantity offer with the price for each hour : (Q,P) ISO Hourly market clear price: MCP; 24 hours: MCP vector Retailer Unit Cost=MCP, retailers price for customers while max their profit under this MCP vector. Declares an Interval demand for each hour: [D L, D u ] Submit piecewise offer with the price for each hour: [Q(D), P(D)] Hourly supply curve (Demand based piecewise MCP function): MCP=f(D) Unit cost=piecewise MCP function. Retailers price for customers under each MCP vector. Then the customers response demand function is obtained and sent to the ISO: D vector=f(mcp vector) Finally the ISO finds the hourly match equilibrium between demand and supply as the result of the proposed mechanism. No loops. ISO The estimated customers hourly demand is the new demand of each hour. Match it with the supply, if not balance loop again. ISO. The ISO then aggregates these functions to a demand function. Finally, the ISO finds the match equilibrium between demand and supply as the result of the proposed mechanism. The details of each part are illustrated in the next section. Figure 4.2 illustrates the process of our proposed mechanism, Table 4.1 compares the mechanism in the current research is with our new mechanism. 4.3 Proposed Market Model In this section, we present a detailed mathematical representation for our proposed mechanism. In subsection 4.3.1, we describe how the ISO gets the MCP function in the wholesale market. Then we describe how the retailer sets prices for its customers under a demand based MCP function while maximizing the profit. This method is described in subsection Finally in subsection a 110

111 CHAPTER 4. A NEW MARKET MECHANISM genetic algorithm-based approach which is used to find the hourly match equilibrium between the supply and the demand is provided Calculating the MCP function In the proposed new mechanism, the first stage involves the ISO declaring a interval demand to the wholesale market for each hour as we showed in Figure 4.2. Then, based on the informed hourly interval demand, each generator submits its bid curve (a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,2,, m i ) which is sorted by price in increasing order by the bid price) to the ISO for each hour. The m i is the number of segments in this generator s bid curve. At the third stage of our mechanism, the ISO uses Algorithm 2 (which is shown below) to aggregate all the bidding curves to a supply curve of hour h and, based on this bidding curve, the ISO obtains a MCP function in hour h. In order to reach the requirement of economic dispatch, the production cost on the generation side should be minimized and the demand from the retailer should be fulfilled. Therefore the MCP functions should minimize the production costs under any amount of demand. Algorithm 2 shows how we get the MCP function for hour h. In Algorithm 2, i is the number of generators; s h is the number of total segments in all generators bidding curves of this hour, where s h = i m i ; s h,k is the k-th segment of the proposed step function in hour h; MCP h k is the corresponding unit price of s h,k ; D h,k is the corresponding supply quantity of s h,k. Algorithm 2 Calculate the MCP function of hour h Input: Each generator s bid curve of hour h: generator i s bid curve can be represented as a set of bid segments ( p i,mi, (q i,mi, q i,mi ], m i = 1,2,, m i ) sorted by price in increasing order by the bid price; Output: MCP function of hour h; 1: For each generator i, get the unit price p i,mi and the corresponding quantity bids q i,mi ( q i,mi = q i,mi q i,mi ) of each segment in the 111

112 CHAPTER 4. A NEW MARKET MECHANISM generator i s bidding curve of hour h. 2: Let p 1 p 2 p Sh be the ordered unit prices p i,mi of all segments in generators bidding curves and q 1, q 2,, q Sh be the corresponding quantity bids q i,mi, where s h is the number of total segments in all generators bidding curves. 3: Assigns the value of p 1 and q 1 to MCP h,1 and D h,1 respectively, assigns 0 and q 1 to the lower and upper bound of s h,1 respectively. 4: For k=2 to S h do 5: Assigns the value of p k and q k to MCP h,k and D h,k respectively; assign D h,k 1 + q k to the upper bound of s h,k and assign the value of upper bound of D h,k 1 to the lower bound of s h,k. 6: End for 7: A new set of segments (MCP h,k, (D h,k, D h,k ], k = 1,2,, s h ) is obtained, which can be plotted as a step function in the coordinate system. This new step function is the MCP function in hour h (equation 4.1). Through using Algorithm 2, the ISO can obtain the MCP function (4.1) for each hour and sends these functions to all retailers. MCP 1 h ; D h (D h,1, D h,1 ] MCP h = { ; k = 1,2,, s h (4.1) MCP k h ; D h (D h,k, D h,k ] Figure 4.2 shows an example of the resulting MCP function from Algorithm 2. In the Algorithm 2, the ISO sorts all bids segments of all generators bidding curves in the increasing order by the unit price and then starts from the smaller one in order to put them into the aggregated bidding curve. The segments in the aggregated bidding curve are also ordered in increasing order, which means that the amount of electricity which has the lowest unit price is always dispatched first. Therefore we can say that the MCP functions minimize the production cost under any amount of demand. Then the ISO transfers the obtained MCP functions to the format of Table 4.2. Table 4.2 is the MCP table with the corresponding supply segments under all MCP functions. For each hour h, we use 112

113 CHAPTER 4. A NEW MARKET MECHANISM Table 4.2: The MCP table and corresponding supply segments hour 1 hour h MCP supply segments MCP supply segments 1 MCP 1 (D s 1 1,1, D 1,1] MCP h (D s h,1, D h,1] 2 MCP 1 (D s 2 1,2, D 1,2] MCP h (D s h,2, D h,2] MCP 1 s 1 s (D 1,s1 s, D 1,s1 ] MCP h s h s (D h,sh s, D h,sh ] Figure 4.3: An example of the MCP function s h to present the maximize number of segments in the MCP function of hour h (for any two different hours i and j in (1,, H), s i could be different to s j ) Retail Side On the retail side, we use the same optimal pricing model in Section For reasons of the customers acceptability, each retailer needs the revenue constraint, the minimum and maximal retail price constraints. The pricing optimization problem for a retailer j can be expressed as follows: 113

114 CHAPTER 4. A NEW MARKET MECHANISM max PR j = h ((p j h MCP h ) RF h (p j 1, p j 2,, p j H )) (4.2) Subject to p j h RF h (p j 1, p j 2,, p j h H H ) RC j p min h p h p max h, h (1, H) Where RC j is the revenue constraint of the retailer j, which is also the total bill constraint of all retailer j s customers; PR j is the total profit of retailer j within the given period H, which is usually set as 24 hours. Given an MCP vector, the retailer j can find the best prices for maximizing its profit by solving the optimization problem (4.2). As this is a quadratic programming problem, it can be solved with the SCIP solver from MATLAB OPTI TOOLBOX. In our mechanism, the retailer receives the hourly MCP function from the ISO. This means that the MCP h in problem (4.2) is not a number but a function like showed as equation (4.1). So the solution of problem (4.2) will be dependent on each MCP h of MCP function in hour h, which is a different Quadratic programming problem from before. The following subsection will address how we deal with this problem in our mechanism. In the fourth stage of our mechanism, based on hourly MCP functions, the retailer obtains a set of MCP vectors for 24 hours. All the combinations of MCP vectors will be included in this set. Then each retailer optimizes its retail price under each MCP vector for the next 24 hours. Each MCP vector is then matched with a customers response vector. Then the customers response demand function which is dependent on the MCP vector, can be obtained by the retailer and sent to the ISO: Demand vector = f(mcp vector). After that, the ISO aggregates these functions to a demand function. Finally the ISO finds the match equilibrium between demand and supply as a result of the proposed mechanism (The stage 6 in Figure 4.1). The details of how the ISO finds the match equilibrium between demand and supply are illustrated in the next sub-section. 114

115 CHAPTER 4. A NEW MARKET MECHANISM Integrating Supply Side to Demand Model The challenge of how to get the MCP function in the ISO side and how to find the optimal prices in order to maximize the profit for the retailer in the demand side have been discussed and solved in subsection and subsection respectively. In this subsection, we will illustrate how to support the ISO to finds the match equilibrium between demand and supply. In the existing wholesale pricing mechanism, there is always a missing link between the market clearing price (MCP) and the demand. The reason for this phenomenon has been discussed in Section 2.3 which is also listed here. Normally the ISO forecasts the customers demand for the next day and gives this information to the generators. Then the ISO produces an MCP for each hour to the retailers after receiving the bids from the generators. After that, the retailers price the electricity for their customers. The problem is that the demand is price-dependent; this means that customers may react to the retail price with a new demand which is different from that predicted by the ISO. Furthermore, this change in demand also leads to the change of the MCP, this being the unit cost for the retailer. Therefore we state that there is a missing link and mismatch between the MCP and the demand. This weakness means that the current wholesale mechanism cannot handle the unpredictable demand under dynamic retail pricing. To address this issue, we propose a mechanism which solves the problems of how to integrate two markets into one framework and find the hourly match equilibriums between supply and demand. In the sixth stage of our mechanism, the ISO wants to find the match equilibriums between demand and supply for each hour of the next day. Here we will define the match equilibriums at first. 115

116 CHAPTER 4. A NEW MARKET MECHANISM Figure 4.4: An example of the match equilibrium Definition 4.1: If the ISO finds Match equilibriums from all MCP h functions (equation (4.1)), a MCP vector MCP = (MCP 1,, MCP H ), which satisfies the following condition (4.3). Then it is said that the demand and supply reaches the match equilibrium under this MCP vector. D h s (MCP h ) < D h r,j (MCP 1,, MCP H ) j D h s (MCPh ) h (1,, H) (4.3) In equation (4.3), D h r,j j is the demand of all retailers in hour h under(mcp 1,, MCP H ), where D h r,j is the response demand of retailer j s customers: RF h (p 1 j, p 2 j,, p H j ) which can be calculated by solving optimization problem (4.1); D h s (MCP h ) and D h s (MCPh ) are the lower and upper bound of MCP h s corresponding supply segment in Table 4.2 and equation (4.1) respectively. Under the match equilibrium, the total customers response demand of hour h is located within the supply segment: (D h s (MCP h ), D h s (MCPh )] in Table 4.2 when the market clear price is (MCP 1,, MCP H ). Therefore, the balance of the supply and the demand avoids the shortage and surplus of electricity supply. Figure 4.4 shows an example of match equilibrium in hour h. 116

117 CHAPTER 4. A NEW MARKET MECHANISM Although the MCP functions are already known, finding the match equilibriums between demand and supply for each hour of the next day is still a complicated problem for the ISO. Because the combination of MCP vector in the next day is a very large number. For each hour h, we use s h to present the maximize number of segments in the MCP function at hour h. The domain of MCP h in the MCP function is MCP h (MCP 1 h, MCP 2 s h,, MCP h h ), h (1,, H). Then the combination of all possible MCP vectors MCP = (MCP 1,, MCP H ) in the next day is equal to s 1 s 2 s H. For example, when both of s 1, s 2,, s H are equal to 3 and H is equal to 24, the combination of all the MCP vectors for the next day is e+11. It is unrealistic, or impossible for each retailer to solve the problem (4.2) so many times. Even if retailers were able to solve Problem (4.2) for all those combinations, the ISO would cost a lot of time to connect these results with MCP functions and find the match equilibriums. This is the combination explosion challenge in our mechanism. A genetic algorithm-based approach is proposed in order to solve this complicated Quadratic programming problem. A genetic algorithm (GA) is a search heuristic which mimics the process of natural selection. This heuristic is routinely used to generate useful solutions for optimization and search problems [78] [80]. GA has been widely used for the solution of combinatorial optimization problems and is theoretically and empirically proven to provide robust search in complex spaces [111]. GA is an appropriate approach for large-scale global optimization problem. It can impose a series of genetic operations such as selection crossover and mutation on the current population and gradually evolve to the optimal solution [112]. Therefore it could deal with the combination explosion challenge and efficiently find the match equilibriums between supply and demand in our mechanism. 117

118 CHAPTER 4. A NEW MARKET MECHANISM But the question of how to transfer our problem to the form that the genetic algorithm can deal with is the new challenge that we need to solve. More specifically, a typical genetic algorithm requires a genetic representation of the solution domain and a fitness function to evaluate the solution domain. So, we need to find the solution domain of our balancing problem and transfer it to a form of genetic representation that the GA accepts. Furthermore, the question of how to define a proper fitness function which is able to efficiently evaluate a candidate solution, whether good or not, is another problem. In the following section, we describe how to solve these two problems. In genetic algorithm, a population of candidate solutions to an optimization problem has evolved towards finding better solutions. Each candidate solution has a set of properties which can be mutated and altered. In our problem, each combination of MCP vector in MCP table (Table 4.2) is a candidate solution. The candidate solution has 24 properties which means that each hour is a property; the value of MCP h can be mutated and altered. The evolution of the GA algorithm usually starts from a population of randomly generated individuals, and it is an iterative process. The population in each iteration is called a generation. In each generation, the fitness of every individual in the population is evaluated; the fitness is usually the value of the objective function in the optimization problem being solved. The more fit individuals are stochastically selected from the current population, and each individual's genome is modified (recombined and possibly randomly mutated) to form a new generation. The new generation of candidate solutions is then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population. 118

119 CHAPTER 4. A NEW MARKET MECHANISM From the above, we can deduce that a typical genetic algorithm requires a genetic representation of the solution domain and a fitness function to evaluate the solution domain [77]. As we said before, each combination of MCP vector in MCP table is a candidate solution. So the solution domain in our problem is the MCP vector of next 24 hours: MCP v = (MCP 1,, MCP H ), in which for any hour h (1,, H), MCP h (MCP h 1, MCP h 2,, MCP h s h ). The input of the GA program in our problem is the MCP for the next 24 hours. For each MCP vector, the program returns a fitness value according to fitness function which is shown as equation (4.4) and (4.5). The GA can find the best MCP vector among all combinations of MCP vector which has the best fitness value in certain populations and generations. We set the program stops when the fitness value is less than 2% of yesterday s demand or a fixed number of generations is reached. The fitness function Z is set as follows, where MCP v = (MCP 1, MCP H ). z h (MCP v ) = 0; if D h s (MCP h ) < D h r,j (MCP v ) j D h s (MCPh ) j D r,j h (MCP v ) D s h (MCP h ); if D s h (MCP h ) j D r,j h (MCP v )(4.4) { D r,j s s j h (MCP v ) D h(mcph ); if D h(mcph ) < j D r,j h (MCP v ) Z(MCP 1,, MCP H ) = z h h h = 1,, H (4.5) In equation (4.4), z h is equal to zero if the demand of all retailers in hour h satisfied the equation (4.3). When this occurs, it means that the ISO finds the match equilibrium at hour h. Otherwise, z h presents the difference between the retailers demand and the supply demand, the minimum (maximum) supply demands is the lower (upper) bound of MCP h s corresponding segment in Table 4.2. The Algorithm 3 shows the detail process of our GA-based approach. Algorithm 3 GA-based approach Input: MCP function of 24 hours; population size: ps; the max number of generations: g; a match threshold value. Output: A MCP vector (which reaches the match equilibriums); 119

120 CHAPTER 4. A NEW MARKET MECHANISM 1: Population initialization: generating a population of ps MCP vectors randomly; 2: For k= 1 to ps do; 3: The ISO announces k-th MCP vector in the population to each retailer. 4: Through solving problem (4.2), each retailer gets its customers response demand vector for 24 hours and sends it to the ISO. 5: The ISO calculates the aggregated demand vector, and uses fitness functions (4.4) and (4.5) to the get fitness value of k-th MCP vector. 6: End for 7: A new generation of MCP vectors is created by using the selection, crossover and mutation operations. 8: Steps (2-7) are repeated until the stopping condition is reached. Stopping condition: Steps (2-7) are repeated g times or the current optimal fitness value is less than a match threshold value. 9: The ISO announces the final MCP vector to the retailers. For s 1 s 2 s H MCP vectors, we use Algorithm 3 to find a MCP vector which makes the Z equal or close to 0 (this integer optimization problem can be solved by the genetic algorithm solver in MATLAB [79]). According to this MCP vector, the ISO can schedule a generation scheme and retailers can price the electricity in each hour for the next day, the match equilibrium can be reached in the electricity market. In Algorithm 3, the steps of 2-7 are looped at most g times. In each loop, the problem (4,2) is solved ps times. Therefore, the running time T of Algorithm 3 is: T t ps g (4.6) This means that the ISO requires each retailer to calculate optimization problem (4.2) no more than ps g times in the proposed mechanism, where t is the calculation time for solving the problem (4.2). Normally, given the choices of Gaussian mutation, scattered crossover, and tournament selection, a genetic algorithm s time complexity can be shown as below. O(g ps O(fitness function) (Pc O(cossover) + Pm O(mutation))) 120

121 CHAPTER 4. A NEW MARKET MECHANISM (4.7) In equation (4.7), Pc is the crossover probability and Pm is the mutation probability. Normally, Pc and Pm are constant. Transforming the population with the crossover and mutation operations usually takes O(H*ps), where H is the length of the individual in Algorithm 3 and also the number of hours in the proposed mechanism. The time complexities of tournament selection and fitness function (4.5) are O(ps) and O(H) respectively. So we have following equations. Table 4.3: Aggregated bidding curve/mcp function MCP h Segment scale D h s (MCP h ) D h s (MCPh ) Segment number Table 4.4: The fitness value of each hour Hour z h Hour z h Hour z h (MWH)

122 CHAPTER 4. A NEW MARKET MECHANISM O(tournament selection) = O(ps) (4.8) O(fitness function) = O(H) (4.9) O(mutation) = O(cossover) = O(H ps) (4.10) Based on equations (4.8)-(4.10), the time complexity of Algorithm 3 (equation (4.7)) can be simplified to O(ps + g (H + ps H + ps H)), which is on the order of O(g*ps*H). As H is a fixed number in the proposed new mechanism, saying 24, which is not a big number. Then all complexity of Figure 4.5: Running result of the Genetic algorithm Figure 4.6: Running result of GA under new MCP function 122

123 CHAPTER 4. A NEW MARKET MECHANISM Algorithm 3 in fact is dependent on the population size ps and the number of generation g. 4.4 Numerical Results In this section, the numerical results are illustrated. Firstly, we present the parameters in our experiments. All the data in the demand model derived from PJM which includes the day ahead pricing and demand information between 01/01/2011 and 30/11/2012. Table 4.3 is the aggregated bidding curve/mcp function obtained from the generators bid curves. In order to simplify the calculation, here we set that the MCP functions of each hour are the same. The constraints set in problem (4.2) are listed as follows: revenue restriction is 34,347,000 cents and max price p max h is cents/mwh. As stated before, the retailers in the market are always competitive in-between. But, as we are discussing the pricing for the next day and the competition which exists within a day can be ignored. This is due to the fact that the customers switching within one day is very limited, and thus can be ignored. Therefore retailers are pricing independently and there is no cross-impact between the prices of different retailers in this scenario, meaning that each retailer has its own customers. That is, we do not consider the competition among retailers with 24 hours in our experiment. If needed, a retailer can set price differential bounds with its competitors to take the competitive impact of prices into consideration. Therefore we only set one retailer in the market, because setting more retailers in the market would only increase the calculation time in the GA. Figure 4.5 is the running result of the Genetic Algorithm under the MCP function shown in Table 4.3. The max generation is set as 30 and the match threshold value is set as 2% of yesterday s demand: MWh. After

124 CHAPTER 4. A NEW MARKET MECHANISM Figure 4.7: The customers response demand of each hour and z h Figure 4.8: The result MCP vector and the retail price generations, the best penalty value Z (equation (4.5)) is 207,388 MWh, which is quite a substantial number compared with the average demand per hour from all customers: 34,972 MWh. This must be caused by the MCP function which has too many segments. From the history usage data, there never exists an hourly demand that reaches the segment 7, 8 or 9 in Table 4.3; these extra segments increase the number of combination of MCP vectors. That is why the fitness 124

125 CHAPTER 4. A NEW MARKET MECHANISM value cannot converge to a small number at the early generation. Therefore, we select the first 6 segments as the new MCP function. Figure 4.6 shows the running result under the new MCP function. After 28 generations, the best penalty value Z is 8, MWh, which is a smaller than the match threshold value. Table 4.4 and Figure 4.7 show the detail of the best result in Figure 4.6. Table 4.4 shows the value of z h in 24 hours. In most hours z h equals zero, which means that the total customer s response demand in hour h is located within the supply segment: (D s s h (MCP h ), D h(mcph )] in Table 4.2 when the market clear price is MCP h. It also means that the ISO finds the match equilibrium in most hours of the next day. This proves that our new mechanism works well. The Figure 4.7 is the comparison between the z h and the customers response demand (estimated reaction demand) in that hour. Even in the hours which have a difference between customers response demands and minimum or maximum value of MCP s interval (z h is not equal to zero), the difference in each hour is only a small part compared with the customers response demand: 3.26%, 14.23%, 1.18%, 0.31% and 4.24%. The better result could be obtained by increasing the number of generation, but the running time of Algorithm 3 also becomes longer. This paragraph talks about the retailer s optimal pricing model. The revenue of the best result in Figure 4.6 is 29,727,000 cents, which is obviously smaller than the revenue restriction. The MCP vector MCP v and reactor s retail price of next day are shown in Figure 4.8. The retailer increases the price in hours 14, 16, 18, 20 and 21 which are all in the peak-times. This encourages customers to shift the consuming power from peak-times to non-peak hours, which is also why Figure 4.7 shows a lower peak-average-ratio All of this proves that the retailer s pricing optimization model in this Chapter is reasonable. 125

126 CHAPTER 4. A NEW MARKET MECHANISM 4.5 Summary The main work in this Chapter outlines and develops an integrated framework which combines the retail and wholesale market. Under this framework, we have developed and integrated the supply and demand response mechanism by using the new wholesale pricing mechanism and the new demand pricing mechanism. The simulation results in Section 4.4 show that the ISO can find the match equilibriums for most hours of the next day in our mechanism. This proves that our mechanism can effectively handle unpredictable demand under the dynamic retail pricing (demand response programs). A pricing optimization model for the retailer is also given in this Chapter. This pricing model can find the optimal prices of the next day in order to maximize the retailer s profit under the demand based piecewise MCP function by predicting customers reaction to electricity prices. It can change the price-sensitive customers behaviour and help them to save money by switching to cheaper times. It also enables retailers to improve their profits without increasing the revenue. The simulation results of the pricing optimization model show that the retailer can achieve the maximum profit under the integrated framework. A GAbased method is proposed in order to implement our new mechanism and support the ISO to efficiently find the match equilibrium between demand and supply. But our GA-based method has its own limitations. Although a genetic algorithm is a better method in comparison to enumeration, it still needs to calculate the retailer s pricing optimization problem many times and thus takes a long time to obtain an approximate result. For example, under the environment of 8 GB RAM, 3.20 GHz Intel Core processor with parallel processing, Algorithm 3 needs 10 hours to obtain the result when the max generation is set as 30 and the 126

127 CHAPTER 4. A NEW MARKET MECHANISM population size as 60. Furthermore, only an approximated match equilibrium can be found by the genetic algorithm-based approach. Sometimes this solution is a local optimal solution but not the global optimal solution. However, these limitations are not difficult to solve for the ISO and retailers in the real market, as they all have more powerful computers which only require a small amount of time to find the match equilibrium between supply and demand by implementing Algorithm 3. Furthermore, the approximated result has been proven good enough in Section 4.4. The only limitation of our GA-based method implementing in the real market is that the ISO need to ask each retailer to determine the optimal price so many times (1800 times in our experiment), thus requiring huge information exchanges between the ISO and retailers. These limitations are improved upon, and overcome, in Chapter

128 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Chapter 5. An Analytical Optimization Method for Implementing the New Market Mechanism 5.1 Introduction In Chapter 4, we developed an integrated framework which combines the retail and wholesale market. Under this framework, we proposed the new integrated supply and demand response mechanism. This mechanism can efficiently identify the match equilibrium between supply and demand despite the unpredictable demand caused by demand response management. In Chapter 4, a genetic algorithm-based approach was provided in order to implement this new mechanism. Although a genetic algorithm is a better method in comparison to enumeration, it still needs to calculate the retailer s pricing optimization problem many times and thus takes a long time to obtain an approximate result. Furthermore, only an approximated match equilibrium can be found by the genetic algorithm-based approach. Therefore, in this Chapter, we propose an analytical optimization method for the proposed mechanism in order to efficiently obtain an accurate result. This Chapter is adapted from our published paper: [81]. The remainder of this Chapter is organized as follows: Firstly, the problem statement and the proposed algorithm are presented in Section 5.2. Secondly, the proof of the feasibility for the proposed algorithm is presented in Section 5.3. Thirdly, numerical results are provided in Section 5.4, showing that our analytical 128

129 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD optimization method improves the efficiency of the GA-based approach. This chapter is summarized in section Problem Statement and the Proposed Algorithm The methods involving how to obtain the MCP function for the ISO and how to find the optimal prices in order to maximize the profit for retailers on the demand side have been discussed and solved in subsection and subsection respectively. A GA-based approach which is used to implement the proposed mechanism is introduced in subsection In this section, we will propose and develop an analytical optimization method which can accurately find the match equilibrium between supply and demand, and which improves the efficiency of the GA-based approach. In the proposed mechanism (which has been illustrated in Section 4.2 in detail), the ISO wants to find the match equilibrium between demand and supply for each hour of the next day. This means that the ISO finds an MCP vector: MCP = (MCP 1,, MCP H ), MCP h (MCP h 1, MCP h 2,, MCP h s h ), h (1,, H), which satisfies the following equation (5.1). D h s (MCP h ) D h r,j (MCP 1,, MCP H ) j D h s (MCPh ) h (1,, H) (5.1) r,j In equation (5.1), j is j-th retailer in the market; j D h is the demand of all retailers in hour h under MCP h, where D h r,j is equal to the response demand of retailer j s customers: RF h (p 1 j, p 2 j,, p H j ) which can be calculated by solving optimization problem (4.2); D h s (MCP h ) and D h s (MCPh ) are the lower and upper bounds of MCP h s corresponding supply segment in Table 4.2 respectively. Under the match equilibrium, the total customers response demand of hour h is located within the supply segment: (D h s (MCP h ), D h s (MCPh )] in Table 4.2 when 129

130 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD the market clearing price is MCP h. Therefore, the balance of supply and demand is matched and then avoids the shortage and surplus of electricity. Although the MCP functions are already known, finding the match equilibrium between the demand and supply for each hour of the next day remains a complicated problem for the ISO. This is because the combination of MCP vectors for the next day amounts to a really large number. For each hour h, we use s h to present the maximize number of segments in the MCP function (equation (4.1)) at hour h. The domain of MCP h in the MCP function is MCP h (MCP 1 h, MCP 2 s h,, MCP h h ), h H. Then the combination of all possible MCP vectors MCP = (MCP 1,, MCP H ) in the next day is equal to s 1 s 2 s H. For example, when both of s 1, s 2,, s H are equal to 3 and H is equal to 24, the numbers of combination of all MCP vectors for the next day is e+11. It is unrealistic, or impossible, for each retailer to solve the problem (4.2) so many times. Even if retailers were able to solve problem (4.2) for all those combinations, it would cost the ISO a lot of time to connect these results with the MCP functions and find the match equilibriums. We have used the Genetic Algorithm to solve this problem in Chapter 4. Although the genetic algorithm is a better method when compared to the enumeration, it still needs to calculate the pricing optimization problem a huge number of iterations and get an approximate solution. In other words, it takes a long time to obtain an approximate result. For example, the program takes about 10 hours to calculate the problem (4.2) for 1800 times when we set the generation and population size as 30 and 60 respectively. Therefore we need to develop a new algorithm for the proposed mechanism to efficiently gain an accurate result. Based on equation (5.1), (4.4) and (4.5), we proposed the equation (5.2) and (5.3) below to measure whether a MCP vector reaches the match equilibriums. Only when the ISO finds a MCP vector: MCP v = (MCP 1,, MCP H ) which makes 130

131 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Er(MCP 1,, MCP H ) equals to 0, we could say that the match equilibrium can be reached in the electricity market. Er h (MCP v ) = { 0; if D h s (MCP h ) D h r,j (MCP v ) j D h s (MCPh ) j D r,j h (MCP v ) D s h (MCP h ); if D s h (MCP h ) > j D r,j h (MCP v ) D r,j s s j h (MCP v ) D h(mcph ); if D h(mcph ) < D r,j j h (MCP v ) (5.2) Er(MCP 1,, MCP H ) = Er h h (5.3) In equation (5.2), MCP v = (MCP 1,, MCP H ); Er h is equal to zero if the demand of all retailers in hour h satisfies the equation (5.1). When this occurs, it means that the ISO finds the match equilibrium at hour h. Otherwise, Er h presents the difference between the retailers demand at hour h and the supply under MCP h, the minimum (maximize) supply is the lower (upper) bound of MCP h s corresponding supply segment in Table 4.2. For s 1 s 2 s H possible MCP vectors, we want to propose an algorithm to find a MCP vector which makes Er given in (5.3) equal to 0. According to this MCP vector, the ISO can schedule a generation scheme and retailers can price the electricity for each hour of the next day, the match equilibrium can be reached in the electricity market. Equation (5.4) is the estimated customers reaction function in the retailer s pricing model (4.2) which has been introduced in Section RF h (p 1, p 2,, p H ) = α h + β h,1 p β h,h p H (5.4) From equation (5.4), we know that the customers demand for hour h can be affected by the price of this hour and other hours. Normally self-elasticity β h,h is bigger than H c=1,c h β h,c, so we assume that self-elasticity is the main factor affecting the demand of all customers. For hour h, if Er h is greater than 0, this means that the retail market needs more energy from the supply side due to the 131

132 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Figure 5.1: Example of the new method lower MCP (i.e., the lower energy cost which allows the lower price and leads to higher demand). In order to reach the match equilibrium in that hour, MCP h should be increased. Figure 5.1 illustrates this scenario. Under an MCP vector: MCP 1,, MCP H, where MCP h equals to MCP 3 h, i.e. MCP 3 h equals to the value of third segment in hour h s MCP function (equation (4.1)), the total customers response demand RF h is not located within the same segment as MCP h, which means that the match equilibrium in hour h is not reached. Based on our assumption, MCP h should be decreased to MCP h where MCP h equals to 2 MCP h in order to decrease the value of Er h and to reach the match equilibrium in that hour. We can use the same idea when trying the match equilibria for the remaining hours. Based on the above idea, the following process is proposed as our algorithm to reach the match equilibrium: Step 1: Input a MCP vector; every retailer solves problem (4.2) to obtain the customers reaction demand and sends it to the ISO. Then the ISO calculates Er h for each hour and gets a vector: Er = (Er 1, Er 2,, Er H ). Step 2: The ISO finds an hour h which has the 132

133 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD biggest Er h in a period of H hours, then decreases MCP h to the value of the previous segment in Table 4.2 if Er h is less than 0 (for example, if MCP h = MCP h 2, MCP h 2 is the value of second segment in hour h in Table 4.2, then decreases MCP h from MCP h 2 to MCP h 1 ), or increase the MCP h to the value of next segment in Table 4.2 if Er h is greater than 0. Step 3: Each retailer calculates the problem (4.2) to get its customers reaction demand under the new MCP. Step 4, If Er is not equal to 0, repeat step 2 and 3 until Er is equal to 0. Step 5: Output the MCP vector of the next H hours. Algorithm 4 shows the detailed process of our new method. Algorithm 4 Analytical optimization method Input: MCP function of each hour (Table 4.2); A random MCP vector generating from Table 4.2. Output: A MCP vector (which reaches the match equilibriums); 1: Retailers calculate problem (4.2) by using the input MCP vector; 2: The ISO calculates Er h for each hour and gets a vector: Er=(Er 1, Er 2,, Er H ); 3: While h (1,, H), exists Er h 0; 4: Find hour h in the set of (1,, H), where h = argmax{ Er h }; 5: If Er h < 0 &MCP h MCP 1 h ; 6: Decreases MCP h to the value of previous segment in Table 4.2; 7: S If Er h > 0 & MCP h MCP h h ; 8: Increases MCP h to the value of next segment in Table 4.2; 9: Retailers calculate problem (4.2) under the modified new MCP vector and get new Er h for each hour; 10: End while; 11: Output the final MCP vector and each retailer s estimated demand. In order to ensure the correctness and feasibility of this new algorithm in our mechanism, it is necessary to prove the monotonicity in step 3. If we can prove that the property of monotonicity is held in step 3, this means that Er will be decreased after each loop. Then we can say that our algorithm becomes closer to the final result (find a MCP vector MCP v which makes Er(MCP v ) = 0) after each 133

134 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD loop. Besides, we should also prove the convergence of Algorithm 4. If we can prove that property, which means Algorithm 4 can be stopped at the final result (find a MCP vector MCP v which makes Er(MCP v ) = 0) after limited numbers of iterations, and then we can say that our algorithm is feasible to the proposed mechanism. These two properties are proved in the next Section. 5.3 Proof Process In order to ensure the correctness and feasibility of Algorithm 4 in our mechanism, we should prove that the monotonicity and convergence properties are held in the algorithm. Firstly we define the monotonicity in our algorithm: Definition 5.1: For hour h, we have Er h > 0 (or Er h < 0) and MCP h = MCP x h where x (1,, s h 1) (or x (2,, s h )), then change the value of MCP h from MCP x h to MCP x+1 h (or MCP x 1 h ), we use MCP h to present the changed MCP h. If equation (5.5) holds, we can say that the each iteration in step 3 is monotone. Er(MCP h, MCP h ) > Er(MCP h, MCP h ) (5.5) In equation (5.5), h mean all hours in (1,..,H) except for hour h. h = s (1, h 1, h + 1, H). For the scenario of Er h > 0 and MCP h = MCP h h (or Er h < 0 and MCP h = MCP 1 h ), the MCP h cannot be increased (or decreased), that is why we set x (1,, s h 1) (or x (2,, s h )). It means that the preferred customers demand in that hour exceeds the given interval demand from the ISO. When this happens, the match equilibrium may not be reached. Therefore, these scenarios should be avoided by the ISO. The declared hourly interval demand should be longer enough to consider all possible demand from the retail market. 134

135 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD For the reason of simplicity, we set H equal to 2 and j equal to 1(only 1 retailer exists in the electricity market) in our proof process. Normally the retailers in the market are always competitive in-between. But, as we are discussing the pricing for the next day and the competition which exists within a day can be ignored. This is due to the fact that the customers switching within one day is very limited, and thus can be ignored. Therefore retailers are pricing independently and there is no cross-impact between the prices of different retailers in this scenario, meaning that each retailer has its own customers. That is, we do not consider the competition among retailers with 24 hours in our proof process. If needed, a retailer can set price differential bounds with its competitors to take the competitive impact of prices into consideration. Setting more retailers in the market will only increase the complexity to the statement. But the number of retailers in the market makes no difference in the following proof process. In each loop of Algorithm 4, we only change a single hour s MCP (MCP h changes to MCP h ). In the proof process, we split Er into two parts: Er h and Er h, then talk about how the change in MCP h affects these two parts separately. So there is no difference when H is set to 2 or 24, we just consider all hours except h: (1,,h-1,h+1,,H) as one whole part. Therefore, the setting of these two parameters cannot affect the correctness of our proof process. Even if H and j are changed to 24 and 2(or higher), we can use the same method to prove the monotonicity and convergence in Algorithm 4. Therefore we assume H=2 (i.e., the peak hour and off-peak hour) and j=1 without loss of generality in order to simplify the statement. The retailer j s reaction function (5.4) can be written as (5.6) and (5.7), where p j p is the retail price in peak hour and p j o is the retail price in off-peak hour. D p r,j (p p j, p o j ) = α p + β p,p p p j + β p,o p o j (5.6) 135

136 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD D o r,j (p p j, p o j ) = α o + β o,p p p j + β o,o p o j (5.7) β p,p < 0; β o,o < 0; β p,o > 0; β o,p > 0 (5.8) Then the retailer s price optimization problem (4.2) can be written to (5.9): max PR j = D r,j p (p j p, p j o ) (p j p MCP p ) + D r,j o (p j p, p j o ) (p j o MCP o ) (5.9) Subject to D r,j p (p j p, p j o ) p j p + D r,j o (p j p, p j o ) p j o RC j (5.10) p min h p h p max h h {p, o} (5.11) In our proof process, we will use two theorems. In the following section we will prove these two theorems first. Theorem 5.1: In problem (5.9), if MCP p is decreased (increased), then the retailer s optimal price in the peak hour will stay the same or decrease (increase). Proof: The optimal point of p j p has three different scenarios wherein MCP p is unchanged. Because equation (5.9) is a quadratic equation and the decision variables are p j p and p j o, we can plot it as a parabola in the coordinate system. The x axis represents the value of p j p and y axis represents the value of PR j. So, the first scenario is that the optimal solution p j p is located in the peak point of equation (5.9) s parabola when the peak point is satisfied the constraints (5.10) and (5.11). On the contrary, the optimal solution is not the peak point in the parabola in both the second and third scenario: the optimal p j p is located in the boundary of constraint (5.10) and (5.11) respectively. The equation (5.9) can be transferred to: PR j = β p,p p j p p j p + p j p (α p + β o,p p j o MCP p β p,p + β p,o p j o β o,p MCP o ) MCP p (α p + β p,o p j o ) + (p o MCP o )(α o + β o,o p j o ) (5.12) 136

137 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD According to the peak-point equation in the quadratic equation, we know for equation y = ax 2 + bx + c, the peak point is ( b, 4ac b2 2a 4a ). Based on the differential of equation (5.12) of p p j, the value of peak point in x axis P peak is: P peak = α p+β o,p p o j +βp,o p o j βo,p MCP o 2β p,p + MCP p 2 (5.13) From (5.13), we know that P peak is decreased when MCP p is decreased. So, in the first scenario, we can deduce that p p j will be decreased when MCP p is decreased. In the second scenario, the optimal result of (5.9) is located within the boundary of sales price constraint (5.11), which means that p p j is equal to p h min (when p h min > P peak ) or p h max (when P peak > p h max ). We know that P peak is decreased when MCP p is decreased, meaning that equation (5.12) s parabola is moved to the left in the coordinate system. Therefore p p j will stay the same when MCP p is decreased. Now we consider the third scenario, the result of (5.9) is located in the boundary of revenue constraint (5.10). Then the optimal solution p p j must satisfy the equation (5.14). D p r,j (p p j, p o j ) p p j + D o r,j (p p j, p o j ) p o j = RC j (5.14) This equation is also a quadratic equation for p p j. We know for equation 0 = ax 2 + bx + c, a 0, the solution is x 1,2 = ( b ± b 2 4ac) 2a. Therefore we can get the solution of equation (5.14): p l and p u, where p u > p l. In this scenario, the following inequality must be held; otherwise the optimal result of (5.9) is not located in the boundary of revenue constraint. p u > P peak > p l (5.15) 137

138 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Figure 5.2: The solution space of three scenarios for problem 5.9 The solution space of p j p in that scenario is: p j p p l andp j p p u. The optimal p j p either is p l or p u. So when equation (5.12) s parabola is moved to the left in the coordinate system, the optimal p j p is either decreased from p u to p l or is kept the same as before. Therefore, we complete the proof of Theorem 5.1. The Figure 5.2 illustrates the solution space of these three scenarios in problem (5.9): part (b) and (c) represents scenario 1; part (a) represents scenario 3 and part (d) and (e) represents scenario 2. Here we give the definition of demand consistent which used in the second theorem. Definition 5.2: For a given product in a retail market, assume that there are n competitive prices P = {p 1, p 2,, p n } for the sane product and their corresponding demands are D 1, D 2,, D n. It is said that the given product in the market is demand consistent, if for any two sets of prices 138

139 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD P(1) = [p 1 (1), p 2 (1),, p n (1)] and P(2) = [p 1 (2), p 2 (2),, p n (2)] satisfying P(1) P(2) which is defined as p i (1) p i (2)(i = 1,2,, n), the corresponding demands {D 1 (1), D 2 (1),, D n (1)} and {D 1 (2), D 2 (2),, D n (2)} satisfying D(1) = D 1 (1) + D 2 (1) + + D n (1) D(2) = D 1 (2) + D 2 (2) + + D n (2). In other words, the given retail product is demand consistent if its price set P(1) is lower than price set P(2), then the total market demand D(1) under P(1) is higher than the total market demand D(2) under P(2). The second theorem that we used in our proof process is illustrated as follows: Theorem 5.2: The necessary and sufficient condition that electricity is a demand consistent product in the considered retail market is that, for each h (1,2,, H), the following inequality holds. H β h,h + c=1,c h β c,h < 0 (5.16) Poof: We prove the Necessity and Sufficiency in the assumption. Necessity: Assuming that the price of electricity at hour h decreases p, but the prices of other hours during the day remain the same, then for hour h, the demand for electricity will be increased, based on the property of self-elasticity: β h,h < 0, we have: D h r (p 1,, p H ) = D h r,p h (p 1,, p H ) D h r,egular (p 1,, p H ) (5.17) = (α h + β h,h (p h p) + H c=1,c h β h,c p c ) (α h + β h,h p h + H β h,c p c = β h,h p > 0 c=1,c h ) On the other hand, based on the property of cross-elasticity: β h,c > 0, for each other hour c (i.e., c h), the demand for electricity during these hours will decrease when the price of electricity at hour h decreases: D c r (p 1,, p H ) = D c r,p h (p 1,, p H ) D c r,egular (p 1,, p H ) (5.18) 139

140 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD H = (α c + β c,h (p h p) + β c,k p k ) (α c + β c,h p h + β h,k p k ) k=1,k h = β c,h p < 0 k h, k = 1,2,, H H k=1,k h Based on the demand consistency, we can also state that the total demand variation during one day should be non-negative due to the price of electricity in one hour increase and in other hours remain same. Therefore, based on (5.17) and (5.18), we have: Demand Growth in one day (5.19) = Demand with p h decrease Demand without p h decrease = D r h (p 1,, p H ) + H c=1,c h D r c (p 1,, p H ) = (β h,h + H c=1,c h β c,h ) p > 0 Noticing that p > 0, the above inequality implies immediately that: H β h,h + c=1,c h β c,h < 0 (5.20) Which completes the proof of the necessity in theorem 5.2. Sufficiency: For any two sets of prices P(1) = [p 1 (1),, p H (1)] and P(2) = [p 1 (2),, p H (2)] satisfying p h = p 1 (2) p 1 (1) 0(h = 1,2,, H), the difference of total demand under this two prices are: D(2) D(1) = H c=1 [D c (2) D c (1)] (5.21) H H H, ( c, h ph(1) c 1 h 1 h 1 = ac c h ph 2) ac H H H H =,, c h ph c h ph 0 c 1 h 1 h 1 c 1 As p h 0(h = 1,2,, H) and c=1 β c,h = β h,h + c=1,c h β c,h 0 (h = 1,2,, H) are based on inequality Therefore, based on Definition 5.2, we H H 140

141 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD have that electricity is a demand consistent product. This completes the proof of Sufficiency. The two essential theorems have been proved. In the following Section, we prove that the monotonicity and convergence properties are held in Algorithm 4. Theorem 5.3: The monotonicity and convergence properties are held in Algorithm 4. In other words, Algorithm 4 can stop at the final result (find a MCP vector: MCP = MCP 1,, MCP H, which makes Er(MCP 1,, MCP H ) = 0) after a limited number of iterations. Proof: Monotonicity: For peak hours, if Er p is smaller than 0 in that situation the retailer s demand is smaller than the lower bound of MCP p s corresponding supply segment in Table 4.3. According to Algorithm 4, the ISO decreases MCP p to the value of the previous segment in the MCP function (equation (4.1)) (for example, if MCP h = MCP 2 2 h, MCP h the value of second segment in hour h in 2 Table 4.2, then decreases MCP h from MCP h to MCP 1 h ), we use MCP p to represent the changed MCP p. According to Theorem 5.1, the retailer decreases or maintains the unit sales price in the peak hour from p j p to p j p when the unit cost (MCP) in the peak hour from supply side is decreased. According to Table 4.2 and 4.3, we can get the equation (5.22) when MCP p is deceased to MCP p. D s p (MCP p ) > D s p (MCP p ) (5.22) According to equations (5.2) and (5.3), we calculate the change of Er after MCP p is deceased to MCP p : Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.23) = D s p (MCP p ) D s p (MCP p ) + D r,j p (p j p ) D r,j p (p j p ) + Er o (MCP p, MCP o ) Er o (MCP p, MCP o ) 141

142 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD According to the property of self-elasticity, the demand will increase when the retail price in the peak hour is decreased from p j p to p j p, then we can get: D r,j p (p j p ) D r,j p (p j p ) 0 (5.24) There could be two different situations for Er o, either this could be greater than 0 or smaller than 0. If Er o (MCP p, MCP o ) < 0, according to equation (5.6) and (5.7): Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.25) = D s p (MCP p ) D s p (MCP p ) + D r,j p (p j p ) D r,j p (p j p ) + D s o (MCP o ) D s o (MCP o ) + D r,j o (p j p ) D r,j o (p j p ) = D s p (MCP p ) D s p (MCP p ) + (β p,p + β o,p ) (p j p p j p ) If Er o (MCP p, MCP o ) > 0, according to equation (5.6) and (5.7): Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.26) = D s p (MCP p ) D s p (MCP p ) + D r,j p (p j p ) D r,j p (p j s p ) + D o(mcpo ) s D o(mcpo ) D r,j o (p j p ) + D r,j o (p j p ) = D s p (MCP p ) D s p (MCP p ) + (β p,p β o,p ) (p j p p j p ) From Theorem 5.2 and equation (5.8) we know β p,p + β o,p < 0, β p,p β o,p < 0. As p j p p j p, we can deduce that equation (5.26) is held in both of these two situations. Er(MCP p, MCP o ) > Er(MCP p, MCP o ) (5.27) Similarly, we can use the same method to prove the monotonicity when Er p is bigger than 0. According to equation (5.8), Theorem 5.2, Table 4.2 and 4.3, equation (5.27) and (5.28) are held. p j p p j p (5.28) s s D p(mcpp ) < D p(mcpp ) (5.29) Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.30) 142

143 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD s s = D p(mcpp ) D p(mcpp ) D r,j p (p j p ) + D r,j p (p j p ) + Er o (MCP p, MCP o ) Er o (MCP p, MCP o ) According to the property of self-elasticity, the demand will decrease when retail price in the peak hour is increased from p j p to p j p, then we can get: D r,j p (p j p ) D r,j p (p j p ) 0 (5.31) There could be two different situations for Er o. This either be greater than 0 or smaller than 0. If Er o (MCP p, MCP o ) < 0, according to equation 5.6 and 5.7: Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.32) s s = D p(mcpp ) D p(mcpp ) D r,j p (p j p ) + D r,j p (p j s p ) + D o(mcpo ) s D o(mcpo ) + D r,j o (p j p ) D r,j o (p j p ) s s = D p(mcpp ) D p(mcpp ) + (β p,p β o,p ) (p j p p j p ) If Er o (MCP p, MCP o ) > 0, according to equation (5.6) and (5.7): Er(MCP p, MCP o ) Er(MCP p, MCP o ) (5.33) s s = D p(mcpp ) D p(mcpp ) D r,j p (p j p ) + D r,j p (p j s p ) + D o(mcpo ) s D o(mcpo ) D r,j o (p j p ) + D r,j o (p j p ) s s = D p(mcpp ) D p(mcpp ) + (β p,p + β o,p ) (p j p p j p ) Then we can also conduct to equation (5.27) to prove the property of monotonicity is held in Algorithm 4 when the ISO increases MCP p to the value of the next segment MCP p in Table 4.2. As well as proving the monotonicity of Algorithm 4, we should also prove the convergence in Algorithm 4 to ensure that the program will finally stop at the point where Er is equal to 0. Convergence: We set equal to the value of minimum supply segment in the Table 4.2. From equation (5.25), (5.26) and (5.27), we know that: 143

144 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Er(MCP p, MCP o ) Er(MCP p, MCP o ) > D s p (MCP p ) D s p (MCP p ) (5.34) Assuming Er(0) is the result of step 2 in Algorithm 4 and Er(0) = c > 0, t is the looping times of step 3 in the algorithm. From equation (5.34) we know that: Er(t) < Er(t 1) < Er(t 2) 2 < Er(0) t (5.35) If Algorithm 4 will never stop, which means that t +. According to equation (5.35) we can deduce that: Er(t) < Er(0) t < (5.36) From equation (5.3) we know Er(t) 0. But equation (5.36) is contradictory with the condition Er(t) 0. So we can deduce that our algorithm can stop in finite steps, which completes the proof of convergence in the Algorithm 4. After proving the convergence and monotonicity, we can say that through using Algorithm 4, our proposed mechanism could converge to a result which state that the supply and demand are matched in all hours. Then Theorem 5.3 is proven. The running time T of Algorithm 4 is: T (s 1 + s 2 + s h + s H ) t S max H t (5.37) This means that the ISO calculates optimization problem (4.2) no more than S H times in proposed mechanism, where S max is the biggest number in the vector (s 1, s 2 s h s H ) and t is the calculation time for solving the problem (4.2). This is much smaller than the running time of the GA-based approach: t ps g. Normally ps (population size) and g (the max number of generations) are set bigger than 100 and 50 respectively. An improved Algorithm In our test, the results of problem 4.2 between two adjacent steps in Algorithm 4 are very close. In order to avoid repetitive calculations and to 144

145 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD increase the running speed, we made some changes in Algorithm 4. These were made in order to improve efficiency. We modified Step 3 in the analytical optimization method as: for each hour h in the set (1,, H) which has Er h 0, decreases MCP h to the value of the previous segment in Table 4.2 if Er h is greater than 0, or increase MCP h to the value of previous segment in Table 4.2 if Er h is less than 0. Algorithm 5 is the detail process of improved analytical optimization method. Algorithm 5 Improved analytical optimization method Input: MCP function of each hour (Table 4.2); A random MCP vector generating from Table 4.2. Output: A MCP vector ( which reaches the match equilibriums); 1: Retailers calculate problem (4.2) by using the input MCP vector; 2: The ISO calculates Er h for each hour and gets a vector: Er=(Er 1, Er 2,, Er H ); 3: While h (1,, H), exists Er h 0; 4: For i=1 to H, do; 5: If Er i < 0 &MCP i MCP i 1 ; 6: Decreases MCP i to the value of previous segment in Table 4.2; 7: If Er i > 0 & MCP i MCP i S i ; 8: Increases MCP i to the value of next segment in Table 4.2; 9: End for; 10: Retailers calculate problem (4.2) under the modified new MCP vector and get new Er h for each hour; 11: End while; 12: Output the final MCP vector and each retailer s estimated demand. In Algorithm 4, for each hour h in set of (1,,H), if there exists Er h 0, it only modifies one hour s MCP (this hour has the biggest Er h ) in the MCP vector before calculating the problem (4.2). But in Algorithm 5, for all hours in the set of (1,,H) which has Er h 0, it modified these hours MCP. Then the problem (4.2) is calculated. This method decreases the calculation time. As with Algorithm 4, 145

146 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD we need to prove that the properties of monotonicity and convergence are held in Algorithm 5. Definition 5.3: For all the hours h (1,, H) which has Er h > 0& MCP h s MCP h h and MCPh = MCP x h where x (1,2,, s h 1), then change the value of MCP h from MCP x h to MCP x+1 h, and for all the hours h (1,, H) which has 1 Er h < 0 & MCP h MCP h and MCP h = MCP x h where x x (2,, s h ), then x change the value of MCP h from MCP h to MCP x 1 h. We use MCP h present the changed MCP h for each hour h (1,, H). If the equation (5.38) holds, we can say that the running result of step 3 in Algorithm 4 is monotone. Er(MCP h, MCP h ) > Er(MCP h, MCP h ) (5.38) Similarly with Theorem 5.3, here we propose Theorem 5.4. Theorem 5.4: the monotonicity and convergence properties are held in Algorithm 5. In other words, Algorithm 5 can stop at the final result (find a MCP vector: MCP = MCP 1,, MCP H, which makes Er(MCP 1,, MCP H ) = 0) after a limited number of iterations. Proof: First we consider the scenario of 2 hours, if both of Er p and Er o are less than 0, which means the retailer s demand in this two hours are smaller than the lower bound demand of MCP p s and MCP o s supply segment in Table 4.2 respectively. According to Algorithm 5, the ISO decreases MCP p and MCP o to the value of previous segments MCP p and MCP o in Table 4.2 respectively. This process is similar with Algorithm 4; the change of MCP vector from (MCP p, MCP o ) to (MCP p, MCP o ) can be split into two steps: change MCP vector from (MCP p, MCP o ) to (MCP p, MCP o ), then change MCP vector from (MCP p, MCP o ) to (MCP p, MCP o ). These two steps are the same as the step 3 of the analytical optimization method. Due to the property of monotonicity in Algorithm 4, we can obtain the following equations: 146

147 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Table 5.1: Aggregated bidding curve/mcp function MCP h Segment scale D h s (MCP h ) D h s (MCPh ) Segment number Table 5.2: The fitness value of each hour of new method Hour z h Hour z h Hour z h Er(MCP p, MCP o ) > Er(MCP p, MCP o ) (5.39) Er(MCP p, MCP o ) > Er(MCP p, MCP o ) (5.40) According to equation (5.39) and (5.40), we can deduce that: Er(MCP p, MCP o ) > Er(MCP p, MCP o ) (5.41) Equation (5.41) proves the monotonicity of Algorithm 4. In a scenario of 24 hours, we can use a similar method to prove the monotonicity of Algorithm 4: Er(MCP h, MCP h ) > Er(MCP h, MCP h ) > > Er(MCP h, MCP h ) (5.42) 147

148 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD The convergence property in Algorithm 5 can be proved using the same method in the proof process as Theorem 5.3. Therefore we can say that, through using Algorithm 5, our proposed mechanism could converge to a result in which the supply and demand are matched throughout all hours, which completes the proof of Theorem 5.4. The running time T of Algorithm 5 is: T S max t (5.43) 5.4 Numerical Results In this section, the numerical results are illustrated. Firstly, we present the exact parameters. All the data is in the demand model derived from PJM which includes the day-ahead pricing and demand information between 01/01/2011 and 30/11/2012. Table 5.1 is the MCP function obtained from the aggregated bidding curve. In order to simplify the calculation, here we set the MCP functions of each hour to be the same. After numbers of loops, the final value of Er in our experiment is equal to 0. Table 5.2 shows the running results of Algorithm 4. Compared with running result of GA (Table 4.4) which under the same MCP function, in every hour Er h is equal to zero, which means that the customers reaction demand for hour h is located in the same segment with the MCP of this hour. It also means that the ISO finds the match equilibrium throughout all the hours of the next day. This proves that the mechanism of this Chapter works well. The running time of GA is approximately 10 hours. In contrast, the running time of Algorithm 4 is about 15~45 mins, showing that our new method is more efficient. The Figure 5.3 shows the customers response demand (estimated reaction demand) in the final result. 148

149 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD Figure 5.3: Customers response demands Figure 5.4: The resulting MCP vector and retailer s retail price This paragraph addresses the pricing model of the retailer. The constraints set in problem (4.2) are listed as follows: Revenue restriction is 34,347,000cents max and max price p h is cents/MWh. The revenue of the final result in Table IV is 29,728,216 cents, which is obviously smaller than the revenue restriction. The MCP vector and the reactor s sales price for the next day are shown in Figure 5.4. The retailer increases the price in hours 14, 16, 18, 20 and 21 which are all within peak-times. This encourages customers to shift the consuming power 149

150 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD from peak-times to non-peak hours, which is also why Figure 5.3 shows a lower peak-average-ratio From Figure 5.4, we can also see that the retailer set their unit cost (MCP) as the retail price during off-peak hours (hour 1-10, 23-24). The reason for this phenomenon is the revenue constraint that we set in problem (4.2). If we do not set the revenue constraint, even with the retail price constraint within each hour, the retailer will set the price of the electricity as high as they can in order to achieve more profit. This will lead to the result that the optimal price in the hours with small price elasticities (off-peak hours) will be set very close to the upper bounds of the retail price constraint. Also, the revenue and profits of the retailer are very high, and this is not acceptable to the government. So, the revenue constraint in problem (4.2) guarantees that the retailer can achieve the maximum profit only under the condition of unincreased revenue. For peak hours 12, 13, 15, 17 and 19, the retailer does not increase the price either. This incentivises the customers to shift their demand of other peak hours to these hours and helps the ISO to flatten the demand profile during peak hours. This is also reflected in Figure 5.3. The demands in hours 13, 15, 17 and 19 are slightly higher than in the adjacent peak hours. If the retailer increases all peak hours prices, then the customer has no choice to shift their peak hours demand to. All the above proves that the pricing optimization method which we have designed in this paper is reasonable. 5.5 Chapter Summary The main work presented in this Chapter outlines and develops an analytical optimization method for implementing our integrated supply and demand response mechanism. In order to ensure the correctness and feasibility of this new method, we prove that the monotonicity and convergence properties are 150

151 CHAPTER 5. AN ANALYTICAL OPTIMIZATION METHOD held in the algorithm. The simulation results in Section 5.4 show that the ISO can find the match equilibriums throughout all the hours of the next day in our mechanism in short times by using the analytical optimization method. This prove that our mechanism could effectively handle unpredictable demand under the dynamic retail pricing (demand response programs). But our analytical optimization method has a limitation. That is, the ISO needs to require each retailer to determine the optimal price several times (at the most 9 times in our experiment which is much better than the GA-based method). It thus needs a number of information exchanges between the ISO and retailers. Although this is not a problem in the real market, especially with the two-way, cyber-secure communication infrastructure, we want each retailer to return hourly response demand function to the ISO when it receives the MCP function of 24 hours. The ISO can then find the match equilibrium between the aggregated demand functions and MCP functions. This process only requires the ISO and the retailer to exchange the information twice, as this is more efficient. This limitation will be considered in our future research. 151

152 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Chapter 6. Demand Based Bidding Strategies under Interval Demand for Integrated Demand and Supply Management 6.1 Introduction In the previous chapters, we have developed an integrated framework which combines the retail and wholesale market. Under this framework, we proposed the new integrated supply and demand response mechanism by using the new wholesale pricing mechanism and the new demand pricing mechanism. This mechanism can effectively handle unpredictable demand under the dynamic retail pricing by finding the match equilibrium between supply and demand. In this proposed new mechanism, the ISO declares an interval demand to the wholesale market for each hour at first. Each generator is required to submit a bidding curve under the informed interval demand. Then the ISO calculates the aggregated MCP curve (step function) of each hour after receiving the bid curve from each generator. This step function can directly reflect the MCP which was under minimum production cost when it received the demand from retailers. However, the question of how to select the best optimal bidding strategy still remains an unsolved problem for generators. The selection of bids is extremely important for all generators in the wholesale market since their profits are based on the result of the dispatch of their units and on the market clearing price. This kind of question can be referred as the optimal bidding strategy problem [9]. 152

153 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Therefore, we mainly solve the generator s optimal bidding problem under our mechanism in this Chapter. This Chapter is adapted from our paper: [82]. The rest of this Chapter is organized as follows: Firstly, the problem statement is presented in Section 6.2. Secondly, the analysis of optimal bidding strategies for the generator under interval demand is presented in Section 6.3. Thirdly, numerical results are given in Section 6.4, showing that, with the help of the MCP forecasting model and the proposed confidence interval, the generator can use the bidding strategies which are given in Section 6.2 to maximize its profit. This chapter is summarized in Section Context and Motivation In our proposed new mechanism, the ISO declares an interval demand to the wholesale market for each hour at first. As aforementioned (Section and 4.2), the interval demand is more robust than a single demand figure, and this enables the ISO to consider the demand variates under different prices. Then based on this hourly interval demand, generators submit a bid curve (bid price vs. bid quantity). A bid curve or supply function of generator i is a set of bid segments (p i,mi, (q i,mi, q i,mi ], m i = 1,, I ) sorted by price in increasing order, which can be represented as a piecewise constant curve. This step function can represent this bid format. This interval is called a segment in the bid curve. Each segment in the bid curve of the generator i represents the bid information of a generation unit which belong to generator i. When it has received the hourly interval demand, each generator decides a bid for each of its generation unit. A bid is a pair of price and quantity (p ik, q ik ) and this is the bidding information of generator i s generation unit k at an hour, where k I, I denotes the set of the generator i s generation units. Then all the bids of the generator i s generation 153

154 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES units can be sorted by price in increasing order and represented as a piecewise constant curve. This piecewise constant curve is the generator i s submitted bid curve in this hour. The ISO calculates the aggregated MCP curve (step function) of each hour after receiving the bid curve from each generator. This step function can directly reflect the MCP which was under minimum production cost when it received the demand from retailers. The method of how to generate a MCP curve under the generators submitted bids for each hour has been presented in Section Therefore, the problem of how to select the optimal bidding strategies for the generator under the declared interval demand from the ISO is the main challenge which needs to be solved in this Chapter. We use the variable MCP d to define the market clearing price in the wholesale market in a hour under demand d. The electricity awarded to each bidder (generator) is then determined based on its bid curve and the MCP. All the electricity awards will be compensated at MCP d. The profit of each generation unit corresponds to (MCP d c k ) q k at each time step (normally set as hour), for k K, where c k and q k represents its unit production cost and the electricity award after the bidding respectively, K denote the set of all generation units and k is a generation unit in set K. For a generator i, which owns several different generation units, its profit in this hour represent as formula (6.1). Where I denote the set of indexes associated to generation units belonging to generator i, I K. MCP d k I q k C( k I q k ) (6.1) Prior to submitting its bid curve, generator i determines a quantity bid along with a price for each of its generation units that aims to build an optimal bid curve for this hour. Each quantity bid along with a price is a segment in generators i s bid curve. Solution approaches for this problem have been 154

155 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES presented in many papers as we have analysed in Chapter 2. Although the problem description differs in the detail of each paper, they all focus on the object of maximizing the profit for a generator subject to some common constraints: the bidding decision must consider the anticipated MCP, generation award and costs, and competitors decisions. To summarise, we can say that the models for solving the optimal bidding strategy problem can be broadly classified into two classes. The first class of model selects the best strategy for a generator s profit by estimating the bidding behaviour of other generators (competitors). The second class of model focuses on predicting the MCP of the next day, based on the prediction for deciding the best bids for a generator. In the SG, using the first method to solve the optimal bidding problem for the generator is unrealistic. Two reasons are given in support of this view. Firstly, the information for other generators (such as bidding history, capacity, etc.) is not available to any single company, as most auctions in the wholesale market are uniform-price auctions which require all bidders to simultaneously submit sealed bids to the ISO. Secondly, due to the fact that until recently most markets were structured as monopolies or duopolies, the wholesale market has become more competitive. More and more generators have entered the market. It is hard, or impossible, for a single generator to estimate all the competitors bidding behaviours, especially when the number of generators is so huge within the wholesale market. Therefore, forecasting the MCP in the wholesale electricity market is the most essential task and basis for any decision-making for the generator. The most common way for predicting the market behaviours is regression. The basic idea is to use the history and other available information for tomorrow to predict the MCP of tomorrow. A well-established nonlinear regression method is artificial neural network (ANN). An ANN forecasting model is introduced in Section 6.3 in detail. 155

156 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Since the predicted value can t be 100% precisely, which means the predicted MCP always has a prediction error. In some scenarios, this part of error could influence the result of bidding auction. Therefore we introduce the notion of confidence interval to the ANN forecasting model and use it to help the generator to choose the best bidding strategy. This part is also presented in Section 6.3. With the help of the ANN forecasting model and the proposed confidence interval, how to select the optimal strategies for the generator under an interval demand is also given in Section 6.3 as the contribution of this Chapter. 6.3 Strategic Bidding in Wholesale Market In this section, we present a detailed bidding model for a generator. Firstly, the problem statement is set out in Section The generator s best bidding strategies under an interval demand are then discussed in Section At the end of this section, the neural network forecasting model is presented Problem Statement and the Proposed Approach In the auction, a generation unit can win the bidding only if its bid price is lower than MCP. It is straightforward to verify that there is always an optimal solution for the generator s bidding problem to maximize the profit, makes all of its generation units which have a lower corresponding unit production cost compared with MCP to win the bidding, and to produce electricity to their capacity. In order to reach this optimal scenario, these generation units bidding price must be lower than the MCP and they bid their capacity as bidding quantity. For the rest of units, they bid their unit production cost and capacity as the bidding price and quantity respectively to avoid a loss. Let S denote the set of generator i s generation units whose production cost c k satisfy the equation (6.2), 156

157 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES where MLP k represent the minimum marginal profit that generation unit k can accept, S I. MCP c k + MLP k (6.2) Then the profit maximization problem for generator i within an hour can be represented as problem (6.3), where q k is the bidding quantity of generation unit k, here q k is equal to its capacity. max k S p k q k c k q k (6.3) subject to p k MCP k S For all generation units in set S, problem (6.3) decides an optimal bidding price p k for each of them based on the MCP in order to maximize the profit of generator i. Since their bidding prices are lower than or equal to MCP, they could win the bidding and be dispatched in its capacity. For the rest of units, they bid their unit production cost and capacity as the bidding price and quantity respectively in order to avoid a loss. The profit of generator i is then maximized since all its units in set S win the bidding and are paid based on MCP. Given the MCP, this linear programming problem (6.3) can be solved by the OPTI toolbox in MATLAB [72] [73]. But the main problem that we encounter is the fact that MCP is not available to any single generator at the time of its bid. Therefore forecasting the MCP in the wholesale electricity market is the most essential task and dorms the basis for any decision-making for the generator. The common method in the current research for predicting the market behaviours is the neural network. In current research, they only predict a single number for the MCP. But the prediction cannot be 100% precisely, even using the neural network, which means that the optimal solution in linear programming problem (6.3) cannot be ensured by a single prediction number. Therefore, in order to avoid this 157

158 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Figure 6.1: The Regression function (solid line) and its 95% confidence band (the region bounded by dashed lines) weakness, we introduce the confidence and prediction interval to the neural network. This is the first difference between the current research and ours. A classical confidence interval definition is given as follows. If a univariate sample V consists of values a 1,, a N, one can also consider an interval [φ L (V), φ U (V)] such that there is a 95% probability that a new value a N+1 drawn randomly from the population will occur within the interval. This interval [φ L (V), φ U (V)] is called a 95% prediction interval for a N+1. A confidence interval is a type of interval estimation of a population parameter; whereas a prediction interval is a type of interval estimate a single value from the population. As an example, for sample a 1,, a N, where a 1,, a N are continuously valued, the 95% prediction interval for a N+1 is given by [4]: a ± t 0.025[N 1] (sd 1 + 1) (6.4) N Where t 0.025[N 1] is the required critical value of Student's t-distribution (N-1 degrees of freedom) [114], and sd is the standard deviation of the sample [113]. 158

159 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Confidence interval and prediction interval can also be applied to regression, which also called as error bars. A set of confidence intervals constructed continuously over a rage of an input x produces a two-dimensional confidence band. Figure 6.1 illustrates an example of a 95% confidence band for a regression function. The solid line is the regression function and the region bounded by dashed lines is the 95% confidence band for a regression function. If we can obtain the predicted MCP μ and the 95% confidence interval [μ L, μ U ]for real MCP, this means that the range of the real MCP could be estimated. Generator i could set the bid price of one of its generation unit as p k, where p k μ L, this ensures there is 95% or more probability that bid price p k small than real MCP. Let S denote the set of generator i s generation units whose production cost c k satisfies equation (6.5), where k S I, μ L is the lower bound of confidence interval for real MCP. μ L c k + MLP k (6.5) Then the profit maximization problem (2) for generator i can be changed as follows, where Pr is the probability of equation p k MCP. For all generation units in set S, problem (6.6) decides an optimal bidding price p k for each of them based on the MCP in order to maximize the profit of generator i. For the rest of units, they bid their unit production cost and capacity as the bidding price and quantity respectively. max k S p k q k c k q k (6.6) subject to Pr(p k MCP) 95% k S Since we know the μ L (lower bound of confidence interval for MCP), the problem (6.6) is a linear problem which can be solved by the OPTI toolbox in MATLAB [72] [73]. But, in our mechanism, this is even more complicated when 159

160 CHAPTER 6. DEMAND BASED BIDDING STRATEGIES Figure 6.2: The process of proposed strategic bidding model the generator is bidding under an interval demand. The MCP is not a figure but an MCP function (step function) as we showed in equation (4.1). If we know the MCP function under this interval demand, the optimal bidding strategies for a generator under an interval demand can be analysed. Therefore, forecasting the MCP function under an interval demand is the most essential task to consider. So we build a forecasting model in Section which can predict a MCP function under the declared interval. This is the second difference between the current research and ours. We also propose a strategic bidding model for the generator. Here we present Figure 6.2 to show the process of our strategic bidding model for the generator. In our model, the ISO s declared interval demand of an hour and the generator i s production cost curve (step function) are the input parameters. Through using our proposed neural network forecasting model, the generator predicts a MCP function. Then, according to the predicted MCP function, the generator i selects the best bidding strategy (i.e. it decides on a bid for each of its generation units). All the bids of the generation units can be sorted by price in increasing order and represented as a piecewise constant curve. Finally the 160