Ph.D. Defense: Resource Allocation Optimization in the Smart Grid and High-performance Computing Tim Hansen

Ph.D. Defense: Resource Allocation Optimization in the Smart Grid and High-performance Computing Tim Hansen Department of Electrical and Computer Engineering Colorado State University Fort Collins, Colorado, USA outline introduction to resource allocation resource allocation in Smart Grid resource allocation in high-performance computing conclusions and future directions 1 May 8, 2015

Introduction to Resource Allocation resource allocation: assignment of limited resources to perform useful work optimal resource allocation problems, in general, are NP-Complete in high-performance computing (HPC): allocate HPC resources to parallel applications in electric power systems: allocate generation resources to energy consumers 2

Outline Resource Allocation in Smart Grid introduction to resource allocation resource allocation in Smart Grid background and motivation non-myopic home energy management system aggregator-based residential demand response demand response visualization co-simulation framework resource allocation in high-performance computing conclusions and future directions 3

4 Traditional Bulk Power System

5 Traditional Bulk Power System

6 Traditional Bulk Power System

7 Traditional Bulk Power System

Traditional Bulk Power System transmission level 8

Traditional Bulk Power System transmission level distribution level 9

Bulk Power Market owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 10

Bulk Power Market demand owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 11

Bulk Power Market demand owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 12

Bulk Power Market demand price owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 13

Bulk Power Market demand price owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 14

Bulk Power Market demand price owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 15

Bulk Power Market demand price owners of generators bid into the bulk power market as demand increases, more expensive generators are needed to meet the demand 16

17 Example Variation of Price

Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions 18

Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions load shifting: electric energy consumed by a load (e.g., electric vehicle charging) is moved from one time (e.g., peak) to another total energy consumed is approximately the same 19

Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions load shedding: electric energy consumed by a load is permanently removed from the system total energy consumed is reduced 23

Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions 26

Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions sample demand response (DR) programs: direct load control (DLC) air conditioner electric water heater time-varying pricing (indirect) real-time pricing (RTP) time-of-use pricing (ToU) 27

Outline Home Energy Management introduction to resource allocation resource allocation in Smart Grid background and motivation non-myopic home energy management system aggregator-based residential demand response demand response visualization co-simulation framework resource allocation in high-performance computing conclusions and future directions Under Preparation, Timothy M. Hansen et al., A Partially Observable Markov Decision Process Approach to Customer Home Energy Management Systems, IEEE Transactions on Smart Grid. 30

HEMS Research Overview residential home energy management system (HEMS) goal: determine when to use energy within the household to minimize the electricity bill difficulties: uncertainty in the time-varying price of electricity as a customer, the benefit of changing energy use must exceed the inconvenience caused 31

System Model Overview single residential household simulation household is in a real-time pricing market for electricity dynamically arriving appliances that need to be used goal: determine when to run each appliance to minimize the total cost of electricity 32

Real-time Pricing Model modeled after ComEd real-time pricing (RTP) in Chicago price of electricity changes every hour provided a forecast for the next day at 4:30 p.m. actual price of electricity is known at the start of the hour 33

Appliance Model desired usage pattern (W) appliances arrive into the system given a desired usage pattern models appliance energy usage of a household use queueing theory 34

Appliance Queue Model model the arrival of appliances as an M t /G/ queue arrive with Markovian probability, varying with time appliance run-times are generally distributed household modeled as an infinite capacity compute server at time t let: λ(t) be the arrival rate of the appliances l(t) be the desired load D be a random variable describing appliance run-times L be a random variable describing appliance loads λ t = l(t + E D ) E L E D 35

Example Generated Usage Pattern appliance arrivals averaged over 500 trials 36

Problem Statement given: a time-varying RTP modeled after ComEd dynamically starting appliances that statistically follow the provided usage pattern constraints: goal: each appliance has a known: power rating duration start time deadline appliances must be started before their start time deadline minimize the total cost of electricity of using all appliances 37

Comparison Methods immediate start each appliance at their usage start time (status quo) minimum forecast cost schedule appliances to run at the lowest forecast cost POMDP Gaussian estimation autoregressive estimation lower bound exact minimum cost of running all appliances 38

Markov Decision Process (MDP) MDP: decision making framework where outcomes have uncertainty goal: find a policy that maps actions to states to maximize reward 39

Partially Observable Markov Decision Process what if we do not know the underlying state? MDP partially observable MDP (POMDP) instead of states, you have belief states a probabilistic estimate of the underlying state 40

POMDP Optimization Q-value Approximation goal: maximize total reward over a long time horizon non-myopic, trade-off between immediate and total reward use Q-value approximation with Bellman s equation for state x and taken actions a let: Q(x, a) be the total expected reward R(x, a) be the immediate reward x be the next state V(x) be the optimal reward value following initial state x find the policy (action to state mapping) to maximize Q(x, a) Q x, a = R x, a + E V x x, a immediate reward future reward 41

POMDP State the underlying state is represented by: the current appliances ready to start (observable) the real-time price (unobservable) use a measurement method and filter to determine the belief state of the real-time price two measurement methods: Gaussian noise autoregressive time-series model (GARCH) use a particle filter to determine conditional probability of the state sequential Monte Carlo sampling method 42

POMDP State the underlying state is represented by: the current appliances ready to start (observable) the real-time price (unobservable) 43

POMDP State the underlying state is represented by: the current appliances ready to start (observable) the real-time price (unobservable) measurement 44

POMDP State the underlying state is represented by: the current appliances ready to start (observable) the real-time price (unobservable) measurement measurement filter 45

POMDP State the underlying state is represented by: the current appliances ready to start (observable) the real-time price (unobservable) measurement measurement filter action selector 46

AR Process autoregressive (AR) process output depends linearly on previous values at time t, let: x t be the output of the AR process c be a constant x t i be the i th previous output γ i be the coefficient corresponding to x t i m be the number of modeled coefficients ε t be the error (usually Gaussian noise) m x t = c + γ i x t i + ε t i=1 47

GARCH Process generalized autoregressive conditional heteroskedastic (GARCH) process used to model process error (e.g., AR process) at time t, let: ε t be the error σ t be the standard deviation z t be a Gaussian random variable with mean 0, standard deviation of 1 ε t = σ t z t heteroskedastic: the variance of the error (i.e., σ t 2 ) varies over time 48

GARCH Heteroskedasticity at time t, let: k be a constant 2 σ t i be the i th previous standard deviation 2 p i be the coefficient corresponding to σ t i GARCH terms P be the number of GARCH terms 2 ε t j be the j th previous square-error 2 q j be the coefficient corresponding to ε t j ARCH terms Q be the number of ARCH terms σ t 2 = k + P i=1 2 p i σ t i Q 2 + q j ε t j j=1 49

AR + GARCH Model for POMDP recall AR process: x t = c + m i=1 γ i x t i + ε t recall GARCH process: ε t = σ t z t for POMDP, (number of coefficients from literature): m = 504 ε t is a GARCH process P = 1, Q = 3 50

Action Selector Rollout recall: Q x, a = R x, a + E V x x, a immediate reward future reward a = {now, later} R x, a cost of using appliance now E V x x, a expected cost of waiting determined with a particle filter and the AR + GARCH process 51

Example Result January 2011 50 trials of a one-month long simulation of January 2011 each trial is compared to the lower bound 52

53 Change in Energy Usage

Contributions design of a non-myopic HEMS using a sequential decision making technique (POMDP) creation of a new appliance energy usage pattern based on queueing theory comparison of the POMDP HEMS against the lower bound using month-long simulation studies 54

Outline Aggregator-based Demand Response introduction to resource allocation resource allocation in Smart Grid background and motivation non-myopic home energy management system aggregator-based residential demand response demand response visualization co-simulation framework resource allocation in high-performance computing conclusions and future directions Timothy M. Hansen et al., Heuristic Optimization for an Aggregator-based Resource Allocation in the Smart Grid, IEEE Transactions on Smart Grid, 10 pages, accepted 2015, to appear. DOI: 10.1109/TSG.2015.2399359. 55

Recall: Demand Response demand response: peak demand reduction by shifting or shedding loads in response to system or economic conditions sample demand response (DR) programs: direct load control (DLC) residential assets time-varying pricing (indirect) real-time pricing (RTP) time-of-use pricing (ToU) aggregator-based demand response 56

Cyber-physical Social System (CPSS) transmission level aggregator DR distribution level 57

Aggregator-based Residential Demand Response for-profit aggregator entity offers all customers time-varying price for participating in DR customer customer incentive price (CIP) competitive rate owns a set of DR assets schedulable smart appliances pays CIP for allowing aggregator use of DR assets generally cheaper than utility price 58

Assumptions price is exogenous at the load levels one aggregator changes, bulk price changes marginally retail electricity market is fully deregulated allows customer to choose supplier control and communication infrastructure exchange of information and control of DR assets customer willingness to participate 59

Smart Grid Resource Allocation Smart Grid Resource Allocation (SGRA) from the point-of-view of the aggregator given set of customers information about customer loads constraints customer constraints availability of loads to be rescheduled incentive pricing requirements system objective aggregator find customer incentive pricing and schedule of loads to maximize aggregator profit 60

Simulation Setup 5,555 customers each customer has a threshold for determining whether the CIP offers enough discount each customer defines a time period to which each load can be rescheduled 56,498 schedulable loads probabilistically generated to simulate use of an average household pricing data bulk power spot market price from PJM utility price from ComEd genetic algorithm used as optimization method 61

Results Demand Response Load Shifting peak reduction of 2.66MW (12.6%) aggregator profit $947.90 total customer savings $794.93 62

63 Customer Incentive Pricing

Contributions alternative customer pricing structure customer incentive pricing heuristic optimization framework mathematical models for the customer and aggregator entities large-scale test simulation consisting of 5,555 customers and ~56,000 schedulable loads used real pricing data from ComEd and PJM showed that aggregator optimizing for economic reasons: 64 benefits participating customers benefits aggregator benefits non-participating customers system peak reduced as a common good

Outline Demand Response Visualization introduction to resource allocation resource allocation in Smart Grid background and motivation non-myopic home energy management system aggregator-based residential demand response demand response visualization co-simulation framework resource allocation in high-performance computing conclusions and future directions Timothy M. Hansen et al., A Visualization Aid for Demand Response Studies in the Smart Grid, The Electricity Journal, vol. 28, no. 3, pp. 100-111, Apr. 2015. 10.1016/j.tej.2015.03.008 65

Power Systems Visualization as new power systems technologies become more pervasive, the amount of data available increases dramatically abundance of data makes it difficult to: assess power system states in a fast, user-friendly manner find key information application of visualization methods provide unique insight and information to users goal: design new visualization methods for evaluating demand response programs 66

System Model use the system model and results from the aggregator-based DR work aggregator profit, P, is composed of three pieces: N income received by aggregator for selling negative load to spot market at times load rescheduled from S income received by aggregator for selling electricity to customer at times load scheduled to B cost paid by the aggregator for buying electricity from spot market at times load scheduled to P = N + S B 67

Visualization Method Overview DR actions normally visualized as 2D load curves three new methods for DR visualization: heat map representation 3D load curve spatially distributed DR 68

Heat Map Representation color at a point (x,y) indicates the load moved from time x to time y (magnitude given by associated color bar) 69

3D Load Curve Representation at each surface point (x,y,z), the magnitude z represents the load moved from time x to time y 70

Spatially Distributed GIS Representation mapped a distribution system spatially onto Fort Collins spread aggregator customers across system 71

Profit Heat Map shows the difference in profit between the forecast and actual real-time price recall: N income, rescheduled from B cost, scheduled to 72

Contributions design of two new temporal visualization techniques for DR that answer: did the DR plan work effectively? when did the DR entity make a profit or loss? how does differences in the real-time price affect profit margin? creation of a spatial visualization technique for DR that answers where in the distribution system did the DR affect load discussion of how the methods can be used to analyze the effectiveness of DR optimization techniques 73

Outline Co-simulation Framework introduction to resource allocation resource allocation in Smart Grid background and motivation non-myopic home energy management system aggregator-based residential demand response demand response visualization co-simulation framework resource allocation in high-performance computing conclusions and future directions Timothy M. Hansen et al., Bus.py: A GridLAB-D Communication Interface for Smart Distribution Grid Simulations, IEEE Power and Energy Society General Meeting 2015, 5 pages, accepted 2015, to appear. 74

Co-simulation in Power Systems co-simulation: multiple individual tools, each specializing in a specific domain, interact while running simultaneously 75

Bus.py introduce bus.py a transmission-level bus simulator and communication interface 76

Bus.py introduce bus.py a transmission-level bus simulator and communication interface enables co-simulation between: 77

Bus.py introduce bus.py a transmission-level bus simulator and communication interface enables co-simulation between: 78

System Model aggregator controls the individual loads within a household each house is represented on a GridLAB-D distribution feeder GridLAB-D a PNNL distribution system simulator 79

Peak Load Reduction reduction in peak load of 19.2 kva (total available at peak time) 80

Cost Minimization in Time-of-Use Pricing change in schedulable load when minimizing cost in a time-of-use market 81

Contributions design of bus.py, a software transmission bus interface for use in Smart Grid co-simulation studies demonstration of bus.py interfacing with GridLAB-D simulating a small set of customers on a distribution feeder and an aggregator entity 82

Outline Resource Allocation in HPC introduction to resource allocation resource allocation in Smart Grid resource allocation in high-performance computing combined dual-stage resource allocation conclusions and future directions Timothy M. Hansen, Florina M. Ciorba et al., Heuristics for Robust Allocation of Resources to Parallel Applications with Uncertain Execution Times in Heterogeneous Systems with Uncertain Availability, in 2014 International Conference on Parallel and Distributed Computing, pp. 536-541, July 2014, received the best paper award. Florina M. Ciorba, Timothy M. Hansen et al., A Combined Dual-Stage Framework for Robust Scheduling of Scientific Applications in Heterogeneous Environments with Uncertain Availability, in 21st Heterogeneity in Computing Workshop, pp. 187-200, May 2012. 83

Overview 84

Motivation obtain the most performance from a system given a batch of parallel scientific applications i.e., maximize the investment in the computing infrastructure computing systems today are often heterogeneous in nature scheduling becomes more difficult with: uncertain application execution times e.g., varies based on input data uncertain processor slowdown e.g., system jitter, sharing of resources 85

Optimization Methods processor balance random (PB-R) smart (PB-S) robustness floor min-min (RF Min-Min) min-max (RF Min-Max) duplex (RF Duplex) upper bound on robustness 86

(higher is better) robustness Typical Result 32 applications, 4 processor types, high system slowdown each scenario was run over 48 trials between trials, the application characteristics changed i.e., parallel speedup and execution time 87

Performance Trend Number of Processor Types (lower is better) normalized robustness normalized robustness = (upper bound robustness)/upper bound a value of zero is better (i.e., equal to upper bound) 32 applications, low system slowdown RF Duplex used for these results 88 # of proc. types

Contributions design of a model for moldable parallel applications with stochastic execution times running in a heterogeneous computing environment design of a new robustness metric that models the effect of two uncertainties design and analysis of three novel iterative-greedy heuristics that mitigate the effects of the two uncertainties 89

Outline Conclusions introduction to resource allocation resource allocation in Smart Grid resource allocation in high-performance computing conclusions and future directions 90

Conclusions: Resource Allocation in Two Domains resource allocation in Smart Grid system-view with aggregator-based demand response individual house view with POMDP HEMS resource allocation in HPC dual-stage scheduling of applications to HPC systems the demand response methods were shown to reduce peak demand reduction in peak demand can: reduce the cost of electricity reduce the output of dirty diesel peaking generators defer building new transmission lines 91

Future Directions combine the system and end-user approaches for a unique multi-level multi-scale end-to-end DR simulation leverage the use of HPC study market responses (as opposed to assuming an exogenous price) model more end-user assets study sustainability through the use of long-term simulations new metrics such as reduction in capacity factor of dirty generators 95

Questions and Discussion collaborators: CV available: www.engr.colostate.edu/~hansentm 96