A Model For The Effect of Aggregation on Short Term Load Forecasting

Transcription

1 A Model For The Effect of Aggregation on Short Term Load Forecasting Raffi Avo Sevlian and Ram Rajagopal Department of Electrical Engineering, Stanford University Department of Civil and Environmental Engineering, Stanford University Abstract In this work, we propose a simple empirical scaling law that describes load forecasting accuracy at different levels of aggregation. e show that for the short term forecasting problem, aggregating more users will improve the relative forecasting performance up to a point. Beyond this point, no more improvement in relative performance can be obtained. I. INTRODUCTION Load forecasting is a very mature field and has focused primarily on predicting loads for transmission power buses with aggregate loads in the order of gigawatts or load service regions with aggregates in the order of several megawatts or more: [15], [8]. The recent deployment of advanced metering infrastructure (AMI) and widespread adoption of modern communication systems has enabled the acquisition of load data from individual homes and commercial buildings in real time. Coupled with the increasing adoption of consumer-side technologies such as storage, microgrids requires forecasting at smaller aggregation levels. Typical homes consume 1 to 2 kh of energy during an hour and commercial buildings are in the 10kh to 100kh range. Load forecasting can be broadly classified into three groups according to the forecast horizon and forecasted quantity: very short term forecasting (predicting loads 15 min to 1 hour ahead); day ahead forecasting (predicting hourly loads for the next day) [9]; and day ahead forecasting of peak and total loads. Long term forecasts of up to a year are also possible [10]. Accurate very short term forecasting is a basic requirement for the operation of reliability services for microgrids and efficient operation of technologies such a storage. Typical forecast errors at the aggregation level of substations and areas are quite low (1% to 2%). In contrast, recent work shows that for individual homes, forecast errors are much higher (up to 50%) indicating a lack of predictability. Significant effort has been expended in improving such forecasts, but the observed performance differences when predicting loads of different sizes has not been explored. It is folk knowledge that the mean absolute percentage error () should decrease with increase in load size, as variability reduces. This papers starts the development of a fundamental understanding of how forecast errors scale with aggregation levels. In particular, we quantify the effect of aggregation by proposing a set of scaling laws relating kh load level and forecasting error. e focus in very short term forecasting and predict consumption an hour ahead. Using real data from more than 100,000 residential customers, we construct datasets of varying aggregation levels and build a model for the scaling of forecasting error with respect to aggregate size. The remainder of the paper is organized as follows. Section II develops a model for aggregation and proposes a set of scaling laws for common forecasting metrics. The scaling laws are then /14/$ IEEE verified in Section III, IV. e touch upon providing a justification for this effect in Section V. II. MODELING LOAD AGGREGATION A. Qualitative understanding of load aggregation 1 user 5 users 20 users 40 users 60 users 80 users Fig. 1. Hourly consumption level for various aggregation levels. Consumption pattern of a single user generally has little structure to be exploited. Aggregating more and more users smoothes the signal so that it can be more predictable. Aggregation level of 20 or more residential customers shows a predictable pattern. Plots are not scaled to each other. Aggregation reduces the inherent variability in consumption resulting in increasingly smooth load shapes. Figure 1 illustrates this effect. It is clear in this example that the higher aggregation levels are easier to predict. Yet, it is less clear how to quantify the improvements in forecasting. The main goal of this paper is to develop an appropriate scaling model for forecasting performance with respect to aggregation size. In particular, we identify the most appropriate way to measure aggregation and then propose a simple model to explain the scaling phenomena. e start by reviewing typical performance metrics used for quantifying forecasting. B. Forecasting performance metrics A very popular forecasting performance metric is the Mean Absolute Percentage (). Given the true measurements x = {x(1),...,x(t)} and the corresponding forecasts ˆx = {ˆx(1),...,ˆx(T )}, (x, ˆx) is defined as (x, ˆx) = 100 T T x(t) ˆx(t) x(t). (1) t=1

2 is generally chosen as one of the reported metrics in forecasting literature as it is a relative metric and potentially would allow comparing studies that rely on distinct consumption data. However, as we show in our work, this is not valid for short term forecasting of electricity demand. The level of aggregation needs to be taken into account. C. Forecasting scaling laws In the following section our treatment should not depend on which relative metric we use. So the following proposed law is applicable to and CV. Consider a set of N customers with consumption given by a time-series x n (t). The time average consumption for customer n is n = 1 T T t=1 x n(t). e randomly select a subset A {1, 2,...,N} of customers and form a group that consumes x A (t) = n A x n (t), (2) which has mean consumption A = n A n. e build an hour ahead predictor for the aggregate time-series x A that outputs the predicted sequence ˆx A and evaluate (x A, ˆx A ). e postulate that the population average scales as a function of A according to ( )=E[(x A, ˆx A ) A = ] (3) = p + α 1. (4) Some key observations about this empirical law are: is the average over all groups with mean consumption. The choice of using mean consumption as a proxy for group size is deliberate. An alternative choice would be the number of customers in the group. But we observe that in fact customers with higher absolute consumption tend to be more predictable. (c) α 0 / p represents improvement in relative error from aggregation. It represents the underlying variance of the signal that is reduced from aggregation. α 1 is the irreducible relative error. (e) The exponent p shows empirically how quickly the relative variance is reduced. ly, p =1, but as we show this is empirically not the case. The proposes scaling law segments the forecasting problem into two regimes: hen α 1, relative error improves considerably due to aggregation. e can approximate e.q. (4) as ( )= /. hen α 0 / α 1, ( ) α 1. There is no improvement in forecasting from aggregation. In the remainder of the paper we show how this scaling law can be applied to short term load forecasting experimenting with various machine learning methods relying on a real dataset. III. EXPERIMENT SETUP A. Description of data The data used in this paper is anonymized hourly power consumption from the Pacific Gas and Electric company representing northern California. The data consists of N = 116, 438 residential customers for one year (T = 8760 samples per customer). The Energy Consumption [kh] M T TH F ST Day of eek # Users Mean User Consumption [kh] Fig days of consumption for single residential customer. Empirical Histogram of yearly mean consumption of all residential customers. data represents 408 zip codes in California providing geographic diversity of consumption. The average consumption for this set of customers is 1.05 kh. The average consumption per customer never exceeds 4kh. Figure 2 illustrates a typical weekly consumption profile of a household, where there is high variability in consumption. Figure 2 shows the yearly average consumption of the population in histogram form. This is the empirical distribution of n in the population. B. Forecasting models e study the scaling using three commonly used methods for short term load forecasting: Seasonal Auto Regressive Moving Average (SARMA), Support Vector Regression (SVR) [6] and Feed Forward Neural Networks [2]. Each model is described next. 1) Seasonal Auto Regressive Models: Seasonal Auto Regressive Moving Average (SARMA) [4] fits a linear model between current power demand and previously observed demands. A general form SARMA(p, q) (P, Q) s captures autoregressive order p, P and moving average order q and Q. In this work, we apply a restricted class with no moving average component (q,q=0). The generative model is p P x(t) = θ i x(t i)+ φ i x(t sk)+ɛ(t). (5) k=1 k=1 e assume that ɛ(t) N(0,σ 2 ) is a independent and identically distributed normal variable. The term p corresponds to the order of the autoregressive (AR) component. Normally, values of p larger than 4 are not recommended [12], since it will lead to over-fitting. Seasonality is captured by the parameters s, and P. For this work, seasonality of s =24hours is used. The SARMA model is applied at each time step by learning the linear model using a pre-set model size. This constitutes an adaptive SARMA model. 2) Support Vector Regression: Support Vector Regression (SVR) works by building a non-linear learning method for a training dataset {(x 1,y 1 ),...,(x N,y N )}. The training set comprises of N response y i and predictor x i pairs. The SVR data fitting method will output a predictor of the form: ŷ i = w T Φ(x i )+b. A given training factor variable x i will be mapped to a higher dimensional space with a kernel function Φ(x i ).Thevaluesw and b are determined in the training process. The user will set the appropriate kernel function as well as particular tolerances in the training process. 3) Feed Forward Neural Network: Feed forward neural networks are a subset of artificial neural networks where each layer

3 of the network connects directly to each forward layers. FFNNs build a general non-linear mapping between the input and output of the training data of the form and can handle any number of hidden layers. For the case of single hidden layer, the input/output relationship can be expressed as the following. ( ) f(x) =Σ M2 m w 2=1 m 2 Ψ m2 Σ M1 m w 1=1 m 1,m 2 x m1 + w m2,0 (6) Here, inputs x m1 are weighted with w m1,m 2 andsuppliedtoa set of activation functions Ψ m ( ). The activation function and the choice of inputs must be performed in the model selection step. During the training phase, the weight s are computed to minimize the training error. C. Generating Aggregate Residential Consumption Aggregate consumption time series are generated by randomly sampling consumers from the total available smart meter dataset. 56 group sizes were chosen ranging from single users to 100,000 users. For each chosen group size A, 50 time series are generated by randomly choosing A elements from the database. Random selection ensures geographic diversity. Based on our process a total of 2800 time series are generated. The mean consumption for the different groups ranges between 1 kh and 100 Mh and peak consumption ranges between 3 kh and 180 Mh. IV. EXPERIMENTAL RESULTS A. Empirical with Aggregation Level Fig. 3. Mean absolute percentage error () for varying size of residential consumers. SARMA model M 1 is applied to data. against Mean Load for each experiment is shown in green markers. Best fit (blue) has p =0.88. Dashed line indicated error scaling with ideal aggregation effect (no irreducible error and p =1). Transition point, = 1810 kh. Irreducible error is α 1 =1.52. e start our analysis by investigating the performance of the model SARMA(1, 0) (1, 0) 24 in predicting the data one hour ahead. For each random choice of time-series, compute the forecast ˆx A ),(x A, ˆx A ) and hourly mean A to form a pair ( A, (x A, ˆx A )). Figure 3 shows how scales with the mean load of the grouped time series ( A ). Eq. (4) is fit using least squares using the observed pair ( A, (x A, ˆx A )). The fit parameters are shown in Table I. Figure 3 traces the curve obtained from data. Suppose no bias was observed in the forecaster predictions. Then the curve in red represents the scaling law. Notice that errors would continues to decrease beyond the curve with a bias. The fit identifies two regimes of aggregation: 1 kh to 1 Mh: At this load level (1 to 1000 homes), relative error improvement due to aggregation dominates the decrease in. This corresponds to α 1. The approximation ( ) α 0 / p /2 holds quite well in this regime. Visually the red line matches the blue fit and green markers. 2 Mh to 100 Mh: In this regime, the modeling error for a given forecaster (bias) dominates the averaged aggregation error. In this case α 1 and we can approximate the decrease with ( ) α 1. An important observation is that to achieve 12% error only about 20kh of aggregation (roughly 20 homes). This quantity is quite meaningful since 20 residential consumers corresponds to loads connected to a small distribution transformer. In fact, at the level of 20 users, more finely tuned forecasting methods could potentially achieve even more accurate forecasting. Forecasting accuracy of a single transformer can be of use when integrating smart meter data and line measurements in outage detection as done in [16]. B. Linear Scale Observations Group Mean [kh] Means Group Size Group Size Fig. 4. boxplot of randomly generated groups of N customers. Group mean load vs number of customers. e can also consider forecasting accuracy of with respect to the number of consumers. Figure 4 shows how group size and mean load relate. Data from all sampled groups is plotted. The relationship is close to linear but the dispersion of means increases as group size increases. Figure 4 plots empirical against group size measured as number of homes in the aggregate. Forecasting individual residential loads is highly inaccurate. The mean for individual residential consumer is 29%. This is very similar to results presented in previous work. Note also that at the individual level, there is very high variability. The median error is 33% while the 25% to 75% quantiles are 23% 56%. This large variability is explained by noting that the average hourly consumption for a single home ranges from 1kh to 4kh. Plugging these values into Eq. (4) and using the parameters for model M 1 from Table I, we obtain that the ranges from 24% to 45% which is close to the quantiles in the data. This gives empirical evidence for the validity of chosen kh as the baseline. Accuracy at the larger levels ( 2000) usersisclose to 2% which is not very different from the scale of 50Mh.

4 C. Critical Load The transition point (critical load) is defined as the unique solution to p = α 1. (7) This gives us a method to automatically identify the two regimes of the behavior. From the values shown in Table I for model M 1 is 1.8 Mh of load. This critical load has significant implications. The 100 kh to 1 Mh range encompasses most commercial buildings. Any work on forecasting short term energy demand for buildings should keep the aggregation level in mind. For example when comparing one research paper using a 500 kh average load dataset to a paper using a 25 kh average load dataset, it is difficult to draw meaningful conclusions. D. Comparison of Different Models Model p TABLE I SCALING LA FIT FOR α1 (95% CI) M ( ) 2250 M ( ) M ( ) M ( ) 168 M ( ) 498 (c) (d) Fig. 5. Approximate model fit to ( ). Model M 2 is a seasonal ARMA model: (1, 0) (1, 0) 24. Model M 3 is a seasonal ARMA model: (2, 0) (3, 0) 24.(c)M 4 (SVR) and (d) M 5 (FFNN). Note that for short term forecasting, non-linear methods might not always provide the most accurate methods. They are also more difficult to train. In Section IV-A only a single forecasting model (M 1 )isshown with associated model fitting. (i.e. Figure 3). Here we show that that a number of other forecasting methods will behave similarly. The three classes of models we applied described in III-B. Models M 1, M 2, M 3 are standard SARMA models of varying orders. For a single hour ahead forecaster, we see that under the fit parameters M 3 has the best asymptotic performance. Figure 5 shows the log() vs log(mean Load) for the different models used. Also, Table I compares values attained from fitting the data with the approximate model. In comparing different models, there are two important points to consider. (1) Betweenen different models, variation in the α 0 are not very different. (2) The aggregation independent terms α 1 should be taken to represent a more fundamental measure of forecast accuracy. Notice in general that the lower the α 1, the larger the critical load. This is seen clearly for the FFNN and SVM models. Since they have much higher irreducible error, the calculated critical load is an order of magnitude lower than that of M 3. This is because the critical load calculation is very sensitive to the α 1. Also, this can be seen visually: the aggregation-error curve for M 3 flattens out far to the right The model that seems to work best across all scales of loads is the M 3 which is the the most complex seasonal linear model used among the others. Although not documented here, higher order seasonal models were applied with worse results. It is interesting to note that the non-linear estimation methods: SVR and FFNN perform quite poorly. As indicated in Figure 5 the error s flatten at 4% for the SVR and 2.4% for FFNN. Under more finely tuned analysis, SVR has been reported outperform linear models in some work [6]. It has also been been shown to perform poorly as well [14]. The experience of using non-linear methods for short term forecasting produced errors larger than that of linear models. Additionally, the non-linear methods are more difficult to train, understand and ensure stability of training. Another major issue for large scale use is that they are generally too slow and unstable to train adaptively: i.e. retrain the model for every out of sample forecast. E. Previous work on Short Term Electricity Load Forecasting 100 % 10 % 1 % 0.1 % [7][17] [14] [6] 95 % Int. [6] 1 G 10 G [13] [1] [3] [11] [5] [2] Fig. 6. Forecasting performance on varying levels of aggregation using seasonal model M 3. Superimposed on experimental data is previous work on short term load forecasting. Data suggests that previous work agrees with model suggested here.

5 Forecasting smart meter data from homes and small businesses is a recent goal in analytics and power communities. Recent work shows a fundamental limitation to the predictability of individual customers. [6] performs one hour ahead forecasting using hourly energy data. They implement a variety of machine learning algorithms on 3 datasets. The first dataset is a commercial building used in the Building Energy Predictor Shootout hosted by ASHRAE. The mean load for this dataset is approximately 700 kh. Three residential homes are used as well with a mean consumption of around 1.5 kh. The methods achieve a of 1.61% to 13.41% for small businesses and 15% to 30% for residential data. In [17], a number of machine learning methods are used on three homes. The mean consumption level is approximately 1 2 kh They report varying improvement as compared to the baseline load forecast. Estimates of their reported relative error are on the order of 25%. In [14] the authors instead use machine learning methods to forecast home peak demand. Their results show that seasonal autoregressive models are the most simple and most accurate over the range of homes used in evaluating. They report and average error rate of 30% for the seasonal autoregressive models, and much higher error rates (50-70%) for more advanced data mining methods. In [7] the authors apply a kalman filter based forecaster and achieve 30% relative error on a home with a mean kh usage of 0.8 kh. At the far opposite end of aggregation a number of work report very low errors. In [1] the authors use an artificial neural network to forecast a mean load of 2.5 Gh. The achieved ranges from 1.73% %. In [3] the authors apply wavelet multiscale decomposition based autoregressive approaches. They attain values of 0.7% to 3.5% depending on the exact method developed. The data used is that of a state controlled utility with a mean load of 9 Gh. In [13], the authors apply artificial neural networks to the Okinawa Electric Power Company load data which has a mean consumption of 800 Mh. The they achieve ranges from 1.11% %. In [2] the authors apply artificial neural networks to electricity load data from the western operation al area of the Saudi electric company. The mean consumption is close to 8 Gh. The they achieve ranges from 0.81% %. In [5] the authors apply a novel ANN architecture to two utility datasets with peak loads of 4.4 Gh. They report a one hour ahead forecast of 0.8% to 1.5%. Similarly, [11] applies artificial neural networks to attain an error rate of 1.7%, for a load of 7 Gh. Figure 6 shows the previous work on short term forecasting superimposed on the scaling curve obtained using model M 3. It is clear that the work on the level of homes 1 to 5 kh, have results that are below the average and are within the 95% confidence interval of the experimental fit. For [6], the dataset with bigger loads does much worse than our fit predicts. For the previous work at Gh level data, the accuracy is neither better or worse than the fit. V. AN ARGUMENT FOR AGGREGATION VS ERROR CURVE If we assume that every user is the sum of two components: x n (t) = p n (t) +e n (t). p n (t) can be though of as a daily routine specific to each energy consumer. Additionally, e n (t) is a deviation from an underlying daily pattern which is difficult to estimate. is approximately residual variance/signal mean. Therefore if variance grew linearly with the size of the problem (assume signal mean grows linearly) we have 1 N dependence between error and kh. However, if at some 1 point error variance begins to grow quadratically then we end up with the curves where error no longer improves. This occurs for two reasons: (1) specific correlation structures in the deviation e n (t) among the population. (2) bias between a forecast ˆx and the estimable daily routine vectors p n (t). VI. CONCLUSION AND CONSEQUENCES OF AGGREGATION Analyzing the dependence of forecasting provides some interesting insights as well as some questions regarding forecasting. As shown in Figure 3 and 5 we attribute the flat portion of the erroraggregation curve to effects which grow the error quadratically with the size of the problem. This brings about the question, is it possible to get close to the ideal aggregation line? Perhaps this motivates the need to segmentation and aggregation which separates the problem into smaller subproblems of loads that are more like each other and possibly a new application for smart meter data. Currently, metering data is used only for billing. Perhaps with intelligent use of these data, there can be a push for more accurate forecasters, with provable lower bounds. REFERENCES [1] G.A. Adepoju, S.O.A. Ogunjuyigbe, and K.O. Alawode. Application of neural network to load forecasting in nigerian electrical power system. The Pacific Journal of Science and Technology, 8(1):68 72, [2] A.J. Al-Shareef, E.A. Mohamed, and E. Al-Judaibi. One hour ahead load forecasting using artificial neural network for the western area of saudi arabia. International Journal of Electrical Systems Science and Engineering, 1(1):35 40, [3] D. Benaouda, F. Murtagh, J.L. Starck, and O. Renaud. avelet-based nonlinear multiscale decomposition model for electricity load forecasting. Neurocomputing, 70(1): , [4] G. Box, G. M. Jenkins, and G. C. Reinsel. Time series analysis: forecasting and control. iley Publisher, [5] I. Drezga and S. Rahman. Short-term load forecasting with local ann predictors. IEEE Transactions on Power Systems, 14(3): , [6] R. E. Edwards, J. New, and L. E. Parker. Predicting future hourly residential electrical consumption: A machine learning case study. Energy and Buildings, 49: , [7] M. Ghofrani, M Hassanzadeh, M. Etezadi-Amoli, and MS. Fadali. Smart meter based short-term load forecasting for residential customers. In North American Power Symposium (NAPS), 2011, pages 1 5. IEEE, [8] C. Harris. Electricity markets: pricing, structures and economics, volume 565. iley Publishing, [9] T. Hong. Short term electric load forecasting. PhD thesis, North Carolina State Univ., [10] T. Hong, J. ilson, and X. Jingrui. Long term probabilistic load forecasting and normalization with hourly information. IEEE Transactions on Smart Grid, 5(1): , Jan [11] K.Y. Lee, Y.T. Cha, and J.H. Park. Short-term load forecasting using an artificial neural network. IEEE Transactions on Power Systems, 7(1): , [12] A. Pankratz. Forecasting with univariate Box-Jenkins models: Concepts and cases, volume 224. iley Publishing, [13] T. Senjyu, H. Takara, K. Uezato, and T. Funabashi. One-hour-ahead load forecasting using neural network. IEEE Transactions on Power Systems, 17(1): , [14] R. P. Singh, P. X. Gao, and D. J. Lizotte. On hourly home peak load prediction. In IEEE International Conference on Smart Grid Communications, [15] R. eron. Modeling and forecasting electricity loads and prices: A statistical approach, volume 403. iley. com, [16] Y. Zhao, R. Sevlian, R. Rajagopal, A. Goldsmith, and H.V. Poor. Outage detection in power distribution networks with optimally-deployed power flow sensors. In Power and Energy Society General Meeting (PES), 2013 IEEE, pages 1 5, July [17] H. Ziekow, C. Goebel, J. Struker, and H. Jacobsen. The potential of smart home sensors in forecasting household electricity demand. In IEEE International Conference on Smart Grid Communications, 2013.