Streamflow Prediction Based on Least Squares Support Vector. Machines

Transcription

1 Streamfow Prediction Based on Least Squares Support Vector Machines Nian Zhang Chares Wiiams Esther Ososanya Wagdy Mahmoud University of the District of Coumbia Department of Eectrica and Computer Engineering 4200 Connecticut Ave. NW Washington, DC, Abstract: We used the east squares support vector machine (LS-SVM) agorithm to forecast the future streamfow discharge using the past streamfow data and gage height, which is cosey reated to reguarization networks and Gaussian processes. A Gaussian Radia Basis Function (RBF) kerne framework was buit on the data set to optimize the tuning parameters and to obtain the moderated output. The prediction capabiity with different kerne functions was aso expored to evauate the impact of the streamfow discharges and gage height for ong-term prediction of fow rates and its accuracy. The training process of LS-SVM invoves the seection of both kerne parameters and reguarization constants. The tuning parameters were found by using a combination of couped simuated anneaing (CSA) and a standard simpex method. The CSA finds good starting vaues and these vaues were passed to the simpex method in order to fine tune the resut. One of the parameters, γ is the reguarization parameter, determining the trade-off between the training error minimization and smoothness. The other parameter, σ represents the squared bandwidth. The USGS rea-time water data at various ocations were used as time series input. 50% of the data were used for training, and 50% were used for testing. The experimenta resuts showed that the LS-SVM agorithm is a reiabe and efficient method for streamfow prediction, which has an important impact to the water resource management fied. Key words: Support vector machine, water quantity prediction, time series prediction Introduction: Land deveopment activities inevitaby change watershed conditions, primariy due to an increase in the impervious area through paving, construction, adding drainage systems and remova or ateration of vegetation which resuts in water quantity and quaity probems of oca receiving bodies. Reiabe estimation of stream fows at various ocations is very important from the water resources management viewpoint. The study area is focused on the Potomac River watershed, as shown in Fig. 1. The Potomac River is one of the east dam-reguated arge river systems in the eastern United States [1]. Approximatey 90% of DC area drinking water comes from Potomac River. The Washington Aqueduct is ocated directy adjacent to the Potomac River, as shown in Fig. 2. It produces drinking water for approximatey one miion citizens iving, working, or visiting the District of Coumbia, Arington County, Virginia, and the City of Fas Church, Virginia, and its service area [2].

2 Given the existing fow conditions of Potomac River, there is need to anayze the fow conditions at specific ocations for future fow, specificay streamfow rate, and a reiabe estimate under changing cimactic conditions. It is extremey important to investigate state-of-the-art computationa inteigence techniques that have the potentia to improve the forecasting of cycic natura events, ike stormwater runoff or soar radiation eves. Method: Support Vector Machines (SVMs) are a powerfu kerne based statistica earning methodoogy for the soving probems of noninear cassification, pattern recognition and function estimation [3]. Least Squares Support Vector Machines (LS-SVM) are an advanced version of the standard SVMs which incorporates unsupervised earning and recurrent networks. Recent deveopments of LS-SVM are especiay reevant to the fieds of time series prediction, kerne spectra custering, and data visuaization [4]-[14]. The preiminary resuts show that the LS-SVM modeing method is promising for time series prediction, thus we want to study the present a current LS-SVM toobox run through Matab to impement a number of LS-SVM agorithms. Support Vector Machines are a new and potentia data cassification and regression instrument. The basic idea of SVM is based on Mercer core expansion theorem which maps sampe space to a high dimension or even unimited dimension feature space (Hibert space) via noninear mapping φ. And it wi boi agorithm which searches for optima inear regression hyper pane down to a convex programming probem of soution of a convex restriction condition. And it wi aso obtain overa situation optimum soution so as to use the method of inear earning machine in feature space to sove the probem of high-degree noninear regression in sampe space [15]. In Least Squares Support Vector Machines, the probem of optimization is described as foows: min w,b,ε L w, b, ε = 1 w 2 + c 2 ε 2 2 i=1 i (1) Such that: y i = w t x i + b + ε i (i=1,2,,) The extreme point of Q is a sadde point, and differentiating Q can provide the formuas as foows, using Lagrangian mutipier method to sove the formuas: i=1 (2) = w w i x i = 0 b = i= 0 i=1 = wt x i + b + ε i y i = 0 = Cε ε i i = 0 i From formuas above: 1 2 i=1 i (x i ) j x j j =1 2C i=1 i + b i=1 i = i=1 i y i (3) The formua above can be expressed in matrix form: 0 e T e Ω + C 1 I b = 0 Y (4)

3 T where e = [1,,1] x Ω ij = K x i, x j = (x i ) T (x j ) Formua (4) is a inear equation set corresponding to the optimization probem and can provide us with α and b. Thus, the prediction output decision function is: y x = i=1 i K x i x + b (5) where K (x i,x) is the core function. We used the LS-SVM method to cacuate and predict the USGS water data, specificay using time-series data prediction. After oading the data into Matab, we first buid the training and testing sets from the data. Next we cross-vaidate based upon a feed-forward simuation on the vaidation set using a feed-forwardy trained mode. This wi suppy us with the tuning parameters: γ (gamma) which is the reguarization parameter and σ2 (sigma squared) or the squared bandwidth. Once the parameters are cacuated, we are abe to pot the function estimation or use the predict function to predict future vaues of the data. By using ony a subset of the tota data avaiabe, we can compare the predictions against rea vaues to see how accurate the prediction is. Experimenta resuts: We used a sampe set of about 35,000 data points, a taken at a reguar time intervas. The discharge is the voume of water fowing past a certain point in a water-fow. For exampe, the amount of cubic feet passing through a drain per second is a measure of discharge. Fig. 3 is a pot of the discharge vs. time. To test LS-SVM predictions on the water data we seected a random portion of the discharge data, 500 data points from the origina sampe of about 35,000. The LS-SVM agorithm is known to be very resource efficient, meaning it can process arge amounts of data without using too much processor or memory power. By even this agorithm woud take a very ong time to process more than a few thousand data points. After oading the data, we tune the hyper-parameters gamma and sigma squared with the tunessvm command, which generates the foowing vaues after 10 iterations, as shown in Fig. 4. Once the hyper-parameters are tuned, we just assigned apha and b with the trainssvm command and finay use the predict function to make a prediction for the next set of vaues. The fina pot generated is shown in Fig. 5, with rea USGS discharge datapoints shown in bue whie the LS- SVM prediction is the red ine. Concusion: This paper documents a Least Squares Support Vector Machine impementation of a prediction agorithm used for time series data. This method expored a new kerne-based soution for monitoring and anayzing this type of data set. The experimenta resuts demonstrated that the proposed LS-SVM based predictive mode and the training agorithm ensure an accurate prediction of LS-SVM, and by association any natura measurabe system. This provides an exceent prediction method for this type of data, and if correcty impemented can be an invauabe too in predicting natura weather events. Even outside of storm-water and soar

4 radiation appications, this agorithm coud be very usefu to engineers who wish to deveop a resource efficient prediction mode for any quantifiabe data set. By continuing this research and testing a wider variety of data sets, we can get a better understanding to the strengths and weaknesses of this system. Whie it is a reativey new method within the computationa sciences, LS-SVM is a very promising and exciting avenue for computer scientists interested in regression, function estimation, cassification, and prediction. Acknowedgments: The authors woud ike to express thanks to the University of the District of Coumbia STEM Center (NSF/HBCU-UP/HRD ) grant and DC Water Resources Research Institute (WRRI) Grant. References: 1. Potomac Conservancy, State of the Nation s River, Potomac Watershed Avaiabe: 2. US Army Corps of Engineers, 3. Support Vector Machines Toobox, 4. Z. Liu, X. Wang, L. Cui, X. Lian, J. Xu, Research on Water Boom Prediction Based on Least Squares Support Vector Machine, 2009 WRI Word Congress on Computer Science and Information Engineering, vo.5, pp , Y. Xiang, L. Jiang, Water Quaity Prediction Using LS-SVM and Partice Swarm Optimization, Second Internationa Workshop on Knowedge Discovery and Data Mining, pp , X. Wang, J Lv, D. Xie, A hybrid approach of support vector machine with partice swarm optimization for water quaity prediction, Internationa Conference on Computer Science and Education (ICCSE), pp , W. Liu, K. Chen; L. Liu, Prediction mode of water consumption using east square support vector machines optimized by hybrid inteigent agorithm, 2011 Second Internationa Conference on Mechanic Automation and Contro Engineering (MACE), pp , L. Yu, X. Wang, Q. Ming, H. Mu, Y. Li, Appication and comparison of severa modeing methods in spectra based water quaity anaysis, th Chinese Contro Conference (CCC), pp , L. Liang, F. Xie, Appied research on wastewater treatment based on east squares support vector machine, 2011 Internationa Conference on Remote Sensing, Environment and Transportation Engineering (RSETE), pp , X. Zhang, S. Wang, Y. Zhao, Appication of support vector machine and east squares vector machine to freight voume forecast, 2011 Internationa Conference on Remote Sensing, Environment and Transportation Engineering (RSETE), pp , R.J. Liao, J.P. Bian, L.J. Yang, S. Grzybowski, Y.Y. Wang, J. Li, Forecasting dissoved gases content in power transformer oi based on weakening buffer operator and east square support vector machine Markov, Generation, Transmission & Distribution, IET, vo. 6, no. 2, pp , L. Hou, Q. Yang, J. An, An Improved LSSVM Regression Agorithm, Internationa Conference on Computationa Inteigence and Natura Computing, vo. 2, pp , X. Zhang, Y. Zhao, S. Wang, Reiabiity prediction of engine systems using east square support vector machine, 2011 Internationa Conference on Eectronics, Communications and Contro (ICECC), pp , Stoarski T A. System for wear prediction in ubricated siding contacts. Lubrication Science, 1996, 8 (4): Suykens J A K,Vandewae J. Least squares support vector machine cassifiers. Neura Processing Letter, 1999, 9(3):

5 Discharge Potomac River Chesapeake Bay Atantic Ocean Fig. 1. Sateite andsat photo of Potomac River watershed. The Potomac river is a key entry point to the Chesapeake Bay for miions iving in or visiting metropoitan Washington. Fig. 2. The ocation of Washington Aqueduct, which is indicated by a red capita etter A. It is right beside the Potomac River. 8 USGS Data: Discharge vs. Time Time x 10 4 Fig. 3. Pot of entire discharge data set vs. time.

6 Discharge Fig. 4. Output generated from tunessvm command operating on USGS water data RBF LS-SVM 2 = , = LS-SVM Prediction USGS Data Points Time Fig. 5. USGS discharge data set (bue dots) and LS-SVM prediction (red).