A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series

Transcription

1 Ths s the Pre-Publshed Verson A comparson of performance of several artfcal ntellgence methods for forecastng monthly dscharge tme seres Wen-Chuan Wang Ph.D., Insttute of hydropower system & Hydronformatcs, Dalan Unversty of Technology, Dalan, , P.R. Chna Faculty of Water conservancy Engneerng, orth Chna Insttute of Water Conservancy and Hydroelectrc Power, Zhengzhou , P.R.Chna Kwok-Wng Chau Assocate Professor, Department of Cvl and Structural Engneerng, Hong Kong Polytechnc Unversty, Hung Hom, Kowloon, Hong Kong (Correspondng author): E-mal address: cekwchau@polyu.edu.hk Tel.: (+852) Chun-Tan Cheng Professor, Insttute of hydropower system & Hydronformatcs, Dalan Unversty of Technology, Dalan, , P.R. Chna Ln Qu Professor, Insttute of envronmental & muncpal engneerng, orth Chna Insttute of Water Conservancy and Hydroelectrc power, ZhengZhou, , P.R.Ch Abstract. Developng a hydrologcal forecastng model based on past records s crucal to effectve hydropower reservor management and schedulng. Tradtonally, tme seres analyss and modelng s used for buldng mathematcal models to generate hydrologc records n hydrology and water resources. Artfcal ntellgence (AI), as a branch of computer scence, s capable of analyzng long-seres and large-scale hydrologcal data. In recent years, t s one of front ssues to apply AI technology to the hydrologcal forecastng modelng. In ths paper, autoregressve movng-average (ARMA) models, artfcal neural networks (As) approaches, adaptve neural-based fuzzy nference system (AFIS) technques, genetc programmng (GP) models and support vector machne (SVM) method are examned usng the long-term observatons of monthly rver flow dscharges. The four quanttatve standard statstcal performance evaluaton measures, the coeffcent of correlaton (R), ash-sutclffe effcency coeffcent (E), root mean squared error (RMSE), mean absolute percentage error (MAPE), are employed to evaluate the performances of varous models developed. Two case study rver stes are also provded to llustrate ther respectve performances. The results ndcate that the best performance can be obtaned by AFIS, GP and SVM, n terms of dfferent evaluaton crtera durng the tranng and valdaton phases. Key words: monthly dscharge tme seres forecastng; ARMA; A; AFIS; GP; SVM 1

2 42 1. Introducton The dentfcaton of sutable models for forecastng future monthly nflows to hydropower reservors s a sgnfcant precondton for effectve reservor management and schedulng. The results, especally n long-term predcton, are useful n many water resources applcatons such as envronment protecton, drought management, operaton of water supply utltes, optmal reservor operaton nvolvng multple objectves of rrgaton, hydropower generaton, and sustanable development of water resources, etc. As such, hydrologc tme seres forecastng has always been of partcular nterest n operatonal hydrology. It has receved tremendous attenton of researchers n last few decades and many models for hydrologc tme seres forecastng have been proposed to mprove the hydrology forecastng. These models can be broadly dvded nto three groups: regresson based methods, tme seres models and AI-based methods. For autoregressve movng-average models (ARMA) proposed by Box and Jenkns (1970), t s assumed that the tmes seres s statonary and follows the normal dstrbuton. ARMA s one of the most popular hydrologc tmes seres models for reservor desgn and optmzaton. Extensve applcaton and revews of the several classes of such models proposed for the modellng of water resources tme seres were reported (Chen and Rao, 2002; Salas, 1993; Srkanthan and McMahon, 2001). In recent years, AI technque, beng capable of analysng long-seres and large-scale data, has become ncreasngly popular n hydrology and water resources among researchers and practcng engneers. Snce the 1990s, artfcal neural networks (As), based on the understandng of the bran and nervous systems, was gradually used n hydrologcal predcton. An extensve revew of ther use n the hydrologcal feld s gven by ASCE Task Commttee on Applcaton of Artfcal eural etworks n Hydrology (ASCE, 2000a; ASCE, 2000b).The As have been shown to gve useful results n many felds of hydrology and water resources research (Campolo et al., 2003; Chau, 2006; Muttl and Chau, 2006). The adaptve neural-based fuzzy nference system (AFIS) model and ts prncples, frst developed by Jang (1993), have been appled to study many problems and also n hydrology feld as well. Chang & Chang (2001) studed the ntegraton of a neural network and fuzzy arthmetc for real-tme streamflow forecastng and reported that AFIS helps to ensure more effcent reservor operaton than the classcal models based on rule curve. Bazartseren et al. (2003) used neuro-fuzzy and neural network models for short-term water level predcton. Dxon (2005) examned the senstvty of neuron-fuzzy models used to predct groundwater vulnerablty n a spatal context by ntegratng GIS and neuro-fuzzy technques. Other researchers reported good results n applyng AFIS n hydrologcal predcton (Cheng et al., 2005; Keskn et al., 2006; ayak et al., 2004). Genetc Programmng (GP), an extenson of the well known feld of genetc algorthms (GA) belongng to the famly of evolutonary computaton, s an automatc programmng technque for evolvng computer programs to solve problems (Koza, 1992). GP model was used to emulate the ranfall-runoff process (Whgam and Crapper, 2001) and was evaluated n terms of root mean square error and correlaton coeffcent (Long et al., 2002; Whgam and Crapper, 2001). It was shown to be a vable alternatve to tradtonal ranfall runoff models. The GP approach was also employed by Johar et al (2006) to predct the sol-water characterstc curve of sols. GP s employed for modellng and predcton of algal blooms n Tolo Harbour, Hong Kong (Muttl and 2

3 Chau, 2006) and the results ndcated good predctons of long-term trends n algal bomass. The Darwnan theory-based GP approach was suggested for mprovng fortnghtly flow forecast for a short tme-seres (Svapragasam et al., 2007). The support vector machne (SVM) s based on structural rsk mnmzaton (SRM) prncple and s an approxmaton mplementaton of the method of SRM wth a good generalsaton capablty (Vapnk, 1998). Although SVM has been used n applcatons for a relatvely short tme, ths learnng machne has been proven to be a robust and competent algorthm for both classfcaton and regresson n many dscplnes. Recently, the use of the SVM n water resources engneerng has attracted much attenton. Dbke et al. (2001) demonstrated ts use n ranfall runoff modelng. Long and Svapragasam (2002) appled SVM to flood stage forecastng n Dhaka, Bangladesh and concluded that the accuracy of SVM exceeded that of A n one-lead-day to seven-lead-day forecastng. Yu et al.(2006) successfully explored the usefulness of SVM based modellng technque for predctng of real tme flood stage forecastng on Lan-Yang rver n Tawan 1 to 6 hours ahead. Khan and Coulbaly (2006) demonstrated the applcaton of SVM to tme seres modelng n water resources engneerng for lake water level predcton. The SVM method has also been employed for stream flow predctons (Asefa et al., 2006; Ln et al., 2006). The major objectves of the study presented n ths paper are to nvestgate several AI technques for modellng monthly dscharge tme seres, whch nclude A approaches, AFIS technques, GP models and SVM method, and to compare ther performance wth other tradtonal tme seres modellng technques such as ARMA. Four quanttatve standard statstcal performance evaluaton measures,.e., coeffcent of correlaton (R), ash-sutclffe effcency coeffcent (E), root mean squared error (RMSE), mean absolute percentage error (MAPE), are employed to valdate all models. Bref ntroducton and model development of these AI methods are also descrbed before dscussng the results and makng concludng remarks. The performances of varous models developed are demonstrated by forecastng monthly rver flow dscharges n Manwan Hydropower and Hongjadu Hydropower Descrpton of Selected Models Several AI technques employed n ths study nclude As, AFIS technques, GP models and SVM method. A bref overvew of these technques s presented here Artfcal eural etworks (As) Snce early 1990s, As, and n partcular, feed-forward back-propagaton perceptrons have been used for forecastng n many areas of scence and engneerng (Chau and Cheng, 2002). An A s an nformaton processng system composed of many nonlnear and densely nterconnected processng elements or neurons, whch s organzed as layers connected va weghts between layers. An A usually conssts of three layers: the nput layer, where the data are ntroduced to the network; the hdden layer or layers, where data are processed; and the output layer, where the results of gven nput are produced. The structure of a feed-forward A s shown n Fg. 1. A mult-layer feed-forward back-propagaton network wth one hdden layer has been used 3

4 throughout the study (Haykn, 1999). In a feed-forward back-propagaton network, the weghted connectons feed actvatons only n the forward drecton from an nput layer to the output layer. These nterconnectons are adjusted usng an error convergence technque so that the network s response best matches the desred response. The man advantage of the A technque over tradtonal methods s that t does not requre nformaton about the complex nature of the underlyng process under consderaton to be explctly descrbed n mathematcal form Adaptve neural-based fuzzy nference system (AFIS) The AFIS used n the study s a fuzzy nference model of Sugeno type, and s a composton of As and fuzzy logc approaches (Jang, 1993; Jang et al., 1997). The model dentfes a set of parameters through a hybrd learnng rule combnng the back-propagaton gradent descent and a least squares method. It can be used as a bass for constructng a set of fuzzy IF-THE rules wth approprate membershp functons n order to generate the prevously stpulated nput-output pars (Keskn et al., 2006). The Sugeno fuzzy nference system s computatonally effcent and works well wth lnear technques, optmzaton and adaptve technques. As a smple example, we assume a fuzzy nference system wth two nputs x and y and one output z. The frst-order Sugeno fuzzy model, a typcal rule set wth two fuzzy If-Then rules can be expressed as: Rule 1:If x s A 1 and y s B 1,then f1 p1x q1 y r1 Rule 2:If x s A 2 and y s B 2,then f2 p2x q2 y r2 The resultng Sugeno fuzzy reasonng system s shown n Fg. 2. It llustrates the fuzzy reasonng mechansm for ths Sugeno model to derve an output functon (f) from a gven nput vector [x, y]. The correspondng equvalent AFIS archtecture s a fve-layer feed forward net work that uses neural net work learnng algorthms coupled wth fuzzy reasonng to map an nput space to an output space. It s shown n Fg.3. The more comprehensve presentaton of AFIS for forecastng hydrologcal tme seres can be found n the lterature (Cheng et al., 2005; Keskn et al., 2006; ayak et al., 2004) Genetc programmng (GP) GP s a search methodology belongng to the class of ntellgent methods whch allows the soluton of problems by automatcally generatng algorthms and expressons. These expressons are codfed or represented as a tree structure wth ts termnals (leaves) and nodes (functons). GP, smlar to GA, ntalzes a populaton that compounds the random members known as chromosomes (ndvdual). Afterward, ftness of each chromosome s evaluated wth respect to a target value. GP works wth a number of soluton sets, known collectvely as a populaton, rather than a sngle soluton at any one tme; the possblty of gettng trapped n a local optmum s thus avoded. GP dffers from the tradtonal GA n that t typcally operates on parse trees nstead of bt strngs. A parse tree s bult up from a termnal set (the varables n the problem) and a functon set (the basc operators used to form the functon). GP s provded wth evaluaton data, a set of prmtves and ftness functons. The evaluaton data descrbe the specfc problem n 4

5 terms of the desred nputs and outputs. They are used to generate the best computer program to descrbe the relatonshp between the nput and output very well (Koza, 1992). The representaton of GP can be vewed as a parse tree-based structure composed of the functon set and termnal set. The functon set s the operators, functons or statements such as arthmetc operators ({+, -,, /}) or condtonal statements (f then ) whch are avalable n the GP. The termnal set contans all nputs, constants and other zero-argument n the GP tree. An example of such a parse tree can be found n Fg. 4. Once a populaton of the GP tree s ntalzed, the followng procedures are smlar to GAs ncludng defnng the ftness functon, genetc operators such as crossover, mutaton and reproducton and the termnaton crteron, etc. In GP, the crossover operator s used to swap the subtree from the parents to reproduce the chldren usng matng selecton polcy rather than exchangng bt strngs as n GAs. The genetc programmng ntroduced here s one of the smplest forms avalable. A more comprehensve presentaton of GP can be found n the lterature (Borrell et al., 2006; Koza, 1992) Support vector machne (SVM) SVM s the state-of-the-art neural network technology based on statstcal learnng (Vapnk, 1995; Vapnk, 1998). The basc dea of SVM s to use lnear model to mplement nonlnear class boundares through some nonlnear mappng of the nput vector nto the hgh-dmensonal feature space. The lnear model constructed n the new space can represent a nonlnear decson boundary n the orgnal space. In the new space, SVM constructs an optmal separatng hyperplane. If the data s lnearly separated, lnear machnes are traned for an optmal hyperplane that separates the data wthout error and nto the maxmum dstance between the hyperplane and the closest tranng ponts. The tranng ponts that are closest to the optmal separatng hyperplane are called support vectors. Fg. 5 exhbts the basc concept of SVM. There exst uncountable decson functons,.e. hyperplanes, whch can effectvely separate the negatve and postve data set (denoted by x and o, respectvely) that has the maxmal margn. Ths ndcates that the dstance from the closest postve samples to a hyperplane and the dstance from the closest negatve samples to t wll be maxmzed. Gven a set of tranng data {( x, )} (x s the nput vector, d s the desred value and s the d total number of data patterns), the regresson functon of SVM s formulated as follows: y f ( x) w ( x) b (1) where (x) s the feature of nputs, and w and b are coeffcents. The coeffcents ( w and b) are estmated by mnmzng the followng regularzed rsk functon (Vapnk, 1995; Vapnk, 1998): r ( C) C L ( d, y ) (2) where d y f d y L ( d, y) 0 otherwse (3) 5

6 In Eq. (2), the frst term s the emprcal error (rsk). They are measured by Eq. (3). L ( d, y) s called the -nsenstve loss functon, the loss equals zero f the forecast value s wthn the -tube and Fg. 6. The second term s used as a measure of the flatness of the functon, Hence, C s referred to as the regularzed constant and t determnes the trade-off between the emprcal rsk and the regularzaton term. Increasng the value of C wll result n an ncreasng relatve mportance of the emprcal rsk wth respect to the regularzaton term. s called the tube sze and t s equvalent to the approxmaton accuracy placed on the tranng data ponts. Both C and are user-prescrbed parameters, two postve slack varables and, whch represent the dstance from actual values to the correspondng boundary values of -tube (Fg. 6), are ntroduced. Then, Eq. (2) s transformed nto the followng constraned form. Mnmze: Subject to 1 2 C ( ) (4) 2 ( x ) b d, 1,2,, d ( x ) b, 1,2,,,, 1,2,3,, Ths constraned optmzaton problem s solved usng the followng prmal Lagrangan form: 1 2 L C( ) [ ( x ) b d ] 2 1 [ d ( x ) b ] ( ) (5) Eq. (5) s mnmzed wth respect to prmal varables,b, and, and maxmzed wth respect to the nonnegatve Lagrangan multplers and, Fnally, Karush-Kuhn- Tucker condtons are appled to the regresson, and Eq. (5) has a dual Lagrangan form: (, ) 1 wth the constrants, 1 d ( ) 1 ( ) ( ) 0 And a, [0, C], 1,2,, ( )( ) K( x, x In Eq. (6), the Lagrange multplers satsfy the equalty 0, The Lagrange multplers and are calculated, and the optmal desred weght vector of the regresson hyperplane s ( ) K( x, x ) (7) 1 Therefore, the regresson functon can be gven as f ( x,, ) ( ) K( x, x ) b (8) 1 j j j ) (6) 6

7 Here, K ( x, x ) s called the Kernel functon. The value of the Kernel s nner product of the two vectors x and x j n the feature space (x) and ( x j ), so K( x, x j ) ( x) ( x j ), and functon that satsfes Mercer s condton (Vapnk, 1998) can be used as the Kernel Functon. In general, three knds of kernel functon are used as follows: Polynomal: K ( x, x ) Radal bass functon (RBF) K( x, x Two-layer neural networks n j ) ( x x j 1 (9) 2 2 ) exp( x / 2 ) (10) j x j K ( x, x ) n j ) tanh( kx x j (11) Study area and data In ths study, Manwan Hydropower n Lancangjang Rver s selected as a study ste. The monthly flow data from January 1953 to December 2004 are studed. The data set from January 1953 to December 1999 s used for calbraton whlst that from January 2000 to December 2004 s used for valdaton (Fg.7). Lancangjang Rver s a large rver n Asa, whch orgnates from Qngha-Tbet Plateau, penetrates Yunnan from northwest to the south and passes through Laos, Burma, Thaland, Camboda and Vetnam, ngresses nto South Chna Sea fnally. The rver s about 4,500 km long and has a dranage area of 744,000 km 2. Manwan Hydropower merges on the mddle reaches of Lancang Rver and at borders of Yunxan and Jngdong countes. The catchment area at Manwan dam ste s 114,500 km 2, the length above Manwan s 1,579 km, and the mean elevaton s 4,000 km. The average yearly runoff s 1,230 cubc meters per at the dam ste. Ranfall provdes most of the runoff and snow melt accounts for 10%. early 70% of the annual ranfall occurs from June to September. Locatons of Lancang Rver and Manwan Hydropower are shown n Fg.8 (A). The second study ste s at Hongjadu Hydropower on Wujang Rver n southwest Chna. The monthly flow data from January 1951 to December 2004 are studed. The data set from January 1951 to December 1994 s used for calbraton whlst that from January 1995 to December 2004 s used for valdaton (Fg.9). Wujang Rver, orgnatng from Wumeng foothll of Yun-Gu Plateau, s the bggest branch at the southern bank of Yangtze Rver, whch covers 87,920km 2, total length of 1,037km, centralzed fall of 2,124m, and wth approved nstalled capacty 8,800MW. owadays, Hongjadu hydropower staton s the master regulaton reservor for the cascade hydropower statons on Wujang Rver. The catchment area at Hongjadu dam ste s 9,900 km 2 and the average yearly runoff s 155 cubc meters at the dam ste. Ranfall provdes most of the runoff. Locatons of Wujang Rver and Hongjadu Hydropower are shown n Fg.8 (B). In A, AFIS and SVM modelng processes, large attrbute values mght cause numercal problems because the neurons n A and AFIS are combned Sgmod functon as exctaton functon, and the kernel values n SVM usually depend on the nner products of feature vectors, such as the lnear kernel and the polynomal kernel. There are two man advantages to normalze 7

8 features before applyng A, AFIS and SVM to predcton. One advantage s to avod attrbutes n greater numerc ranges domnatng those n smaller numerc ranges, and another advantage s to avod numercal dffcultes durng the calculaton. It s recommended to lnearly scale each attrbute to the range [-1, +1] or [0, 1]. In the modelng process, the data sets of rver flow were scaled to the range between 0 and 1 as follow: q q q ' mn (12) qmaz qmn where q s the scaled value, ' the mnmum and maxmum of flow seres. q s the orgnal flow value and q mn, q maz are respectvely Predcton modelng and nput selecton We are nterested n hydrologcal forecastng model that predct outputs from nputs based on past records. There are no fxed rules for developng these AI technques (A, AFIS, GP, SVM), even though a general framework can be followed based on prevous successful applcatons n engneerng (Cheng et al., 2005; Ln et al., 2006; ayak et al., 2004; Sudheer et al., 2002). The objectve of studes focus on predctng dscharges usng antecedent values s to generalze a relatonshp of the followng form: m Y f ( X ) (13) where X m s a m-dmensonal nput vector consstng of varables x 1,,x, x m, and Y s the output varable. In dscharge modelng, values of x may be flow values wth dfferent tme lags and the value Y s generally the flow n the next perod. Generally, the number of antecedent values ncluded n the vector X m s not known a pror. In these AI technques, beng typcal n any data-drven predcton models, the selecton of approprate model nput vector would play an mportant role n ther successful mplementaton snce t provdes the basc nformaton about the system beng modeled. The parameters determned as nput varables are the numbers of flow values for fndng the lags of runoff that have a sgnfcant nfluence on the predcted flow. These nfluencng values correspondng to dfferent lags can be very well establshed through a statstcal analyss of the data seres. Statstcal procedures were suggested for dentfyng an approprate nput vector for a model (Ln et al., 2006; Sudheer et al., 2002). In ths study, two statstcal methods (.e. the autocorrelaton functon (ACF) and the partal autocorrelaton functon (PACF)) are employed to determne the number of parameters correspondng to dfferent antecedents values. The nfluencng antecedent dscharge patterns can be suggested by the ACF and PACF n the flow at a gven tme. The ACF and PACF are generally used n dagnosng the order of the autoregressve process and can also be employed n predcton modelng (Ln et al., 2006). The values of ACF and PACF of monthly flow sequence (1953/1~1999/12) s calculated for lag 0 to 24 n Manwan, whch are presented n Fg.10. Smlarly, the values of ACF and PACF of monthly flow sequence (1951/1~1994/12) s calculated for lag 0 to 24 n Hongjadu, whch are presented n Fg.11. From Fg.10(a) and Fg.11(a), the ACF exhbts the peak at lag 12. In addton, Fg.10(b) and Fg.11(b) showed a sgnfcant correlaton of PACF at 95% confdence level nterval up to 12 months of flow lag. Therefore twelve antecedent flow values have the most nformaton to predct future flow and are consdered as nput for 8

9 304 monthly dscharge tme seres modelng Model performance evaluaton Some technques are recommended for hydrologcal tme seres forecastng model performance evaluaton accordng to publshed lterature related to calbraton, valdaton, and applcaton of hydrologcal models. Four performance evaluaton crtera used n ths study are computed as n the followng secton. The coeffcent of correlaton (R) or ts square, the coeffcent of determnaton (R 2 ): It descrbes the degree of collnearty between smulated and measured data, whch ranges from -1 to 1, s an ndex of the degree of lnear relatonshp between observed and smulated data. If R =0, no lnear relatonshp exsts. If R=1 or -1, a perfect postve or negatve lnear relatonshp exsts. Its equaton s R 1 n n 1 1 n n 1 ( Q ( ) Q ( Q ( ) Q ) )( Q 1 n f n 1 ( ) Q ( Q f f ) (14) ( ) Q R and R 2 have been wdely used for model evaluaton (Ln et al., 2006; Santh et al., 2001; Van Lew et al., 2003), though they are oversenstve to hgh extreme values (outlers) and nsenstve to addtve and proportonal dfferences between model predctons and measured data (Legates and McCabe, 1999). ash-sutclffe effcency coeffcent (E): The ash-sutclffe model effcency coeffcent s used to assess the predctve power of hydrologcal models (ash and Sutclffe, 1970). It s a normalzed statstc that determnes the relatve magntude of the resdual varance ( nose ) compared to the measured data varance and ndcates how well the plot of observed versus smulated data fts the 1:1 lne (Moras et al., 2007). It s defned as: n ( Q0 ( ) Q f ( )) 1 E 1 n (15) ( Q ( ) Q ) ash-sutclffe effcences ranges between (-, 1]: E=1 corresponds to a perfect match of forecastng dscharge to the observed data; E=0 shows that the model predctons are as accurate as the mean of the observed data; and - <E<0 occurs when the observed mean s a better predctor than the model, whch ndcates unacceptable performance. Root mean squared error (RMSE): It s an often used measure of the dfference between values predcted by a model and those actually observed from the thng beng modeled. RMSE s one of the commonly used error ndex statstcs (Ln et al., 2006; ayak et al., 2004) and s defned as: f ) RMSE 1 n n 1 ( Q f ( ) Q0 ( )) 2 (16) Mean absolute percentage error (MAPE): The MAPE s computed through a term-by-term comparson of the relatve error n the predcton wth respect to the actual value of the varable. 9

10 Thus, the MAPE s an unbased statstc for measurng the predctve capablty of a model. It s a measure of the accuracy n a ftted tme seres value n statstcs and has been used for rver flow tme seres predcton evaluaton (Hu et al., 2001). It usually expresses accuracy as a percentage and s defned as: n 1 Q f ( ) Q0 ( ) MAPE 100 (17) n Q ( ) 1 0 where Q 0( ) and Q f () are, respectvely, the observed and forecasted dscharge and Q 0, Q denote ther means, and n s the number data ponts consdered. f Development of models ARMA model uses the drect dependence of the prevous measurements and depends on the prevous nnovaton of the process n a movng average form. The monthly dscharge seres, whch do ft a normal dstrbuton wth respect to the skewness coeffcent, can be normalzed usng a log-transformaton functon n order to remove the perodcty n the orgnal record (Keskn et al., 2006). In order to choose the approprate ARMA (p, q) model, the Akake nformaton crtera (AIC) are used to select the value of p and q, whch represent respectvely the number of autoregressve orders and the number of movng-average orders of the ARMA model. In ths study, the models ARMA (5, 8), (6, 7), (8, 7), (9, 8) and (11, 8), have a relatvely mnmum AIC value based on flow seres n Manwan, and the models ARMA (5, 9), (6, 10), (7, 9), (8, 9) and (10, 11) have a relatvely mnmum AIC value based on flow seres n Hongjadu. Table 1 and Table 2, respectvely, show ther AIC values and the performance of alternatve ARMA models. Hence, accordng to ther performance ndces, ARMA (8, 7) s selected as the ARMA model n Mamwan, and ARMA (6, 10) s selected as the ARMA model n Hongjadu. In ths study, a typcal three-layer feed-forward A model (Fg. 1) wth a back-propagaton algorthm s constructed for forecastng monthly dscharge tme seres. The back-propagaton tranng algorthm s a supervsed tranng mechansm and s normally adopted n most of the engneerng applcaton. The prmary goal s to mnmze the error at the output layer by searchng for a set of connecton strengths that cause the A to produce outputs that are equal to or closer to the targets. The neurons of hdden layer use the tan-sgmod transfer functon, and the lnear transfer functon for output layer. A scaled conjugate gradent algorthm (Moller, 1993) s employed for tranng, and the tranng epoch s set to 500. The optmal number of neuron n the hdden layer was dentfed usng a tral and error procedure by varyng the number of hdden neurons from 2 to 13. The number of hdden neurons was selected based on the RMSE. The effect of changng the number of hdden neurons on the RMSE of the data set s shown n Fg. 12 and Fg. 13. It can be observed that the effect of the number of neurons assgned to the hdden layer has nsgnfcant effect on the performance of the feed forward model. The numbers of hdden neurons were found to be four and four for Manwan and Hongjadu, respectvely. The AFIS apples a hybrd learnng algorthm that combnes the backpropagaton gradent descent and the least squares estmate method, whch outperforms the orgnal backproagaton algorthm. An essental part of fuzzy logc s fuzzy sets defned by membershp functons and rule bases. Shapes of the fuzzy sets are defned by the membershp functons. The adjustment of 10

11 adequate membershp functon parameters s facltated by a gradent vector. After determnng a gradent vector, the parameters are adjusted and the performance functon s mnmsed va least-squares estmaton. For the proposed Sugeno-type model, the overall output s expressed as lnear combnatons of the resultng parameters. The output f n Fg. 3 can be rewrtten as: 380 f w (18) 1 f1 w2 f 2 ( w1 x) p1 ( w1 y) q1 ( w1 ) r1 ( w2 x) p2 ( w2 y) q2 ( w2 ) r The resultng parameters (p 1, q 1, r 1, p 2, q 2, r 2 ) are computed by the least-squares method. Consequently, the optmal parameters of the AFIS model can be estmated usng the hybrd learnng algorthm. For more detal, please refer to Jang and Sun (Jang et al., 1997). GP has the ablty to generate the best computer program to descrbe the relatonshp between the nput and output. In ths study, n order to fnd the optmal monthly flow seres forecastng model, the selecton of the approprate parameters of GP evoluton s necessary. Although the fne-tunng of algorthm was not the man concern of ths paper, we nvestgated varous ntalzaton and run approaches and the adopted GP parameters are presented n Table 3. Ths setup furnshed stable and effectve runs throughout experments. The evolutonary procedures are smlar to GAs ncludng defnng the ftness functon, genetc operators such as crossover, mutaton and reproducton and the termnaton crteron, etc. In GP, the crossover operator s used to swap the subtree from the parents to reproduce the chldren usng matng selecton polcy rather than exchangng bt strngs as n GAs. A kernel functon has to be selected from the qualfed functons n usng SVM. Dbke et al. (2001) appled dfferent kernels n SVR to ranfall- runoff modelng and demonstrated that the radal bass functon (RBF) outperforms other kernel functons. Also, many works on the use of SVR n hydrologcal modelng and forecastng have demonstrated the favorable performance of the RBF (Khan and Coulbaly, 2006; Ln et al., 2006; Long and Svapragasam, 2002; Yu et al., 2006). Therefore, the RBF s used as the kernel functon for predcton of dscharge n ths study. There are three parameters n usng RBF kernels: C, ε and σ. the accuracy of a SVM model s largely dependent on the selecton of the model parameters. However, structured methods for selectng parameters are lackng. Consequently, some knd of model parameter calbraton should be made. Recently, there are several methods developed to dentfy the parameters, such as the smulated annealng algorthms (Pa and Hong, 2005), GA (Pa, 2006) and the shuffled complex evoluton algorthm (SCE-UA) (Ln et al., 2006; Yu et al., 2004). The SCE-UA method belongs to the famly of evoluton algorthm and was presented by Duan et al. (1993). In ths study, the SCE-UA s employed as the method of optmzng parameters of SVM and a more comprehensve presentaton can be found by Ln et al. (2006). To reach at a sutable choce of these parameters, the RMSE was used to optmze the parameters. Optmal parameters (C, ε, σ) = ( , e-004, ) and (C, ε, σ) = (0.5045, e-004, ) were obtaned for Manwan and Hongjadu, respectvely Results and dscusson The Manwan Hydropower, has been studed by Cheng et al. (2005) usng AFIS wth dscharges of monthly rver flow dscharges durng , and by Ln et al. (2006) usng SVM wth dscharges of monthly rver flow dscharges durng In ther study, the R and RMSE were employed for evaluaton model performance. In ths paper, n order to dentfy more 11

12 sutable models for forecastng future monthly nflows to hydropower reservors, the monthly dscharge tme seres data of two study stes n dfferent rvers are appled. For the same bass of comparson, the same tranng and verfcaton sets, respectvely, are used for all the above models developed, whlst the four quanttatve standard statstcal performance evaluaton measures are employed to evaluate the performances of varous models developed. Tables 4 and 5 present the results of Manwan and Hongjadu study stes respectvely, n terms of varous performance statstcs It can be observed from Tables 4 and 5 that varous AI methods have good performance durng both tranng and valdaton, and they outperform ARMA n terms of all the standard statstcal measures. For Manwan hydropower, n the tranng phase, the AFIS model obtaned the best R, RMSE, and E statstcs of 0.932, , and 0.869, respectvely; whle the SVM model obtaned the best MAPE statstcs of Analyzng the results durng testng, t can be observed that the SVM model outperforms all other models. Smlarly, for Hongjadu hydropower, n the tranng phase, the AFIS model obtaned the best RMSE and E statstcs of and 0.564, respectvely; whle the SVM model obtaned the best R and MAPE statstcs of and 28.25, respectvely. Analyzng the results durng testng, the SVM model obtaned the best R and MAPE statstcs of and 33.77, respectvely; whle the GP model obtaned the best RMSE, and E statstcs of and 0.654, respectvely. RMSE evaluates the resdual between observed and forecasted flow, and MAPE measures the mean absolute percentage error of the forecast. R evaluates the lnear correlaton between the observed and computed flow, whle E evaluates the capablty of the model n predctng flow values away from the mean. Accordng to the fgures n Tables 4 and 5, we can conclude that the best performance of all AI methods developed n ths paper s dfferent n terms of the dfferent statstcal measures. In addton, n the valdaton phase as seen n Tables 4 and 5, the values wth the AFIS, GP and SVM model predcton were able to produce a good, near forecast, as compared to those wth ARMA and A model, whlst t can be concluded that the AFIS model obtaned the best mnmum absolute error between the observed and modeled maxmum and mnmum peak flows n Manwan Hydropower, and the GP and SVM model obtaned the best mnmum absolute error between the observed and modeled maxmum and mnmum peak flows, respectvely, n Hongjadu Hydropower. In the valdaton phase, the SVM model mproved the ARMA forecast of about 6.06% and 20.12% reducton n RMSE and MAPE values, respectvely; Improvements of the forecast results regardng the R and E were approxmately 1.22% and 1.69%, respectvely n Manwan Hydropower. In Hongjadu Hydropower, the GP model obtaned the best value of RMSE durng the valdaton phase decreases by 8.77% and the best value of E ncreases by 11.99% comparng wth ARMA; whle, the SVM model obtaned the best value of R durng the valdaton phase ncreases by 4.71% and the best value of MAPE decreases by 29.69% comparng wth ARMA. Thus the results of ths analyss ndcate that the AFIS or SVM s able to obtan the best result n terms of dfferent evaluaton measures durng the tranng phase, and the GP or SVM s able to obtan the best result n terms of dfferent evaluaton measures durng the valdaton phase. Furthermore, as can be seen from Tables 4 and 5 that the vrtues or defect degree of forecastng accuracy s dfferent n terms of dfferent evaluaton measures durng the tranng phase and the valdaton phase. SVM model s able to obtan the better forecastng accuracy n terms of dfferent evaluaton measures durng the valdaton phase not only durng the tranng phase but also durng the valdaton phase. The forecastng results of AFIS model durng the valdaton phase are 12

13 nferor to the results durng the tranng phase. GP s n the mddle or lower level n tranng phases, but the GP model s able to obtan the better forecastng result n valdaton phases, and especally the GP model s able to obtan the maxmum peak flows among all models developed n Hongjadu Hydropower. The performances of all predcton models developed n ths paper durng the tranng and valdaton perods n the two study stes are shown n Fg. 14 to Conclusons An attempt was made n ths study to nvestgate the performance of several AI methods for forecastng monthly dscharge tme seres. The forecastng methods nvestgated nclude the As AFIS technques, GP models and SVM method. The conventonal ARMA s also employed as a benchmarkng yardstck for comparson purposes. The monthly dscharge data from actual feld observed data n the Manwan Hydropower and Hongjadu Hydropower were employed to develop varous models nvestgated n ths study. The methods utlze the statstcal propertes of the data seres wth certan amount of lagged nput varables. Four standard statstcal performance evaluaton measures are adopted to evaluate the performances of varous models developed. The results obtaned n ths study ndcate that the AI methods are powerful tools to model the dscharge tme seres and can gve good predcton performance than tradtonal tme seres approaches. The results ndcate that the best performance can be obtaned by AFIS, GP and SVM, n terms of dfferent evaluaton crtera durng the tranng and valdaton phases. SVM model s able to obtan the better forecastng accuracy n terms of dfferent evaluaton measures durng the valdaton phase durng both the tranng phase and the valdaton phase. The forecastng results of AFIS model durng the valdaton phase are nferor to the results durng the tranng phase. GP s n the mddle or lower level n tranng phases, but the GP model s able to obtan the better forecastng result n valdaton phases. The AFIS and GP model obtan the maxmum peak flows among all models developed n dfferent studes stes, respectvely. Therefore, the results of the study are hghly encouragng and suggest that AFIS, GP and SVM approaches are promsng n modelng monthly dscharge tme seres, and ths may provde valuable reference for researchers and engneers who apply AI methods for modelng long-term hydrologcal tme seres forecastng. It s hoped that future research efforts wll focus n these drectons,.e. more effcent approach for tranng mult-layer perceptrons of A model, the ncreased learnng ablty of the AFIS model, the fne-tunng of algorthm for selectng more approprate parameters of GP evoluton, savng computng tme or more effcent optmzaton algorthms n searchng optmal parameters of SVM model etc to mprove the accuracy of the forecast models n terms of dfferent evaluaton measures for better plannng, desgn, operaton, and management of varous engneerng systems. 495 Acknowledgements Ths research was supported by the Central Research Grant of Hong Kong Polytechnc Unversty (G-U265), the atonal atural Scence Foundaton of Chna (o ), Doctor Foundaton of hgher educaton nsttutons of Chna (o ). 13

14 499 Reference ASCE Task Commttee., 2000a. Artfcal neural networks n hydrology-i: Prelmnary concepts. Journal of Hydrologc Engneerng, ASCE 5(2): ASCE Task Commttee., 2000b. Artfcal neural networks n hydrology-ii: Hydrologcal applcatons. Journal of Hydrologc Engneerng, ASCE5(2): Asefa, T., Kemblowsk, M., McKee, M. and Khall, A., Mult-tme scale stream flow predctons: The support vector machnes approach. Journal of Hydrology, 318(1-4): Bazartseren, B., Hldebrandt, G. and Holz, K.P., Short-term water level predcton usng neural networks and neuro-fuzzy approach. eurocomputng, 55(3-4): Borrell, A., De Falco, I., Della Coppa, A., codem, M. and Trautteur, G., Performance of genetc programmng to extract the trend n nosy data seres. Physca a-statstcal Mechancs and Its Applcatons, 370(1): Box, G.E.P. and Jenkns, G.M., Tmes seres Analyss Forecastng and Control. Holden-Day, San Francsco. Campolo, M., Soldat, A. and Andreuss, P., Artfcal neural network approach to flood forecastng n the Rver Arno. Hydrologcal Scences Journal, 48(3): Chang, L.C. and Chang, F.J., Intellgent control for modellng of real-tme reservor operaton. Hydrologcal Processes, 15(9): Chau, K.W., Partcle swarm optmzaton tranng algorthm for As n stage predcton of Shng Mun Rver. Journal of Hydrology, 329(3-4): Chau, K.W. and Cheng, C.T., Real-tme predcton of water stage wth artfcal neural network approach. Lecture otes n Artfcal Intellgence, 2557: 715. Chen, H.L. and Rao, A.R., Testng hydrologc tme seres for statonarty. Journal of Hydrologc Engneerng, 7(2): Cheng, C.T., Ln, J.Y., Sun, Y.G. and Chau, K.W., Long-term predcton of dscharges n Manwan hydropower usng adaptve-network-based fuzzy nference systems models, Advances n atural Computaton, Pt 3, Proceedngs. Lecture otes n Computer Scence. Sprnger-Verlag Berln, Berln, pp Dbke, Y.B., Velckov, S., Solomatne, D. and Abbott, M.B., Model nducton wth support vector machnes: Introducton and applcatons. Journal of Computng n Cvl Engneerng, 15(3): Dxon, B., Applcablty of neuro-fuzzy technques n predctng ground-water vulnerablty: a GIS-based senstvty analyss. Journal of Hydrology, 309(1-4): Duan, Q.Y., Gupta, V.K. and Sorooshan, S., Shuffled complex evoluton approach for effectve and effcent mnmzaton. Journal of Optmzaton Theory and Applcatons, 76(3): Haykn, S., eural networks: a comprehensve foundaton. 2nd ed. Upper Saddle Rver, ew Jersey. Hu, T.S., Lam, K.C. and g, S.T., Rver flow tme seres predcton wth a range-dependent neural network. Hydrologcal Scences Journal, 46(5): Jang, J.-S.R., AFIS: Adaptve-etwork-based Fuzzy Inference Systems. IEEE Transactons on Systems, Man, and Cybernetcs, 23(3): Jang, J.-S.R., Sun, C.-T. and Mzutan, E., euro-fuzzy and Soft Computng: A 14

15 Computatonal Approach to Learnng and Machne Intellgence. Prentce-Hall, Upper Saddle Rver, J. Johar, A., Habbagah, G. and Ghahraman, A., Predcton of sol-water characterstc curve usng genetc programmng. Journal of Geotechncal and Geoenvronmental Engneerng, 132(5): Keskn, M.E., Taylan, D. and Terz, O., Adaptve neural-based fuzzy nference system (AFIS) approach for modellng hydrologcal tme seres. Hydrologcal Scences Journal, 51(4): Khan, M.S. and Coulbaly, P., Applcaton of support vector machne n lake water level predcton. Journal of Hydrologc Engneerng, 11(3): Koza, J., Genetc Programmng: On the Programmng of Computers by atural Selecton. MIT Press, Cambrdge, MA. Legates, D.R. and McCabe, G.J., Evaluatng the use of "goodness-of-ft" measures n hydrologc and hydroclmatc model valdaton. Water Resources Research, 35(1): Ln, J.Y., Cheng, C.T. and Chau, K.W., Usng support vector machnes for long-term dscharge predcton. Hydrologcal Scences Journal, 51(4): Long, S.Y. et al., Genetc Programmng: A new paradgm n ranfall runoff modelng. Journal of the Amercan Water Resources Assocaton, 38(3): Long, S.Y. and Svapragasam, C., Flood stage forecastng wth support vector machnes. Journal of the Amercan Water Resources Assocaton, 38(1): Moller, M.F., A scaled conjugate gradent algorthm for fast supervsed learnng. eural etworks, 6(4): Moras, D.. et al., Model evaluaton gudelnes for systematc quantfcaton of accuracy n watershed smulatons. Transactons of the ASABE, 50(3): Muttl,. and Chau, K.W., eural network and genetc programmng for modellng coastal algal blooms. Internatonal Journal of Envronment and Polluton, 28(3-4): ash, J.E. and Sutclffe, J.V., Rver flow forecastng through conceptual models part I A dscusson of prncples. Journal of Hydrology, 10(3): ayak, P.C., Sudheer, K.P., Rangan, D.M. and Ramasastr, K.S., A neuro-fuzzy computng technque for modelng hydrologcal tme seres. Journal of Hydrology, 291(1-2): Pa, P.-F., System relablty forecastng by support vector machnes wth genetc algorthms. Mathematcal and Computer Modellng, 43(3-4): Pa, P.-F. and Hong, W.-C., Support vector machnes wth smulated annealng algorthms n electrcty load forecastng. Energy Converson and Management, 46(17): Salas, J.D., Analyss and modelng of hydrologc tme seres. In: Madment, D.R., Edtor, The McGraw Hll Handbook of Hydrology, pp Santh, C. et al., Valdaton of tbe swat model on a large rver basn wth pont and nonpont sources. Journal of the Amercan Water Resources Assocaton, 37(5): Svapragasam, C., Vncent, P. and Vasudevan, G., Genetc programmng model for forecast of short and nosy data. Hydrologcal Processes, 21(2): Srkanthan, R. and McMahon, T.A., Stochastc generaton of annual, monthly and daly clmate data: A revew. Hydrology and Earth System Scences, 5(4): Sudheer, K.P., Gosan, A.K. and Ramasastr, K.S., A data-drven algorthm for constructng 15

16 artfcal neural network ranfall-runoff models. Hydrologcal Processes, 16(6): Van Lew, M.W., Arnold, J.G. and Garbrecht, J.D., Hydrologc smulaton on agrcultural watersheds: Choosng between two models. Transactons of the Asae, 46(6): Vapnk, V., The ature of Statstcal Learnng Theory. Sprnger, ew York. Vapnk, V., Statstcal learnng theory. Wley, ew York. Whgam, P.A. and Crapper, P.F., Modellng Ranfall-Runoff Relatonshps usng Genetc Programmng. Mathematcal and Computer Modellng 33: Yu, P.S., Chen, S.T. and Chang, I.F., Support vector regresson for real-tme flood stage forecastng. Journal of Hydrology, 328(3-4): Yu, X.Y., Long, S.Y. and Babovc, V., EC-SVM approach for real-tme hydrologc forecastng. Journal of Hydronformatcs, 6(3):

17 Table.1. AIC value and performance ndces of alternatve ARMA models for Manwan hydropower (p, q) AIC Tranng Valdaton R E RMSE MAPE R E RMSE MAPE (5, 8) (6, 7) (8, 7) (9, 8) (11, 8) Table.2. AIC value and performance ndces of alternatve ARMA models for Hongjadu hydropower (p, q) AIC Tranng Valdaton R E RMSE MAPE R E RMSE MAPE (5,9) (6,10) (7,9) (8,9) (10,11) Parameter Table 3. Values of prmary parameters used n GP runs Value Termnal set Varable x, random (0,1) Functon set Populaton: +, -,, /, sn, cos, ^ 2000 ndvduals The maxmum number of generatons: 100 Crossover rate: 0.9 Mutaton rate: 0.05 Selecton: Tournament wth eltst strategy Intal populaton: Ramped-half-and-half The maxmum depth of tree representaton 9 17

18 Table.4. Forecastng performance ndces of models for Manwan hydropower Model Tranng Valdaton R RMSE MAPE E R RMSE MAPE E Mn Max Observed ARMA A AFIS GP SVM otes: Mn means mnmum peak flows, and Max means maxmum peak flows Table.5. Forecastng performance ndces of models for Hongjadu hydropower Model Tranng Valdaton R RMSE MAPE E R RMSE MAPE E Mn Max Observed ARMA A AFIS GP SVM otes: Mn means mnmum peak flows, and Max means maxmum peak flows 18

19 Lst of all fgures Fg.1. Archtecture of three layers feed-forward back-propagaton A Fg.2. Two nputs frst-order Sugeno fuzzy model wth two rules 19

20 Fg.3. Archtecture of AFIS Fg. 4. GP parse tree representng functon ( b ac b 2 4 ) 2a 20

21 Fg. 5. The bass of the support vector machnes Fg.6. The soft margn loss settng for a lnear SVM and ε-nsenstve loss functon 21

22 Fg. 7. Monthly dscharge at Manwan Reservor 22

23 Fg. 8 Locaton of study stes 23

24 Fg. 9 Monthly dscharge at Hongjadu Reservor Fg.10. (a) the autocorrelaton functon of flow seres. (b)the partal autocorrelaton functon of flow seres n Manwan 24

25 Fg.11 (a) The autocorrelaton functon of flow seres. (b)the partal autocorrelaton functon of flow seres n Hongjadu Fg. 12 Senstvty of the number of nodes n the hdden layer on the RMSE of the neural network for Manwan hydropower 25

26 Fg. 13 Senstvty of the number of nodes n the hdden layer on the RMSE of the neural network for Hongjadu hydropower Fg.14 Forecasted and observed flow durng tranng perod by ARMA, A, AFIS, GP and SVM for Manwan hydropower 26

27 Fg.15 Forecasted and observed flow durng tranng perod by ARMA, A, AFIS, GP and SVM for Hongjadu hydropower Fg.16 Forecasted and observed flow durng valdaton perod by ARMA, A, AFIS, GP and SVM for Manwan hydropower 27