Classification and Regression Trees and MLP Neural Network to Classify Water Quality of Canals in Bangkok, Thailand

Transcription

1 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 Classfcaton and Regresson Trees and MLP eural etwork to Classfy Water Qualty of Canals n Bangkok, Thaland Srlak Areerachakul, Srpun Sanguansntukul Chulalongkorn Unversty, Thaland Srlak.Ar@Student.chula.ac.th, srpun.s@chula.ac.th Abstract Water qualty s one of the major concerns of countres around the world. Ths study endeavors to automatcally classfy water qualty. The water qualty classes are evaluated usng 6 factor ndces. These factors are ph value (ph), Dssolved Oxygen (DO), Bochemcal Oxygen Demand (BOD), trate trogen (O 3 ), Ammona trogen (H 3 ) and Total Colform (T-Colform). The methodology nvolves applyng data mnng technques usng classfcaton and regresson tree (CART) compared wth multlayer perceptron (MLP) neural network models. The data conssted of 88 canals n Bangkok, Thaland. The data s obtaned from the Department of Dranage and Sewerage Bangkok Metropoltan Admnstraton durng The results of classfcaton trees perform better than multlayer perceptron neural network. Classfcaton trees exhbt a hgh accuracy rate at 99.96% n classfyng the water qualty of canals n Bangkok. Subsequently, ths encouragng result could be appled wth plan and management source of water qualty. Keywords: water qualty, classfcaton and regresson tree, multlayer perceptron neural network. 1. Introducton Water qualty s a major concern around the world. In Thaland, Bangkok has been known as the Vence of the East from ts many canals and rvers. Bangkok s the captal cty, as well as, the economc center of Thaland. Its actvtes, whch nclude commercal, ndustral and servce have caused the expanson of the cty and ts populaton to accumulate envronmental polluton to the pont that nature cannot cope wth the polluton loadng, especally for water qualty. Presently, the water qualty s below standard due to populaton ncrease and ndustral growth. Addtonally, the canals and rvers around Bangkok are used for transportaton, toursm, and consumpton. In order to montor water qualty, people, lab nstruments and sensors have been used, but the cost and tme are expensve. The am of ths nvestgaton s to fnd an automated methodology that can quckly and effcently classfy the water qualty of canals n Bangkok. Recently, several machne learnng algorthms have been used to fnd patterns to classfy water qualty such as decson tree and artfcal neural networks (As). Classfcaton and regresson tree (CART) s a type of decson tree methodology. Classfcaton and regresson tree have the advantage of expressng regulartes explctly and thus beng convenence to nspect for water qualty valdty [17]. Artfcal eural etworks have become the central focus of many scentfc dscplnes, such as ecology [8], analytcal chemstry [9], and water qualty. Lterature on modelng water qualty usng As nclude [1][][1][13]. In ths study, usng classfcaton and regresson tree method compared wth multlayer perceptron neural network and appled to k-foldss cross valdaton to fnd effcently model classfy water qualty of canals n Bangkok. Ths paper s organzed as follows: Secton descrbes the materals used n the experments. Secton 3 demonstrated the methodologes used n the experments. Secton 4 contans the smulaton n the experments. The results and dscusson are shown n Secton 5. Fnally, Secton 6 concludes the paper.. Materals Data and surface water qualty standards are descrbed n ths secton..1. Ste Descrpton and Data Durng the years , Bangkok was comprsed of 88 canals. Fgure 1 [16] shows the network of the canals, mportant for the daly lfe of the people n Bangkok. These canals are used for consumpton, transportaton and recreaton. Especally a result of Bangkok beng the captal of Thaland, the rapd growth of ndustry, condomnums, hgh-rse and low-rse buldngs, and other nfrastructures, have had a sgnfcant effect on Copyrght 010, Infonomcs Socety 43

2 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 Fgure 1. A map of canals n Bangkok the canals water qualty. In order to mprove canal water qualty, classfcaton dfferent levels of water qualty becomes a major concern. The understandng of dfferent levels of water qualty can be utlzed n water management and treatment systems. In ths study, water qualty data are provded by Department of Dranage and Sewerage Bangkok Metropoltan Admnstraton durng There are 11,80 records of data. Each record conssts of 6 parameters namely; ph value (ph), Dssolved Oxygen (DO), Bochemcal Oxygen Demand (BOD), trate trogen (O 3 ), Ammona trogen (H 3 ) and Total Colform (T-Colform). The classfcatons of canal water qualty are based on surface water standards [15].The lower the number of class, the better the qualty of water qualty... Surface Water Qualty Standards Many parameters can nfluence the surface water qualty. Sx parameters are selected for the nvestgatons. In Thaland, the surface water qualty can be classfed as n Table 1[16]. Generally, surface water qualty can be dvded nto fve classes; class I, extra clean fresh surface water resources use for conservaton that are not necessary to pass through water treatment processes and requre only ordnary processes for pathogenc destructon and ecosystem conservaton where basc organsms can breed naturally; class II, very clean fresh surface water resources use for consumpton that requre ordnary water treatment processes before use by aquatc organsms n conservaton, fsheres and recreaton; class III, medum clean fresh surface water resources use for consumpton, but are passed through an ordnary treatment process before use; class IV, farly clean fresh surface water resources use for consumpton, but requres specal water treatment processes before use; and class V, the sources whch are not wthn class I to class IV and are used for navgaton. Table1. Surface Water Qualty Standards [16] Pollutants Index ph DO BOD O 3 H 3 T-Colform (MP) Class I II III IV V < >9 >6 6 4 < < >4 < >5 < >0.5 < >00 >00 Copyrght 010, Infonomcs Socety 44

3 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June Methodology In ths secton, we demonstrated methodology n ths experment. Classfcaton and Regresson tree and Multlayer perceptron neural network Classfcaton and Regresson Tree Classfcaton and regresson tree (CART) s a type of decson tree methodology. CART analyss s a form of bnary recursve parttonng []. The term bnary mples that each group of data, represented by a node n a decson tree can only be splt nto two groups. Thus, each node can be splt nto two chld nodes whch case the orgnal node s called parent node. The term recursve refers to the fact that the bnary parttonng process can be appled over and over agan. Therefore, each parent node can gve rse to two chld nodes and n turn each of these chld nodes may themselves be splt formng addtonal chldren. The term parttonng refers to the fact that the dataset s splt nto sectons or parttoned. CART analyss conssts of four basc steps []. The frst step conssts of tree buldng, durng whch a tree s bult usng recursve splttng of nodes. Each resultng node s assgned a predcted class based on the dstrbuton of classes n the learnng dataset whch would occur n that node and the decson cost matrx. The assgnment of a predct class to each node occurs whether or not that node s subsequently splt nto chld nodes. The second step conssts of stoppng the tree buldng process. At ths pont a maxmal tree has been produced whch probably greatly overfts the nformaton contaned wthn the learnng data set. The thrd step conssts of tree prunng whch results n the creaton of a sequence of smpler and smpler trees through the cuttng off of ncreasngly mportant nodes. The fourth step conssts of optmal tree selecton durng whch the tree whch fts the nformaton n the learnng dataset but does not overft the nformaton, s selected from among the sequence of pruned trees. 3.. Multlayer perceptron etwork The artfcal neural network (A), or neural network n short, s nspred by smulatng the functon of a human bran. A neural network can be used to represent a nonlnear mappng between nput and output vectors. eural networks are among the popular sgnal-processng technologes. In 0engneerng, neural networks serve two mportant functons: as pattern classfers and as nonlnear adaptve flters [5] [6]. A general network conssts of a layered archtecture, an nput layer, one or more hdden layers and an output layer [1]. Fgure shows a typcal archtecture of a multlayer perceptron network. The Multlayer perceptron (MLP) s an example of an artfcal neural network that s used extensvely to solve a number of dfferent problems, ncludng pattern recognton and nterpolaton [4][7]. Each layer s composed of neurons, whch are nterconnected wth each other by weghts. In each neuron, a specfc mathematcal functon called the actvaton functon accepts nput from prevous layers and generates output for the next layer. In the experment, the actvaton functon used s the hyperbolc tangent sgmod transfer functon [13] whch s defned as n equaton (1): where n s w 1 are nput values. x s 1 e f n = s 1 e (1), n whch w are weghts and x The MLP s traned usng the Levenberg Marquardt technque as ths technque s more powerful than the conventonal gradent descent technques [4]. Fgure. A typcal Multlayer Perceptron A Archtecture The Levenberg-Marquardt (LM) algorthm [14] s the most wdely used optmzaton algorthm. It outperforms smple gradent descent and other conjugate gradent methods n a wde varety of problems. If a functon V(x) s to be mnmzed wth respect to the parameter vector x, then ewton s method would be: 1 x [ vx] vx () Copyrght 010, Infonomcs Socety 45

4 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 vx s the Hessan matrx and vx the gradent. If v x reads: where v x 1 e then t can be shown that: where J and x x x xex s (3) T v J (4) T x J xj x Sx v (5) J x s the Jacoban matrx e 1 x e x e x x x x 1 e1 x x x x 1 e x x x x s 1 x e x e e 1 x e1 x e x e x For the Gauss-ewton method t s assumed that, and equaton () becomes: s x 0 T 1 T J xj x J xex (6) (7) x (8) The Lavenberg-Marquardt modfcaton to the Gauss-ewton method s: T 1 T J xj x I J xex x (9) The parameter s multpled by some factor (β) whenever a step would result n an ncreased Vx. when a step reduces V x, s dvded by β. When the scalar s very large the Levenberg- Marquardt algorthm approxmates the steepest descent method. However, when μ s small, t s the same as the Gauss-ewton method. Snce the Gauss- ewton method converges faster and more accurately towards an error mnmum, the goal s to shft towards the Gauss-ewton method as quckly as possble. The value of s decreased after each step unless the change n error s postve;.e. the error ncreases. For the neural network-mappng problem, the terms n the Jacoban matrx can be computed by a smple modfcaton to the backpropagaton algorthm [3]. 4. Smulatons Ths secton dscusses data preprocessng, expermental data and model n experment Preprocessng Data At the ntal stage of the experment, data was scaled or normalzed usng equaton (10) x x mn x new (10) xmax xmn Where x s the orgnal data pont, x mn and x max are the mnmum and maxmum values n the data set, respectvely. Ths s done n order to ensure that the mnmum value n the data set s scaled to zero, and that the maxmum value s scaled to one [11]. 4.. Expermental Data In ths study, we use the water qualty data of canal n Bangkok, over a perod of fve years from 003 to 007. The man water qualty ndces nclude. Accordng to the above ndces, the water qualty can be classfed nto 5 categores based on surface water qualty standards n Thaland. 11,80 samples are avalable for the analyss n water qualty classfcaton. Frst step, the rato of the tran and test set employed n the experment s 60:40 randomly. Ths means that wth 11,80 data record, there are 7,09 records for the tran set and 4,78 records for the test set. Second step, we make several dfferent dvsons of the observed data nto tranng set and testng set. K-folds cross valdaton s used to measure the performance of CART and MLP neural network. K- folds cross valdaton s one of the most adopted crtera for assessng the performance of a model and for selectng a hypothess wthn a class. An advantage of ths method, over the smple tranng and testng data splttng, s the repeated use of the whole avalable data for both buldng a learnng machne and for testng t [4]. In ths study, a 3 fold partton of the data set was created. Splt data to 3 fold; hold out successve blocks of observaton as test sets. Each fold s held out n turn and learnng scheme traned on the remanng second-thrd, then the error rate s calculated on the hold out set. Thus, the learnng procedure s executed a total 3 tmes on dfferent tranng sets. Ths means that wth 11,80 Copyrght 010, Infonomcs Socety 46

5 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 data record, there are 7,880 records for the tran set and 3,940 records for the test set CART model and MLP eural etwork Model As n ths secton, shown classfcaton and regresson tree model and MLP neural network model Classfcaton and regresson tree Model An example of the decson tree that generated from the classfcaton and regresson tree algorthm s shown n fgure 3. Classfcaton and regresson tree start wth parent node whch s BOD. The ndependents parameters contan of 5 parameters. These are ph, DO, O 3, H 3 and T-Colform. The CART procedure examnes all possble ndependent, varables and selects one that results n bnary group. In fgure 3, ode 1 (parent node, BOD) splt nto node, 3 whch s H 3 and class 5 of surface water standard. Wthn these nodes, node (H 3 ) become parent node and splt nto nodes whch are node 4 (DO) and node 5 (surface water standard class 5). Smlarly, the tree growng MLP eural etwork Model The Levenberg-Marquardt algorthm uses nput vectors and correspondng target vectors to tran neural networks. All the tranng records were fed nto the network to make t learn the potental relatonshps between water qualty ndces and ther correspondng categores. Accordngly, the 6 nput layer nodes represent 6 water qualty ndces, whle the 5 output layer nodes represent the 5 dfferent class categores. The traned neural networks can provde an output representng the specfc class for each of water qualty ndces. The testng samples are used to verfy ts classfcaton ablty. Many expermental nvestgatons are conducted. The number of hdden nodes that provded the optmal result s 4 hdden nodes. Therefore, the archtecture of the network s The target mean square error (MSE) s after 5000 teratons. Equaton (11) shows the mean square error. MSE = 1 n e n 1 (11) Fgure 3. Example of Classfcaton and Regresson Tree Model Copyrght 010, Infonomcs Socety 47

6 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June Results and Dscusson Comparng wth the usng, CART as a methodology lead to better result than MLP neural net work methodology of confuson matrx as followng fgure 4 and fgure 5. Fgure 4. Example of Confuson Matrx of CART Model Fgure 4 llustrates example of the confuson matrx of CART model and fgure 5 shown example of the confuson matrx of MLP neural network model for the experment results. The confuson matrx demonstrates nformaton about the target (actual class of surface water standard) and the output (predcted class by the network). In the matrx, each column of the matrx represents a target (actual) class, whereas, each row represents an output (predcted) class. From fgure 4 performance of the classfcaton and regresson tree approach can be evaluated usng data n the matrx. The examples of nterpretatons nclude: Vertcally readng from Target Class V, there are 3641 records classfed correctly. The accuracy percentage s % Table. Result of Percentage Accuracy n the Test Set Method Accuracy Percentage (%) Wthout appled to K-foldss Cross Valdaton CART MLP Vertcally readng from Target Class III, there are 13 records classfed correctly. The accuracy percentage s 81.5% Vertcally readng from Target Class IV, there are 83 records classfed correctly. The accuracy percentage s % Wth appled to K-foldss Cross Valdaton Copyrght 010, Infonomcs Socety 48

7 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 Table demonstrates the comparson percentage accuracy of canal water qualty classfcaton between CART methodology and MLP neural network methodology wth and wthout K-foldss cross valdaton. It can be seen that classfcaton and regresson tree and multlayer perceptron neural network hgher performance after appled K-foldss croess valdaton. The CART hgh correctly classfed than multlayer perceptron neural network, accuracy percentage s 99.96%. 6. Concluson In ths paper, k-folds cross valdaton appled to classfcaton and regresson tree (CART) and multlayer perceptron (MLP) neural network usng the Levenberg-Marquardt algorthm are appled to classfy the water qualty of canals of Bangkok, Thaland. The results ndcate that the CART performs wth a hgh accuracy classfcaton percentage of 99.96%, whle the MLP neural network shows percent accuracy of These encouragng results may be appled to automate water qualty classfcatons. As a result, the cost and tme of water resource management could be mnmzed. Applcaton of the CART and MLP neural network ndcates that t s robust and remarkably mproves the effcency of the classfcaton of water pollutons whch are useful for planner and watershed management nutrent loadng, sedmentaton and also water treatment process. 7. Acknowledgment The authors would lke to thank Suan Sunandha Rajabhat Unversty for scholarshp support. Thanks to Department of Dranage and Sewerage Bangkok Metropoltan Admnstraton for the provded data. 8. References [1] A.ajah, A.Elshafe,O.Karm and O.Jaffar Predcton of Johor Rver Water Qualty Parameter Usng Artfcal eural etworks, Journal 0f Scentfc Research, EuroJournals Publshng, 009, pp [] Breman L, FredmanJH, Olshen RA, and Stone CJ Classfcaton and Regresson Tree Champman &Hall (Wadsworth, Inc.), ewyork, [3] Ch Zhou, Lang Gao and Chuanyong Peng, Pattern Classfcaton and Predcton of Water Qualty by eural etwork wth Partcle Swarm Optmzaton, Proceedngs of the 6 th World Congress on Intellgent Control and Automaton, Chna, June 006, pp [4] D. Anguta, S.Rdella and F.Rvecco, K-folds Generalzaton Capablty Assessment for Support Vector Classfers, Proceedng of Internatonal Jont Conference on eural etwork, Canada,005,PP [5] D.Marquardt, An Algorthm for Least Squares Estmaton of on-lnear Parameter, J. Soc. Ind. Appl. Math., pp [6] L.Fausett, Fundamentals of eural etworks Archtecture.Algorthms and Applcatons, Pearson Prentce Hall, USA, [7] L-hua Chen, and Xao-yun Zhang, Applcaton of Artfcal eural etwork to Classfy Water Qualty of the Yellow Rver, Journal 0f Fuzzy Informaton and Engneerng, Sprnger-Verlag, Jan 009, pp [8] L, Y., Jang, J.H., Chen, Z.P., Xu, C.J., Yu, R.Q.: A ew Method Based on Counter Propagaton etwork Algorthm for Chemcal Pattern Recognton, 1999, pp [9] L.Khuan,.Hamzah and R Jalan, Predcton of Water Qualty Index(WQI) Based on Artfcal eural etwork(a),conference on Research and Development Proceedngs, Malasa, 00, pp [10] L.Khuan,.Hamzah and R Jalan, Water Qualty Predcton Usng LS-SVM wth Partcle Swarm Optmzaton,Second Internatonal Workshop on Knowledge Dscovery and Data Mnng, Chna, 009, pp [11] Martn T.Hagen and Mohammad B.Menhaj, Tranng Feedforward etworks wth the Marquardt Algorthm, IEEE Transactons on eural etworks,vol.5, no.6,ov 1994, pp [1] M.J. Damantopoulou, V.Z. Antonopoulos and D.M. Papamchal The Use of a eural etwork Technque for the Predcton of Water Qualty Parameters of Axos Rver n orthern Greece, Journal 0f Operatonal Research, Sprnger-Verlag, Jan 005, pp [13] S.Areerachakul and S.Sanguansntukul Water Classfcaton Usng eural etwork: A Case Study of Canals n Bangokok, Thaland, The 4th Internatonal Conference for Internet Technology and Secured Transactons (ICITST-009), Unted Kngdom, 009. [14] S.H.Musav and M.Golab Applcaton of Artfcal eural etworks n the Rver Water Qualty Modelng: Karoon Rver,Iran, Journal 0f Appled Scences, Asan etwork for Scentfc Informaton, 008, pp [15] Smon Haykn, eural etworks:a Comprehensve foundaton second edton, Pearson Prentce Hall, Delh Inda, 005. [16] S.Lek, Guegan, J.F. (eds.): Artfcal eural etworks: Applcaton to Ecology and Evoluton. Sprnger, Berln, 000. [17] Walley, W.J., DZerosk,S Bologcal montorng A Comparson between Bayesan, neural and machne learnng methods of water qualty classfcaton, Copyrght 010, Infonomcs Socety 49

8 Internatonal Journal of Intellgent Computng Research (IJICR), Volume 1, Issue 1/, March/June 010 Internatonal Symposum on Envronmental Software System, [18] Mnstry of atural Resources and Envronment: [19] Department of Dranage and Sewerage Bangkok Metropoltan Admnstraton: wqm/tha/home.html Copyrght 010, Infonomcs Socety 50