Outliers and/or measurement errors on the permanent sample plot data. M. Hordo, A. Kiviste, A. Sims, M. Lang

Transcription

1 Outlers and/or errors on the permanent sample plot data M. Hordo, A. Kvste, A. Sms, M. Lang Estonan Agrcultural Unversty, Insttute of Forestry and Rural Engneerng, Kreutzwald 5, Tartu, Estona. Emal: Abstract Tree-wse re-measured permanent sample plot data are needed to create and mantan modern forest growth models for Estona. For that purpose an all-estonan network of forest sample plots was establshed by the Department of Forest Management at Estonan Agrcultural Unversty n Tree speces, storey, azmuth, dstance from plot centre, two breast heght dameters and faults were determned for each tree, and total heght, crown base and heght of dry branches were measured for sample trees. Routne nspecton of forest plot data for outlers was necessary because t provded nformaton about the causes and consequences of errors. At the moment, 715 forest sample plots have been establshed. A total of 749 outlers have been checked for errors on 11 plots n 004. In ths study several statstcal methods were tested for the detecton of outlers. Emprcal dstrbutons of most tree varables by speces and storey were analyzed usng Grubb s test, Dxon s test and the -sgma rule. Multvarate methods (resdual dagnostcs) were used for detectng outlers n the nteracton of several varables. Analyzng emprcal dameter dstrbutons, Dxon s test for small samples and -sgma rule for large samples were more effectve than Grubbs test. The results of regresson dagnostcs of nfluental observatons showed that Covrato method was the most senstve one for outler detecton n nonlnear model and Dffts method n lnear models. The most effectve method for checkng outlers from heght-dameter data was the studentzed resdual. Outler detecton plays mportant role n modelng, nference and even data processng because outlers can lead to model msspecfcaton, based parameter estmaton and poor forecast. Introducton In forestry stand and tree-wse growth and yeld modelng s a relevant queston today. The key to the successful sustanable tmber management s a proper understandng of the 1

2 growth processes, and one of the objectves of forest development modelng s to provde the tools that enable foresters to compare alternatve slvcultural approaches. The current Estonan forest structure models are poor (Nlson, 1999). Both adequate forest stand descrptons and stand growth and structure models are needed for the effectve use of the system. The development of forest growth and yeld based on stand-level models, dameter dstrbuton models and ndvdual tree models s a response to changng management objectves (Pretzsch et al., 00). Growth modelng s also an essental prerequste for evaluatng the consequences of a partcular management acton for the future development of an mportant natural resource, such as a woodland ecosystem. Forest models represent average experence on how trees grow and how forest structures are modfed (Gadow and Hu, 001). The accuracy of the growth and mortalty models, s known, the nterest usually les n the development of stand volume (or other stand characterstcs), the accuracy of whch cannot be drectly stated. It s very mportant to know the accuracy of predcted characterstcs (Kangas, 1997). So the outler detecton plays an mportant role n modelng, nference and even data processng because outlers can lead to model msspecfcaton, based parameter estmaton and poor forecasts (Tsay et al., 000; Fuller, 1987) Tree-wse re-measured permanent sample plot data are needed to create and mantan modern forest growth models for Estona. As 715 forest sample plots have been establshed at the moment, routne nspecton of forest plot data for outlers s necessary because t provdes nformaton about the causes and consequences of errors (Hordo, 004 1, ). Statstcans recommend that data be routnely nspected for outlers, because outlers can provde useful nformaton about the data. At the same tme, methods for dentfyng outlers need to be used carefully, as t s qute easy to confuse dscordant random observatons and outlers (Iglewcz and Hoagln, 1993; Carroll et al., 1995). In the study the term outler means an observaton that has a substantal dfference between ts actual and predcted dependent varable (a large resdual) values or between ts ndependent varable values and those of other observatons. The objectve of denotng outlers s to dentfy observatons that are napproprate representatons of the populaton from whch the sample s drawn, so that they may be dscounted or even elmnated from the analyss as unrepresentatve (Har et al, 1998). The ams of ths paper are: Overvew of several tests for the detecton of outlers from permanent sample plot data;

3 Comparson and estmaton of effcency of dfferent outler tests for detectng errors. Materal and methods Network of permanent sample plots Data on the network of permanent forest growth plots were used. A method of establshng a network of sample plots was developed at the Department of Forest Management of the Estonan Agrcultural Unversty. The method of establshng permanent forest growth plots s manly based on the experence of the Fnnsh Forest Research Insttute (Gustavsen et al., 1988). Some recommendatons from Curts (1983) were taken nto consderaton. In ths study data on 715 sample plots were ascertaned. On each plot, s of more than 100 trees were recorded n The check program was appled on 715 sample plots and was re-measured on 11 sample plots n the summer 004. Each forest element (storey and tree speces) on sample plot was checked as ndependent sample. Tree speces, storey, azmuth, dstance from the plot centre, two perpendcular breast heght dameters and faults were measured for each tree, and total heght, crown length and heght of dry branches were measured for model trees (Kvste, Hordo, 003). Analyss In the study, several methods were used to detect outlers for the purpose of fndng out errors n the permanent sample plot database arsng from data collectng, enterng or from wrong codng. In ths study, authors were focused on outler detecton methods, whch may be dstngushed as follows: dstrbuton base tests and regresson dagnostcs. Dstrbuton based tests Dstrbuton based tests were used for detectng outler from dameter dstrbuton and were appled all trees. Emprcal dstrbutons of most tree varables by speces were summarzed and several statstcal crtera were elaborated to detect outlers n the data set. All the methods frst quantfy how far the outler s from the other values. Ths can be the dfference between the outler and the mean of all ponts, the dfference between the outler and the mean of the remanng values, or the dfference between the outler and the next closest value (Grubbs, 003). The followng tests were used to detect outlers/ error from dameter dstrbuton: 3

4 Grubbs test s used to detect outlers n a unvarate data set (sample N > 3). It s based on the assumpton of normalty. That s, you should frst verfy whether your data could be reasonably approxmated by a normal dstrbuton before applyng the Grubbs test. ( ) d d The Grubbs test statstc s defned as: G = max where d and s d are the sample s mean and standard devaton. The Grubbs test statstc s the largest absolute devaton from the sample mean n unts of the sample standard devaton (Motulsky, 1999; Grubbs, 003). Test statstc was decded n terms of the bound gven n the table by Grubbs (003). d Dxon s test (sample N 3...5) s used to fnd out whether the mean ( ) of some data set dffers substantally from the k 1 mutually dfferent mean of other data sets, orderng them by magntude - n ncreasng order f the mean n queston s the smallest, n descendng order f t s the largest. The sgnfcance bounds depended on the sample N. d1 d3 Then the test statstc D = was computed (e. g., for 3 N 7) and decded n d d terms of the bound gven n the table by Sachs (198). 1 max d The -sgma regon was recommended by Sachs (198), Sheskn (000), Iglewcz and Hoagln (1993) for sample sze larger than N = 5 the extreme values and was tested by x µ means of the test statstc, T = 1 where x 1 s the supposed outler and µ s mean- δ value and σ s standard devaton. The value s dscarded as an outler f T equals or exceeds the regon µ ± σ where the mean and standard devaton are computed wthout the value suspected of beng an outler. Resdual dagnostcs heght-dameter check An assumpton of regresson s that the predctor varables are measured wthout errors. Several resdual dagnostcs were for detectng outlers from heght dameter dataset used and were appled on sample trees dataset. Two regresson (heght curve) models were used to detect heght errors, outlers from sample trees: logarthmc functon H a + b ln( D) = ; smple lnear functon H = a + b D. Regresson dagnostcs were developed to measure varous ways n whch a regresson relaton mght derve largely from one or two observatons. Observatons whose ncluson or excluson result s n substantal changes n the ftted model (coeffcents, ftted values) are sad to be nfluental (Har et al., 1998; Tsay et al., 000; Hordo, 004 1, ). Outlers n the 4

5 response varable represent model falure. The followng methods are used for detectng outlers from sample plots: Leverage method h ( x x) 1 n = +, where v = ( 1 n v = x x), n - sample sze, x th observaton, x - mean. Outler was detected f h > 0.5. Leverage - an observaton that has substantal mpact on the regresson results n ts dfferences from other observatons on one or more of the ndependent varables. The most common measure of the leverage pont s the hat value, contaned n the hat matrx (Har et al., 1998; Iglewcz and Hoagln, 1993). Standardzed resdual s = sˆ e 1 h, where e resdual, ŝ - standard devaton of resduals, h leverage of th observaton. Outler was detected then s > ±. Standardzed resduals are the resduals dvded by the estmates of ther standard errors. Ther mean s 0 and standard devaton s 1. There are two common ways to calculate the standardzed resdual for the -th observaton. One uses the resdual mean square error from the model ftted to the full model (nternally studentzed resduals). The other uses the resdual mean square error from the model ftted to all the data except the th observaton (externally studentzed resduals). The externally standardzed resduals follow a t-dstrbuton wth n-p- df. They can be thought of as testng the hypothess that the correspondng observaton does not follow the regresson model that descrbes the other observatons (Har et al., 1998; Iglewcz and Hoagln, 1993). Studentzed resdual Stud = r = sˆ e 1 h (), where e resdual, ŝ() - standard devaton of resduals, h leverage of th observaton. Outler was detected then Stud > ± (Har et al., 1998; Iglewcz and Hoagln, 1993). Cook Dstance D r = p h ( 1 h ), where r studenzed resdual, p number of parameters, h leverage of th observaton. Outler was detected then D >. Cook s n Dstance for the th observaton s based on the dfferences between the predcted responses from the model constructed from all the data and the predcted responses from the model constructed by settng the th observaton asde. For each observaton, the sum of squared resduals s dvded by (p+1) tmes the Resdual Mean Square from the full model (Becker, 000; Har et al., 1998). 5

6 Dffts test F ˆ µ ˆ µ () =, where µˆ - predcted value for th observaton, µˆ () - s h () predcted value for th observaton wthout th observaton, wthout th observaton. Outler was detected then s() - standard devaton of resdual F >. Dffts measure of an n observaton s mpact on the overall model ft. Dffts s the scaled dfference between the predcted responses from the model constructed from all the data and the predcted responses from the model constructed by settng the th observaton asde (Har et al., 1998). ( ) ( ) 1 1 ' s() X () X () Covrato test C =, where X ' () - observaton matrx wthout the s X X observaton, detected then s() - standard devaton of resduals wthout the th observaton. Outler was C 1 > 3p. Covrato measure of the nfluence of a sngle observaton on n the entre set of estmated regresson coeffcents (Har et al., 1998). th Dfbetas test DFBETAS observaton, j, ˆ β j ˆ β j = s C () jj (), where - regresson coeffcent wthout βˆ j() - regresson coeffcent wth th observaton, s() - standard devaton of resduals wthout the th observaton, C jj βˆ j - dagonal element of Covrato matrx. Outler was detected then DFBETAS J I, >. Dfbetas measure of the change n a regresson n coeffcent when an observaton s omtted from the regresson analyss. Dfbetas are smlar to Dffts. Instead of lookng at the dfference n ftted value when the th observaton s ncluded or excluded, Dfbetas looks at the change n each regresson coeffcent (Har et al., 1998). th Results A Vsual FoxPro program was elaborated to detect errors and outlers n the forest sample plot data. In 004 total of 11 sample plots were re-measured. Usng data measured n 1999 on those 11 sample plots, totally of 749 outlers were detected by the test program. Applyng at the same tme by test program several (dstrbuton and resdual dagnostc based) outlers detecton methods were detected more than one outler on tree. Durng re- were outler trees carefully checked, was t error or not. Addtonally were detected errors by measurer, what the test program was 6

7 not able to detect. Totally 415 errors of dameter (Table ) were detected by re- and 508 errors of heght (Table 3). Dstrbuton based tests Grubbs test s unversal for outler detecton for small (Fgure 1) and large sample (Fgure ); however only one observaton from sample sze can have detected as outler. In the study each forest element was checked as sample. As an example, n the Fgure 1 s shown vsual pcture of sample plot number 19, frst storey pne. In the Fgure 1 was sample of fve dameters tested and the maxmum dameter was as outler detected, but actually t was correct observaton. In the Fgure, accordng Grubbs test for sample sze n > 50, also maxmum dameter was detected as outler, but that was actually error. By Grubbs test was detected 43 outlers (Table ), and only three s were detected as errors from outlers. Grubbs' Sample plot 19; Scots pne,0 1,8 1,6 1,4 1, 1,0 0,8 0,6 0,4 0, 0, Dameter cm Fgure 1. Sample plot number 19 and sample sze n = 5 then G > 1.71; wth dot s marked outler (not error). Grubbs' Sample plot 16; Norway spruce 4,0 3,5 3,0,5,0 1,5 1,0 0,5 0, Dameter cm Fgure. Sample plot number 16 and sample sze n > 50 then G > 3.61; wth cross + s marked outler/ error. 7

8 Dxon s test can be used for smaller sample (3 n 5), and more than outler can be detected. As an example, n the Fgure 3, all Dxon s test statstcs values above the crtcal lne (Dxon > 0.5) were outlers, and n ths case t s also error. By Dxon s test were 33 outlers and only 5 dameter errors of outlers detected (Table ). 1,8 Sample plot 175; Aspen 1,6 1,4 1, Dxon 1,0 0,8 0,6 0,4 0, 0, Dameter cm Fgure 3. Sample plot number 175 and sample sze n = 15 then T > σ; wth cross + s marked outler/ error. -sgma rule can be used for large sample (n 5). As an example, n the Fgure 4, outlers were detected and they all were errors. In addton, four other errors were found durng the re-, but test was not able to detect them. By -sgma rule were 74 outlers and only 19 dameter errors of outlers detected (Table ). -sgma regon 6 5,5 5 4,5 4 3,5 3,5 1,5 1 0,5 Sample plot 19; Norway spruce Dameter cm 8

9 Fgure 4. Sample plot number 19 and sample sze N > 5 then D > 0.55; wth crosses + are marked outler/ errors, and wth are errors (not detected as outlers). Table 1 shows a comparson of results from the dameter dstrbuton tests. Regardng the detecton of errors for dameter dstrbuton, Dxon s method (sutable for small samples) and the -sgma method (sutable for bgger samples) detected more outlers than Grubb s test. Grubbs test detects one outler at a tme. Ths outler s expunged from the dataset and the test s terated untl no outlers are detected. Table 1. dameter dstrbuton outlers on permanent sample plots excluded from the dfferent tests. 0 no outlers detected, 1 outlers detected sample N Grubbs' test Total Dxons' test Total sample N > 5 Grubb's test Total -sgma rule Total Results from dameter dstrbuton are presented n Table. Grubb s test s neffcent, method were 43 outlers detected, but only three were n the same tme errors. Dxon s test for smaller and -sgma regon for larger sample sze seems to be better. Table. Results from dameter dstrbuton control. Method outlers errors from outlers % Total number of errors errors from outlers % Dxon's test (3 n 5) sgma regon (n 5) Grubb's test (n > 3) Heght dameter control 9

10 A total of 7580 heght-dameter from sample trees were analyzed. In the study on two regresson models lnear and logarthmc for outler detecton (usng resdual dagnostcs) was studed. For each sample as forest element (storey and tree-wse) was predcted model parameters and was used n resdual dagnostc. For resdual dagnostc were used several methods. In ths study, for checkng conformty between outler methods and errors were used ch-square test. As an example, n the Fgure 5 shows vsually, detected outlers wth leverage method les above the lne. Ths presents all observatons (sample trees) from the dataset. Leverage method shows nfluental observatons. Accordng Table 3 and Table 4 by leverage method were detected 149 outlers usng lnear model and 73 outlers usng logarthmc model. Chsquare was hgher for functon of lnear regresson (Table 3) 39.6 and for functon of logarthmc functon only Leverage Heght m Leverage Heght m Fgure 5. Leverage method, accordng lnear (left) and logarthmc (rght) model. In the Fgure 6, as an example, are presented standardzed resduals, outsde the lnes (crtera ± ) are outlers. For next test, n the Fgure 7, studentzed resduals also observatons outsde the lnes are outlers, used for all observatons. From lnear model were detected 189 outlers wth standardzed resdual test and from logarthmc model 35 outlers. Accordng Table 3 and Table 4 the ch-square were almost equal, respectvely 85. and 83.. Standardzed resduals Heght m Standardzed resduals Heght m Fgure 6. Standardzed resduals, accordng lnear (left) and logarthmc (rght) model. 10

11 Studentzed resduals Heght m Studentzed resduals Heght m Fgure 7. Studendzed resduals accordng lnear (left) and logarthmc (rght) model. Cook s Dstance method was used also for lnear and logarthmc models (Fgure 8). In ths case crtcal value depends on sample sze. In the Fgure 8 shows, usng lnear model for Cook s D testng, two outlers were detected, they both occur errors. Addtonally were detected durng re- four errors what were not outlers. And usng logarthmc model was detected only one outler/ error. 0,8 Sample plot 103; Brch Sample plot 103; Brch 0,7 0,6 0,5 Cook's D Cook's D 0,4 0,3 0, 0, , , Heght m Heght m Fgure 8. Cook s Dstance method, from lnear (left fgure) and logarthmc (rght fgure) models (wth cross + s marked outler/ error and wth error, not detected as outler). Table 3. Results from heght control, functon of lnear regresson were used. Method outlers errors from outlers % Total number of errors errors from outlers % Ch-sq Leverage Standardzed resdual Studentzed resdual Cook's D Covrato Dffts

12 Dfbetas From analyzed several resdual dagnostc methods, Table 3 and Table 4 show the results of resdual dagnostc from heght-dameter check. Comparng lnear and logarthmc model wth ch-square test, a result show that on both cases the studentzed tests were best. The results of regresson dagnostcs of nfluental observatons show that the Covrato method was the most senstve one for outler detecton. Covrato method was detected the hghest number of outlers 676 and 655, respectvely from lnear and logarthmc regresson Table 4. Result from heght control, functon of logarthmc regresson was used. errors from outlers % Total number of errors errors from outlers % Ch-sq Method outlers Leverage Standardzed resdual Studentzed resdual Cook's D Covrato Dffts Dfbetas Dscusson Many statstcal technques are senstve to the presence of outlers. Checkng for outlers should be a routne part of any data analyss. Potental outlers should be examned to see f they are possbly erroneous. If a data pont s n error t should not be deleted wthout careful consderaton. In general the outler dentfcaton procedure presumes data from normal dstrbuton, as appears from a comparson of Grubbs test wth Dxon s test and -sgma regon method. For Grubbs test t s hghly recommended that result data come from normal dstrbuton (Iglewcz, Hoagln, 1993). Accordng to Sachs (198) Dxon s test s rather prone to devatons from normalty and varance homogenety, snce by the central lmt theorem, the means of non-normally dstrbuted data sets are themselves approxmately normally dstrbuted. In practcal stuatons dameter data are hghly skewed, usually to the rght. As an example, Fgure 9 shows the tree mean dameter dstrbuton s a unmodal on sample plots; dstrbuton s skewed to rght (skewness 0.996). Accordng to Iglewcz and Hoagln (1993) n such stuatons, a lognormal dstrbuton s frequently more approprate than a normal dstrbuton. 1

13 Fgure 9. Dameter dstrbuton of trees on the 715 permanent sample plots. Heght-dameter regresson models are more complcated, the relatonshps n models are not lnear. The varous dagnostc measures am at enablng the user of least-squares regresson to cope wth the fact that all observatons, no matter how dscrepant, have an mpact on the regresson lne (Iglewcz and Hoagln, 1993). Outlers n (heght-dameter) data may take more form than n unvarate data, and detectng them offers more challenges. The stuaton s more challengng because some (heght-dameter) data ponts have more mpact than others on the ftted regresson lne. Iglewcz and Hoagln (1993) have recommended specalzed dagnostc technques, and to use robust methods, nstead of least squares. Carroll et al. (1995) have summarzed some of the known results about the effect of error n lnear regresson, but a comprehensve account of lnear error models can be found n Fuller (1987). Usually focusng on smple lnear regresson and arrvng at the concluson that the effect of error s to bas the slope estmate n the drecton of 0 (Carroll et al., 1995). Conclusons At the moment, 715 forest sample plots have been establshed n Estona. A total of 749 outlers have been checked for errors on 11 plots n 004. Measurement error enters nto forestry n many dfferent forms. The errors can have very negatve effects on model parameters, model estmates, and the varances of model parameters and model estmates. In ths study several statstcal methods were tested for outler detecton. Emprcal dstrbutons of most tree varables by speces and storey were analyzed usng Grubb s test, Dxon s test and the -sgma regon method. Multvarate methods (resdual dagnostcs for 13

14 lnear and nonlnear models) were used for detectng outlers n the nteracton of several varables. Analyzng emprcal dameter dstrbutons, Dxon s test for small samples and -sgma rule for large samples are more effectve than Grubbs test. The results of regresson dagnostcs of nfluental observatons showed that Covrato method was the most senstve one for outler detecton n nonlnear model and Dffts method n lnear models. The most effectve method for checkng outlers from heght-dameter data was studentzed resdual. In ths study, n the outler analyss t was dscovered that most of the outlers tested and put out by the control program were caused by stand natural dsturbance regme, lke mortalty, competton, spatal locaton, dseases, effect of weather (frost damage, temperature), etc. As well, a large amount of the detected outlers n the changes of tree dmensons was caused by ungulate herbvores (lke elk eatng spruce bark), effect of the harvest (ntensty) as well as the effect of deforestaton, understory, etc. Outler detecton plays mportant role n modelng, nference and even data processng because outlers can lead to model msspecfcaton, based parameter estmaton and pore forecast. Acknowledgements The establshment of the network of permanent forest growth plots was supported by the Estonan Scence Foundaton (Grants No 4813 and 5768), the Estonan State Forest Management Centre and the Centre of Envronmental Investments. References Barnett, V. (1978) The Study of Outlers: Purpose and Model. Appled Statstc 7, No 3. pp Becker, C. (000) Performance Crtera for Multvarate Outlers Identfcaton Procedures. Lecture notes. pp Carroll, R. J., Ruppert, D. and Stefansk, L. A. (1995) Measurement Error n Nonlnear Models. Chapman & Hall/CRC. Curts, R. O. (1983) Procedures for Establshng and Mantanng Permanent Plots for Slvcultural and Yeld Research. USDA General Techncal Report PNW-155, 56 pp. Fuller, W. A. (1987) Measurement error models. John Wley & Sons. Gadow, K. v. and Hu, G. (1999) Modellng forest development. Kluwer Academc Publshers. Grubbs (003) 14

15 Gustavsen, H. G., Roko-Jokela, P. and Varmola, M. (1988) Kvennäsmaden talousmetsen pysyvat (INKA ja TINKA) kokeet. Suunntelmat, mttausmenetelmät ja anestojen rakenteet. Metsäntutkmuslatoksen tedonantoja 9, 1 pp. [n Fnnsh] Har, J. F. Jr., Anderson, R. E., Tatham, R. L. and Black, W. C. (1998) Multvarate Data Analyss, 5 th edn. Prentce-Hall, New Jersey. Hordo, M. (004) 1 Erndte dagnostka pustu kasvukägu püsproovtükkde andmestkul. MSc thess, Estonan Agrcultural Unversty. [n Estonan] Hordo, M. (004) Outler dagnostcs on permanent sample plot network n Estona. Research for Rural Development 004. Internatonal Scentfc Conference Proceedngs. Latva Unversty of Agrculture, Jelgava, pp Iglewcz, B. and Hoagln, D. C. (1993) How to Detect and Handle Outlers. Amercan Socety of Qualty, Statstcs Dvson, 16, Wsconsn. Kangas, A. S. (1997) On the predcton bas and varance n long-term growth projectons. Forest Ecology and Management, 96. pp Kangas, A. S. (1998) Uncertanty n growth and yeld projectons due to annual varaton of dameter growth. Forest Ecology and Management, 108. pp Kvste, A. and Hordo, M. (003) The network of permanent sample plots for forest growth modellng n Estona. Research for rural development 003. Internatonal Scentfc Conference Proceedngs. Latva Unversty of Agrculture. Pp Motulsky, H. J. (1999) Analyzng Data wth GraphPad Prsm. GraphPhad Software Inc., San Dego CA. Nlson, A Pdev metsakorraldus ms see on. Pdev metsakorraldus. EPMÜ Metsandusteaduskonna tometsed nr. 3. Lk [n Estonan] Pretzsch, H., Bber, P., Ďurský, J., von Gadow, K., Hasenauer, H., Kändler, G., Kenk, G., Kubln, E., Nagel, J., Pukkala, T., Skovsgaard, J. P., Sodtke, R. and Sterba, H. (00) Recommendatons for Standardzed Documentaton and Further Development of Forest Growth Smulatons. Forstwssenschaftlches Centralblatt, 11. Berln, pp Sachs, L. (198) Appled Statstcs: A Handbook of Technques. Sprnger Seres n Statstcs, Sprnger-Verlag, New York. Sheskn, D. J Handbook of parametrc and nonparametrc statstcal procedures. nd ed. Chapman & Hall/CRC. The Unted States of Amerca. Sokal, R. R. and Rohlf. F. J. (1997) Bometry: the prncples and practce of statstcs n bologcal research. 3 rd edn. W. H. Freeman and Company, New York. Tsay, R. S., Pena, D. and Pankratz, A. E Outlers n multvarate tme seres. Bometrka. 87, 4, pp