OBJECT TRACKING IN VIDEOS BY EVOLUTIONARY CLUSTERING AND LOCALLY LINEAR NEURO-FUZZY MODELS

Transcription

1 F. Saadian Paćenje pedmea u video snimkama pomoću modela evolucijskog gupianja i lokalno lineanih neuo-fuzzy modela ISSN (Pin), ISSN (Online) DOI:.7559/TV OBJECT TRACKING IN VIDEOS BY EVOLUTIONARY CLUSTERING AND LOCALLY LINEAR NEURO-FUZZY MODELS Faemeh Saadian Oiginal scienific pape In his pape a new mehod based on evoluionay cluseing and locally linea neuo-fuzzy (LLNF) models is poposed fo he poblem of objec acking in videos. This appoach uilizes cluseing on colo feaue space o obain a model of objec which is given a he iniial fame. To achieve he opimal cluseing, evoluionay opimizaion mehods ae used. Based on he esuls of cluseing, paamees of LLNF model is deemined so i can be used as an idenifie of objec duing he eal video seaming. To ack he objec, a swam of weighed evolving linea models ae used o esimae he locaion and size of he objec a nex fame based on is cuen and pevious saes. The pefomance of he poposed mehod is evaluaed on a benchmak daa se and compaed o ohe mehods pefomed on he same daa se. The esuls show ha he accuacy of he poposed mehod is supeio o pevious mehods. Keywods: cluseing; evoluionay compuaion; Locally Linea Neuo-Fuzzy model; objec acking; swam opimizaion Paćenje pedmea u video snimkama pomoću modela evolucijskog gupianja i lokalno lineanih neuo-fuzzy modela Izvoni znansveni članak U adu se za poblem paćenja objeka u video snimkama pedlaže nova meoda koja se zasniva na modelima evolucijskog gupianja i lokalno lineanim neuo-fuzzy modelima (LLNF). Taj pisup koisi gupianje na posou kaakeisike u boji kako bi se dobio model pedmea zadanog u počenom okviu. Za posizanje opimalnog gupianja, pimijenjene su meode evolucijske opimizacije. Na emelju ezulaa gupianja odeđeni su paamei LLNF modela e se on može abii kao idenifikao objeka ijekom video pijenosa u ealnom vemenu. Za paćenje objeka koisi se oj pondeianih izvedenih lineanih modela za pocjenu lokacije i veličine objeka u sljedećem okviu na emelju njegovog posojećeg i pehodnih sanja. Učinkovios pedložene meode pocijenjena je na efeennom nizu podaaka i uspoeđena s dugim meodama povedenim na isom nizu podaaka. Rezulai pokazuju da je po očnosi pedložena meoda bolja od pehodnih. Ključne iječi: evolucijski izačun; gupianje; lokalno lineani neuo-fuzzy model; opimizacija oja; paćenje pedmea Inoducion The advancemens of machine vision echnology in ecen decades have povided a boad ange of new capabiliies fo human beings. Cuenly, he applicaions of machine vision, image and video pocessing include, bu ae no limied o, biology and medical applicaions, space and aeonauics, suveillance and affic conol, oboics echnology, spos, as and media, ec. [, 2] One of he mos impoan and challenging issues in machine vision is acking of moving objecs in videos in eal applicaions. Seveal aspecs of his poblem make i elaively had fo sysem developes: I is ofen necessay o pefom eal compuaions, even somes wih a fase oupu geneaion han he sensoy sampling of he sysem, which means one needs he oupu of acking algoihm befoe he aival of nex image in he sequence of video ecoding. Objecs wih high speeds and vaiaions in hei movemen ae naually moe difficul o be acked. Changes in he posiion of camea, he sucue, lighing and sceney of he backgound, and also vaiaions in shape, appeaance and oienaion of he objecs ae ohe possible challenges in a acking poblem. The nex secions of his pape summaize as follows. In secion 2 he pevious woks and lieaue elaed o he poblem of objec acking in videos ae eviewed which include eseach in eihe applicaions o mehods. Secion 3 inoduces he poblem of objec acking in a moe fomal and definiive way. Tha pa also descibes seveal challenges in applicaion of diffeen appoaches o his poblem. In secion 4 of he pape he feaues as used o model and idenify objec in an image (fame in video) ae descibed, and he poblem of cluseing fo he feaue space is inoduced. Also in ha secion he evoluionay appoach o he cluseing poblem is descibed. The conceps of finess funcion, geneic algoihm, and paicle swam opimizaion ae pesened o be uilized fo cluseing of poins in he feaue space. Locally Linea Neuo-Fuzzy (LLNF) model as a main pa of he poposed acking mehod in his pape is pesened in secion 5. Secion 6 povides a holisic descipion of he poposed acking mehod which combines conceps of evoluionay algoihms, cluseing, and LLNF. In secion 7 he daa used fo evaluaion, he esuls of opeaing he poposed mehod on daa se, and compaison wih ohe appoaches and mehods ae pesened. Secion 8 povides he concluding emaks. 2 Relaed woks The poblem of objec acking has been sudied by many eseaches duing ecen decades, and wih a divese ange of appoaches [3]. The challenges of his poblem fo diffeen applicaions and siuaions ae no he same. Many of pevious woks have been focused on acking of human face in videos [4, 5]. Ohes have been concened on oboic applicaions of objec acking [6, 7, 8]. One of he mos familia appoaches o objec acking is Kalman file acking and is vaious exended vesions [9,, ]. Kalman files assume linea models fo sae evoluion of a sysem and y o esimae he undelying dynamics based on sequenial obsevaions of oupus. Bu he need fo genealizaion o nonlinea sysems and non-gaussian noise on obsevaions has Tehnički vjesnik 24, 3(27),

2 Objec acking in videos by evoluionay cluseing and locally linea neuo-fuzzy models made eseaches o popose moe complex mehods [2 5]. In he conex of objec acking in videos, paicle files have aaced eseaches in ecen yeas [6 8]. One of he challenges in objec acking is o povide a simple bu obus model of backgound and/o he objec iself. Many of pevious woks have ied o model he backgound of videos [9, 2], bu his appoach only woks well when he backgound is nea seady, fo example when camea is no moving iself. In any case, i is almos necessay o segmen objec in he backgound. To achieve such segmenaion, cluseing appoaches have been used in pevious woks [2, 22]. Weng e al. poposed a combinaion of k-means cluseing and exended Kalman files fo objec acking [23]. Recenly, uilizaion of neual newoks [24] and neuo-fuzzy models [25] have been also inoduced in lieaue fo he poblem of acking. The benefis of hese models include hei high capabiliy in mapping nonlinea elaions and hei genealiy of applicaion. Using boosed leaning is also one of he advanced appoaches o he poblem of objec acking in videos [26]. To implemen hese geneal mehods fo he poblem of acking, one needs o se hei paamees as a ask of leaning. The appoach of using evoluionay compuaion fo leaning of fuzzy neual newoks is one of he mos eliable saegies [27]. 3 Objec acking poblem Geneally, in objec acking asks one ies o idenify, locae and pedic he moion of one o seveal objecs in a video signal as a sequence of odeed images (fames). Based on diffeen cicumsances, he poblem of acking can be consideed in diffeen vesions. In some of he acking asks, he objec of inees is pedefined o he algoihm by use, bu in ohe acking asks he objec may no be pedeemined o he sysem. As an example, in some suveillance sysems, he deecion of a suspicious moion in he seady backgound may be consideed as a pa of he sysem which deemines he moving objec auomaically, wihou he need fo pedeeminaion by a use. Anohe issue is he movemen of camea, as fo he seady camea sysems one may have a simple backgound model. In single objec acking, he sysem ies o locae he posiion of a single objec, bu in ohe muli-objec acking poblems hee could be seveal objecs of inees. In his secion we fomalize he definiion of objec acking poblem ha we addess in his pape. A video signal is a funcion of discee and wo dimensions of posiion (we do no conside deph in his pape) fom poins in a subspace of N 3 o a gey (o colo) inensiy vaiable c which is geneally in R (o R 3 ) space. We define T as he sequence (se) of naual numbes fom o f which couns he numbe of fames. Also conside sceen S as a subspace of N 2 which defines he se of all possible pixel posiions in a fame. Then a video V maps an inensiy value I(, x, y) = c C o each pixel (x, y) S a a T: V = I(, x, y) : T S C () F. Saadian whee TT N, SS N 2, and CC R (o R 3 fo RGB colo signals). An objec OO a he is a subse of pixels in sceen S. Then acking is o find an esimaion OO in a way ha Ô O i.e. Ô O O and Ô O O (2) i means we wan o find he se of pixels OO as much as equivalen o OO. Pacically in implemenaions, we will define bounding boxes BBBB and BBBB fo OO and OO, and y o maximize hei muual ovelap. In addessing such acking poblems, hee ae seveal consideaions which should be aken ino accoun. In some of he siuaions one migh be able o incopoae some assumpions ino he poblem o make i easie o solve. On he ohe hand, in some siuaions camea movemens, he vaiaions in moion paen, shape and oienaion of he objec, and also vaiaions in lighing siuaions may need o be aken ino accoun and he soluion mehod can be moe difficul. In a complee acking poblem, seveal seps have o be pefomed. Fis is he definiion o deecion of he objec. In some sysems, he objec will be deeced by is novely o disinguishable movemen in he backgound. In mos of such siuaions, a nea seady model of backgound can be defined. Then evey egion of a fame which is no coelaed o ha backgound model can be suspeced as a foegound objec of inees. In a simple siuaion, one may assume a seady backgound and use he diffeenial image (he diffeence of wo consecuive fames) o find he moving objec. Bu in such assumpions, one has o be caeful abou issues like empoal sopping of he objec, moving in deph, o oaions. Anyway, eihe by auomaic deecion o pedefiniion of objec by use, he nex sep is o idenify he objec in each fame, by esimaing he locaion of pixel ses assigned o i on he sceen se. In his pape, we use modeling of he objec based on is feaues, and hen find egions of sceen wih same feaue conen o he objec. The ohe impoan sep in acking is he pedicion of he locaion of he objec in he nex fame. I is no only impoan because of is applicaion elaed necessiy, bu is impoan because i can cause a significan impovemen in he speed of compuaions since i educes he seach space in he sceen fo he objec a he nex fame. As an example fo applicaion necessiy of locaion pedicion (in many woks his sep is acually called he acking sep) conside designing a obo am wih ask of gasping a moving ball. 4 Cluseing in colou feaue space using evoluionay mehods In his secion we descibe he pocedue of exacing a model fo he objec, which is assumed o be given by use in ou consideaions in his pape. Theefoe, a he beginning of he acking pocedue, a egion in he fis fame (o a bounding box aound i) is deemined by use of he objec. Then auomaically, he sysem exacs a model based on seveal feaues of he objec in ode o be uilized in nex fame as he idenificaion ool of he 8 Technical Gazee 24, 3(27), 89-86

3 F. Saadian Paćenje pedmea u video snimkama pomoću modela evolucijskog gupianja i lokalno lineanih neuo-fuzzy modela objec. In he conex of image analysis, one can have a vey boad ange of feaues of he signal. These include fequency domain feaues like Fouie coefficiens o wavele coefficiens, Hough ansfoms, colo feaues, geomeic feaues, ec. The selecion of appopiae feaue epesenaion fo each specific poblem is a vey sensiive, impoan, and almos inuiive ask. A feaue epesenaion vey useful fo one siuaion may be no good fo anohe one a all. In addiion, ying o use an opimally fewe numbe of feaues in modeling he objec is no only effecive in educion of compuaion, bu also may incease he genealizaion of he model. A good feaue epesenaion has o be obus o vaiaions of lighing, shape, and oienaion of he objec. In his pape we addess he acking of objecs in colo videos, so he use of colo feaues o exac a model of objec seems saighfowad. Anyway, in his pape we develop a mehod fo acking of objecs based on cluseing and LLNF models, and hen in ohe applicaions wih ohe appopiae feaue epesenaions he same appoach can be used. In fac, as we inoduce a geneal pocedue of cluseing, hen his would be nonelevan ha wha kinds of feaues one ies o use fo modeling, as long as i can be locally defined on egions and be used in cluseing. 4. Local colou feaues Local feaues ae exaced by consideing small neighbohoods aound pixels, and calculaion of some quaniies fom hese iny images. In his pape we use colo feaues of such local neighbohoods. Fo each pixel in he RGB colo image, hee ae hee values of inensiy fo each componen {c, c2, c3} = {, g, b}. We can also epesen pixels in hei HSV epesenaion {c4, c5, c6} = {h, s, v} [28]. Then conside a ecangula d- neighbohood DD(xx, yy) aound pixel (xx, yy): DD(xx, yy) = {(xx dd, yy dd), (xx dd +, yy dd),, (xx, yy),, (xx + dd, yy + dd ), (xx + dd, yy + dd)} (3) Then we can assign aveage values of c o c6 fo DD(xx, yy) as he feaue epesenaion assigned o (xx, yy). Ohe quaniies like vaiances, o even all he c values in DD(xx, yy) can be also consideed as feaues. 4.2 Cluseing of samples Cluseing is used in he poposed mehod of his pape fo leaning (seing he paamees) of LLNF as he model of objec. In his egad, on an iniial fame, fo which he objec is deemined by use, seveal andom posiions ae seleced, and he feaue values assigned o hem ae calculaed. Conside a andom selecion of N sample poins (xx ii, yy ii ) on sceen whee ii =,2,, NN. Then hee ae NN feaue vecos ff(xx ii, yy ii ) wih lengh 6. Also fo each poin (xx ii, yy ii ) a label LL is assigned based on he use deeminaion of he objec OO in iniial fame:, ( x, y ) O f ( x, y ) = i i i i (4), ( xi, yi ) O The goal of cluseing is o paiion he whole se of sample poins ino seveal subses (cluse), and assign a cluse cene o each cluse in a way minimizing an objecive funcion which is mainly he summaion of disances of poins in each cluse o is assigned cene. Ohe kinds of objecives like maximizaion of disances beween cenes may be incopoaed. Many mehods of daa cluseing have been inoduced in lieaue. The mos famous one of cluseing mehods is k-means algoihm [23]. This algoihm is fas and useful o many poblems, bu i has seveal dawbacks. In fac, i may be incapable of finding he globally opimum soluion o he cluseing poblem in some cases, and be no vey accuae confoning highly mixed daases. The appoach of his pape o cluseing is using evoluionay mehods. In ecen decades, evoluionay compuaion has been gown majoly and found a vey divese ange of applicaions in engineeing, science, managemen, economics, and even as. Geneally, in evoluionay mehods, one geneaes iniial soluions o a poblem almos andomly a he beginning. Then by means of seveal appopiae evoluion opeaos pefoming ieaively on he soluions, ies o find bee soluions based on he finess (objecive) funcion of he poblem. Mos of famous evoluionay mehods have been inspied fom biological pocesses in naue like geneic evoluions, o flocking of oganisms. In he nex subsecion, main ideas of evoluionay mehods of geneic algoihms and paicle swam opimizaion ae descibed, and hei use in cluseing of daa is explained. 4.3 Evoluionay cluseing Fo any evoluionay mehod o be used, we need o define an appopiae epesenaion of he possible soluion fo he poblem, and also define a finess funcion which maps each possible soluion o a eal value as he measue of finess of ha soluion o be he bes one fo he poblem in hand. In mos of cases he soluion epesenaion is an aay of values, wih each elemen epesening one aspec o dimension of ha soluion [29 32]. In geneic algoihm (GA) [33] such aay is called chomosome, and in paicle swam opimizaion (PSO) [34] i is called paicle posiion. Le XX = (XX, XX 2,, XX mm ) epesen such aay in any of hose algoihms, whee m is he dimension of he poblem. Finess funcion is a eal value FF(XX), which has o be minimized o maximized by he algoihm. In minimizaion poblem, one wans o find XX fo which FF(XX ) < FF(XX) fo all XX in he domain of he poblem. The specific hing o each algoihm is he way hey evolve he iniial soluions ieaively o find XX o a leas a good esimaion of i. In convenional GA, hee ae wo ypes of geneic opeaos used o evolve chomosomes: cossove, and muaion. Cossove opeao combines a paial chomosome wih he complemenay pas of anohe chomosome o geneae a new offsping chomosome (paen paen2 child). Fo muaion opeao, one o seveal genes (elemens of chomosome aay) ae seleced and changed andomly. The selecion of paen chomosomes is done based on hei assigned selecion pobabiliies which depend on hei finess values. In his Tehnički vjesnik 24, 3(27),

4 Objec acking in videos by evoluionay cluseing and locally linea neuo-fuzzy models way, being bee a chomosome, he moe is is chance o poduce new geneaion. The nex populaion of chomosomes would be composed of he bes chomosomes among paen, offsping, and muaed chomosomes accoding o hei coesponding finess values. The evoluion of soluions in PSO is inspied fom he flocking of bids (namely paicles). Iniial soluions ae consideed as posiion vecos of seveal paicles wih some iniial andom velociies vv ii. Based on finess funcion, a globally bes posiion gbes among all paicles in ieaion is found. Also each paicle i has is own memoy of bes expeienced posiion pbes ii fom beginning o he cuen ieaion. Then he velociies of all paicles ae updaed as v ) = c ( g X ) + c ( p X ) (5) i( bes i 2 2 bes i Then he updaed posiion of paicle is X i ( + ) = X i () + v i (). This pocedue uilizes he endency of paicles owad he bes expeienced posiion, and so duing hei movemen hey may find new bee posiions. To use evoluionay mehods in ou cluseing poblem of sample poins in he iniial fame of he video, we ake he posiions of cluse cenes in colo feaues space (6-dimensional in ou choice of feaues) as he soluion aay (chomosome o paicle posiion). The finess funcion ha we y o minimize is G f j, i w2 G j Gk + F = w j w3 σ (6) j,i The fis em in finess funcion is he sum of disance beween a cluse cenes GG jj o is conneced sample poins ff jj,ii. Each sample poin is consideed as conneced o is closes cluse cene. The second em is he sum of disances beween cluse cenes, and i has negaive sign because we wan i o be maximized. The hid em is he sum of vaiances of labels in cluses. In fac, he hid em is consideed o ensue he maximized unifomiy of labels wihin cluses. The weighs ww ii ae inoduced o make all ems in same ode of magniudes; hese ae se by inuiive consideaions on numbes of cluses and daa poins. j,k 5 Locally linea neuo-fuzzy model The goal of neuo-fuzzy sysems is o combine and ake he advanages of aificial neual newoks wih fuzzy infeence. Aificial neual newoks (ANN) ae mahemaical models capable of epesening a mapping beween inpus and oupus. To obain such mapping, seveal leaning algoihms have been inoduced in lieaue. Thee ae also divese ypes of ANNs including feed-fowad newoks and adial basis funcion (RBF) newoks. Fuzzy infeence sysems (FIS) ae mehods used fo decision abou a value by means of seveal ules based on fuzzy logic in an envionmen fo which he daa can be unceain. The concep of fuzzy membeship funcions helps o genealize he noion of membeship of elemens in ses. In fac, in fuzzy logic an elemen can be j 2 j F. Saadian a membe of a se by membeship degee of uu (wih < uu < ), and a he same be a non-membe of i wih degee uu. Locally linea neuo-fuzzy (LLNF) sysems [35] ae RBF like neual newoks wih each neuon accompanied wih a se of (usually Gaussian) fuzzy membeship funcions and a linea funcion of inpus. I is locally linea because he fuzzy funcions ac as selecive weighing funcions fo he linea pa, and weigh high fo some egions of inpu space and weigh low fo ohe pas. Fo a n-dimensional veco of inpus x, he oupu of he h neuon is calculaed by ules decision A and wih weigh w : = + (7) A a x a T w = exp[ ( x c ) ( x c )] (8) whee he n-veco aa and numbe aa define he linea ule; c is he mean n-veco, and he Σ is he n n vaiance maix of he n dimensional Gaussian funcion. The oveall oupu of he newok can be calculaed by nomalized weighed summaion w A y = w (9) The leaning of sysem means o find he appopiae values fo paamees cc, ΣΣ, aa, and aa in a way poviding he minimum eo in mapping beween inpus and oupus. The appoach of his pape o se hose paamees is using evoluionay algoihms and cluseing, wih he goal of finding he bes model fo he objec pesened by use in he fis fame of he video. The poposed mehod is descibed in he nex secion of pape. 6 Poposed mehod In ode o consuc a model of objec o disinguish i fom he backgound based on is colo feaues, in his pape he cluseing of sampled poins fom he iniial fame is done by means of evoluionay opimizaion of he cluseing finess funcion. Then, he esul of cluseing is used o se he paamees of LLNF model. The oupu of LLNF fo he objec sample poins has o be equal o, and fo he backgound poins o be. Then duing he acking ask when video is seaming, he ained LLNF model is used o decide fo each sample poin in a fame is a pa of objec o i is no. I is emphasized ha his mehod decides a poin being foegound (objec) o no, bu i does no decide ha a poin is in backgound. In ohe wod, a backgound model is no consuced in his mehod, because he backgound geneally can be vaying. The way he esuls of cluseing define he paamees of LLNF is descibed as follows. Le s assume we have R cluses found by means of evoluionay cluseing as descibed in pevious secions. Then we 82 Technical Gazee 24, 3(27), 89-86

5 F. Saadian Paćenje pedmea u video snimkama pomoću modela evolucijskog gupianja i lokalno lineanih neuo-fuzzy modela conside R neuons fo he LLNF model. The mean veco c of Gaussian funcion of -h neuon is se o be equal o he cene poin of ha cluse. The Σ maix is consideed as a diagonal maix wih he i h diagonal elemen equal o he squae of maximum disance in he i h dimension wihin he cluse fom is cene poin. Fo linea ule coefficiens, a fis a is se equal o zeo veco, and aa is equal o L, which is he label of cluse i.e. he aveage label of poins in ha cluse. If he esuls ae saisfacoy, hee would be no need o complicae he model wih incopoaion of non-zeo a. Howeve, i is always possible o un hose coefficiens on and y o se hem opimally as a pa of evoluionay opimizaion sep. Fo he acking of he objec in video seam, wo asks have o be done fo each fame; pedicing he locaion and size of he objec in he new fame befoe is obsevaion, and idenifying he objec when he fame is obseved. Fo he second pa, he LLNF as ained by cluseing is used by is opeaion on andom sample poins on he sceen of he new fame. Fo he ohe ask, pedicing he behavio of he objec, a combinaion of weighed linea pedicos wih evoluionay seach is used in his pape. If he locaion and size sae of he bounding box of he objec a is, X = (x, y, w x, w y ) hen he nex sae is calculaed by a linea pedico being X () + = B + B X + B2 X whee BB, BB and BB 2 ae 4, 4 4, and 4 4 maices especively. To achieve he goal of evolving pedico, we use a swam of hose linea pedicos and assign a weigh o each. The weigh of each model depends on is pefomance in pedicing he objec. In fac, a each fame, he bes linea pedico obains he highes weigh. Then he coefficien paamees of he i h model (he elemens of BB, BB and BB 2 ), if all epesened in one 36 aay P i will be updaed as wi ( ) wbes ( ) Pi ( + ) = Pi ( ) + δ ( Pbes ( ) Pi ( ) ) () wi i whee he index "bes" epesens he cuen bes one of linea pedicos wih highes weigh, and δδ is a andom coefficien. Fo he nex sep, he new B maices ae obained again fom he new P aays. This pocedue is simila o he idea of PSO algoihm, and i povides he evoluion of pedico model owads bee pefomance in pedicing he objecs behavio. 7 Resuls The daase of videos used fo evaluaion of he poposed mehod is named BoBoT (Bonn Benchmak on Tacking) [26]. This daa se is puposefully geneaed fo esing objec acking algoihms. I includes cases wih colo videos of moving objecs, moving cameas, low and fas movemens, igid and non-igid objecs, indoo and oudoo sceneies. A fis a disibuion of sample poins in colo feaue space obained fom one fame of a video, which is shown in Fig., is geneaed by andom speading sample windows on he image. The sample poins in space of 3- feaues {, g, b} and in 3-feaues {h, s, v} ae shown in Fig. 2. The poins fom objec and fom backgound ae maked wih diffeen makes. I is clea ha many poins fom objec have vey disincive feaues fom he poins of backgound. Of couse, hee ae some ovelaps beween he wo classes, bu he idea of he poposed mehod is o weigh moe on disincive egions by means of LLNF. I should be noed ha in nex fames hee may be seveal poins in backgound wih close feaues o some of objec poins, and hen be classified as foegound duing he acking pocedue. This effec is educed by uilizaion of pedico model o limi he seach egion on he sceen. In addiion, by sampling enough poins, he aggegaed esuls will eliminae effecs of such false foegound esimaion poins Figue A fame fom one of he videos of daa se, objec shown in bounding box [26] Backgound Objec Backgound Objec.6.6 Blue.4 Value Geen Red.6.4 Sauaion a) b) Figue 2 Sampled poins fom iniial fame shown in (a) RGB colou space, and (b) HSV colou space. Foegound (objec) and backgound poins ae disinguished wih diffeen makes. Tehnički vjesnik 24, 3(27), Hue

6 Objec acking in videos by evoluionay cluseing and locally linea neuo-fuzzy models F. Saadian 2 8 Seq A 6 Seq B a) b) Seq C c) d) Seq E Seq D Seq F e) f) Seq G Seq H g) Seq I h) i) Figue 3 The acking pefomance esuls in he fom of ovelapping pecen of esimaed and acual bounding boxes aound he objecs fo sequences A o I fom he daa se. To compae he esuls of acking wih ohe appoaches, a pefomance scoe is used which calculaes he pecen of ovelapping beween acked (esimaed) and acual bounding box of he objec. I should be noed ha, fo fames wih full occlusion effecs, his scoe should be calculaed as non-exising. Fo he simulaions, 84 Technical Gazee 24, 3(27), 89-86

7 F. Saadian Paćenje pedmea u video snimkama pomoću modela evolucijskog gupianja i lokalno lineanih neuo-fuzzy modela he following seings have been used. Fo all sequences, cluses have been found fo 5 sample poins of iniial fame. To find opimal cluseing, PSO is used wih 3 paicles and 2 ieaions. GA is no used in final esuls, because PSO was fase in his poblem. Thee was no need o inoduce coefficiens of LLNF ules moe han a, based on he saisfacoy esuls. When acking he objec duing he video seam, sample poins fom he sceen wee used fo each fame, and 8 linea pedicos have been uilized accompanied wih coevoluion of iniial andom paamees as descibed. Fo seveal videos of he daase, he acking esuls in he fom of pefomance scoes vesus ae shown in Fig. 3. Vaious behavios on plos ae obained based on he vaious siuaions of coesponding videos. Fo example, in video sequence F, hee ae seveal full occlusions, and on he plo i is seen as non-exising values fo he pefomance scoe. As fo sequence H, hee is no acual movemen of objec o camea, hen he pedico model conveges o a sable posiion and hen pefomance end wih is consan. To compae he esuls, in Tab. he aveage pefomance fo each sequence is pesened fo seveal mehods compaed o he esuls of he poposed mehod. The esuls show ha he pefomance of he poposed mehod is bee han ohe appoaches, fo mos of he sequences. Only fo 3 of 9 sequences, he esuls of he poposed mehod ae slighly lowe han ohe mehods. Based on he aveage of all esuls, he highe accuacy of poposed mehod is concluded. Table Compaison of acking pefomances fo vaious mehods [26] Aveage pefomance scoe of mehods (%) Sequence Hisogam Muli-comp N. ad. hcs Adap. hcs Adap. P. pa. hcs Poposed mehod A 7,73 63,24 38,35 65,6 59,35 82,8 B 67,2 5,73 6,2 79, 77,38 8,85 C 47,58 63,7 89,33 9,66 9,33 74,56 D 63,35 76,39 62,78 7,2 75,2 77,87 E 78,2 77,42 83,2 84,49 86,32 82,6 F 44,43 4,2 63,99 6,82 68,32 86,66 G 46,27 49,62 34,34 77,3 7,6 78,87 H 62,9 86,5 95,79 94,4 94,47 9,7 I 68,94 47,63 48,97 75,2 56,33 89,3 Aveage 6,97 6,7 58,8 77,54 75,54 82,73 8 Conclusion The objec acking mehod poposed in his pape combines he ideas of cluseing, evoluionay compuaion, and neuo-fuzzy models. The evoluionay opimizaion is used o obain a good cluseing of sample poins on he iniial fame accompanied wih he deeminaion of he objec. The cluseing esuls se he paamees of locally linea neuo-fuzzy model in a simple way. The esuled idenifie is used o segmen he objec based on andom sample poins duing he video seam. Anohe swam of linea models co-evolve based on hei pefomances fo esimaing he behavio of objec and pedic he nex locaion. The pefomance analysis and compaison o pevious mehods on same daa se showed ha he accuacy of he poposed mehod is highe han ha of pevious mehods. Fo he exension of his wok, ohe neuo-fuzzy models, cluseing, and evoluionay compuaion of objec acking can be used. Fuue sudies would exend he efficien opimizaion models o he poblem of objec acking in videos. 9 Refeences [] Hu, W.; Tan, T.; Wang, L.; Maybank, S. A suvey on visual suveillance of objec moion and behavios. // Sysems, Man, and Cybeneics, Pa C: Applicaions and Reviews, IEEE Tansacions on. 34, 3(24), pp DOI:.9/TSMCC [2] Kolle, D.; Daniilidis, K.; Nagel, H.-H. Model-based objec acking in monocula image sequences of oad affic scenes, // Inenaional Jounal of Compue., 3(993), pp [3] Wu, Y.; Lim, J.; Yang, M.-H. Online objec acking: A benchmak. // in Compue vision and paen ecogniion (CVPR), 23 IEEE Confeence on. 23, pp [4] Vadakkepa, P.; Lim, P.; De Silva, L. C.; Jing, L.; Ling, L. L. Mulimodal appoach o human-face deecion and acking. // Indusial Eleconics, IEEE Tansacions on. 55, 3(28), pp DOI:.9/TIE [5] Liu, M.; Yao, H.; Gao, W. Real-Time human face acking in colo images. // Chinese Jounal of Compues-Chinese Ediion. 2, (998), pp [6] Allen, P. K.; Timcenko, A.; Yoshimi, B.; Michelman, P. Auomaed acking and gasping of a moving objec wih a oboic hand-eye sysem. // Roboics and Auomaion, IEEE Tansacions on. 9, 2(993), pp DOI:.9/ [7] Papanikolopoulos, N. P.; Khosla, P. K. Adapive oboic visual acking: Theoy and expeimens. // Auomaic Conol, IEEE Tansacions on. 38, 3(993), pp DOI:.9/9.24 [8] Rai, N.; Rai, B.; Rai, P. Compue vision appoach fo conolling educaional oboic am based on objec popeies. // in Emeging Technology Tends in Eleconics, Communicaion and Newoking (ET2ECN), 24 2nd Inenaional Confeence on. 24, pp. -9. [9] Shanaiya, S.; Vema, K.; Meha, K. Muliple Objec Tacking using Kalman File and Opical Flow. // Euopean Jounal of Advances in Engineeing and Technology. 2, 2(25), pp [] Li, X.; Wang, K.; Wang, W.; Li, Y.A muliple objec acking mehod using Kalman file. // in Infomaion and Auomaion (ICIA), 2 IEEE Inenaional Confeence on. 2, pp [] Li, X.; Zhang, T.; Shen, X.; Sun, J. Objec acking using an adapive Kalman file combined wih mean shif. // Opical Engineeing. 49, 2(2), pp DOI:.7/ [2] Risic, B.; Aulampalam, S.; Godon, N. Beyond he Kalman file. // IEEE Aeospace and Eleconic Sysems Magazine. 9, 7(24), pp DOI:.9/MAES [3] Lippiello, V.; Siciliano, B.; Villani, L. Adapive exended Kalman fileing fo visual moion esimaion of 3D objecs. // Conol Engineeing Pacice. 5, (27), pp DOI:.6/j.conengpac [4] Liu, C.; Shui, P.; Wei, G.; Li, S. Modified unscened Kalman file using modified file gain and vaiance scale faco fo highly manoeuving age acking. // Jounal of Tehnički vjesnik 24, 3(27),

8 Objec acking in videos by evoluionay cluseing and locally linea neuo-fuzzy models Sysems Engineeing and Eleconics. 25, 3(24), pp DOI:.9/JSEE [5] Chen, S. Y. Kalman file fo obo vision: a suvey. // Indusial Eleconics, IEEE Tansacions on. 59, (22), pp DOI:.9/TIE [6] Nummiao, K.; Kolle-Meie, E.; Van Gool, L. An adapive colo-based paicle file. // Image and Vision Compuing. 2, (23), pp DOI:.6/S (2)29-4 [7] Gonczaek, A.; Tomczak, J. M. Manifold egulaized paicle file fo aiculaed human moion acking. // In Advances in Sysems Science / Spinge Inenaional Publishing. 24, pp [8] Lucena, M.; Fuees, J. M.; de la Blanca, N. P. Opical flow-based obsevaion models fo paicle file acking. // Paen Analysis and Applicaions. 8, (25), pp DOI:.7/s [9] KaewTaKulPong, P.; Bowden, R. An impoved adapive backgound mixue model fo eal- acking wih shadow deecion. // In Video-based suveillance sysems / Spinge US. 22, pp [2] Rische, J.; Kao, J.; Joga, S.; Blake, A. A pobabilisic backgound model fo acking. // In Compue Vision ECCV 2 / Spinge Belin Heidelbeg. 2, pp [2] Gu, B.l Mu, A.; Tekalp, A. M. Tempoal video segmenaion using unsupevised cluseing and semanic objec acking. // Jounal of Eleconic Imaging. 7, 3(998), pp DOI:.7/ [22] Heisele, B.; Kessel, U.; Rie, W. Tacking non-igid, moving objecs based on colo cluse flow. // In Compue Vision and Paen Recogniion, Poceedings 997 IEEE Compue Sociey Confeence on. 997, pp [23] Weng, S. K.; Kuo, C. M.; Tu, S. K. Video objec acking using adapive Kalman file. // Jounal of Visual Communicaion and Image Repesenaion. 7, 6(26), pp DOI:.6/j.jvci [24] Yang, F.; Paindavoine, M. Implemenaion of an RBF neual newok on embedded sysems: eal- face acking and ideniy veificaion. // IEEE Tansacions on Neual Newoks. 4, 5(23), pp DOI:.9/TNN [25] Angelov, P.; Sadeghi Tehan, P.; Ramezani, R. An appoach o auomaic eal novely deecion, objec idenificaion, and acking in video seams based on ecusive densiy esimaion and evolving Takagi Sugeno fuzzy sysems. // Inenaional Jounal of Inelligen Sysems. 26, 3(2), pp DOI:.2/in.2462 [26] Klein, D. A.; Schulz, D.; Finop, S.; Cemes, A. B. Adapive eal- video-acking fo abiay objecs. // In 2 IEEE/RSJ Inenaional Confeence on Inelligen Robos and Sysems (IROS). 2, pp DOI:.9/IROS [27] Leng, G.; McGinniy, T. M.; Pasad, G. Design fo selfoganizing fuzzy neual newoks based on geneic algoihms. // IEEE Tansacions on Fuzzy Sysems. 4, 6 (26), pp DOI:.9/TFUZZ [28] Cheng, H. D., Jiang, X. H.; Sun, Y.; Wang, J. Colo image segmenaion: advances and pospecs. // Paen ecogniion. 34, 2(2), pp DOI:.6/S3-323()49-7 [29] Simunovic, G.; Svalina, I.; Simunovic, K.; Saic, T.; Havlisan, S.; Vukelic, D. Suface oughness assessing based on digial image feaues. // Advances in Poducion Engineeing & Managemen. (26), 2, pp DOI:.4743/apem [3] Tang, M.; Gong, D.; Liu, S.; Zhang, H. Applying muliphase paicle swam opimizaion o solve bulk cago po scheduling Poblem. // Advances in Poducion Engineeing & Managemen., 4(26), pp DOI:.4743/apem F. Saadian [3] Chandasekaan, M.; Devaasiddappa, D. Aificial neual newok modeling fo suface oughness pedicion in cylindical ginding of Al-SiCp meal maix composies and ANOVA analysis. // Advances in Poducion Engineeing & Managemen. 9, 2(24), pp. 59-7, DOI:.4743/apem [32] Saic T.; Simunovic G.; Simunovic K.; Svalina I. Esimaion of Machining Time fo CNC Manufacuing Using Neual Compuing. // Inenaional Jounal of Simulaion Modelling. 5(26), 4, pp DOI:.257/IJSIMM5(4)7.359 [33] Goldbeg, D. E.; Holland, J. H. Geneic algoihms and machine leaning. // Machine leaning. 3, 2(988), pp DOI:.23/A: [34] Kennedy, J. Paicle swam opimizaion. // in Encyclopedia of Machine Leaning, Spinge, 2. [35] Nelles, O. Nonlinea sysem idenificaion: fom classical appoaches o neual newoks and fuzzy models. Spinge Science & Business Media, 2. DOI:.7/ Auho s addess Faemeh Saadian, Sama Technical and Vocaional Taining College, Islamic Azad Univesiy, Nou Banch, Nou, Ian faemeh.saadian@chmail.i 86 Technical Gazee 24, 3(27), 89-86