AN ACTOR-BASED SIMULATION FOR STUDYING UAV COORDINATION

AN ACTOR-BASED SIMULATION FOR STUDYING UAV COORDINATION Myeong-Wuk Jang, Smtha Reddy, Predrag Tosc, Lpng Chen, Gul Agha Department of Computer Scence Unversty of Illnos at Urbana-Champagn Urbana, IL 61801, USA E-mal: { mjang, sreddy1, p-tosc, lchen2, agha } @cs.uuc.edu KEYWORDS Actor, Smulaton, Unmanned Aeral Vehcle (UAV), Coordnaton. ABSTRACT The effectveness of Unmanned Aeral Vehcles (UAVs) s beng ncreased to reduce the cost and rsk of a msson [Doherty et al. 2000]. Snce the advent of small szed but hgh performance UAVs, the use of a group of UAVs for performng a jont msson s of major nterest. However, the development of a UAV s expensve, and a small error n automatc control results n a crash. Therefore, t s useful to develop and verfy the coordnaton behavor of UAVs through software smulaton pror to real testng. We descrbe how an actor-based smulaton platform supports dstrbuted smulators, and present three cooperaton strateges: self-nterested UAVs, sharng-based cooperaton, and team-based coordnaton. Our expermental results show how communcaton among UAVs mproves the overall performance of a collecton of UAVs on a jont msson. 1. INTRODUCTION The effectveness of Unmanned Aeral Vehcles (UAVs) s beng ncreased to reduce the cost and rsk of a msson [Doherty et al. 2000]. Some mltary UAVs, such as the Predator and the Global Hawk, were already used durng the wars n Afghanstan and Iraq. Decreasng sze of the UAVs and ncreased demand for more ntellgent and autonomous behavor of UAVs are pavng the way for consderaton of a group of UAVs performng a jont msson. Whle the cost of UAVs s lower than that of real planes, the development cost of a UAV s stll very hgh, and a small error n automatc control may result n a crash. Therefore, when we consder a large number of UAVs workng together, t s necessary to desgn and verfy the behavor of UAVs through software smulaton pror to real testng. Many smulators have been developed as sngle process smulators. However, a sngle process smulator has several lmtatons. Frst, the performance of a smulaton depends on the computatonal power of one computer. Second, a sngle process smulator has an extensblty ssue when a specal component requres ts own specfc process. For example, f we want to smulate the coordnaton behavor of many vrtual UAVs wth a few real UAVs, each real UAV s workng as an ndependent process. In ths knd of smulaton, a sngle process smulator cannot work well. Therefore, a concurrent object-based dstrbuted smulator provdes a better smulaton envronment. It s commonplace to say that human bengs are dsposed to cooperate. Bology and ethology show that kn-altrusm and recprocal-altrusm can ground cooperatve behavor n anmals, such as wolves surroundng prey, termtes nest buldng, and brds flockng. Drawng a parallel, ntellgent UAVs that cooperate wth one another are of hgh nterest for ther ablty to search, detect, dentfy, and handle targets together. The old age tenets of pre-plannng and central control have to be reexamned, gvng way to the dea of coordnated executon. In ths paper, we descrbe and analyze three dfferent strateges to coordnate tasks among UAVs n a dynamc envronment to acheve ther goals. The outlne of ths paper s as follows. Secton 2 sketches a smulaton scenaro and explans basc concepts about the actor model and the metrcs n our smulaton. Secton 3 descrbes archtecture for our smulaton, and three cooperaton strateges for a jont msson are presented n Secton 4. Secton 5 explans our mplementaton and expermental results. Then, n Secton 6 and 7, we dscuss related work and our future work. Fnally, we conclude ths paper wth a summary of our smulaton framework and our major contrbutons. 2. TERMINOLOGY 2.1 UAV Smulaton Scenaro Pror to embarkng on the archtecture of our UAV smulator, we present a smple scenaro n order to explan the meanng of basc terms. The applcaton of our smulaton s a UAV survellance msson. For example, 50 UAVs mght be launched nto a certan area by Ground Control System (GCS) to detect targets n the area. For example, targets may be cvlans to be rescued. In the smulaton, UAVs have the autonomy to perform ther msson wthout nteracton wth the GCS, except durng the ntal stage when message exchange s necessary to get each UAV started by sendng them some default ar routes. When UAVs are launched, the UAVs do not have any nformaton about locatons of targets. However, each

UAV s equpped wth some sensors whch can detect objects wthn the certan range. We assume that all UAVs start from the same locaton, called an ar base. Controllng the sequence of takeoffs and landngs of UAVs s managed by the control center, called Ar Base System (ABS). The man task of a UAV s to detect locatons of targets n a msson area and nvestgate them. Therefore, even though they navgate accordng to the gven ar routes, they can change ther trajectores to handle targets once they detect those targets. In addton, when UAVs encounter obstacles, such as tall towers or arplanes, they should change ther ar routes to avod a collson. Therefore, n our UAV smulaton, there are fve types of mportant components: Ground Control System (GCS), Ar Base System (ABS), Unmanned Aeral Vehcles (UAVs), targets, and obstacles. 2.2 Actor Our UAV smulator s based on the Actor system, a concurrent object-based dstrbuted system, and hence, we use the actor model to descrbe each component n the smulaton. An actor s a self-contaned actve object whch has ts own control thread and communcates wth other actors through asynchronous message passng [Agha 1986; Agha et al. 1997]. In addton, an actor can create other actors, just as an object can create other objects. In our UAV smulator, each component, such as a UAV or a target, s mplemented as an actor. Snce these components n real stuatons operate concurrently and communcate wth one another, ther behavor can be captured very well by the actor model. Each software component n the smulaton progresses ts state ndependently of the progress of others n response to the envronment nformaton gathered ether through ts own sensor or by communcatng wth others. 2.3 Attractve Force Value and Utlty Value In our UAV smulaton, each target has ts own value. Ths value could be nterpreted n several dfferent ways. The value mght correspond to the number of solders or the mportance of a buldng. Also, we can consder ths value as the tme requred to nvestgate a target by a UAV. For the smplcty of our smulaton, we use a sngle numerc value nstead of symbolc nformaton or tme nformaton about a target. In our smulaton, we make the followng assumptons. A UAV handles only one target at a tme, although the UAV holds and manages nformaton about several targets. In the advent of multple targets to be handled, the UAV should select one of them. For ths purpose, a UAV uses the attractveness functon to decde on a target. The attractveness functon maps the value of a target to the attractve force value, whch represents a UAV s preference. Ths functon depends on the value of the target and the dstance between tself and the UAV, and s used to select the best target as follows: Θ ( t) = arg Π j ( t) max j x ( t) ( t) ψ j where Π (t) denotes the value of target j at tme t, j x (t) s the locaton of UAV at tme t, and ψ j (t) s the locaton of target j at tme t. If target j s statonary, ψ j (t) s always the same regardless of tme. The value between braces s called the attractve force value of target j, and Θ (t) returns the ndex of the target that has the maxmum attractve force value to UAV at tme t. As a UAV approaches a target, the UAV starts consumng the value of the target once the UAV s wthn a certan dstance of the target. We call the value consumed by the UAV the utlty value. The utlty value functon and the target value functon of the target at tme step t+1 are defned as follows: U ( t + 1) = Π(0) Π ( t + 1) { Π ( t) d n ( ),0} Π ( t + 1) = max t where U (t) means the utlty value of the target at tme t, d s a dscount factor, and n (t) s the number of UAVs whch are near to the target at tme t. Therefore, n our smulaton, when several UAVs are wthn the range of a target, the value of the target s consumed more quckly. After a UAV reaches a target, t wll fly around the target untl the whole value of the target s consumed, ether by the UAV alone or n conjuncton wth a group of UAVs. In our UAV smulaton, one purpose of collectve behavor of UAVs s to maxmze the accumulated utlty value wthn as short a tme as possble. Here, the accumulated utlty value means the whole value of targets consumed by all the UAVs. 3. SIMULATION ARCHITECTURE Our dstrbuted smulaton s comprsed of three layers: user nterface, UAV smulator, and actor-based dstrbuted platform (Fgure 1). The user nterface layer conssts of two programs: Confguraton Interface Program and Smulaton Vewer. Confguraton Interface Program provdes an easy means of defnng mportant attrbutes for the smulaton. Smulaton Vewer s a tool to check and verfy the smulaton results. All task orented components, such as UAVs and targets, and smulaton orented components, such as Smulaton Control Manager (SCM) and Actve Broker (AB), are mplemented as actors n the UAV smulator layer, whch wll be further explaned n secton 3.2.2. Each actor has ts own thread to progress ts state. The thread executon and communcaton of actors are

controlled by the Actor Foundry, an actor-based dstrbuted platform. User Interface Confguraton Interface set up parameters Smulaton Vewer show the result UAV Smulator Task orented components UAVs, targets, obstacles, etc Smulaton orented components SCM, AB, etc Actor-based Dstrbuted Platform Actor Foundry actor thread control communcaton among actors actor mgraton Fgure 1: Three-layered Archtecture for Dstrbuted Smulaton The actor-based dstrbuted platform s a mddleware to support several dstrbuted applcatons and s not talor made for a specfc smulaton, such as a UAV smulaton. The UAV smulator defnes specfc behavors of UAVs, but does not nclude all the parameters to test and verfy a behavor. These parameters are defned n user nterface programs by a user and used n the UAV smulator. The functons of each layer are explaned n detal below. 3.1 Actor-based Dstrbuted Platform The Actor Foundry s mplemented n the Java programmng language, and supports actor executon, communcaton between actors, and actor mgraton [Astlery 1999; Clausen 1998]. In the Actor Foundry, an actor s created by another actor or by a user. When an actor s created, the actor name of the new actor s returned. Ths name would be used to refer to the recever actor n message passng or delver the reference of another actor to the recever actor. The actor name s unque n the actor world. Therefore, even though an actor mgrates from one host to another, the name s always transparent to other actors, and hence, other actors can contnuously use the same name to refer to the gven actor rrespectve of that actor s current locaton, thereby provdng a means for locaton transparency. An actor n the Actor Foundry s runnng as a Java thread, and an actor communcates wth other actors through asynchronous message passng. Ths s the man dfference between the Actor Foundry and other object-based dstrbuted platforms, such as CORBA and DCOM [Grmes 1997; OMG 2002]. In other object-based dstrbuted platforms, one thread control s assumed: when an object s called by another object, the caller object s blocked untl the called object returns the thread control. In the Actor Foundry, snce every actor has ts own control thread to perform ts operaton and communcates wth others through the asynchronous communcaton, the executon of an actor does not depend on those of others. Due to these features, we can easly use the power of dstrbuted systems. Smulaton components mplemented as actors run on dfferent computers ndependently, and they can communcate wth others through the unque actor name, even though the dstrbuted platform mgrates some components from one host to another. When dstrbuted components nteract wth each other through asynchronous communcaton, analyzng the delvery sequence of communcaton messages s burdensome because asynchronous communcaton does not guarantee the message delvery order requrements, such as FIFO order, causal order, or total order [Hadzlacos and Toueg 1993]. Our dstrbuted platform makes a log for message passng among actors, so that users can easly analyze the delvery sequence of messages. 3.2 UAV Smulator All smulaton components n our UAV Smulator can be classfed nto two categores: task orented components and smulaton orented components (Fgure 2). Task orented components smulate objects n real stuatons. For example, a UAV component maps to a UAV object n a real stuaton whle a target component maps to a target object. For the purpose of smulaton, we need some vrtual components, such as Smulaton Control Manager and Actve Broker. The followng sub-sectons explan these two categores of components n detal. Task Orented Components Unmanned Unmanned Unmanned Aeral Vehcle Aeral Vehcle Aeral Vehcle Statc Statc Statc Target Target Target Statc Statc Statc Obstacle Obstacle Obstacle Ground Control System Smulaton Orented Components Smulaton Control Manager Actve Broker Moble Moble Moble Target Target Target Moble Moble Moble Obstacle Obstacle Obstacle Ar Base System Sensor Smulator Fgure 2: Smulaton Components n UAV Smulator

3.2.1 Task Orented Components Task orented components n our UAV smulator consst of fve types: Ground Control System (GCS), Ar Base System (ABS), Unmanned Aeral Vehcles (UAVs), obstacles, and targets. GCS s a central manager of UAVs and s aware of the msson area so as to ndcate each UAV ts ar route n the area. However, GCS may not communcate contnuously wth UAVs to decde behavor of the UAVs at each tme step because UAVs are supposed to perform ther msson autonomously. ABS represents a control center of an ar base and controls the sequence of take-offs and landngs of UAVs. UAVs perform a gven msson autonomously wthn certan restrctons, such as ther knematcs and communcaton capablty. Obstacles represent objects n whch UAVs are not nterested and wth whch a collson can happen. Accordng to whether an obstacle can move or not, they are classfed nto two classes: a moble obstacle, such as an arplane, and a statc obstacle, such as a tall tower or a buldng. Targets represent objects of nterest for the UAVs, such as, cvlans to be rescued. Accordng to ts moblty characterstcs, there are moble targets and statc targets. 3.2.2 Smulaton Orented Components 3.2.2.1 Smulaton Control Manager. Each component manages ts vrtual tme because each actor has ts own control thread. However, ths stuaton can cause nconsstency n vrtual tmes of components. To mantan consstency between vrtual tmes, Smulaton Control Manager (SCM) manages local vrtual tmes of the smulaton components. When every component starts ts executon, the ntal value of each local vrtual tme s set to 0. After every component starts, SCM broadcasts a vrtual tme clock message to the other components. When a component receves the message, the component ncreases ts local tme and performs a small porton of ts task that should be completed durng the predefned tme slce unt. For example, when a UAV receves the message, t updates ts locaton and drecton vector, and also checks whether or not new objects, such as other UAVs, targets, or obstacles, are detected. If a new neghborng UAV s detected, the UAV mght exchange some nformaton wth the new neghborng UAV. After a component fnshes ts computaton, t sends a reply message to SCM. When SCM receves reply messages from all the other components, SCM ncreases ts vrtual tme, and rebroadcasts another vrtual tme clock message. 3.2.2.2 Actve Broker In order for a UAV to perform a group msson, the UAV needs to communcate wth ts neghborng UAVs through local broadcastng. Actve Broker smulates a local broadcastng mechansm. In general, the brokerng servce supports attrbute-based communcaton. For example, f every UAV regsters nformaton about ts current flyng area wth ts actor name on a shared space, then when a UAV requests a broker for a message passng wth a template that descrbes a certan area, the broker delvers the message to other UAVs whch are n the area. However, ths approach s not very accurate for fndng the neghborng UAVs. Therefore, we have extended the functon of the brokerng servce. In the actve brokerng servce, every UAV regsters nformaton about ts current locaton wth ts actor name on the shared space, and a UAV sends a specal object nstead of the template to request a message delvery to Actve Broker. The object ncludes a specfc method to be called by Actve Broker. The method computes the dstance between the locaton of the sender UAV and other UAVs and selects some whch are wthn the local communcaton range. When the method returns actor names of neghborng UAVs, Actve Broker delvers to them the message receved from the sender UAV. 3.2.2.3 Sensor Smulator Although each real UAV s supposed to be equpped wth ts own radar sensor, the radar sensors of all UAVs s smulated by a sngle smulaton orented component, Sensor Smulator. In the smulaton, UAVs, targets, and obstacles regster ther current locatons on a shared space every second n vrtual tme. Sensor Smulator perodcally computes the dstance between any two objects. If some components are wthn the sensor range of a UAV, Sensor Smulator reports nformaton about these components to the UAV. Each UAV regards ths nformaton as ts sensor nput. 3.2.3 UAV Archtecture The most mportant smulaton component s a UAV component. Therefore, we explan the archtecture and the man behavor of a UAV n ths subsecton. A UAV s comprsed of four modules: the physcal process module, the trajectory plannng module, the cooperatve module, and the global control module (Fgure 3). sensor UAV global wayponts local wayponts trajectory plannng module global control module next wayponts request target handlng obstacle handlng physcal process module cooperatve module local nformaton, next waypont request GCS UAV Fgure 3: The Archtecture of the Unmanned Aeral Vehcle Actor

When a UAV starts ts msson, t does not have any nformaton about ts ar route or the msson area. In our smulaton, an ar route s defned as a set of wayponts that need to be traversed by the UAV. Therefore, the frst task of a UAV s to request the wayponts from GCS. The global control module of a UAV takes charge n communcatng wth GCS and managng the wayponts receved. We call these wayponts global wayponts. When a UAV detects targets or/and obstacles, ths nformaton s delvered to the trajectory plannng module from Sensor Smulator. The trajectory plannng module handles them accordng to the predefned rules. For example, when a UAV detects several targets, t selects one target whch has the best attractve force value, and then modfes ts ar route to reach the target. Ths functon s performed by addng a waypont to the lst of UAV s current wayponts. The set of wayponts used nclusve of the addtonal wayponts are called local wayponts. The cooperatve module s used when several UAVs want to handle a set of targets. To decde whch UAV handles whch target, the UAVs communcate wth each other through the cooperatve module. The knematcs of a UAV s mplemented n the physcal process module. Therefore, whenever ths module receves a vrtual tme clock message, the physcal process module computes the next locaton and the next drecton of the UAV. When a UAV reaches the current waypont, ths module starts a turn toward the next waypont accordng to the predefned knematcs. 3.3 User Interface If we have to modfy the UAV smulator whenever we execute t wth dfferent parameters, t s qute burdensome. Besdes, modfcaton at the code level requres comprehenson makng t hard for novce users to modfy the code. In our archtecture of UAV smulaton, we separate the parameter modfcaton part from the UAV smulator code as the user nterface layer. Moreover, we separate the smulaton checkng part from smulator code. Therefore, the user nterface layer conssts of two programs: Confguraton Interface Program and Smulaton Vewer. Fgure 4: Confguraton Interface Program 3.3.2 Smulaton Vewer Because of the characterstcs of large scale smulatons whose duratons may sometmes be so long that we cannot montor the smulaton results contnuously, we have separated the smulaton checkng from the smulaton executon. Therefore, we look at and check the smulaton results through Smulaton Vewer (Fgure 5). Another advantage of ths approach s that the smulaton results can be vewed back and forth wth respect to the smulaton vrtual tme. Whle our UAV smulator s runnng accordng to the gven parameters, the smulator generates smulaton results on data fles. The data fles contan the locatons and drectons of UAVs, targets, and obstacles at every smulaton vrtual tme step. The Smulaton Vewer s used to check and verfy the smulaton results. 3.3.1 Confguraton Interface Program For the convenence of novce users, we have separated the confguraton for UAV smulaton parameters from the smulator code as a confguraton fle. Ths fle can be modfed by the Confguraton Interface Program (Fgure 4). Therefore, although a user does not look at and understand the source code for UAV smulaton, they can change mportant parameters of smulaton and run t wthout recomplng the source code. Wth ths program a user can set up the number of UAVs, the sze of msson area, the attrbutes of targets and obstacles, maxmum smulaton tme, and the sze of smulaton tme slce unt. Fgure 5: Smulaton Vewer

4. COOPERATION AND COORDINATION AMONG UAVS Cooperaton among the UAVs s essental n drectng the adjustment of polces n the globally most benefcal drecton. In addton to cooperatve dssemnaton of nformaton, coordnaton of actons n larger teams s essental. Wth elements of uncertanty exstng n the envronment, coordnaton among UAVs has to be adaptve. The UAVs need to dynamcally allocate responsbltes for dfferent subtasks dependng on the changng crcumstances of the overall stuaton. For example, f addtonal targets are detected durng a group msson, a team of UAVs needs to be able to handle them ether by recrutng new member UAVs or changng the prevous assgnment of targets. In our UAV smulaton, we use three strateges: the self-nterested UAV strategy, the sharng-based cooperaton strategy, and the teambased coordnaton strategy. 4.1 Self-nterested UAVs In the self-nterested UAV strategy, a UAV senses a target and approaches t wth the ntenton of consumng ts entre value. When another UAV detects the same target, t also proceeds to consume the value of the target, rrespectvely of what other UAVs do. Incessant pollng of the target value tll such tme t s consumed completely serves as a means of nteracton among the UAVs. It s not unusual to have more than one UAV concentrated on a target resultng n qucker consumpton of ts value, but also possbly n duplcaton of servce. 4.2 Sharng-based Cooperaton In ths strategy, once a UAV has dscovered and located a target, t broadcasts ths nformaton so that other UAVs could drect ther attenton to the remanng targets. Recepton of such nformaton wll result n the UAVs purgng the targets that were advertsed. Ths approach allows for a larger set of targets to be vsted n a gven tme nterval and s thus expected to be faster n accomplshng the msson goal. Exchange of nformaton between UAVs referrng to the same target wll result n a UAV wth a lower dentfcaton number to determne the UAV that would be responsble for ths target based on parameters such as the dstance from the target. 4.3 Team-based Coordnaton In the team-based coordnaton strategy, certan UAV takes on the mantle of the leader of ts team and dctates course of acton to the other UAVs about the targets they need to vst. A team s dynamcally formed and changed accordng to the set of targets detected;.e. when a UAV detects more than one target, the UAV tres to handle the targets together wth ts neghborng UAVs. At ths tme, the man concern s how to select an optmum UAV and decde the number of UAVs requred to accomplsh a task, when there are a suffcent number of neghborng UAVs. As the basc coordnaton protocol, we use the Contract Net protocol [Smth 1980; Smth and Davs 1981]. The UAV ntatng the group msson works as the group leader UAV, and the other partcpant UAVs are called member UAVs. When a member UAV detects another target, the UAV delvers nformaton about the new target to the leader UAV, and the leader UAV wll add the target to the set of targets to be handled. The leader UAV consders the dstance between a target detected and neghborng UAVs to assgn the target. When a member UAV consumes the entre value of a target the UAV secedes from ts group. 5. EXPERIMENTAL RESULT We have developed the UAV smulator and two nterface programs n Java programmng language. Our UAV smulator s runnng on the Actor Foundry, but nterface programs do not requre the Actor Foundry. In order to smulate the flyng and turnng behavor of UAVs, we use the basc knematcs model of arplanes, but we abstract away the detaled dynamcs and knetcs of arcraft. For the UAV smulaton, the sze of the smulaton area s set to 1,000,000 800,000 8,000 cubc feet (length wdth alttude), sze of the msson area to 400,000 500,000 8,000 cubc feet, the radus of local broadcast communcaton of a UAV to 50,000 feet, and the radus of radar sensor to 25,000 feet. There are 50 targets n the msson area, and they are normally dstrbuted. Half of the targets are statc and the others are dynamc targets. When a UAV s wthn 1,000 feet from a target, the UAV consumes the value of the target. The ntal value of each target s 100, and the dscount factor d n the target value functon s 5 per second. To nvestgate how dfferent cooperaton strateges nfluence the performance of a jont msson, we use Average Servce Cost (ASC) defned as follows: ASC = n ( NT MNT ) n where n s the number of UAVs, NT means navgaton tme of UAV, MNT (Mnmum Navgaton Tme) means average navgaton tme of all UAVs requred for a msson when there are no targets. Fgure 6 shows Average Servce Cost for three dfferent cooperaton strateges. When the number of UAVs s ncreased, ASC s decreased n every case. However, the sharng-based cooperaton strategy and the team-based coordnaton strategy are better than the self-contaned UAV strategy. From ths result, we conclude that communcaton of UAVs s useful to handle targets, even though UAVs n the self-

contaned UAV strategy consumes quckly the value of a target when they handle the target together. Another nterestng result s the performance of the team-based coordnaton strategy s smlar to that of the sharngbased cooperaton strategy, even though the algorthm of the sharng-based cooperaton strategy s much smpler. The overall ASC of the team-based coordnaton strategy s 3 or 5 seconds faster than that of the sharng-based cooperaton strategy. When n (t) n the target value functon s not used, the performances of the sharng-based cooperaton strategy and the team-based coordnaton strategy are not changed very much whle that of the selfnterested UAV strategy s decreased (Fgure 7). ASC ASC 400 350 300 250 200 150 100 50 0 20 25 30 35 40 45 50 Number of UAVs Self- nterested UAVs Sharng- based Cooperaton Team- based Coordnaton Fgure 6: Average Servce Cost (ASC) for three dfferent coordnaton strateges. 400 350 300 250 200 150 100 50 0 20 25 30 35 40 45 50 Number of UAVs Self- nterested UAVs Sharng- based Cooperaton Team- based Coordnaton Fgure 7: Average Servce Cost when n (t) s not used. 6. RELATED WORK Johnson and Mshra present a flght smulaton tool for GTMax (Georga Tech R-Max VTOL UAV) [Johnson and Mshra 2002]. Barney Pell and hs colleagues descrbe the NMRA (New Mllennum Remote Agent), archtecture for a UAV. The NMRA ntegrates realtme montorng and control wth plannng and schedulng, handles fault recovery and reconfguraton of component models, and smulates the autonomy of a UAV [Pell et al. 1997]. However, the type of the GTMax UAV s a helcopter, and both papers do not handle cooperaton among UAVs. Altenburg and hs colleagues present an agent based smulator to smulate UAV cooperatve control [Altenburg et al. 2002]. In ther approach, agents are reactve agents whle UAV components n our smulaton are delberatve agents. Therefore, ther agents drectly respond to sgnals from envronment, whle our agents change ther ntenton about targets and automatcally and proactvely select a dfferent acton. Also, ther agents communcate wth others ndrectly through the envronment whle our agents communcate wth each others drectly. Kolek and hs colleagues present a smulaton framework to evaluate the performance of real tme tactcal rado networks wth a UAV [Kolek et al. 1998]. In ths paper, the authors explan how much dstrbuted smulaton could be appled to solve mltary problems, but they do not handle the autonomy of UAVs and coordnaton among UAVs. 7. FUTURE WORK The Actor system supports dstrbuted computatonal envronment and actor moblty. In the current platform, t s the programmer s role to determne actor placement. However, ths s hard to do when we do not know the CPU speed and the communcaton speed among dfferent machnes. Specfcally, when the communcaton pattern among actors s changed, the ntal placement of actors mght prove to be a deterrent to cross boundary communcaton. For ths, we are developng dynamc actor reconfguraton algorthm. In the new actor platform, the communcaton pattern among actors wll be montored, and actors wll be dynamcally reallocated by the platform. Another problem of the current actor system s the exstence of Smulaton Control Manager (SCM) to control the vrtual tmes of UAVs globally. Ths component may be a bottleneck of the dstrbuted smulaton, and f ths component were to fal, the smulaton would collapse completely. To counter ths, the Jefferson s vrtual tme [Jefferson 1985] based actor platform can be used. In ths actor platform, each actor mantans ts own vrtual tme, and when an actor communcates wth another actor and the tme dfference s more than the gven threshold, the platform performs the rollback. As another extenson, we are lookng to merge a few real UAVs nto UAV smulaton. That s, we are gong to buld a UAV smulator wth the possblty of real tme nput from real UAVs and vrtual UAVs. In ths smulaton, a real UAV can communcate wth other real UAVs and vrtual UAVs to perform a vrtual task. Ths approach can overcome the problem of computer smulaton, such as the naccuracy of UAV knematcs and the communcaton delay defned by programmers. In our smulaton, we use Contract Net Protocol. It means f a UAV accepts the order from a leader UAV,

the UAV must handle the target. However, the belef about envronment changes when UAVs detects more targets or addtonal UAVs become avalable after havng consumed value of ther respectve targets. Therefore, when any change n the envronment s detected or any UAV becomes avalable, ths nformaton s delvered to the leader UAV, and the leader UAV may reconsder and change the target assgnment. Also, a member UAV may secede from ts team to handle a new target wth a more attractve force value. Ths dea s motvated from the leveled commtment n Contract Net Protocol [Sandholm and Lesser 1995]. 8. CONCLUSIONS In ths paper, we have descrbed the desgn and development of a dstrbuted UAV smulator usng an actor-based platform, a utlty functon, and Contract Net Protocol. The three layered archtecture for our UAV smulaton s presented: the actor-based dstrbuted platform, the UAV smulator, and the user nterface layer. We have descrbed three strateges to perform a jont msson: the self-nterested UAVs strategy, the sharng-based coordnaton strategy, and team-based cooperaton strategy. Ths has been supplemented by our expermental results and outlne of the future work. Our UAV smulator s workng on an actor-based dstrbuted platform, and hence, t naturally adapts to the behavor of a dstrbuted and concurrent stuaton. We can easly mprovse the executon envronment wthout changng the UAV smulator by separatng the dstrbuted platform from the smulator. For example, we can mgrate some actors from a computer to another durng the executon tme. Other possble means for mprovsng the workng envronment have been presented n the future work secton. When we consder multple UAVs workng together, ther cooperaton mechansms are of utmost mportance. In ths paper, we have presented three dfferent approaches, and compared and contrasted them. The expermental results suggest that cooperaton and coordnaton strateges are better than the selfnterested UAV strategy. Last but not least, we have ntroduced the actve brokerng servce to support applcaton orented searchng. ACKNOWLEDGEMENT Ths research s sponsored by the Defense Advanced Research Projects Agency under contract number F30602-00-2-0586. Vews and conclusons contaned n ths document are those of the authors and should not be nterpreted as representng offcal polces, ether expressed or mpled, of the Defense Advanced Research Projects Agency or the Unted States Government. REFERENCES Agha, G.A. 1986. Actors: A Model of Concurrent Computaton n Dstrbuted Systems. MIT Press, Cambrdge, Mass. Agha G.A.; I.A. Mason; S.F. Smth; and C.L. Talcott. 1997. A Foundaton for Actor Computaton. Journal of Functonal Programmng, Vol. 7, No. 1, 1-69. Altenburg K.; J. Schlecht; and K. Nygard. 2002. An Agentbased Smulaton for Modelng Intellgent Muntons. In Proceedngs of the Second WSEAS Internatonal Conference on Smulaton, Modelng and Optmzaton, Skathos, Greece (Sep). Avalable at http://www.cs.ndsu.nodak.edu/~nygard/research/munt ons.pdf Astlery M. 1999. Actor Foundry. Department of Computer Scence, Unversty of Illnos at Urbana-Champagn, IL (Feb. 9). Avalable at http://yangtze.cs.uuc.edu/foundry Clausen T.H. 1998. Actor Foundry a QuckStart. Department of Computer Scence, Insttute of Electronc Systems, Denmark (Nov. 9). Avalable at http://yangtze.cs.uuc.edu/foundry Doherty P.; G. Granlund; K. Kuchcnsk; E. Sandewall; K. Nordberg; E. Skarman; and J. Wklund. 2000. The WITAS Unmanned Aeral Vehcle Project. In Proceedngs of the 14th European Conference on Artfcal Intellgence (ECAI 2000), Berln, Germany (Aug), 747-755. Grmes R. 1997. Professonal DCOM Programmng. Olton, Brmngham, Canada, Wrox Press. Hadzlacos V. and S. Toueg. 1993. Fault-Tolerant Broadcastng and Related Problems. In Dstrbuted Systems, S. Mullender (Ed.). ACM Press, New York, 97-145. Jefferson D. 1995. Vrtual Tme. ACM Transactons on Programmng Languages and Systems, Vol. 7, No. 3 (Jul), 404-425. Johnson E.N and S. Mshra. 2002. Flght Smulaton for the Development of an Expermental UAV. In Proceedng of the AIAA Modelng and Smulaton Technologes Conference and Exhbt, Monterey Calforna, CA (Aug), 5-8. Kolek S.R.; S.J. Rak; and P.J. Chrstensen. 1998. Battlefeld Communcaton Network Modelng. The DIS Workshop on Smulaton Standards. Avalable at http://dss.ll.mt.edu/dss.web/98f-siw-143.html OMG. 2002. The Common Object Request Broker Archtecture: Core Specfcaton. Verson 3.0.2 (Dec). Pell B.; D.E. Bernard; S.A. Chen; E. Gat; N. Muscettola; P.P. Nayak; M.D. Wagner; and B.C. Wllams. 1997. An Autonomous Spacecraft Agent Prototype. In Proceedngs of the Frst Internatonal Conference on Autonomous Agents, Marna del Rey, CA, 253-261. Sandholm T. and V. Lesser. 1995. Issues n Automated Negotaton and Electronc Commerce: Extendng the Contract Net Framework. In Proceedngs of the 1st Internatonal Conference on Multagent Systems, San Francsco, CA, 328-335. Smth R.G. 1980. The Contract Net Protocol: Hgh-Level Communcaton and Control n a Dstrbuted Problem Solver. IEEE Transactons on Computers, Vol. 29, No. 12, 1104-1113. Smth R.G. and R. Davs. 1980. Frameworks for Cooperaton n Dstrbuted Problem Solvng. IEEE Transactons on Systems, Man and Cybernetcs, Vol. 11, No. 1, 61-70.

AUTHOR BIOGRAPHIES MYEONG-WUK JANG s a doctoral canddate and research assstant n the Open Systems Laboratory at the Unversty of Illnos at Urbana-Champagn. Hs research nterests nclude mult-agent system and task allocaton n open dstrbuted computng. He receved a BS n Computer Scence from Korea Unversty n 1990 and an MS n Computer Scence from KAIST (Korea Advanced Insttute of Scence and Technology) n 1992. He worked for ETRI (Electroncs and Telecommuncatons Research Insttute), Korea, untl 1998. Hs web page can be found at http://www.uuc.edu/~mjang/. SMITHA REDDY s a Master/PhD student and research assstant n the Open Systems Laboratory at the Unversty of Illnos at Urbana-Champagn. Her research nterests nclude dstrbuted systems, hgh speed networks, and dynamc resource sharng. She receved a BE n Computer Scence from Unversty of Pune n 1999. PREDRAG TOSIC s a doctoral canddate and research assstant n the Open Systems Laboratory at the Unversty of Illnos at Urbana-Champagn. He receved a BS n Mathematcs and Physcs and an MS n Appled Mathematcs, both at Unversty of Maryland Baltmore County, UMBC, n 1994 and 1995, respectvely, and also holds an MS n pure Mathematcs from Unversty of Illnos at Urbana- Champagn n 1997. LIPING CHEN s a doctoral canddate and research assstant n the Open Systems Laboratory at the Unversty of Illnos at Urbana-Champagn. GUL A. AGHA s Drector of the Open Systems Laboratory at the Unversty of Illnos at Urbana- Champagn and Professor n the Department of Computer Scence. Hs research nterests nclude models, languages, and tools for parallel computng and open dstrbuted systems. He receved a BS n an nterdscplnary program from the Calforna Insttute of Technology, an MA n Psychology from the Unversty of Mchgan, Ann Arbor, and an MS and PhD n Computer and Communcaton Scence, from the Unversty of Mchgan, Ann Arbor.