COOPERATIVE REAL-TIME TASK ALLOCATION AMONG GROUPS OF UAVS

Transcription

1 Chapter 1 COOPERATIVE REAL-TIME TASK ALLOCATION AMONG GROUPS OF UAVS Yan Jn Maros M. Polycarpou Al A. Mna Department of Electrcal & Computer Engneerng and Computer Scence Unversty of Cncnnat Cncnnat, OH Abstract Unnhabted autonomous vehcles(uavs) are an ncreasngly mportant part of battlefeld envronments, and may soon be common n cvlan applcatons such as dsaster relef, envronmental montorng and planetary exploraton. Such vehcles may be arborne, land-based or aquatc, though the focus so far has been on arborne vehcles for mltary applcatons, and ths s the focus of the research presented here. We consder a heterogeneous group of UAVs drawn from several dstnct classes and engaged n a search and destroy msson over an extended battlefeld. Durng the msson, the UAVs perform Search, Confrm, Attack andbattle Damage Assessment (BDA) tasks at varous locatons. The tasks are determned n real-tme by the actons of all UAVs and ther consequences (e.g., sensor readngs), so that the task dynamcs are stochastc. The tasks must, therefore, be allocated to UAVs n real-tme as they arse, whle ensurng that approprate vehcles are assgned to each task. Each class of UAVs has ts own sensng and attack capabltes, so the need for approprate assgnment s paramount. We present a smple cooperatve approach to ths problem, based on dstrbuted assgnment medated by centralzed msson status nformaton. We also suggest methods for decentralzng the allocaton process usng a mnmum dsturbance approach termed MDAS (Mnmum Dsturbance Allocaton Strategy). Ths work was supported by the AFRL/VA and AFOSR Collaboratve Center of Control Scence (Grant F ). 1

2 2 1. Introducton Over the last decade, unmanned vehcles such as arborne drones or mnesweepng robots have become an ncreasngly feasble part of the battlefeld envronment, and may soon be common n cvlan applcatons such as dsaster relef, envronmental montorng and planetary exploraton. However, the unmanned vehcles n current use, such as the Predator, are not autonomous, and requre remote gudance by a team of human operators. Not only s ths expensve and rsky, t also places a fundamental lmt on the scalablty and range of such systems. Motvated by recent advances n ntellgent systems and cooperatve control, many researchers have been studyng large groups of truly autonomous vehcles called unmanned autonomous vehcles or UAVs actng n concert to accomplsh dffcult mssons n dynamc, poorly known and/or hazardous envronments (Passno, 2002; Chandler and Pachter, 1998; Chandler et al., 2000; Chandler and Pachter, 2001; Chandler et al., 2001; Chandler, 2002; Beard et al., 2000; Beard et al., ; Bellngham et al., 2001; Jacques, 1998; L et al., 2002; McLan and Beard, 2000; McLan et al., 2002; Motra et al., 2001; Polycarpou et al., 2001; Polycarpou et al., 2002; Schumacher et al., 2001). The work reported here presents an approach to ths problem. We consder a heterogeneous group of UAVs drawn from several dstnct classes and engaged n a search and destroy msson over an extended battlefeld wth known and unknown targets. The UAVs must cooperatvely search the envronment, confrm known targets, dscover and confrm new ones, attack them wth approprate muntons, and confrm ther destructon. Ths defnes a tme-varyng set of tasks over the envronment that must be accomplshed by UAVs of the approprate classes such that the overall msson completon tme s mnmzed. Snce the tasks are determned by the actons of the UAVs and ther stochastc consequences, the process of task generaton s stochastc and not predctable ahead of tme. Ths, n turn, requres that the UAVs select tasks n the battlefeld as they arse. The prmary determnants of ths selecton are: 1) The locatons of tasks and UAVs; and 2) the UAV capabltes requred for each task. Ths creates a problem smlar to many other assgnment problems, but most closely related to the dynamc vehcle routng problem (Schouwenaars et al., 2001; Tan et al., 2002), albet of much greater complexty gven the several types of vehcles and tasks. Our goal, ultmately, s to develop a truly decentralzed approach to the effcent heurstc soluton of ths problem. However, n ths paper,

3 we present a smple, somewhat prelmnary model where UAVs choose tasks autonomously n a decentralzed way but use a central cogntve map that provdes an nstantaneous and accurate summary of the current stuaton as known to the UAV team. Future work wll address the problem of decentralzng the cogntve map and the consequences of ths decentralzaton. The approach we follow s motvated by the semnal work of Chandler and Pachter, and ther collaborators (Chandler and Pachter, 1998; Chandler et al., 2000; Chandler and Pachter, 2001; Chandler et al., 2001; Chandler, 2002; Schumacher et al., 2001). It s also related closely to recent work by several other researchers (Beard et al., 2000; Beard et al., ; Bellngham et al., 2001; Jacques, 1998; L et al., 2002; McLan and Beard, 2000; McLan et al., 2002). A comprehensve overvew of the research problems assocated wth UAV teams s avalable n (Passno, 2002) Scenaro We consder a L x L y cellular envronment, N UAVs, M statonary targets, γ, = 1,..., M wth locatons, (x γ, yγ ), and no threats. Though we requre that the UAVs completely search the entre envronment, we do not nclude any hdden targets n the smulatons reported here. A seres of tasks need to be accomplshed at each target locaton as descrbed below. A canoncal task set, T T, s defned to comprse all the tasks that the UAVs can undertake at a target locaton. In ths formulaton, we have: 3 T T = {Search, Conf rm, Attack, BattleDamageAssess(BDA), Ignore} 1.2. UAV Model Every UAV, u, s characterzed by an expertse vector, that gves nformaton on the UAV s capabltes wth regard to the tasks n the task set, T T. The expertse vector for UAV u s ξ = {ξ j }, j = 1,..., n, 0 ξ j 1, where ξ j ndcates the UAV s expertse for task T j. The expertse reflects the qualty of UAV s capabltes. Ths formulaton can be used to specfy classes of UAVs wth specfc functonal repertores (e.g., reconnassance UAVs and attack UAVs), but UAVs can have ndvdual expertse profles n the general case. These profles may also change wth experence representng learnng. The matrx Ξ wth expertse vectors as rows s termed the expertse matrx for ths problem. UAVs move autonomously n the envronment, scannng, communcatng wth other UAVs, makng decsons, and performng tasks.

4 System Dynamcs At any tme, t, every cell, (x, y), n the envronment has an assocated task status, T (x, y, t), ndcatng what a UAV needs to do n that cell. The task status of all cells, T (t) = {T (x, y, t)}, represents the state of the envronment, termed the task state. The dynamcs of the task state s medated by the target occupancy probablty (TOP), P (x, y, t), of each cell, (x, y), defned as the estmated probablty that the cell contans a lve target. UAVs move n the envronment seekng to accomplsh the tasks cued at the cells they occupy. In the current (centralzed) model, t s assumed that all UAVs know the current task state, and actvely plan ther paths to perform tasks suted to ther capabltes, bddng for tasks and makng commtments n concert wth the team. Actons and observatons by UAVs change the TOP, and the task status of each cell s updated based on TOP thresholds as descrbed n the next secton. The confrm, attack and BDA tasks form the set of assgnable tasks,.e., tasks for whch the UAVs are assgned explctly. These UAVs move purposvely to the locatons of ther assgned tasks and perform them. search and gnore comprse the set of automatc tasks,.e., any UAV passng through a cell wth one of these task statuses automatcally performs the ndcated task possbly wth varyng qualty among dfferent UAVs for search. However, UAVs do not actvely bd for these tasks. The search task does have an effect on UAV movements as descrbed later. All locatons whose task status at tme t corresponds to an assgnable task form the set, L(t), of current target locatons (CTLs). The task, τ j, at each CTL, (x j, y j ), has an assgnment status, A j, whch can take on the values from the set {avalable, assocated, assgned, actve, complete}. The assgnment status ndcates whether the task s open for bddng (avalable), has been provsonally assgned to a UAV (assocated), has been frmly assgned to a UAV (assgned), s beng currently performed by a UAV at the locaton (actve), or has been fnshed (complete). A completed task s accompaned by an mmedate transton n the task status of the CTL, possbly to the same task. The state, S (t), of a UAV, u, at tme t comprses two parts: A physcal state, whch ncludes nformaton on ts poston, λ (t), and orentaton, δ (t). A functonal state, whch ndcates the dentty and locaton of the specfc task (f any) to whch the UAV s commtted or has bd for, the correspondng commtment status (see below), and the UAV s

5 expected cost for performng ths task. The commtment status, K (t), of UAV u takes on values from the set: {open, competng, commtted} ndcatng whether the UAV has no commtment (open), has bd on a task or been assocated wth one (competng), or s assgned to a task and, possbly, s performng t (commtted). The functonal state of an open UAV has NULL values n ts other felds. The search and gnore tasks requre no commtment, and correspond to a NULL functonal state. As u moves n the envronment, t performs an acton, a (t), n each cell, (x (t), y (t), that t vsts at tme t. The actons are drawn from a canoncal lst, and nclude such acts as takng varous knds of sensor readngs and frng varous types of muntons. Dong nothng s also a possble acton, and s termed the null acton. The acton performed by the UAV n cell (x )t), y )t)) s selected from ths canoncal lst by an acton selecton functon, G a (t) = G(T (x (t), y (t), t), S (t)) (1) Thus, the selected acton s based on the current task status of the cell and the UAV s own state n terms of capabltes and commtment. If the acton s a sensor readng, t returns an observaton value, b (t), whch s a stochastc quantty. Whle actons are performed by UAVs, we also denote by a(x, y, t) the set of actons (ncludng null actons) performed by all UAV s n cell (x, y) at tme t, and by b(x, y, t) the set of observatons (f any) taken by the UAVs. a(x, y, t) = {a (t) s.t. λ (t) = (x, y)} (2) b(x, y, t) = {b (t) s.t. λ (t) = (x, y)} (3) Ths determnes the updates of the TOP value at (x, y) through a possbly stochastc TOP update functon, F : P (x, y, t + 1) = F (P (x, y, t), T (x, y, t), a(x, y, t), b(x, y, t)) (4) If a(x, y, t) and b(x, y, t) have several elements (because the cell was occuped by more than one UAV at tme t), the TOP update terates over them. The TOP value, n turn, determnes the dynamcs of the cell s task status, whch s updated va a determnstc automaton whose transtons depend on threshold crossngs n P (x, y, t) (Fgure 1): T (x, y, t + 1) = H(T (x, y, t), P (x, y, t + 1); θ) (5) 5

6 6 where the parameter vector θ represents the set of threshold values used for transtons. Together, Equatons (1) (5) defne the dynamcs of the system. Note that the actons of the UAVs cause state transtons n the envronment whch, n turn, drves the actons of the UAVs. The dynamcs s made stochastc by the stochastcty of b(x, y, t) and the TOP update functon, F (see below). The TOP Update Functon. As descrbed above, the task status of each cell s updated based on the crossng of preset thresholds by ts TOP. Thus, the TOP update functon, F ( ), s crucal for the system s dynamcs. As ndcated n Equaton 4, the TOP update depends on a cell s current task status, and we defne F ( ) separately for each case. 1 Task 1: Search: A UAV, u, engaged n the search task makes a sensor readng ntended to detect targets. The resultng observaton s gven by b (x, y, t) = 1 f the sensor detects a target and by b (x, y, t) = 0 s t does not. The sensor s assumed to be mperfect, wth a parameter α characterzng ts detecton accuracy: α = P (b t A) P (b t A) where A s the event that a target s actually located n the cell beng scanned. When the UAV s search sensor reports a target present n cell (x, y),.e., b (x, y, t) = 1, the update s: P (x, y, t + 1) = αp (x, y, t) 1 + (α 1)P (x, y, t) (6) When the UAV s search sensor reports there s no target n cell (x, y),.e., b (x, y, t) = 0: (x, y), P (x, y, t + 1) = 1 P (x, y, t) 1 + (α 1)P (t) (7) These update equatons are derved based on Bayesan nference (see Appendx). A cell wth the search status remans n ths status untl ts TOP falls below the resoluton threshold, p r, or exceeds the suspcon

7 threshold, p s. In the former case, t transtons to gnore and n the latter to confrm. 2 Task 2: Confrm: Ths status s nvoked when the TOP, P (x, y, t), for a cell (x, y) wth status search reaches the suspcon threshold, p s, ndcatng a sgnfcant possblty of a target beng present there. The cung of a confrm task at cell (x, y) ndcates that a UAV wth the approprate sensors should move purposvely towards the cell and scan t. All cells where targets are suspected at msson ncepton are ntalzed wth the confrm task and gven a TOP of p s. The conf rm task s functonally dentcal to search, but s assgnable to UAVs wth the approprate expertse whereas search s not. The TOP update functon s as gven n Equatons (6) and (7). However, the sensors used need not be the same as those used n search, and may have a dfferent value of α. The cell remans n the confrm status untl ts TOP falls below p s (as a result of falure to confrm suspcons) or exceeds the certanty threshold, p c. In the former case, the status changes back to search, n the latter to attack. 3 Task 3: Attack: If P (x, y, t) at cell (x, y) wth status confrm exceeds the certanty threshold, p c, ts status transtons to attack, ndcatng that an approprately armed UAV should proceed to the locaton and attack t wth the correct munton. Once ths acton s performed, the UAV changes the TOP for that locaton n accordance wth an nternal model (see Appendx). The update functon gven by the smple model s: P (x, y, t + 1) = P (x, y, t) (1 P s ) (8) 7 where 0 P s 1 s the probablty that the target s destroyed n the attack. Dfferent types of UAVs can have dfferent values of P s for dfferent target types. A cell wth status attack remans n ths status untl ts TOP exceeds the ext threshold, p e, whch causes a transton to status BDA. 4 Task 4: BDA:

8 8 If, as a result of an attack n a cell (x, y) wth an attack status, the cell s TOP falls below the ext threshold, p e p s, the cell transtons to the BDA status. The task here s to verfy that the TOP has ndeed fallen below p e. The BDA task, lke search and conf rm s purely observatonal, and uses the same update equatons (6) and (7). If the result of the update produces P (x, y, t + 1) p e, the cell transtons back to attack; f p r P (x, y, t + 1) p e, t transtons to search; and f P (x, y, t + 1) < p r, the cell transtons to gnore. The value p r (resoluton threshold) corresponds to a value so low as to exclude any possblty of a target. Note that settng p r = 0 effectvely elmnates the gnore state. However, gnore s a useful state snce t allows the specfcaton of regons that should specfcally be excluded from the search. It also allows a msson termnaton condton to be defned concsely. 5 Task 5: Ignore: As dscussed above, ths state apples to cells that do not even need to be scanned. Ths may be because they are known a pror to harbor no targets, because they have been scanned suffcently to be excluded, or because known targets there have been destroyed verfably. Fgure 1 shows the transtons between states usng an automaton formulaton Certanty Dynamcs As TOP estmates are updated va search, t s mportant also to quantfy the relablty of these estmates. For example, f P (x, y, t) for a locaton s close to zero after several UAVs have scanned t, one can be more certan that t has no target than f the TOP s based on an ntal guess. Indeed, whle the confrm, attack and BDA tasks are drven by the TOP, the search task must be drven by ths confdence factor We quantfy t by defnng a certanty varable, χ(x, y, t) [0, 1], for each (x, y). The ntal value, χ(x, y, 0), s based on the a pror nformaton about the occupancy of (x, y) (e.g., f all targets are land-based, locatons correspondng to a lake may begn wth P (x, y, 0) = 0 and χ(x, y, 0) = 1). Most locatons would typcally begn wth a certanty of zero. Each tme a UAV vsts (x, y) and makes an observaton, the certanty changes as χ(x, y, t + 1) = χ(x, y, t) + 0.5(1 χ(x, y, t)) = 0.5(1 + χ(x, y, t)) (9)

9 9 p < P <= p s c P > p c P > p e Confrm Attack P > p s P <= p e P <= p s P > p e Search BDA p <= P <= p e r p <= P <= p r s P < p r P < p r Ignore P <= 1 Fgure 1.1. Task Dynamcs. Where, Ps=suspcon threshold, Pc=certanty threshold, Pe=ext threshold, Pr=resoluton threshold. Ths formulaton, orgnally proposed n (Yang et al., 2002b), provdes a smple way to track the number of useful looks each locaton has had and captures the noton of dmnshng returns wth each look.

10 10 2. Algorthm Descrpton We consder a team of UAVs, u, drawn from two known classes wth well-defned capabltes (target recognton (TR) and attack (A)). All UAVs are assumed to have sensors needed for search. The UAV classes are represented by dstnct expertse vectors, ξ j, wth respect to the fve tasks n the task set: TR Class UAVs have ξ j = {ξ S 1, ξc 1, ξa 1, ξb 1, ξi 1 }. A Class UAVs have ξ j = {ξ S 2, ξc 2, ξa 2, ξb 2, ξi 2 }. All ξk T are between 0 and 1. In keepng wth the capablty desgnatons, we set: ξ S > ξ2 S, ξc > ξ2 C, ξa < ξ2 A, ξb > ξ2 B, ξi = ξ2 I. The UAVs operate n a regon where certan targets are suspected to exst. No threats are consdered n ths prelmnary model. The UAVs msson s to search all cells that are not desgnated gnore, and to perform confrm, attack and BDA tasks on each target known or dscovered through search. For each task, the team must try to use UAVs best suted to t Informaton Base All UAVs have nstantaneous and nose-free access to a centralzed nformaton base (IB), whch comprses the followng tems: 1 The TOP map P (x, y, t) (x, y). 2 The certanty map χ(x, y, t) (x, y). 3 The task status map T (x, y, t) (x, y). 4 The assgnment status map A(x, y, y) CT L(x, y). 5 The UAV state vector, S(t) = {S (t)} u. Each UAV reads and updates the IB at each step Intalzaton The msson begns wth an externally suppled TOP map for the envronment. Typcally, ths would nclude P (x, y, 0) p s for regons where targets are unlkely (or mpossble), P (x, y, 0) = p s for suspected target locatons, and p s < P (x, y, 0) < p r for other locatons. Thus, the suspected target locatons are cued wth avalable conf rm tasks and the rest wth search or gnore tasks. The UAVs ntal postons are also gven. All UAVs ntally have the open status.

11 2.3. Intal Assgnment Gven the ntal tasks and UAV postons, the frst step s to assgn an ntal task to each UAV. We denote the current set of assgnable tasks by T s = {τ k }, and use j k to denote the dentty of task τ k,.e., whether t s confrm (j k = 2), attack (j s = 3), or BDA (j s = 4). The assgnment s done as follows: 1 Each UAV u calculates a cost value, h k, wth respect to all avalable or assocated assgnable tasks, τ k : 11 h k = ω 1 d k + (1 ω 1 ) exp( ξ jk ) (10) where ω 1 s a postve parameter, 0 ω 1 1, d k s the normalzed dstance between UAV u and the locaton of task τ k. ξ jk s the expertse of UAV u for task j k. 2 Each UAV, u, chooses a task τ k, whch satsfes h k < h k for all the other τ k s. Note that already assgned tasks are not consdered n ths process. 3 Each UAV, u, reports ts preferred task τ k, the correspondng value of the dstance d k, and the cost value h k to the nformaton base. If the status of the task τ k s avalable (.e., the task has not been bd for by any UAV), then u s consdered for ths task. If the dstance to the task locaton s wthn a threshold (d k θ A ), task τ k s assgned to UAV u, whose status changes to commtted. The task s removed from the pool over whch UAVs are competng. If d k > θ A, task τ k s assocated wth UAV u, whose status s set to competng. The task, n ths case, stll remans open for competton. If the status of the task τ k s assocated (.e., some other UAV, u l, has been provsonally gven ths task), then the costs h k and h lk are compared and the UAV wth the smaller cost s assgned or assocated wth the task under the threshold rule. The UAV that loses the competton stays n the competng pool for the remanng tasks. 4 The process contnues teratvely untl all UAVs have been assgned a task (we assume that the number of tasks exceeds the number of UAVs).

12 Assgnment Update When the ntal assgnment s completed, each UAV begns to move towards ts assgned or assocated task. As t passes each cell, the TOP n that cell s updated n accordance wth the update dynamcs descrbed above. When t gets to ts assgned task, t performs the task and leads to a TOP update there. A new task s then cued at the CTL, and the UAV s status reverts to open. Dependng on whether the last acton caused the TOP to cross a task transton threshold, the new ask may be the same as the prevous one or not. Locatons that prevously dd not have suspected targets can become CTLs f search rases ther TOP above p s. Ths corresponds to the dscovery of a new target. Each new assgnable task whether at an exstng CTL or a new one s cued wth an avalable status. At all tmes, all open and competng UAVs are beng consdered for all avalable and assocated tasks. The UAVs are processed n a randomzed sequence accordng to the same algorthm as that used for the ntal assgnment. The process contnues untl all locatons have an gnore status or some tme threshold s meet UAV Movement At all tmes, open UAVs move by followng the most locally productve search drecton, whch s determned va the certanty varable. We use a partcularly smple model n ths paper, where the UAV compares the certanty values for all possble next postons and always moves to the one wth the lowest certanty. Tes are broken randomly. In other work (Yang et al., 2002b; Yang et al., 2002a), we have consdered more sophstcated approaches for determnng effcent search paths. Competng and commtted UAVs follow the most drect path to ther target locatons. 3. Performance Measures The goal for the UAV team s to cover the envronment as rapdly as possble n such a way that all cells reach the gnore task status,.e., all cells are completely searched and all targets neutralzed. Specfcally, we measure two tmes to quantfy performance: 1 The target neutralzaton tme (TNT), whch s the tme needed to neutralze all a pror known targets. 2 The jotal msson tme (TMT), whch s the total number of steps needed to brng all cells to the gnore status.

13 4. Smulaton Results One of the prmary ssues consdered n the smulatons we report s the effectveness of search n combnaton wth target neutralzaton. As descrbed earler, the movement of uncommtted UAVs n the envronment s drven by the need to search, medated by the certanty varable. We use smulatons to quantfy the effectveness of ths search-drven (SD) polcy n comparson wth the nave random movement (RM) polcy. Note that the dfferent polces apply only to UAVs that are not assocated wth or commtted to a task; other UAVs always take the shortest path to ther desgnated target locaton. In the frst smulaton (Fgures 2 and 3), we consder a cellular envronment wth 10 UAVS 5 ATR unts and 5 attack unts. The number of targets s vared systematcally from 10 to 50. The data for each case s averaged over ten ndependent runs wth random target confguratons. Fgure 2 shows that the SD and RM polces lead to no sgnfcant dfference n the TNT. The tme taken to neutralze all known targets appears to scale lnearly wth the number of targets, whch s to be expected. Fgure 3 shows the TMT wth the two polces, and here t s clear that the SD polcy provdes an extremely sgnfcant mprovement. Thus, usng the SD approach gves up nothng n attack effectveness whle greatly ncreasng search effcency. Fgures 4 and 5 show the results for the case wth 8 ATR UAVs and only 2 attack UAVs. Whle the actual msson duraton s dfferent because of the small number of attack UAVs, the results are qualtatvely smlar to the prevous case. 5. Decentralzaton Approach As descrbed above, the current formulaton s a partally centralzed one n that all UAVs use the same, globally and nstantaneously updated cogntve map. However, the UAVs make ther commtment decsons autonomously, and ths s the bass for the possblty of decentralzaton. We have been developng a decentralzaton approach that we term the mnmum dsturbance allocaton strategy (MDAS), whch has several components: 13 1 Optmal, off-lne ntal assgnment: In ths stage, UAVs are assgned to all known targets usng a powerful but possbly expensve optmzaton procedure such as a genetc algorthm or nteger programmng. However, ths s feasble because t s done off-lne usng powerful computers. Snce the actual dynamcs of the msson s stochastc and not known a pror, ths ntal as-

14 Number of steps to destroy all the targets Number of targets Fgure 1.2. TNT for 15*15 cellular envronment, 5 ATR UAVs and 5 Attack UAVs. Sold lne s for the search-drven polcy and dashed lne for random movement Number of steps to search the whole regon Number of targets Fgure 1.3. TMT for 15*15 cellular envronment, 5 ATR UAVs and 5 Attack UAVs. Sold lne s for the search-drven polcy and dashed lne for random movement.

15 Number of steps to destroy all the targets Number of targets Fgure 1.4. TNT for 15*15 cellular envronment, 8 ATR UAVs and 2 Attack UAVs. Sold lne s for the search-drven polcy and dashed lne for random movement Number of steps to search the whole regon Number of targets Fgure 1.5. TMT for 15*15 cellular envronment, 8 ATR UAVs and 2 Attack UAVs. Sold lne s for the search-drven polcy and dashed lne for random movement.

16 16 sgnment would use a typcal unfoldng of the msson based on a model usng known targets, etc. 2 Decentralzed, opportunstcally updated cogntve map: Rather than a sngle centralzed map, each UAV would carry ts own cogntve map bult based on ts own experence and on nformaton communcated by UAVs that t happened to pass close to. The latter s what we term opportunstc. Clearly, any ndvdual UAV s map wll be varously ncomplete, naccurate and out-of-date, leadng to greater performance challenges compared to the centralzed map. 3 Opportunstc, decentralzed n-feld adjustment and assgnment: After the UAVs enter the envronment and begn to follow ther ntal assgnments, these assgnments are modfed locally by ndvdual UAVs n response to developng crcumstances. As the msson unfolds, t creates the actual task dynamcs; new targets are dscovered; new threats emerge. Each UAV, usng ts own lmted, possbly ncorrect, cogntve map, proposes changes to ts plan that would not alter that plan drastcally. Then, as t gets closer to ts new target, t negotates wth other UAVs that may also have ndependently volunteered themselves for the same task. The M- DAS prncple dctates that, n such a negotaton, the UAV whose assgnment would lead to the least overall dsrupton of the ntal plan s preferred. The man ssue here s to develop a negotaton protocol and trggerng mechansm that leads to mnmal overall dsrupton. For example, ths may nvolve lookng at dfferent condtons for volunteerng and dstance thresholds for commtment. The system descrbed n ths paper can be seen as a prelmnary verson of the thrd component of the MDAS approach. However, the crucal element of ncremental negotaton s not yet ncluded. 6. Concluson and Future Work The model presented above s only a smple, frst-cut attempt to formalze the UAV search-and-destroy problem n a way that s amenable to decentralzaton. The results are promsng, and suggest several avenues for further exploraton. These nclude: Incluson of ntally unknown targets and pop-up threats. Use of more comprehensve cost functons, accountng for UAVspecfc capabltes. Consderng the exstence of threats.

17 17 Lettng each UAV bd for more than one target. Usng two or more stages of commtment for UAVs, and multple thresholds for transton of commtment. Usng more realstc UAV expertse profles and target behavor. Work on these areas wll be reported n the future, as wll the work on the decentralzaton wth MDAS.

18 18 Appendx: Dervaton of TOP Update Equatons To obtan the update functons (6) and (7), consder the case where a UAV takes a measurement n cell (x, y) at tme t. Defne the followng for a cell (x, y): A s the event that a target s located n cell (x, y). b t s the bnary sensor readng taken by the UAV, where b t = 1 ndcates target detecton and b t = 0 non-detecton. B t 1 s the vector of all sensor readngs for cell (x, y) by all UAVs taken up to tme t 1 (.e., before tme t). Based on the above defntons, P (A B t 1) s the probablty of target exstence n cell (x, y) at tme t 1 and P (A B t 1, b t) s the updated probablty after obtanng the new readng, b t. Thus we have P (t 1) = P (A B t 1) (A.1) P (t) = P (A B t 1, b t) (A.2) We assume that the sensors measurements n any cell are condtonally ndependent gven the state of the cell,.e. P (b 1, b 2,..., b n A) = n P (b A) =1 (A.3) Based on the above defntons and assumptons, the updatng functon (6) and (7) follow drectly from Bayes rule (Moravec, 1988). Accordng to Bayes rule, P (A B t 1, b t ) P (A B t 1, b t ) = P (A Bt 1 ) P (A B t 1 ) P (bt A Bt 1 ) P (b t A Bt 1 ) (A.4) whch can be smplfed by vrtue of the condtonal ndependence assumpton to: P (A B t 1, b t ) P (A B t 1, b t ) = P (A Bt 1 ) P (A B t 1 ) P (bt A, Bt 1 ) P (b t A, Bt 1 ) P (A Bt 1 ) = P (A B t 1 ) P (bt A ) P (b t A ) (A.5) (A.6) By solvng (A.6) for P (A B t 1, b t) usng the fact that P (Ā Bt 1, bt) = 1 P (A Bt 1, bt), we get Defnng P (b t A ) P (b t A ) P (A B t 1, b t) = 1 [ 1 + ] 1 P (bt A ) P (A Bt 1 ) P (b t A (A.7) ) P (A B t 1 ) = α and usng equaton (A.7), (A.1), (A.2), we can obtan the update equaton (6) and (7) by exchangng P (A B t 1), P (A B t 1, b t) wth P (t) and P (t + 1) correspondngly. The update functon n (8) s obtaned as follows:

19 APPENDIX A 19 P (x, y, t + 1) P rob(target present at (x, y) at step t + 1 target attacked) = P rob(target present at (x, y) at step t AND not destroyed target attacked) P rob(target present at (x, y) at step t)p rob(target not destroyed target attacked) = P (x, y, t)[ 1 P rob(target destroyed target attacked) ] = P (x, y, t)(1 P s) (A.8) where P s s the probablty that the target s destroyed n the attack.

20

21 References Beard, R., McLan, T., and Goodrch, M. (2000). Coordnated target assgnment and ntercept for unmanned ar vehcles. Proc. ICRA 2000, pages Beard, R., McLan, T., Goodrch, M., and Anderson, E. Coordnated target assgnment and ntercept for unmanned ar vehcles. IEEE Trans. On Robotcs and Automaton. Bellngham, J., Tllerson, M., Rchards, A., and How, J. (2001). Multtask allocaton and path plannng for cooperatve uavs. Conference on Coordnaton, Control and Optmzaton. Chandler, P. and Pachter, M. (1998). Research ssues n autonomous control of tactcal uavs. Proc. ACC 1998, pages Chandler, P. and Pachter, M. (2001). Herarchcal control for autonomous teams. Proc. GNC 2001, pages Chandler, P., Pachter, M., and Rasmussen, S. (2001). Uav cooperatve control. Proc. ACC Chandler, P., Rasmussen, S., and Pachter, M. (2000). Uav cooperatve path plannng. Proc. GNC 2000, pages Chandler, P. e. a. (2002). Complexty n uav cooperatve control. Proc ACC Jacques, D. (1998). Search, classfcaton and attack decsons for cooperatve wde area search muntons. Proc. Cooperatve Optmzaton and Control Workshop. L, S.-M., Boskovc, J., Seereeeram, S., Prasanth, R., Amn, R., Mehra, R., and Beard, R. a. M. T. (2002). Autonomous herarchcal control of multple unmanned combat ar vehcles (ucavs). Proc. ACC 2002, pages McLan, T. and Beard, R. (2000). Trajectory plannng for coordnated rendezvous of unmanned ar vehcles. Proc. GNC 2000, pages McLan, T., Beard, R., and Kelsey, J. (2002). Expermental demonstraton of multple robot cooperatve target ntercept. Proc GNC

22 22 Motra, A., Szczerba, R., Ddomzo, V., Hoebel, L., Mattheyses, R., and Yamrom, B. (2001). A novel approach for the coordnaton of multvehcle teams. Proc. GNC 2001, pages Moravec, H. (1988). Sensor fuson n certanty grds for moble robots. AI Magazne, 9: Passno, K. (2002). An ntroducton to research challenges n cooperatve control for unnhabted autonomous vehcles. preprnt. Polycarpou, M., Yang, Y., and Passno, K. (2001). A cooperatve search framework for dstrbuted agents. Proc IEEE ISIC, pages 1 6. Polycarpou, M., Yang, Y., and Passno, K. (2002). Cooperatve control of dstrbuted mult-agent systems. IEEE Control Systems Magazne. Schouwenaars, T., De Moor, B., Feron, E., and How, J. (2001). Mxed nteger programmng for mult-vehcle path plannng. Proc. ACC Schumacher, C., Chandler, P., and Rasmussen, S. (2001). Task allocaton for wde area search muntons va network flow optmzaton. Proc. GNC 2001, pages Tan, K., Lee, L., Zhu, Q., and Ou, K. (2002). Heurstc methods for vehcle routng problem wth tme wndows. Intellgent n Engneerng, pages Yang, Y., Mna, A., and Polycarpou, M. (2002a). Decentralzed opportunstc learng n uav s performng cooperatve search. Proc. GNC Yang, Y., Polycarpou, M., and Mna, A. (2002b). Opportunstcally cooperatve neural learnng n moble agents. Proc. IJCNN 2002.