Evolving Lage Scale UAV Communication System Adian Agogino UCSC at NASA Ames ail Stop 69-3 offett Field, CA 94035 Adian.K.Agogino@ nasa.gov Chis HolmesPake Oegen State Univesity 04 oges Hall Covallis, O 97331 holmespc@onid.ost.edu Kagan Tume Oegen State Univesity 04 oges Hall Covallis, O 97331 Kagan.Tume@ oegonstate.edu ABSTACT Unmanned Aeial Vehicles (UAVs) have taditionally been used fo shot duation missions involving suveillance o militay opeations. Advances in batteies, photovoltaics and electic motos though, will soon allow lage numbes of small, cheap, sola poweed unmanned aeial vehicles (UAVs) to fly long tem missions at high altitudes. This will evolutionize the way UAVs ae used, allowing them to fom vast communication netwoks. Howeve, to make effective use of thousands (and pehaps millions) of UAVs owned by numeous dispaate institutions, intelligent and obust coodination algoithms ae needed, as this domain intoduces unique congestion and signal-to-noise issues. In this pape, we pesent a solution based on evolutionay algoithms to a specific ad-hoc communication poblem, whee UAVs communicate to gound-based customes ove a single widespectum communication channel. To maximize thei bandwidth, UAVs need to optimally contol thei output powe levels and oientation. Expeimental esults show that UAVs using evolutionay algoithms in combination with appopiately shaped evaluation functions can fom a obust communication netwok and pefom 180% bette than a fixed baseline algoithm as well as 90% bette than a basic evolutionay algoithm. Categoies and Subect esciptos I..11 [Computing ethodologies]: Atificial Intelligence ultiagent systems eneal Tems Algoithms, Pefomance, eliability Keywods UAVs, Evolution, ultiagent Systems Pemission to make digital o had copies of all o pat of this wok fo pesonal o classoom use is ganted without fee povided that copies ae not made o distibuted fo pofit o commecial advantage and that copies bea this notice and the full citation on the fist page. To copy othewise, to epublish, to post on seves o to edistibute to lists, equies pio specific pemission and/o a fee. ECCO 1, July 7-11, 01, Philadelphia, Pennsylvania, USA. Copyight 01 AC 978-1-4503-1177-9/1/07...$10.00. 1. INTOUCTION Taditionally, unmanned aeial vehicles (UAVs) have been used fo tageted suveillance and militay opeations. Whethe they ae poweed fo many hous by et engine, o fo a faction of an hou by electic moto, UAV missions tend to have the same pofile: They ae launched, they cay out a mission, then they etun [5]. Such missions ae natually time-limited, labo intensive and logistically challenging, especially since the epeated takeoff and landings need to be coodinated with ai taffic contol. Cuently the numbe of simultaneous UAV missions is in the single digits. Howeve, technology may soon change this. Fo moe than a decade, development has pogessed on sola poweed UAVs with battey eseves, allowing them to fly fo a cetain amount of time afte dak. apid industy advancements in battey capacity, electic moto technology and sola cell efficiency, is allowing the pogession of sola poweed UAVs to poceed even faste [17]. Once UAV technology goes ove the citical hump, whee they can fly all night on chages eceived duing the day, the mission pofiles of UAVs will change adically: they will go up, and stay up. UAV missions could last fo months if not yeas, with ust a small tunove fo maintenance. In fact, at the magins, we ae aleady thee as the sola poweed QinetiQ s Zephy has been demoed, flying fo moe than two weeks [15]. Pemanently flying UAVs allow aicaft to be used in domains cuently monopolized by satellite and gound based-systems, including two-way communications, continuous suveillance and boadcasts. In addition the technologies in sola UAVs ae elatively cheap, with continual pice declines. Soon these aicaft will be available to nealy all nations, institutions and pehaps even individuals. Also since they do not need oxygen to bun fuel, they can be designed to fly at altitudes much highe than conventional aicaft can fly, allowing them be used independently of the cuent ai taffic system. ue to these factos, thee will likely be an explosion in the numbe and uses of UAVs in the futue [3]. We believe that genetic and evolutionay algoithms in combination with multiagent techniques will have a key ole in the complex poblem of contolling such a lage divese set of aicaft. ultiagent systems match the distibuted natue of the poblem, while evolution addesses the messy nonlinea contol and coodination issues. While the potential applications of these UAVs ae vast, we believe that evolution and multiagent systems ae applicable to a wide ange of UAV applications that shae these common popeties:
Long distance communication is easy due to UAVs being line-of-sight to each othe and to gound. Communication congestion is sevee due to UAVs being line-of-sight. Coodination of UAVs should be distibuted due to thei lage numbes, thei geogaphic sepaation, and UAVs being owned by diffeent institutions. Hadwae failues and integation failues will be common, due to diffeing age, types, manufactues and shee volume of UAVs. These popeties make efficient contol difficult with a topdown appoach, but ae a natual fo adaptive and distibuted appoaches such as evolutionay algoithms and multiagent systems. In this pape we apply these multiagent techniques combined with evolution to the specific domain of ceating an ai-to-gound communication netwok ove a single channel. In some ways this model is elated to cuent WiFi netwoks that can shae many uses ove a single channel. Howeve, the dynamics of having the signals come fom high altitude UAVs make the poblem consideably diffeent than that of teestial communication: Huge numbes of UAVs may be accessible at once, congestion may be sevee, and many failed o uncoopeative UAVs will be line-of-sight and will have to be dealt with. In these senses, this domain shaes many of the qualities that we expect futue UAV poblems to have. In addition this domain is impotant in itself as it allows UAV based communication netwoks to be used fo such puposes as voice communication and data netwoks without the need of expensive and difficult to maintain goundbased hadwae. In addition it allows such netwoks to be ceated in an adhoc way, whee diffeent aicaft ae owned by diffeent institutions. If setup popely, this paadigm may open the path fo UAV based communications to be as ubiquitous fo long-ange communication, as WiFi is fo shot-ange communication today. The main contibution of this pape is to pesent an application of evolutionay algoithms to the domain of UAV communication that is both useful fo downlink communication and shows the potential fo swams of high-altitude, low-cost UAVs likely to be common in the futue. In Section we descibe cuent and futue UAV systems as well as multiagent communication. In Sections 3 and 4 we descibe details of the UAV communication domain used in this pape. In Section 5 we discuss how the evolving agents can be used to maximize custome communication bit ates. In Section 6 we pesent expeimental esults, showing how the evolving agents ae able to pefom well unde vaying conditions. Finally in Section 7 we povide a discussion on the impact of this application and the implications of the esults.. BACKOUN In the nea futue, sola UAVs will play citical oles in the militay, industial, scientific, and academic communities [14, 17, 18]. These devices have seemingly limitless applications including communications, econnaissance missions, space launch platfoms, and wieless powe beaming [10, 14]. ecent missions including NASA s Pathfinde-Plus and QinetiQ s Zephy (which emained aibone fo ove two weeks nonstop) have advanced the state of the at in sola poweed UAVs, taking them fom limited mission life and enduance to the point they can emain opeational fo weeks at a time [10, 15]. As a esult of the inceasing capabilities and availability of these devices coupled with thei falling costs, a plethoa of novel domains and applications will emege to utilize the newly developed technological capabilities of these platfoms [14]. ethods of contolling and coodinating netwoks of UAVs have been eseached including genetic and evolutionay algoithms and einfocement leaning methods[9,, 16, 13, 11, 7, 6]. enetic algoithms have been used to evolve decision tees that allow UAV teams to collaboate in seach missions, and to facilitate UAV task assignments [16, 19]. In addition evolutionay algoithms have been shown to be effective in single playe and multi-playe UAV path planning domains [8]. enetic pogamming has also been shown effective in UAV multi-task allocation domains when thee is limited communication [4]. In addition to evolutionay and genetic algoithms, swam techniques have also been used to coodinate UAVs. In the coopeative huntes domain a swam of UAVs, using hand coded optimization methods, is used to seach fo one o moe smat tagets []. Anothe UAV contol poblem focuses on an NP-complete task allocation poblem which assigned tasks to swams of UAVs [1]. Fequently UAVs ae utilized in econiassance tasks involving Automatic Taget ecognition (AT). In such domains it is desied to have a balance between high coveage of discoveed tagets and boad aea coveage. One high-pefoming appoach to solving this coodination and contol poblem utilizes ant-based swam methods [1]. 3. UAV TO EATH COUNICATION As we pogess into the infomation age, communication becomes an inceasingly citical component of evey day life. Today, cellula phones, laptops, hand held computes, and othe wieless electonic devices have changed the way we see and inteact with the wold. At the coe of these advancements is a well designed wieless communication netwok, which handles the wokload and facilitates infomation shaing between devices connected to the netwok. Cuent netwoks ely on a seies of adio towes to facilitate this infomation shaing wok load. Taditional towes have woked well to date, but they have seveal key dawbacks: 1. They ae expensive to build.. They ae expensive to maintain. 3. They have limited communication due to obstuctions (cannot communicate aound obstuctions). 4. They have static placement (holes in coveage aeas). In this pape we focus on a subset of this domain whee thee is a set of UAVs that ae flying at fixed locations (flying in small cicles) fo long peiods of time (pehaps months o yeas) and ae tansmitting data to a set of customes below (see Figue 1). UAVs have an advantage in sending data fom high altitude in that they can have line-of-sight communication to many customes. In addition by vitue of being ovehead, such UAVs can focus on what aeas of the suface they will poect most of thei signal powe to, allowing fo bette coveage.
In this domain, each UAV can communicate to multiple customes. In addition communication is done ove a shaed channel (ove the same fequency band) analogous to the way WiFi netwoks tansmit data. Using a shaed channel allows the system to be vey adhoc, whee UAVs can come and go, and can decide whethe o not to paticipate in the system without any need fo channel abitation. Note that fo simplicity we only look at the download poblem, whee UAVs ae sending infomation down to customes. Also we make no assumptions on how the UAVs get thei data feeds. We believe that this half of the poblem is the most impotant, as typical intenet use tends to be dominated by download taffic. Although the uplink poblem is faily simila as long as it is done on a diffeent channel than the downlink. by custome i is simply the sum of the signal fom all the UAVs it is not communicating with: N i = / Ji S i, + k, (3) whee J i is the set of UAVs custome i is communicating with and k is a constant fo backgound noise. The maximum communication ate fo custome i can then be estimated fom the signal-to-noise atio using Shannon s law: C i, = B log (1.0 + S i,/n i), (4) whee B is the bandwidth of the channel in Hz 1. The total data ate fo custome i is the sum of the data ates fo each UAV the custome is communicating with: C i = J i C i,. (5) High ain Low ain Attenuation i, Figue 1: UAV Communications. A set of UAVs at high altitude tansmit data to a set of customes on gound ove a single communication channel. The task of the system is to maximize aveage bitate customes eceive. ultiple UAVs may communicate to single custome. A UAV communicates to at most one custome. 4. SINAL YNAICS We assume that the UAVs ae all at simila altitudes and communicate though diectional antennas pointed towads the gound. The amount of aea on the gound that is coveed by the UAV is detemined by the gain of its antenna. Antennas with low gain, tansmit ove a wide aea, but within that aea the stength of the signal is lowe (see Figue ). Antennas with high gain, have moe signal powe in the cente of thei aea, but tansmit ove a smalle aea. The maximum signal eceived fom a UAV is popotional to the invese squae of the gain adius fo the antenna: S max = ap / (1) whee a is a constant, P is the powe tansmitted fom UAV, and is the signal half-powe adius fo UAV. S max is the amount of signal eceived diectly at the cente of the tansmission. Futhe fom the cente, the amount of signal eceived deceases exponentially accoding to the signal adius: S i, = e b i, S max () whee b is a constant and i, is the distance fom custome i and the cente of UAV s tansmission. The noise eceived Figue : Signal ynamics. UAVs with high-gain antennas thow a stong signal ove a small aea. UAVs with low-gain antennas thow weake signal ove lage aea (Left). The stength of the signal depends on how fa the custome is away fom the cente of the signal cone (ight). 5. SYSTE EVALUATION FUNCTION AN AENTS The obective of this poblem is to maximize the aveage data ate of each custome: = 1 n n C i, (6) i=1 whee thee ae n customes, and is the system evaluation function. Combining equation 1,, 3, 4, 5, we obtain: ap = 1 n e b i, B log n 1.0 +, (7) i=1 J i + k ap / J i e b i, putting ou global obective in tems of ou contol vaiables: UAV powe level, P, and indiectly, i,, though the oientation of the UAV. 1 Fo simplicity, cetain factos, such as elative invese squae distance signal attenuation ae ignoed that wee detemined to have little impact on pefomance.
These contols allow us to change the signal-to-noise chaacteistics at diffeent locations on the gound. Howeve, this is a difficult poblem as inceasing the signal fo one custome may incease the noise fo anothe. It is especially difficult, since we want this communication netwok to be adhoc, whee it is contolled in a distibuted way: UAVs ae enteing and leaving the system, and some UAVs fail to coopeate o opeate coectly. Fotunately, evolutionay algoithms and multiagent techniques ae a natual match to this poblem. Thee ae many possible agent definitions and contols fo the UAVs in ou domain, including altitude, antenna gain, powe levels and antenna angle. Hee we focus on the last two: adusting the powe level P and oientation (the diection the tansmitte points to) of each UAV (see Figue 3). We contol each of these actions though agents. The solution to the full poblem consists of the powe level and oientation values fo all the UAVs. Howeve to simplify the poblem, we beak the task into a multiagent system, whee a single agent contols both powe level and oientation fo each UAV. To pefom contol, each agent makes discete actions. Fo adusting powe, the action is scaled exponentially to the action: P = P e zp, (8) whee P is the base powe and z p is the action of the agent fo UAV contolling its powe. To contol oientation, an agent chooses one of nine diections: eithe staight down, o one of eight cadinal diections aound the UAV. The angle of the pointing is fixed so that the cente of the new oientation is moved a distance of what it would have been if it had pointed stait down (see Figue 3 - ight). As descibed in the next section, the values of the contols fo each agent ae detemined by an evolutionay algoithm. High Powe Low Powe Oientation and a discete value of z o detemining oientation diection. The powe level is discetized to 10 diffeent values. Along with the 9 possible oientations each policy fo each agent has a total of 90 diffeent possible values. An agent s policy is detemined by an evolutionay algoithm, whee an agent evolves a population of policies. At evey time step, an agent chooses a policy fom its population of policies using an epsilon-geedy selecto, whee the best policy is chosen with pobability 1 ɛ and a andom policy is chosen with pobability ɛ. The chosen policy table detemines the powe and oientation of the UAV fo that time step. Once all the powe levels and oientations ae chosen the pefomance of the system is evaluated (see Figue 4). The types of evaluations that can be used is discussed in Section 5.. Once the cuent choice of policy has been evaluated, the evaluation of the policy is updated with a leaning ate alpha: New Value = (1 α) * Old Evaluation + α * New Evaluation. Each agent then updates its population by eliminating the lowest value membe of the population, and then copies the highest value membe of population and mutates it. utation is applied by taking a andom table enty and setting each value to a numbe between 0 and 9 taken fom a unifom andom distibution. Vaious othe foms of mutation wee tied including mutating moe table enties at each evolution step, but they did not impove pefomance. Powe Level, Oientation fo UAV 1 Population 1 Agent (iffeence) Evaluations lobal Evaluation Powe Level, Oientation fo UAV Population Powe Level, Oientation fo UAV n Population n Agent 1 Agent Agent n Figue 3: Agent Actions. An agent can choose powe level of UAV within cetain ange. An agent can also choose oientation of antenna. The Agent must choose powe levels and oientations to balance giving moe signal to thei customes and less noise to othe customes. 5.1 Evolving Agents The obective of each agent is to evolve the best values of powe level and oientations that will lead to the best system fitness evaluation function,. The value of powe level and oientation is detemined by the agent s cuent policy. Each policy contains a discete value of z p detemining powe level Figue 4: Evolving Populations of Agents. Each agent has its own population of policies. At evey step a policy is chosen fom the population, which detemines the powe level and oientation of a single UAV. The choice of powe level and oientation can then be evaluated in two diffeent ways: 1) lobal evaluation looks at the value of all of the actions of all of the agents and etuns the same value fo all agents, ) Shaped agent-specific evaluations (implemented with the diffeence evaluation in this pape) make a sepaate evaluation fo each agent using infomation about all of the actions fom all agents. 5. Agent Evaluation Functions The final issue that needs to be addessed is selecting the fitness evaluation function fo the evolving agents. The fist and most diect appoach is to let each agent eceive the
system pefomance as its evaluation function. Howeve, in many domains using such an evaluation function leads to slow evolution. We will theefoe also set up a second evaluation function based on an agent-specific evaluation. iven that agents aim to maximize thei own evaluation functions, a citical task is to ceate good agent evaluation functions, o evaluation functions that when maximized by the agents lead to good oveall system pefomance. In this wok we focus on diffeence evaluation functions which aim to povide an evaluation function that is both sensitive to that agent s actions and aligned with the oveall system evaluation function [1, 0]. 5..1 iffeence Evaluation Function Conside diffeence fitness evaluation function of the fom [1, 0]: i = (z) (z z i), (9) whee z i is the action of agent i (detemined by its policy) as defined in Subsection 5.1, and z z i ae the actions of all the agents with the action of agent i emoved. The second tem of the diffeence evaluation, (z z i), epesents a countefactual of what the pefomance of the system is like when agent i is emoved fom the system (i.e. its powe level is dopped to zeo). By subtacting the countefactual fom the oiginal evaluation, this diffeence in some sense evaluates the agents contibution to the system. Thee ae two advantages to using : Fist, the second tem emoves a significant potion of the impact of othe agents in the system. This happens since the impact of actions that ae ielevant to agent i ae emoved by the subtaction. This benefit has been dubbed leanability (agents have an easie time evolving) in pevious liteatue [1, 0]. Second, because the second tem does not depend on the actions of agent i, any action taken by agent i that impoves, also impoves. This happens since any action that the agent takes can only affect the fist tem, because its action has been eliminated fom the second tem. This benefit which measues the amount of alignment between two evaluation functions has been dubbed factoedness in pevious liteatue [1, 0]. Substituting Equation 7 into Equation 9, we obtain = 1 n B log n 1.0 + i=1 J i 1 n n B log 1.0 + J i, i=1 ap e b i, ap / J i e b i, ap e b i, ap / J i, e b i, + k, + k whee we ae calculating the diffeence evaluation fo agent. Note that the second tem of the diffeence evaluation both emoves the signal eceived fom this agent s UAV and also emoves its noise. 6. EXPEIENTS To test the effectiveness of evolving agents in this UAV communication domain, we pefom an extensive set of expeiments in simulation. In these simulations, (unless othewise specified, such as in the scaling expeiments) 100 UAVs ae placed at an altitude of 0 miles (epesenting appoximately the maximum altitude a sola poweed aicaft can achieve). These UAVs ae placed above a 10x10 mile squae aea. The task of the UAVs is to tansmit data to customes within this 10x10 mile aea. In all expeiments the channel bandwidth is B = 1hz and the noise floo is k = 0.. In addition, the gain adii,, ae distibuted andomly, unifomly between 0.35 miles and 1.05 miles to epesent a heteogeneous set of UAVs. Fo evolution, α = 0. and ɛ = 0.5. All expeiments esults ae pefomed ove 30 tials. In additional all of ou mao pefomance conclusions ae statistically significant with p < 0.05. In this setup we test the pefomance of the evolving agents in cases whee: 1. Agents contol UAVs with no failues and full communication.. UAVs (i) fail; (ii) do not coodinate; o (iii) ae incompatible. 3. UAVs have estictions on thei obsevation capabilities. 4. The numbe of UAVs is scaled to 1000 UAVs. In all of ou scenaios, the numbe of customes is the same as the numbe of UAVs. We do this to model the situation whee a gound-based custome is likely to be a hotspot o signal epeate. In situations whee customes wee individuals, the numbe of customes would likely be consideably moe than the numbe of UAVs. Fo each of the cases above, we epot esults on fou diffeent types of agents to contol the powe and oientation of the UAVs: Static agents always choose maximum powe, and stait down oientation (). andom agents have andom evaluation function (). Evolving agents diectly maximize system evaluation function (). Evolving agents use diffeence evaluation function (). The fist two fom baselines to asses pefomance. The next two compae leaning ates of taditional agents maximizing a common system evaluation function, and agents indiectly maximizing the system evaluation function by diectly maximizing the diffeence evaluation function. 6.1 UAV Pefomance In the fist set of expeiments, we have a single agent contol both powe level and oientation fo a UAV. The esults displayed in Figue 5 show how the fist baseline algoithm is able to achieve a data ate of 00Kbits/s to each custome. The esults also show that evolving agents maximizing the system evaluation function diectly () pefom significantly bette, pefoming up to Kbits/s. This esult shows that evolution can be vey helpful in choosing contol policies in this lage system. Howeve, agents evolved using the evaluation function ae able to pefom even bette, nealy doubling the pefomance of the system. The impoved pefomance of the diffeence evaluation has to do with it being moe specific to the actions of the agent. When an agent chooses a good policy that policy is likely to be evaluated well using the diffeence evaluation. In contast when using
Kbits/s pe Custome 550 450 350 6..1 Failue to Tansmit Fist we conside the case whee UAVs must tun off all tansmissions due to failue o in ode to conseve powe. As agents fail, othe agents must find ways to adapt to make up fo the loss. As seen in Figue 6, agents using ae able to pefom well, even unde high failue ates. While agents using the baseline policy as well as agents using ae not hut significantly fom the failues, they still pefom wose than agents evolved using up to a 95% failue ate. 50 00 550 150 0 1000 1 000 0 3 0 4 Evolution Steps Figue 5: System of 100 UAVs whee each agent optimizes both powe and oientation. Agents evolved using evaluation function outpefoms all othe methods by nealy -to-1. This is due to the evaluation eliminating ielevant actions. Kbits/s pe Custome 450 350 50 00 to evaluate that policy it may not get a good evaluation, since the evaluation depends equally on the policy choices of the 99 othe agents. 6. obustness to Failues, Non-coodination, and Incompatibility Fo an adhoc UAV communications netwok to function popely, it must be obust to many types of failues. UAVs may be of diffeent ages, be in diffeent states of epai and may fail without notice. Even wose than failing completely, a UAV may still be tansmitting at high powe, but not communicating with any custome, causing them to add noise without any benefit. In this section we show how obust an evolved agent based UAV netwok can be against these vaious foms of failues. Kbits/s pe Custome 00 100 0 0 0 40 60 80 100 Pecent fail to Tansmit Figue 6: Pefomance unde tansmission failues. agents outpefom all othe methods with up to 95% failues. 150 0 0 40 60 80 100 Pecent fail to Coodinate Figue 7: Pefomance when agents fail to coodinate. agents outpefom all othe methods. Kbits/s pe Custome 00 100 0 0 0 40 60 80 100 Pecent Incompatible Figue 8: Pefomance unde incompatible agents. Agents evolved using evaluation function outpefom all othe methods with up to 90% failues. 6.. Failue to Coodinate Next we conside the case whee some UAVs fail to coodinate with the est of the system and end up taking uncoodinated locally geedy actions. To maximize local pefomance, these agents simply maximize thei powe level, without egad fo the noise they ae adding to the system. These agents still contibute to the oveall system bandwidth, but they can potentially ham the system by aising the noise levels of othe agents. As shown in Figue 7, agents evolved using ae able to outpefom all othe methods.
In fact even when 100% of agents geedily maximize thei powe levels, agents evolved using the diffeence evaluation still pefom bette since they ae still able to efficiently choose good oientations. 6..3 Failue of Compatibility In ou final case, we conside the situation whee some of the UAVs ae incompatible with the cuent netwok. In this case, incompatible UAVs still send out noise, but do not actually send data to any of the customes in the system. Such cases may be common, when potocols change, softwae is not witten to specification o thee ae multiple diffeent netwoks communicating in the same channel. Figue 8 shows that this type of failue can be vey hamful. Still, with a modeate numbe of incompatible UAVs, agents evolved using the diffeence evaluation ae able to pefom well. 6.3 Obsevation estictions In ode to demonstate the concept and the suitability of the multiagent evolutionay appoach to this domain, the esults epoted above assume that thee ae no estictions in the obsevational capabilities of the agents (fo example in computing o eceiving the system obective and diffeence evaluation functions, o in communicating signal-to-noise atios fo all of the customes). Because such assumptions ae not ealistic, in this section we exploe the impact of such limitation on the pefomance of each of the algoithms (note that obsevation estictions hee also includes inte-uav communication, but we use the tem obsevation to diffeentiate fom the main application of UAV to gound communication). 550 As seen in Figue 9, agents evolved using and pefom equally when no obsevations ae possible (agents have access to only thei own local infomation). As the level of obsevation inceases and agents eceive moe infomation, an incease in pefomance could be expected. That is not what happens howeve; agents evolved using actually expeience a decease in pefomance as they gain additional infomation. This is because the agents don t know what to do with the exta infomation they eceive. Agents evolved using ae able to handle this infomation in a way that benefits the system pefomance, allowing them to coodinate and make decisions that positively impove system pefomance. Agents evolved using begin to impove pefomance when the obsevation adius is below.5. This happens since the agents ae eceiving enough infomation to make moe infomed decisions, but not enough infomation to negatively impact thei pefomance (too much infomation causes noise on the agents evaluation function). Afte this point howeve, the amount of infomation each agent eceives becomes ovewhelming and they ae unable to coodinate thei actions. 6.4 Scalability While all of the pevious expeiments have been pefomed with 100 UAVs, we expect futue UAV systems to be much lage. In this section we test the scalability of ou appoach by measuing the system pefomance when the numbe of UAVs is scaled fom 10 to 1000. To make scaling compaable, we also scale the numbe of customes and the aea of the land seviced by the same amount. The esults show that fo moe than 100 UAVs, the amount of data each custome eceives is highly stable (see Figue 10). This esult suggests that agents evolved using diffeence evaluation functions should be able to efficiently contol this system, even when thee ae a vey lage numbe of UAVs. 1000 900 800 Kbits/s pe Custome 450 350 Kbits/s pe Custome 700 50 00 00 150 0 4 6 8 10 Obsevation adii Figue 9: Pefomance of a 100 UAV system in the pesence of obsevation estictions. Agents can only obseve actions of othe agents that lie within a given adii fom thei location. Agents using ae able to effectively use additional obsevational infomation to coodinate and impove system pefomance, wheeas agents using can be negatively impacted by an inceased amount of infomation. 100 0 100 00 700 800 900 1000 Numbe of Agents Figue 10: Pefomance vesus scalability whee the numbe of customes, agents, and wold size ae scaled popotionally. As seen, agents evolved using ae able to outpefom all othe methods by nealy 50% to 100% fo any given system size. Agents using with a system size of 1000 UAVs pefom nealy as well as agents evolved using when the system size is 50 UAVs. This is a twenty fold incease in system size while maintaining good pefomance. 7. ISCUSSION AN CONCLUSION In this pape, we pesent an impotant application of multiagent evolution: Coodinating lage numbes of UAVs to
fom an ai-to-gound communication netwok. ue to advances in sola, battey and moto technology, such lage UAV netwoks ae becoming an attactive altenative to both Eath-based and satellite based communication systems. Indeed, such netwoks will be inceasingly poweful, allowing fo voice and data netwoks anywhee in the wold without need fo expensive and bittle gound-based infastuctue, o the need fo expensive-to-launch and maintain satellite systems. In addition, with mino modifications, the UAV coodination algoithms can be adapted fo othe types of lage scale UAV applications, anging fom obsevation systems to micowave powe delivey systems. The application chosen in this pape exhibits a numbe of salient featues that we expect lage UAV netwoks to have. ue to thei high altitude, long-ange point-to-point communication is easy. Howeve, long-ange signal congestion is pevalent. These popeties contast with teestial netwoks whee point-to-point communication tends to be shot-ange and congestion is localized. Such topological diffeence make UAV-based netwoks much diffeent than gound-based netwoks, and necessitates a highe level of coodination. This pape shows how multiagent evolution can be used to effectively coodinate these UAV netwoks. In ou expeiments even basic evolution (i.e. evolved using ) is shown to be helpful, pefoming consideably bette than simple baseline algoithms. Howeve, agents evolving to maximize the diffeence evaluation function achieve twice the level of pefomance. In addition agents using the diffeence evaluation ae able to scale effectively to systems with lage numbes of UAVs. These esults ae also shown to be obust with espect to numeous types of failues, incompatibilities and obsevational estictions that will be common in eal-wold adhoc netwoks. The key to these esults is that they ae based on lage scale UAV coodination, and will extend to othe domains whee simila congestion exists. Fo example, two way communication and UAV-to-UAV communication ae natual extensions. 8. ACKNOWLEENTS This eseach was patially suppoted by NSF gants CNS- 0931591 and IIS-0910358. 9. EFEENCES [1] A. K. Agogino and K. Tume. Analyzing and visualizing multiagent ewads in dynamic and stochastic envionments. Jounal of Autonomous Agents and ulti Agent Systems, 17():30 338, 008. [] Y. Altshule, V. Yanovsky, I. Wagne, and A. Buckstein. The coopeative huntes - efficient coopeative seach fo smat tagets using UAV swams. Second Intenational Confeence on Infomatics in Contol, Automation and obotics (ICINCO), 005. [3] N. Ande. esign of sola poweed aiplanes fo continuous flight. Ph thesis, Technische Wissenschaften, ETH Zuich, 008. [4]. J. Balow, C. K. Oh, and S. F. Smith. Evolving coopeative contol on spasely distibuted tasks fo uav teams without global communication. In ECCO 08, Atlanta, a, July 008. [5] O. Banes, edito. Ai Powe: UAVs: The Wide Context. oyal Ai Foce iectoate of efence Studies, 009. [6] A. Bell. Leaning-based contol and coodination of autonomous UAVs. aste s thesis, Oegon State Univesity, 011. [7] H. Choi, Y. Kim, and H. Kim. enetic algoithm based decentalized task assignment fo multiple unmanned aeial vehicles in dynamic envionments. Intenational Jounal of Aeonautical and Space Science, 1():163 174, 011. [8] J.. de la Cuz, E. Besada-Potas, L. Toe-Cubillo, B. Andes-Too, and J. A. Lopez-Oozco. Evolutionay path planne fo uavs in ealistic envionments. In ECCO 08, pages 1477 1484, Atlanta, a, July 008. [9] P. audiano, B. Shagel, E. Bonabeau, and B. Clough. Contol of UAV swams - what the bugs can teach us. In AIAA, 003. [10] S. Hewitz, L. Johnson, J. Avesen,. Higgins, J. Leung, and S. unagan. Pecision agicultue as a commecial application fo sola-poweed unmanned aeial vehicles. In Ameican Institute of Aeonautics and Astonautics (AIAA), 00. [11] S. Kaaman, T. Shima, and E. Fazzoli. Task assignment fo complex uav opeations using genetic algoithms. In AIAA uidance, Navigation, and Contol Confeence, August 009. [1]. ille, P. asgupta, and T. Judkins. istibuted task selection in multi-agent based swams using heuistic stategies. Lectue Notes in Compute Science, 007. [13] J. Peon, J. Hogan, B. oulin, J. Bege, and. Belange. A hybid appoach based on multiagent geosimulation and einfocement leaning to solve a uav patolling poblem. In Poceedings of the 008 Winte Simulation Confeence, 008. [14] N. Picon. Effects of wieless powe beaming in the space industy: oden applications and futue possibilities. Tiple Helix - Exploations in Science, Society and Law, 011. [15] QinetiQ. www.qinetiq.com, 011. [16].. ichads,. Whitley, and J.. Beveidge. Evolving coopeative stategies fo uav teams. In ECCO 05, pages 171 178, Washington,.C., June 005. [17]. omeo,. Fulla, and E. Cestino. esign of sola high altitude long enduance aicaft fo multi payload and opeations. Aeospace Science and Technology, 10(6):541 550, 006. [18] H. unge, W. ack, A. uiz-leon, and. Heppele. A sola poweed hale-uav fo actic eseach. In 1st CEAS Euopean Ai and Space Confeence, 007. [19] T. Shima, S. J. asmussen, and A.. Spaks. Uav coopeative multiple task assignments using genetic algoithms. In 005 Ameican Contol Confeence, pages 989 994, Potland, O, June 005. [0] K. Tume and. Wolpet, editos. Collectives and the esign of Complex Systems. Spinge, New Yok, 004.