Unifying Agent Systems

Transcription

1 Unifying Agent Systems Mar d nverno Cavendish Shool of Computer Siene University of Westminster ondon, W1M 8S, UK dinverm@wmin.a.u Mihael u Dept of Eletronis and Computer Siene University of Southampton Southampton, S17 1B, UK mml@es.soton.a.u Abstrat Whilst there has been an explosion of interest in multi-agent systems, there are still many problems that may have a potentially deleterious impat on the progress of the area. These problems have arisen primarily through the la of a ommon struture and language for understanding multi-agent systems, and with whih to organise and pursue researh in this area. n response to this, previous wor has been onerned with developing a omputational formal framewor for ageny and autonomy whih, we argue, provides an environment in whih to develop, evaluate, and ompare systems and theories of multi-agent systems. n this paper we go some way towards justifying these laims by reviewing the framewor and showing what we an ahieve within it by developing models of agent dimensions, ategorising ey inter-agent relationships and by applying it to evaluate existing multi-agent systems in a oherent omputational model. We outline the benefits of speifying eah of the systems within the framewor and onsider how it allows us to unify different systems and approahes in general. 1 ntrodution n reent years, there has been an explosion of interest in agents and multi-agent systems. This has not only been in artifiial intelligene but in other areas of omputer siene suh as information retrieval and software engineering. ndeed, there is now a plethora of different labels for agents inluding autonomous agents [32], software agents [22], intelligent agents [59], interfae agents [37], virtual agents [1], information agents [35], mobile agents [57], and so on. The diverse range of appliations for whih agents are being touted inlude operating systems interfaes [21], proessing satellite imaging data [54], eletriity distribution management [31], air-traffi ontrol [34] business proess management [29], eletroni ommere [26] and omputer games [25], to name a few. Moreover, signifiant ommerial and industrial researh and development efforts have been underway for some time [9, 11, 44, 45], and are set to grow further. However, the field of agents and multi-agent systems is still relatively young, and there are many problems that may have a potentially deleterious impat on the progress of the area. These problems have arisen primarily through the la of a ommon struture and language for understanding multiagent systems, and with whih to organise and pursue researh in multi-agent systems. t is, therefore, important to ensure that any suh strutures we generate are aessible if there are going to have any signifiant impat on the way researh progresses [38]. n partiular, we need to be able to relate different theories and approahes within MAS so that different systems and models an be integrated. This an be ahieved in two stages: first we need to be able to isolate the potential inonsistenies in 1

2 definitions of fundamental terms frequently used when disussing multi-agent systems, and seond we need to provide an environment in whih different systems and theories an be developed, evaluated and ompared. 1.1 Formal Framewors We have previously onsidered the requirements for the strutures or framewors that are neessary to provide a rigorous approah to any disipline [19], and in partiular to agents and multi-agent systems [40]. Suh framewors should preisely and unambiguously provide meanings for ommon onepts and terms but in an aessible manner sine only then will a ommon oneptual framewor have a hane of emerging. (f there is a generally held understanding of the salient features and issues involved in the relevant lass of models then we an assert the existene of a ommon oneptual framewor. Another important aspet, and ey to the wor presented in this paper is that it enables models and systems to be expliitly presented, ompared and evaluated. ot only must it provide a desription of the ommon abstrations found within that lass of models, but also it must provide a means of further refinement to speify partiular models and systems. Multi-agent systems are inherently ompliated, and, onsequently, reasoning about the behaviour of suh systems beomes diffiult. The ability to formalise multi-agent systems, to do so in suh a way that allows automated reasoning about agents behaviour and, additionally, to do so in a way that is also aessible, is therefore ritially important. n what follows, we see to show that the formal framewor we have developed satisfies these requirements and provides just suh a base as is neessary for a rigorous and disiplined approah to multi-agent systems researh. ur aim here is not to provide a detailed presentation of our framewor, whih has been presented extensively elsewhere, but instead to show how it may be applied to different systems, and how they may be aommodated within a single overarhing framewor. Similarly, we are not onerned in this paper with the detailed speifiation of agent behaviour, though we have addressed this previously in [16, 18], nor with reasoning about agent behaviour, though again related wor has addressed this, for example in the ontext of Agentis [15, 56]. 1.2 verview n this paper, we review and build on previous wor that has developed a formal agent framewor and extend it to onstrut several agent models. These models range from generi abstrat models to very speifi system models that have been desribed and implemented elsewhere. n the next setion we review the framewor we have developed and in Setion 4 and show how it an be extended to desribe ertain types of agents, desribing autonomous agents, planning agents, memory agents and soial agents. ext, the paper presents three ase studies in applying the framewor. These ase studies have been hosen as exemplars speifially beause they lie at opposite ends of the multiagent system spetrum; one is a relatively new mostly theoretial model, while the other two are well-nown, fully developed and implemented systems. Finally, the paper assesses the signifiane of this researh, and onsiders further wor. 2 Formal Speifiation ne serious problem with many formal models of multi-agent systems is that they are very rarely appliable to building real systems. ndeed the gap between formal theoretial models on the one hand and implemented systems on the other is now a widely anowledged issue [13, 30] and whilst 2

3 there have been several notable efforts to address this (e.g. [46, 58], it is still the ase that most new formal models do not even outline their role in system design and implementation. There is a large number of formal tehniques and languages available to speify properties of software systems [14] inluding state-based languages suh as VDM [33], Z [53] and B [36], proessbased languages suh as CCS [41] and CSP [28], temporal logis [20], modal logis [10], and Stateharts [55]. However, in bringing together the need for formal models on one hand and omputational models that relate to software development on the other, we adopt the Z speifiation language. Z is the most widely used of formal methods from software engineering, and offers arguably the best hane of the agent models developed ahieving a signifiant degree of adoption in the broader ommunity. The Z language is used both in industry and aademia, as a strong and elegant means of formal speifiation, and is supported by a large array of boos (e.g. [2, 27], artiles (e.g. [3, 4] and development tools. ts use in industry is both a refletion of its aessibility (the language is based on simple notions from first order logi and set theory and its expressiveness, allowing a onsistent, unified and strutured aount of a omputer systems and its assoiated operations. Furthermore, Z is gaining inreasing aeptane as a tool within the artifiial intelligene ommunity (e.g. [24, 39, 42] and is onsequently appropriate in terms of standards and dissemination apabilities. 2.1 Z anguage Syntax The ey syntati element of Z is the shema, whih allows speifiations to be strutured into manageable modular omponents. The shema below has a very similar semantis to the Cartesian produt of natural numbers but without any notion of order. n addition, and as an be seen in the example, the shema enables any delared variables to be onstrained. n this ase the shema delares two variables that are both natural numbers, and onstrains them so that the variable is less than or equal to the variable. Modularity is failitated in Z by allowing shemas to be inluded within other shemas. We an selet a state variable,, of a shema, "# $, by writing "# $ &%. For example, it should be lear that ' % refers to the variable ( in the shema. ow, operations in a state-based speifiation language are defined in terms of hanges to the state. Speifially, an operation relates variables of the state after the operation (denoted by dashed variables to the value of the variables before the operation (denoted by undashed variables. perations may also have inputs (denoted by variables with question mars, outputs (exlamation mars *,&-/.01 and a preondition. n the shema below, there is an operation with an input variable, 243 ; if 243 lies between the variables ( and, then the value of is replaed with the value of 243. The original value of ( is the output as.05. The 6 symbol, is an abbreviation ' 87 for ' 19 and, as suh, inludes in this shema all the variables and prediates of the state of ' before and after the operation. 3

4 [ 6 w H o } } m P } } G H } e o r 7 *,&-/.01 : 243; '.<5 = 243 : AD.<5EAF ( A ey type in the speifiation ontained in this paper is the relation type, expressing a mapping between two sets: a soure set and a target set. The type of a relation with soure G and target H is a simply a set of ordered pairs, KM &. More formally, we have the following definition for the relation type. A;ASR GQP TGVU=H8 By way of example, onsider the following relation between natural numbers. $4W.X. Y $4W.X. ZAC[ T\,^]#T_, àtb, \,^ The domain of a relation (dom is the set of related soure elements, while the range (ran is the set of target elements. n the example above we have the following. dom $4W.0 ^. ZAD[ \, _, ba ran $4W.0 ^. ZAD[ `, \,^]# n the following example we show how a set of elements an be defined using set omprehension. dfe hgji>l`4mnm AD[po `,^]E b, q,r r`, The bullet an also be used in prediates suh as the one below, whih states that the square of any natural number less than 10 is less than 100. s tdfea g r 2.2 Z Extensions gur oo n the shema given below, $ W v taes a funtion and a sequene and applies the funtion to eah element of the sequene and $ W& taes a funtion and a set and applies the funtion to eah element of the set. $ GxBHzy W v{ TG H{~ seq G seq $ W n H TG H8j } R G } R s v ; ;R seq $ G n W& v v zad[dzce,z Gx G HC $ W& zac[ % %1 v ( M { *( v ^^ 4

5 G y We have found it useful in this speifiation to be able to assert that an element is optional. For example, in the speifiation given in this paper, whether a soial agent has a model of itself or not is optional. The following definitions provide for a new type, ˆ# 1Š^ ÊŒŽ, for any existing type,. w A;Au[ >R e ˆ# Š ÊŒdŽ G y r1 Most other syntati onstruts in this paper are fairly standard but more omplete treatments of Z an be found elsewhere [53]. 3 The Agent Framewor There are four types upon whih all our notions in the SMART (Strutured and Modular Agents and Relationship Types agent framewor are based. The definitions within this paper will be built up using only these types; they are delared below. w * * K p. d An entity is something that omprises a set of attributes, a set of ations, a set of goals and a set of motivations. The shema below has a delarative part ontaining four variables. First, * is the set of features of the entity. These features are the only harateristis of the entity that are manifest. They need not be pereived by any partiular entity, but must be potentially pereivable in an omnisient sense. Seond, W d. ( is the set of ations of the entity, and is sometimes referred to as the ompetene of the entity. ext, a. and ${d E are the sets of goals and motivations of the entity respetively. Goals are simply states of affairs to be ahieved in the environment, in the traditional artifiial intelligene sense, while motivations are higher-level non-derivative omponents haraterising the nature of the agent, but are related to goals. Motivations are, however, qualitatively different from goals in that they are not desribable states of affairs in the environment. For example, the motivation greed does not speify a state of affairs to be ahieved, nor is it desribable in terms of the environment, but it may (if other motivations permit give rise to the generation of a goal to rob a ban. The distintion between the motivation of greed and the goal of robbing a ban is lear, with the former providing a reason to do the latter, and the latter speifying what must be done. Finally, the prediate part states that the entity must have a non-empty set of attributes. & * * K( ;dr * * K W d. >R a. >R. $ B >R d * * K( š AC[ An objet is any entity that has apabilities (as well as attributes. The shema defining an objet is that of an entity with the further proviso that the objet has a non-empty set of apabilities. œ T & W d. žš AD[ K( 5

6 n our framewor an agent is an objet that is serving some purpose. That is, an agent is an instantiation of an objet together with an assoiated goal or set of goals. The shema for an agent is simply that of an objet but with the further restrition that the set of goals of an agent is not empty. a & œ T a. žš AD[ We define an ŸB$ & ^B$ & A>AjR * to be a set of attributes. K ext we define an interation. An interation is simply what happens when ations are performed in an environment. The effets of an interation on the environment are determined by applying the & K p funtion in the axiom definition below to the urrent environment and the ations taen. This axiom definition is a global variable and is onsequently always in sope. We require only one funtion to desribe all interations, sine an ation will result in the same hange to an environment whether taen by an objet or any ind of agent. *z p K Z Ÿ&$ & } R p } ^B$ 3.1 Agent Pereption and Agent Ation Pereption an now be introdued. An agent in an environment may have a set of perepts available, whih are the possible attributes that an agent ould pereive, subjet to its apabilities and urrent state. We refer to these as the possible perepts of an agent. However, due to limited resoures, an agent will not normally be able to pereive all those attributes possible, and will base its ations on a subset, whih we all the atual perepts of an agent. ndeed, some agents will not be able to pereive at all. n this ase, the set of possible perepts will be empty and onsequently the set of atual perepts will also be empty. To distinguish between representations of mental models and representations of the atual environment, a type,, is defined to be the pereption of an environment by an agent. This has an 2 equivalent type to that of ŸB$ &, but now physial and mental omponents of the same type an be distinguished. 2jA;AjR/ * * K t is also important to note that it is only meaningful to onsider pereptual abilities in the ontext of goals. Thus when onsidering objets without goals, pereptual abilities are not relevant. bjets respond diretly to their environments and mae no use of perepts even if they are available. We say that pereptual apabilities are inert in the ontext of objets., & n the shema for agent pereption,, we add further detail to the definition of a E, agents, and so inlude the shema a &. An agent has a set of pereiving ations, WB ^ whih are a subset of the apabilities of an agent. The funtion, WE Ÿ (, determines the attributes that are potentially available to an agent through its pereption apabilities. When applied, its arguments are the urrent environment, whih ontains information inluding the agent s loation and orientation (thus onstraining what an be pereived and the agent s apabilities. The seond prediate line states that those apabilities will be preisely the set of pereptual apabilities. Finally, 6

7 the funtion, 2.. WE ^ always be applied to its goals. a & Ÿ (@W a & WB ^ a E >dr p WE Ÿ ( ^B$ & } R p 2.. WE Ÿ ( R. } Ÿ&$ & } WB ^ a E D W. s ^B$ & ;R > WE ^ dom B dom 2.. WB ^ >AD[., desribes those attributes that are atually pereived by an agent and will } 2 zä ^B$ & WE Ÿ ( a p E Diretly orresponding to the goal or goals of an agent, is an ation-seletion funtion, dependent on the goals, urrent environment and the atual pereptions. This is speified in, & below, with the first prediate ensuring that the funtion returns a set of ations within the agent s ompetene. ote also that if there are no pereptions, then the ation-seletion funtion is dependent only on the environment. a & a & a E >R. } 2 } ^B$ & } R s ;R. 2 ^B$, & 4 p E D W. B dom a & E ;Au[. 3.2 Agent State The state of an agent desribes an agent urrently situated in some environment and is defined as follows. This inludes two variables, WE1 KWE ^ ªW, desribing those perepts possible in the urrent environment, and. WE ^ ªW, a subset of these whih are the urrent (atual perepts of the funtions. agent. These are alulated using the WE Ÿ ( and 2.. WE Ÿ ( a & «& K ^B$ & n ^B$ & a & Ÿ (@W a & WE1 KWE ^ ªW 8 2. WE ^ ªW 2 2..<d R. WE ^ ªW 8 FWE1 KWE ^ ªW WB ^ a E AC[ W#1 TWE Ÿ (@W >AD[ WE1 KWE ^ ªW ;A WB ^ j 8 Ÿ&$ & WE ^ a E. WE ^ ªW lac2.. WE Ÿ ( 4. W#1 TWE Ÿ (@W 2..<d A, & E '.. WE Ÿ (@W 8 ŸB$ & 7

8 «6 U astly we speify how an agent interats with its environment. As a result of an interation, the environment and the a & «& K hange. a & & K p, & «& K ^B$ & ª9&A *z p & K Z ŸB$ & 2..0d WE1 KWE ^ ªW 9:Ā WE Ÿ ( 8 ^, p B. WE ^ ªW 9:AC2.. WE Ÿ ( 'a. W#1 KWE ^ ªW 9 2..<dd9&A a & p B 'a.. WE ^ ªW 9± 3.3 Tropisti Agents SMART speifies a set of generi arhitetures. The types, funtions and shemas it ontains an be applied to other systems and onepts. n order to illustrate its use in this way, and to show how the model of interation is suffiiently general to apture most types of agents, tropisti agents [23] are reformulated as an example of an agent. t is one of a set of ore agent arhitetures used by Genesereth and ilsson to demonstrate some ey issues of intelligent agent design. The ativity of tropisti agents, as with reflexive agents, is determined entirely by the state of the environment in whih they are situated. First, the original desription of tropisti agents is summarised and then reformulated using elements of SMART. Aording to Genesereth and ilsson, the set of environmental states is denoted by «. Sine agent pereptions are limited in general, it annot be assumed that an arbitrary state is distinguishable from every other state. Pereptions thus partition «in suh a way that environments from different partitions an be distinguished whilst environments from the same partition annot. The partitions are defined by the sensory funtion,, whih maps environments ontained in «to environments ontained in, the set of all observed environments. The effetory funtion,, whih determines how environments hange when an agent performs an ation, taen from the set of the agent ations,, maps the agent s ation and the urrent environment to a new environment. Finally, ation-seletion for a tropisti agent,, is determined by pereptions and maps elements of to elements of. Tropisti agents are thus defined by the following tuple. 8E«a } d «} «Z } Reformulating Pereption The SMART framewor an be applied to reformulate tropisti agents by first defining types: equating ŸB$ & ; the set (as it refers to agent pereptions, to the type the set «to the SMART type, 2 ; and the set,, to the type p. The following type definitions an then be written. «xa;a ŸB$ & 7 A;A 2j7 A;A p Aording to SMART, tropisti agents are not autonomous. Thus the agent-level of oneptualisation is the most suitable level, and these are the models hosen. The funtions defining arhiteture at this level are WE Ÿ (, 2.. WE ^ and a & p B, defining the possible perepts, atual perepts and performed ations, respetively. The effet of ations on environments is independent of the level hosen in the agent hierarhy and defined by z & K. Reall that these funtions have the following type signatures. 8

9 s A } [ WE Ÿ ( 8 ^B$ & } R p WE Ÿ ( 8R. } 2 } 2 a E 8R. } 2 } Ÿ&$ & } *z p K Ÿ&$ & } R } R ŸB$ & These funtions inlude expliit referene to agent goals, whih are not represented in the model of tropisti agents sine they are impliitly fixed in the hard-oded funtions. n what follows, the value of these goals is taen to be and aordingly set all goal parameters of SMART funtions to this value. The goals of a tropisti agent do not onstrain the seletion of its pereptions from those that are available, and 2.. WE Ÿ ( is defined as the identity funtion on observed environments. n SMART, the pereiving ations are used at every pereiving step so that the seond argument of WE ^ a E is always applied to the pereiving ations ( WB ^ of the agents as speified in the, & shema. Aordingly, tropisti pereption is reformulated in the seond prediate below. There is an impliit assumption that tropisti agents are apable pereivers; pereptions are always a subset of the atual environment. This assumption is formalised in the last of the three prediates below that together define tropisti pereption. 2.. WE Ÿ ( zad[ 2 s ;E«s E«> AC2.. WE Ÿ ( 2.. WE Ÿ ( 4 4 Ÿ& &^ WE Ÿ ( 8WE Ÿ ( a E WB ^ WB ^ a E D The set of partitions in «an be alulated using set omprehension. W ( E >A;Au[, Reformulating Ation ŸB$ & e AD > 4²³[ The differene between the SMART framewor and tropisti agent effetory funtions is simply that the former allows for a set of ations to be performed rather than a single ation. s ; ^B$ & d B A *z p & K Z [ The ation seleted by a tropisti agent is dependent solely on its pereptions. n SMART, the ations performed are additionally dependent on goals and the environment. The environment an affet the performane of seleted ations if, for example, an agent has inorret or inomplete pereptions of it. By ontrast it is assumed that a tropisti agent orretly pereives its stati environment and performs ations that are equivalent to those seleted. These assumptions mean that the environment does not affet the performane of ations one they have been seleted. n order to speify this in SMART, the ŸB$ & parameter of, & E is fixed to the empty set, and is defined using, & p E as follows., & p E S Reformulating tropisti agents using SMART highlights several issues of note. First, SMART provides a more intuitive oneptualisation of an agent as an objet with a purpose. Goals are hard-oded into tropisti agent ations and pereption funtions; they are neither asribed to the agents nor are there any expliit mehanisms by whih agent goals diret behaviour. Seond, expliitly inorporating agent goals in SMART provides a more sophistiated design environment. t inorporates the premise 9

10 that agent goals hange over time and that the seletion of ations and pereptions must be adapted aordingly. Clearly, it is ineffiient to have to re-write the funtions defining ation and pereption seletion every time new goals are adopted. Third, features of SMART are more generally appliable than those desribed for tropisti agents, and it an therefore be used to expliitly formalise any assumptions (impliit or otherwise regarding the tropisti agent, its environment, and the interation between them. n this setion we have onstruted a formal speifiation that provides us with a hierarhy where all agents are objets, and all objets are entities, with the distintion between eah ategory made preise. However, the agents that have been defined are not in themselves espeially useful or interesting. Consequently, we must onsider how we an refine this framewor to develop definitions for other, more interesting and more varied inds of agents. The next setion desribes and speifies four types of agents, eah desribing some subset of the agent lass. These are autonomous agents, memory agents, planning agents and soial agents. Eah definition will arise through a refinement of the ideas and shemas presented above. 4 Classes of Agents 4.1 Autonomous Agents ur definition of agents entails the notion that an entity is serving some purpose or, equivalently, that the entity an be asribed some goal. However, we have not as yet onsidered how goals arise in the first plae. n our view, goals are derivative omponents that are onstruted in response to the needs either of the agent itself, or of some other agent. Goals an be adopted and transferred, but if goals are derivative as we laim, then there must be some entities in the world an generate or derive these goals. We define the non-derivative omponents from whih goals are derived as motivations; goals are generated solely in response to motivations. t is this quality that defines autonomy; an agent that has a non-empty set of motivations (from whih goals may be reated is an autonomous agent. An autonomous agent is an agent together with an assoiated set of motivations. T$ a & $ a & B žš AD[ An autonomous agent is defined as an agent with motivations and some potential means of evaluating behaviour in terms of the environment and these motivations. n other words, the behaviour of the agent is determined by both external and internal fators. This is qualitatively different from an agent with goals beause motivations are non-derivative and governed by internal inaessible rules, while goals are derivative and relate diretly to motivations. Autonomous agents also pereive, but motivations, as well as goals, filter relevant aspets of the environment. n the shema below, the funtion T2.. WE ^ is then a more omplex version of an agent s 2.. WB ^, but they are related if we hoose to interpret the behaviour of the agent solely in terms of its agenthood (and therefore its goals then the 2.. W ^ representation is appropriate, and if we wish to interpret its behaviour in terms of its autonomy (and therefore its motivations as well as its goals then the funtion T2.. WE Ÿ ( will be appropriate. evertheless, that whih an autonomous agent is apable of pereiving at any time is independent a ^ ªW from. of its goals and motivations and we just import the definition of WE ^ 10

11 6 R R T$ a & Ÿ (@W T$ a & a & Ÿ (@W T2.. WB ^ R } dom T2.. WE ^ >Au[d$ E. } ŸB$ & } 2 The next shema defines the ation-seletion funtion and inludes the previous shema definitions for a & T${, & and. The ation-seletion funtion for an autonomous agent is produed at every instane by the motivations of the agent, and is always and only ever applied to the motivations of the autonomous agent. T$ a & p T$ a & a & T E 8R } dom T E ;AD[d$ E. } 2 } ^B$ & } R We also define the state of an autonomous agent in an environment by inluding the, & «& K, T${, & ^ ªW T$ a & p and shemas. T$ a & «& K T$ a & Ÿ (@W T$ a & p a & «& K. WE ^ ªW la T2.. WB ^ >$ E a. WE1 KWE ^ ªW 2..<d A T E l$ E 4. p. WE Ÿ (@W { ^B$ ow we an speify the operation of an autonomous agent performing its next set of ations in its urrent environment. otie that while no expliit mention is made of any hange in motivations, they may hange in response to hanges in the environment. 9 Ā A T$ a & & K p T$ a & «& K ^B$ & ª9&A *z p & K Z ŸB$ & 2..0d WE1 KWE ^ ªW WE Ÿ ( 8 ^B$ WE ^, p B WE ^ ªW 9:A T2.. WE Ÿ ( p9${d E 9. 9W#1 KWE ^ ªW 9..<d T p B ;${d E a. 9M p WE Ÿ (@W ŸB$ & The essential feature in distinguishing autonomous agents from non-autonomous agents is the ability to generate their own goals aording to their internal non-derivative motivations. ne goals are generated, they an subsequently be adopted by, and in order to reate, other agents. We an extend the framewor to show how an autonomous agent an generate goals. n order to do so, we require a repository of nown goals, whih apture nowledge of limited and well-defined aspets of the world. These goals desribe partiular states or sub-states of the world with 11

12 eah autonomous agent having its own suh repository. An agent tries to find a way to mitigate motivations, either by seleting an ation to ahieve an existing goal, by reatively performing an ation in diret response to motivations, or by retrieving a goal from a repository of nown goals. The first two of these alternatives were addressed by the T E funtion in the T$, & shema seen earlier, while the last is onsidered briefly below. n order to retrieve goals to mitigate motivations, an autonomous agent must have some way of assessing the effets of ompeting or alternative goals. Clearly, the goals that mae the greatest positive ontribution to the motivations of the agent should be seleted. To do this, an autonomous agent must monitor its motivations for goal generation, and retrieve appropriate sets of goals from a repository of available nown goals. We an define a funtion that taes a partiular onfiguration of motivations and a set of existing goals and returns a numeri value representing the motivational effet of satisfying those goals. Then all that is neessary for goal generation is to find the set of goals in the goalbase that has a greater motivational effet than any other set of goals, and to update the urrent goals of the agent are updated to inlude the new goals. 4.2 Memory Agents The agents onsidered above are very simple and rely solely on the environment, goals and motivations (if available to determine ation. We have not yet speified the way in whih some agents may be able to tae into aount prior experiene (exept through any hanges that arise in goals and motivations. Agents who annot tae past experiene into aount will be extremely limited as a result. By adding further detail to our existing definitions we now provide a desription of agents with the ability to aess an internal store of attributes or memory that an reord, for example, prior experiene and other relevant information. We all an agent with suh an internal store a memory agent. A memory agent does not neessarily reord just those attributes that are urrently available in the external environment, but may also store some other attributes regarding more general learned or aquired information. $ a & $ ${ $ ${, & R * AD[ š K Thus a memory agent differs from the previous agents by having available, in addition to the external environment, an internal store of attributes, both of whih ontribute to forming the agent s urrent view of the world. Clearly, a memory agent will require ertain pereiving ations in order to aess the memory. n this respet it may often be useful to divide the pereiving ations of an agent into internal and external parts and, analogously, the environment may also be split into internal and external omponents, the internal environment being the memory. 2 ote that we refer to the internal set of attributes as an environment, rather than a, sine it is a physial store similar to the external environment. We an now refine the agent shemas to inlude this onept of memory as follows. 12

13 U $, & $, & a & Ÿ (@W & K. WE Ÿ ( a E dr p K. WE Ÿ ( a E dr p $ ${ WE Ÿ ( 8 ^B$ & ^B$ } R } Ÿ&$ & & K. WE Ÿ ( a E µý K. WB ^ a E >Ä WE Ÿ ( a p E & K. WE Ÿ ( a E µ Y K. WB ^ a E >AD[ s ^B$ ;R r,` > $ $ WE ^ zä WB ^ a E dom r,à^ A memory agent s possible perepts are derived from applying its pereiving ations to both its external environment and its internal memory. Depending on its goals, the agent will selet a subset a ^ ªW in the shema. The of these available attributes, as defined previously by 2.. W ( ation that suh an agent selets at any time is also determined in the same way as defined previously in the a & p shema, sine the memory is arried through possible perepts and atual perepts to the ation-seletion funtion, a & p B. $, & a & $, & «& K $, & a & «& K WE1 KWE ^ ªW ;Af$ $ WB ^ ŸB$ & $ $ B WB ^ a E As a result of these refinements, we must also onsider the onsequenes of an ation on the environment and memory. The performane of some set of ations may, in addition to ausing a hange to the external environment, also ause a hange to the memory of the agent. n this respet, we define two funtions for these interations below, where the external environment funtion is exatly as defined earlier, but the memory is updated as a funtion of both the internal environment, external environment and the urrent goals of the agent. Goals are relevant here beause they may onstrain what is reorded in memory, and what is not. K.0*z p & K & K.0*z p & K Ÿ&$ & } R p } Ÿ&$ & } ^B$ & } R } R. } ^B$ ^B$ & We an now refine another shema to tae into aount these hanges. The following shema, whih speifies how a memory agent interats with its environment, is a new version of the a & K shema defined earlier. 13

14 $, & & K $, & 6ž a & & K p K.0 ŸB$ & µ ^B$ & K.0 ŸB$ & ª9M ŸB$ & K.0 ŸB$ & 9 A K (.X & K p K (.X Ÿ&$ & 2..0d $ ${ 9A & K.0*z p K Z$ $ K (.X ^B$ 2..<d>. These memory agents are similar in spirit to the nowledge-level agents of Genesereth and ilsson [23], in whih an agent s mental ations are viewed as inferenes on its database, so that prior experiene and nowledge an be taen into aount when onsidering what ation to tae. 4.3 Planning Agents Planning is the proess of finding a sequene of ations to ahieve a speified goal. ur definition of agents requires the presene of goals, and although we have briefly disussed how goals may be generated, we have not onsidered how agents plan to ahieve them. This setion inreases the omplexity of the agent speifiation so that it aptures the essene of a planning agent Modelling Plans This involves defining first the omponents of a plan, and then the struture of a plan, as shown in Figure 1. The omponents, whih we all plan-ations, eah onsist of a omposite-ation and a set of related entities as desribed below. The struture of plans defines the relationship of the omponent plan-ations to one another. For example, plans may be total and define a sequene of plan-ations, partial and plae a partial order on the performane of plan-ations, or trees and, for example, allow hoie between alternative plan-ations at every stage in the plan s exeution. We identify four types of ation that may be ontained in plans, alled primitive, template, onurrentprimitive and onurrent-template. There may be other ategories and variations on those we have hosen, but not only do they provide a starting point for speifying systems, they also illustrate how different representations an be formalised and inorporated within the same model. A primitive ation is simply a base ation as defined in the agent framewor, and an ation template provides a high-level desription of what is required by an ation, defined as the set of all primitive ations that may result through an instantiation of that ation-template. An example where the distintion is manifest is in dmars (see Setion 7.2, where template ations would represent ation formulae ontaining free variables. ne all the free variables are bound to values, the ation is then a primitive ation and an be performed. We also define a onurrent-primitive ation as a set of primitive ations to be performed onurrently and a onurrent ation-template as a set of template ations that are performed onurrently. A new type, ±- $4W, is then defined as a ompound-ation to inlude all four of these types. Ations must be performed by entities, so we assoiate every omposite-ation in a plan with a set of entities, suh that eah entity in the set an potentially perform the ation. At some stage in the planning proess this set may be empty, indiating that no hoie of entity has yet been made. We define a plan-ation as a set of pairs, where eah pair ontains a omposite-ation and a set of those entities that ould potentially perform the ation. Plan-ations are defined as a set of pairs rather than a single pair so that plans ontaining simultaneous ations an be represented. 14

15 Ô Ô ¾± ÁÀ Â^¼<º0Ã(Ä Å¾ª»ÇÆdÈÊÉ ¼X¾ ÌË À Â^¼<º0Ã(Ä Í Ã(Ä1ÂK M¹Kº<»º<¼<º<½ ¾ žëàmâ^¼<º0ã(ä Í Ã(Ä1žª»±ÆdÈÊÉ(¼X¾ žë&î<ëà Â^¼<º<Ã(ÄaÏ À Â^¼<Ä Í Ã(»±Æ>Ð Ð t :¹Kº<»lÑ ÑÒ M¹Kº<»º<¼<º<½ ¾ Ó Ó Ô Å¾ª»ÇÆBÑ ÑÅ1¾ª»±ÆdÈÊÉ ¼X¾ Ó Ó Ô Í Ã(Ä1ÂK M¹Kº<»lÑ Ñ Í Ã(ÄÂK :¹Kº<»º0¼<ºÕ½ ¾Ó Ó Ô Í Ã(Ä1žª»±ÆBÑ Ñ Í Ã(Ä1žª»±ÆdÈÊÉ(¼X¾Ó Ó Å1à ¼0É(È ÈÊÉ(Ä4 ÈÊÉ(ÄÀ Â^¼<º0ÃÄ seq Ź ¾K¾T MÈÊÉ(ÄÖÐ Ð få,º ÆEÑ ÑÒ MÈÊÉ(ÄpÀMÂ^¼<º0Ã(ÄÓ Ó ÔØ Ã¹TÙÑ Ñ<Ë&Ú^ÎÒ MÈÊÉ(ÄpÀMÂ^¼<º0Ã(Ä>Û Å¹ ¾T¾T ÈÊÉ(ÄÏ Ó Ó MÈÊÉ(Ä Ð Ð Ü MÉ(¹@¼@Ñ ÑÒ É¹ª¼<º0É(È ÈÊÉ(ÄdÓ Ó Å1Ã(¼0É È Ñ ÑÅ1à ¼0É È MÈÊÉ(ÄÓ Ó Å¹ ¾T¾Ñ ÑŹ ¾K¾T MÈÊÉ(ÄÓ Ó ÆdÈÊÉÄÆpÉ(º<¹KÝÐ ÈÊÉ(Ä4Þ Ë MÈÊÉ(ÄpÀMÂ^¼<º0Ã(Ä ÆdÈÊÉľªÄ¼<º<¼<ºX¾ªÝ Ð MÈÊÉ(Ä4ÞuË ßMÄ1¼<º0¼<à^á>Ã^â ¾ŸÈ ÆdÈÊÉÄÉ Â^¼<º0ÃÄ1Ý Ð ÈÊÉ(ÄzÞ ËÀ Â^¼<º<Ã(Ä ÈÊÉÄpÀ Â^¼<º<Ã(Ä4 ÌËÎ ÀMÂ^¼<Ä Í Ã(»ÇÆ8Û'Ëß:ļ<º<¼<àá>Ã^â ¾ŸÈXÏ É¹ª¼<º0É(È ÈÊÉ(Ä4 ÁãªÆÝÐ MÈÊÉ(ÄÀ Â^¼<º0Ã(Ä äu ÈÊÉ(ÄÀ Â^¼<º0ÃÄ Ô^å Éæç Ð MÈÊÉ(ÄpÀMÂ^¼<º0Ã(Äzè Î*ÉæŸÉÏ/é ê ÆÝŸë ì Î*Éæç Ï ê ÆÝŸë8í ÎTç(æ^Éϱé ê ÆÝŸë èçæýî Figure 1: Plan omponents and struture We speify three ommonly-found ategories of plan aording to their struture as disussed earlier, though other types may be speified similarly. Partial Plans. A partial plan imposes a partial order on the exeution of ations, subjet to two onstraints. First, an ation annot be performed before itself and, seond, if plan-ation is before, annot be before. Formally, a partial plan is a relationship between plan-ations suh that the pair is not in the transitive losure and, further, if the pair is in the transitive losure of the relation then the pair is not. Total Plans. A plan onsisting of a total order of plan-ations is a total plan. Formally, this is represented as a sequene of plan-ations. Tree Plans A plan that allows a hoie between ations at every stage is a tree. n general, a tree is either a leaf node ontaining a plan-ation, or a for ontaining a node, and a (non-empty set of branhes eah leading to a tree. These are formalised in Figure 1. Using these omponents we an define a planning agent. Any planning agent must have a set of goals urrently being pursued, a., and a set of plans assoiated with these goals, W. E. The plans assoiated with eah of the goals is given by the There will also be a repository of all goals,., a repository of all plans, W., and a funtion assoiating plans in the W. with the goals in the a., W. aa.. 15

16 ï ï Ä Ä. B a a & a } R. W. B >R. W.,.. } R. a. 8R. W. R. dom ªW. a./a?. W. E ran W.,. W. W. a. ~. dom a. a. The way in whih a planning agent hooses how to at is now also a funtion of its urrent plans. This is shown in the. B a a & shema below.. B a a & p. B a a & a & W. &,, & p E R. } R. } 2 } Ÿ&$ & } R s ;R. W ;R. 2 Ÿ&$ & W. B a a & ' p B 4 W u W d. B dom s W >R W. B. a, & W& > E >AD[ W. B a. a E 4a. W& zajw. E dom Here, we have only extended our desription of an agent to inlude the ability to plan. Further wor is neessary to investigate and speify how the plans of an agent also affet its reasoning. For example, we must address the questions of when an agent should abandon plans, generate new plans, abandon goals beause of a la of appropriate plans, and so on. However, this is beyond the sope of the urrent wor and will not be addressed further in this paper. evertheless, we have provided a framewor within whih suh issues and related theories and systems an be formally presented as we shall show in the next setion when we show how plans are modelled for various existing systems. 4.4 Soiologial Agents Agent Models We have already stated that an agent will have ertain perepts available to it. f an agent an mae sense of these attributes and group ertain sets of them together into entity-desribing models, then we have the beginnings of a soiologial agent, whih we tae to be an agent that is aware of other agents, and their role and funtion. Speifially the agent framewor provides the struture that allows an agent to onstrut meaningful and useful models of these roles and funtions in a very simple but effetive way. Suh models are desribed below. f the agent does not have a memory, then the union of the attributes of the set of entities it models must be a subset of its urrent pereptions. f the agent does have a memory, however, this ondition an be relaxed. 16

17 ne again, we refine the shemas given earlier to onstrut our model of a soiologial agent. The shema below desribes an agent that has grouped attributes into distint entities. a & p. & ( a & «& K & dr & & {š AC[ ï;[,> & ( %Ò * * K(. WE ^ ªW Though the agent does not neessarily have memory, this still onstitutes a model of the world that it possesses, sine it imposes a struture by grouping attributes. This ind of modelling is used by mehanisms suh as a robot arm on a prodution line. The arm is only onerned with the pereptual stimuli needed for it to perform the appropriate ation on an entity. n many ases, it will not need to now about the apabilities of the entity. ow an agent may, in addition, assoiate apabilities with some entity, and its model of the world will therefore be a olletion entities and objets. (This will typially involve the use of memory, but we will not onsider memory further here, so that we may learly differentiate the qualities that arise for distint reasons. a & p. œ * p a & p. & ( * ;R œ * * š AC[ * > C & Similarly, a more sophistiated agent may be able to model the world as a set of entities, objets and agents. a & p., & a & p. œ * p a ;R, & a š AD[ a > u * p At this level of modelling, an agent is aware of the onept of agenthood. That is, it is aware that some of the entities in the world are serving a purpose. However, we annot yet laim this to be a soiologial agent, sine it must be aware not only that a goal is being satisfied, but also 2" the goal exists. n other words, it must now that the goal has been generated by some agent. A soiologial agent must thus understand the onept of autonomous agents, whih are the only agents apable of generating goals. We therefore define a soiologial agent to be any agent that has the ability to model an entity as an autonomous agent. ote that in building up this notion of a soiologial agent, we are providing only a basi foundational onept that allows us to desribe agents apable of modelling others. Further ategories of agents may be onstruted on top of this and, indeed, this notion of soiologial agents is distint from soial agents that interat with others. However, in order for soial behaviour to be effetive, we argue that soiologial apabilities are needed. 17

18 y «.0. a a & p., & T$, & 8R T${ T$, & š AD[ T$, &, &, & Aording to this view, an agent an be soiologial even it has no soial apabilities (suh as rhetorial devies other than that it reognises autonomy. f we expand the previous shema, then a soiologial agent onsiders the world to onsist of entities, objets, agents and autonomous agents, where all autonomous agents are agents, all agents are objets and all objets are entities. n addition, if it an reognise agents that are autonomous and objets that are agents then it will ertainly be able to reognise agents that are not autonomous and objets that are not agents. These are nown as Server Agents and eutral bjets respetively, and are defined as follows. «: a a & $ B AD[ ð. œ * œ T a. AD[ 7Ø$ E ;AD[ A soiologial agent may also have a model of itself. This is given in the following shema. ote that a soiologial agent is not neessarily an autonomous agent, nor is an autonomous agent a soiologial agent. (For a definition of ˆ# Š ÊŒ1Ž the reader is ased to onsult the Appendix A. «.0. a. «.0. a :.< T >dr ð. œ * p, & ;dr «a &. w, & ˆ# Š ÊŒ1Ž a za T$ a & µ t, & * zaf.< * µ a & aturally, if an agent want to tae advantage of its ability to model these agents, it will need to use ertain persuasive devies as desribed in [6], but whih will not be onsidered here. Suppose, for example, that a robot wishes to use a radio. Also, suppose that the radio is already swithed on, and that the robot understands that the radio is serving some purpose. Sine the robot is a soiologial agent and understands autonomy, it is aware that the radio is serving some purpose, ultimately for some autonomous agent. The robot an simply tae ontrol of the radio (if it had the neessary apabilities, fully aware that the radio would not then be serving its original purpose. Alternatively, the robot may deide not to interfere with the radio. n both ases, there is no soial 18

19 behaviour. However, if the robot has the ability to model the relationship between the radio and the agents for whih that radio is serving a purpose, whih may in turn be serving a purpose for other agents, it may be able to identify the autonomous agent at the top of the regressive agent hain. n this ase, the robot may attempt to persuade this agent to release the radio from its original purpose, so that it may use it instead. n other words, we still need an understanding of the relationships between the various entities in the world aording to the purposes they serve. We need to be able to identify the originators of these purposes or goals, the autonomous agents that generated them. This requires an ability to model an agent diretly engaging a server agent, and an autonomous agent ooperating with another autonomous agent. We now speify the soial strutures we all engagements and ooperations that exist in a multi-agent world Engagement and Cooperation A diret engagement taes plae whenever a neutral-objet or a server-agent adopts some goals. n a diret engagement, an agent with some goals, whih we all the lient, uses another agent, whih we all the server, to assist them in the ahievement of those goals. ote that aording to our previous definition, a server-agent is non-autonomous. t either exists already as a result of some other engagement, or is instantiated from a neutral-objet for the urrent engagement. There is no restrition plaed on a lient-agent. We define a diret engagement in the following shema, whih onsists of a lient agent,. &, a server agent,, and the goal that is satisfying for. &. eessarily, an agent annot engage itself, and both agents must have the goal of the engagement. ñ> ^ a, $. a & E«, & a./.. 4š AD a.. & %. ' t %. An engagement hain represents a sequene of diret engagements. For example, suppose a robot uses a omputer terminal to run a program to aess a database in order to loate a library boo, then there is a diret engagement between the robot and the terminal, of the terminal and the program, and of the program and the database, all with the goal of loating the boo. An engagement hain thus represents the goal and all the robots involved in the sequene of diret engagements. n the above example, the agents involved would be as follows: ò a ($. ^d $ ñ8 Speifially, an engagement hain omprises some goal., the autonomous lient-agent that generated the goal, T a, and a sequene of server-agents, ", where eah one in the sequene is diretly engaging the next. For any engagement hain, there must be at least one serveragent, all the agents in involved must share., and the same agent annot be involved more than one. 19

20 A r a a $ & -±" a./. T a & n T${, & " t seq, & a. T, & % a. a. tï>[ ; a µe, > ran " ran " " %. The term ooperation is reserved for use only when the parties involved are autonomous and potentially apable of resisting. f they are not autonomous (and not apable of resisting, then one simply engages the other. A ooperation desribes a goal, the autonomous agent that originally generated that goal, and those autonomous agents who have adopted that goal from the generating agent. Thus in this view, ooperation annot our unwittingly between autonomous agents. - pwe a./., a a & n T${, & WE a a & R T$, & WB,, & {ó s WB,, &. d&%. a. Áa a a % a. A soiologial agent thus views the world as a olletion of engagements, engagement hains and ooperations between the entities in the world. ð 2 «.0. a. «.0. a. a a a $ & R ñ> p a a $ & a " tdr a a $ & -±" WE B 8Rt- pwe These shemas provide useful struture that an be exploited by intelligent agents for more effetive operation. This is only possible if eah agent maintains a model of their view of the world. Speifially, eah agent must maintain information about the different entities in the environment, so that both existing and potential relationships between those entities may be understood and onsequently manipulated as appropriate. 5 Appliation of the Framewor The strutures desribed above will be present to a greater or lesser extent in all multi-agent systems. aturally, these models and the models that an agent has of other entities an beome even more sophistiated. For example, an agent may model other agents as planning agents as we shall see later in this setion. n this way, agents an oordinate their ativities and enlist the help of others in order that plans an be ahieved suessfully and effiiently. n this setion, we omplete the path from our initial framewor through models of varying levels of abstration to detailed formal speifiations of three distint appliations. The first is dmars (the 20

21 distributed Multi-Agent Reasoning System, whih has been applied in perhaps the most signifiant multi-agent appliations to date. The seond is the well-nown ontrat net protool [51, 52, 12], whih again is situated in the domain of pratial implemented systems. The third appliation is the soial dependene networ[49, 50], whih is a struture that forms the basis of a omputational model of Soial Power Theory[7, 8]. These networs allow agents to reason about and understand the olletive group of agents that mae up the multi-agent world in whih they operate. Below, we onsider eah of these in turn, and show how they an be formalised in the ontext of previous models. n order to ahieve this, we reuse, refine and elaborate the shemas presented so far, in order to speify these multi-agent systems at a detailed level of desription. 5.1 Appliation 1: The distributed Multi-Agent Reasoning System (dmars While many different and ontrasting single-agent arhitetures have been proposed, perhaps the most suessful are those based on the belief-desire-intention (BD framewor. n partiular, the Proedural Reasoning System (PRS, has progressed from an experimental SP version to a full C implementation nown as the distributed Multi-Agent Reasoning System (dmars. PRS, whih has its oneptual roots in the belief-desire-intention (BD model of pratial reasoning, has been the subjet of a dual approah by whih a signifiant ommerial system has been produed while the theoretial foundations of the BD model ontinue to be losely investigated. As part of our wor, we have sought to formalise these BD systems through the diret representation of the implementations on the one hand, and through refinement of the detailed models onstruted through the abstrat agent framewor on the other. This wor has inluded the formal speifiation [16] of the AgentSpea( language developed by Rao [46], whih is a programming language based on an abstration of the PRS arhiteture; irrelevant implementation detail is removed, and PRS is stripped to its bare essentials. ur speifiation reformalises Rao s original desription so that it is ouhed in terms of state and operations on state that an be easily refined into an implemented system. n addition, being based on a simplified version of dmars, the speifiation provides a starting point for atual speifiations of these more sophistiated systems. Subsequent wor ontinued this theme by moving to produe an abstrat formal speifiation of dmars itself, through whih an operational semantis for dmars was provided, offering a benhmar against whih future BD systems and PRS-lie implementations an be ompared. Due to spae onstraints, we annot hope to get anywhere near a speifiation of either of these systems, but instead we aim to show how we an further refine the models of plans desribed above to get to a point at whih we an speify the details of suh implementations. The value of this is in the ease of omparison and analysis with the more abstrat notions desribed earlier. We begin our speifiation, shown in Figure 2 by defining the allowable beliefs of an agent in dmars, whih are lie PRG fats. To start, we define a term, whih is either a variable or a funtion symbol applied to a (possibly empty sequene of terms, and an atom, a prediate symbol applied to a (possibly empty sequene of terms. n turn, a belief formula is either an atom or the negation of an atom, and the set of beliefs of an agent is the set of all ground belief formulae (i.e. those ontaining no variables. (We assume an auxiliary funtion. whih, given a belief formula, returns the set of variables it ontains. Similarly, a situation formula is an expression whose truth an be evaluated with respet to a set of beliefs. A goal is then a belief formula prefixed with an ahieve operator or a situation formula prefixed with a query operator. Thus an agent an have a goal either of ahieving a state of affairs or of determining whether the state of affairs holds. The types of ation that agents an perform may be lassified as either external (in whih ase the domain of the ation is the environment outside the agent or internal (in whih ase the domain of the 21