Autonomous Agent Negotiation: Leveled Commitment and Trust

Transcription

1 Autonomous Agent Negotiation: Leveled Commitment Trust Tracking number: E069 Abstract. As agents become more autonomous, agent negotiation motivational attitudes such as commitment trust become more important. In this paper we illustrate the important choice in advanced negotiation applications whether negotiation parameters such as cardinality of interaction, agent attitude, agent architectures are incorporated in the negotiation protocol or in the negotiation strategy. Only in the first case this parameter is fixed in the protocol agents do not have to reason about them when they choose their strategy. We focus on the relation between commitments between agents trust. The illustration is also an example of the use of deontic logic in computer science. Keywords: multi agent systems, autonomous agents, negotiation, socionics, deontic logic 1 Introduction In advanced applications of multi-agent systems agents interact more frequently, deliberate more extensively, in general act more autonomously. For example, in electronic commerce agents are allowed to negotiate make complex decisions autonomously. In these applications reasoning about motivational attitudes how do agents motivate their decisions practical reasoning in general become crucial issues. For example, the agents should be able to reason whether their messages commit them or create any liability [1, 8, 16, 9]. Lack of this reasoning capacity becomes more dangerous as agents become more autonomous. Recently several mal systems (in particular logics) have been introduced to support these ms of reasoning. In this paper we are interested in automated negotiation, i.e. the interaction between rational, self-interested autonomous agents who try to accomplish a mutually acceptable agreement on some issues [12, 2, 7, 11]. These agents are usually designed by different organizations are supposed to interact real time with each other in an open environment, in the sense that may not know the architecture, the identity, the privileges the negotiation strategy of each other. Automated negotiation systems are realized in various fields of electronic commerce like auction houses, electronic shops, or electronic market places. The increase in agent autonomy corresponds to more flexible protocols consequently more complex strategies. Agents start to reason about the behavior of other agents, their trustworthiness [6], the acceptance of commitments, potential violation of obligations. To malize these concepts we use a logic to reason about deontic mental states of agents such as trust in other agents BODI (= beliefs, obligations, desires, intentions) protocols represented by dynamic logic with iteration concurrency to malize finite state machines with effects of actions ( example on deontic mental states). In this paper we present our general framework, which is illustrated by the following relation between obligations trust: 1. when agents can negotiate the stage level of their commitments [1], they must be able to reason about obligations violations of obligations, 2. the consequences of violations of obligations are twofold: short term effects like penalties, long term effects like decrease in trustworthiness. This illustration also gives evidence to the conjecture posed in the community of deontic logic in computer science that the logic of obligations is a useful sometimes necessary ingredient of complex systems [, ]. This article is organized as follows. In Section 2 we discuss the balance between protocol strategy in (automated) negotiation. In Section we sketch our mal framework to reason about negotiation. We illustrate our framework with an example of an electricity negotiation. 2 A Trade-off between Protocol Strategy An automated negotiation system comprises a negotiation protocol the rules of encounter among involved agents a negotiation strategy the actions of an agent at a certain stage of interaction. The protocol determines which strategies can be designed a strategy determines which protocols are required. The protocol is usually represented by a finite state machine where a link represents a negotiation state transition such as e.g. making proposals, accepting or rejecting them, a strategy of an agent is usually represented by a trace of the protocol. Different kinds of negotiations can be characterized by means of various negotiation parameters like e.g. cardinality of negotiation interaction (one-to-one, one-to-many, many-to-many), dimensions of negotiation items (single-attribute item, multi-attribute item), agent attitude (social, selfish, cooperative, competitive), agent architecture (BDI components+commitments+trust) [12]. For example, consider the two negotiation protocols A B shown in Figure 1. In this figure circles represent negotiation states state transitions represent actions. The protocols describe how a washing machine W can negotiate with an energy provider company E to buy electricity. Protocol A describes a one-to-one negotiation system (interaction cardinality parameter) in which two competitive agents (agent attitude parameter) W E negotiate about a multi-attribute item (dimensions of item), e.g. kilo watt electricity some price at some time, by making proposals, counter proposals, accepting or rejecting them. Protocol B differs from protocol A since different BDI components (agent

2 1 Request_W 2 Reject_E Figure 1. Propose_W 6 Protocol A Accept_E Accept_W Reject_W 1 Request_W 2 Propose_W Reject_E 6 Accept_E Accept_W Retract_E Retract_E Retract_W Sanction 7 Protocol B Retract_W Reject_W A restrictive protocol (Protocol A) a flexible protocol (Protocol B) the washing machine scenario. architecture parameter) the involved agents are required. In fact, the BDI components of the agents in protocol B can hle commitments their violations. As we will see in following sections, the commitments their violations can be used to determine the trust of agents in each other (another agent architecture parameter). Protocol B is more flexible than protocol A, because the agent has more options thus can make more complex (strategic) decisions. In negotiation state of protocol A B, the agent W can strategically decide to make a counter proposal rather than rejecting the proposal of the provider ending the negotiation (he may know that the energy provider does not have enough consumers). But only in state of protocol B the washing machine that implicitly has committed to buy energy can retract that commitment. Similarly, it can retract the commitment made by doing a proposal in the negotiation state. Protocol B is theree more useful in practice, because agent W can negotiate with different energy companies in order to find the best possible deal. In such a scenario, agent W may not be able to wait until he receives the proposals of all energy companies (e.g. because of the time limit imposed by the companies) theree he needs to be able to retract an accepted proposal offered by e.g. in order to accept a more beneficial proposal offered by e.g.. Summarizing, Protocol B allows agents to commit, to violate commitments (possibly some sanctions may be imposed), to change their beliefs trust in each other, in general to make better strategic decisions. Negotiation parameters determine the characteristics the behavior of negotiation systems, consequently, they can be incorporated in the negotiation protocol as well as the negotiation strategy. In the protocol component, they specify the space of possible negotiation states the actions that the agents can take. In the strategy component, they m the basis on which agents can decide how to act. We call a parameter a protocol or strategy parameter depending on whether it is incorporated in respectively the protocol or the strategy component. The following definition distinguishes between protocol strategy parameters in negotiation systems. Definition 1 (Parameters) A negotiation parameter is called relevant a negotiation state if in that state at least one action can be taken which is specified in terms of that parameter. A parameter is called a protocol parameter in a negotiation state if it is relevant that state there is only one action possible in that state, i.e. the action specified in terms of the relevant negotiation parameter. Moreover, in a negotiation system a parameter is called a protocol parameter an agent if it is a protocol parameter in all negotiation states in which the agent can act. Likewise, a parameter is called a strategy parameter in a negotiation state if it is relevant that state there are more than one non-equivalent action possible in that state. Moreover, in a negotiation system a parameter is called a strategy parameter an agent if it is a strategy parameter in at least one negotiation state in which the agent can act. Note that relevant parameters which are not strategy parameters are thus protocol parameters vice versa. According to Definition 1, commitment in negotiation protocol A, shown in Figure 1, is a protocol parameter since the agents can only implicitly make certain predefined full commitments by accepting a proposal (state ). On the other h, commitments their violations in the negotiation protocol B, illustrated in Figure 1, can be considered as strategic parameters since the agent can decide to retract them in various negotiation states like in negotiation states,,. Negotiation parameters their allocations in the protocol strategy define the space of possible negotiation systems. The allocation of negotiation parameters in one of negotiation components depends on the desired characteristics behavior of the negotiation system such as computational communicational efficiency the size of negotiation space; enlarging negotiation space may increase the chance to accomplish an agreement. For example, a restrictive protocol that does not allow removing adding agents or does not allow choosing stages levels of commitments [1] during the negotiation process may result in lower computational communicational complexities than when these parameters are considered as strategic parameters theree allowed during the negotiation process. Summarizing, in specifying a negotiation system, there is a tradeoff between assigning negotiation parameters to the protocol strategy component. At one extreme parameters of negotiation are incorporated in the negotiation protocol on the other extreme they are incorporated in the negotiation strategy. The space of negotiation systems defined by the allocation of negotiation parameters to negotiation components is schematically illustrated in Figure 2. This figure should be read as follows. A boc represents a negotiation, an arrow between boxes represents levels of flexibility. In this way, a shift towards the strategy extreme results in more realistic dynamic models of negotiation where most interactions among agents are incorporated in strategy component less in the protocol, a shift towards protocol extreme results in more artificial constrained negotiation in which each agent is limited in choosing actions. For example, agents involved in an English auction are limited in possible actions (a bid should be higher than the previous bid) the stages of taking actions (propose a bid after the auctioneer s call). In contrast, human agents in a market place can decide to propose a bid or make a commitment at a certain time if they believe that the bid at that time would result in higher utility. In the following sections, we concentrate on the design of dynamic negotiation systems by discussing how commitments trust can be incorporated in the strategy component rather then in the protocol component. We also discuss the properties of the resulting negotiating mechanism. In this way, agents are allowed to reason about commitment trust, change them, take appropriate actions to 2

3 Cardinality : Strategy Item-dimension : Protocol Negotiation 1 Cardinality : Protocol Item-dimension : Protocol Negotiation 2 Negotiation Negotiation Cardinality : Strategy Negotiation Cardinality : Strategy Attitude : Strategy Cardinality : Protocol.1 Logic We outline only the structure of the logic do not consider logical properties. The most important elements of the logic are the modalities, whose intended meaning is agent, is more committed, towards agent, to perm than to perm, with the intended meaning the trust of agent in agent after permance of is less than, or equal to the trust after permance of. We stress that defines a state of commitment, not a commit action. Actions such as commit, that result in a commitment will be discussed in section. Given a set of action symbols, a set of proposition symbols, a well med mula is defined through the following BNF: Figure 2. maximize their utilities. The space of possible negotiation systems. Formalizing trust commitment in a dynamic logic setting We investigate the reasoning of an agent involved in a negotiation process. Apart from believes, desires intentions, the reasoning will have to account : 1. the negotiation protocol, that constrains the order choice in negotiation acts available to the agent to the other party, 2. the effects of negotiation acts on normative states (whether or not to be subject to commitments) both parties,. the effects of the actual permance of (non-negotiation) actions, by the agent or by the other party, their effects (these may be action that fulfill commitments, or actions that violate commitments),. the relation between action effects utility both parties,. the possible violation of commitments the effects on mutual trust both parties. To focus on the issues described in section 2, the logic we define leaves out some aspects. We do not model believes desires. This would require embedding in a BDI-logic, instance in the one of Rao Georgeff [1, 1]. Furthermore, the relations between action effects utility between trust utility are left implicit. We just assume a utility function each separate agent. This utility function characterizes what things are valuable to the agent whether it thinks it is important to be trustworthy or not. The focus of the logic is on the new elements their relations in the framework: levelled commitments, obligations, trust. The notions intention, obligation, permission prohibition are available as derived notions. The malization (logic) builds on previous work in the area of dynamic deontic logics [17][10]. The logical notions we define are of the to do type, which contrast with notions of the to be type. This is a common distinction in the area of deontic logic. In the present setting it applies also to notions such as intention, meaning that the logic is about intention to do certain actions, not about the intention to be in a certain state. The machinery of dynamic logic, encompassing choice ( ), sequence ( ) iteration ( ), is especially suited to reason in the context of negotiation protocols. Protocols are usually represented by finite sate machines, which are equivalent in expressive power to the regular actions that are central to the ontology of dynamic logic. The concurrency operator is, instance, used to model the reasoning of an agent that offers some service to different agents simultaneously. Action negation ( ) is used in the malization of the deontic notions obligation, permission prohibition. This according to an approach first taken by Meyer [10]. The denotes an action whose effect is void: doing a does not change the state we are in. It can be used in an expression like to express that by perming our trust is decreased in comparison to the situation where we do nothing (we skip). The -action refrains from action labels completely, which is useful if we want to reason about reachability of states, in a more temporal fashion. The -action is the empty action. The semantics is defined in a fairly stard way. Definition 2 Given a set of action symbols, a set of proposition symbols, a structure is defined as: is a nonempty set of possible states is an -indexed collection of (reachability) relations over is a valuation function that interprets propositions is an ordering over that orders levels of commitment of agent with respect to agent is an ordering over that orders trustlevels of states agent with respect to agent Definition The meaning of well-med mulas in a state structure is given by: all with of a

4 iff iff not iff iff s.t. iff o.i. s.t. o.i. s.t. iff o.i. s.t. o.i. s.t. Validity of a mula on a model general validity are defined as usual. The orderings that interpret are designed to deal with leveled to do -notions concerning nondeterministic actions. We believe these orderings are more adequate than the ones used by instance Liau [8]. The variant of dynamic logic defined is new in the sense that it combines a complete boolean algebra over, with sequence iteration. For mulas we use the usual abbreviations. In addition, we abbreviate instance to..2 Derived notions We define Intention as self-commitment: We think this identification has a strong intuitive appeal to it. It shows that the notion of intention has a normative (deontic) flavor, which is in accordance with the fact that one can violate ones intentions. Other normative notions are also available as reductions to commitment. Extending an idea of Meyer [10], we define obligation to perm as the commitment to perm rather than : We define permission to do as the absence of an obligation to do, prohibition as the negation of permission: The first mula says that trust is increased by the fulfillment of an obligation, the second that trust is decreased by permance of a bidden action (in comparison to doing nothing). This distinguishes trust from commitment in the malization. Basically the distinction expresses that while commitment in each stage of an action trace orders states along the choice-dimension, trust also orders states along the sequence direction of action traces. This is the mal counterpart of saying that trust is a notion that recalls the past. Example We show how to apply the logic to examples as given in section 2. First we say more about the many different negotiation acts that are used in protocols. These negotiation acts in most cases have commitments as a result..1 Negotiation acts in protocols We distinguish between negotiation actions, services. The first have a predefined meaning, their possible occurrences are governed by the negotiation protocol. Typical examples are propose accept. The second type of actions, are the ones that the negotiation is about. They are services of one agent to the other. The effects of these actions are valuable to agents, as determined by their utility functions. Since negotiation acts such as propose are about services, they will have to be indexed with symbols indicating what services are negotiated. Usually two services are involved in one negotiation act. An example is the proposal to perm a service in return a payment action. Also, negotiation acts will have to be indexed with parameters that denote which agent perms the act which agent is subject to it. This results in a propose-act of the m: propose The mula means that agent proposes to perm in return. Likewise we denote an acceptance of agent of this proposal by: accept. Relating norms trust One of the main purposes of this article is to argue that commitment trust are interrelated. Theree we constrain the semantics of the relation between commitment trust by imposing some minimal properties. First of all, it is clear that the levels of commitment, will induce levels of trust: violating a strong commitment decreases one trustworthiness more than violating a weak commitment. In other words: if the commitment to do is higher than the commitment to do, the trust after permance of is higher than the trust after permance of. In a mula this is expressed as: This suggests a close correspondence between the notions of commitment trust. But there is an important difference: the trust modality there is an a priori relation between the trustworthiness in the current state, in the state resulting after complying to, or violating a commitment. This is expressed by: Both a propose an accept action create an obligation, respectively the agent proposing, the agent accepting. In the logic of the previous section this is defined as: propose accept The mula expresses that the obligation of the proposing agent is conditional on the acceptance of it by the second agent, that the obligation of the accepting agent is a direct consequence of the acceptance. The part of the mula that is within the modal box, is related to the negotiation protocol. In this example it is extremely simple, saying that there can be a proposal followed by an acceptance nothing else. More elaborate examples involve operators like iteration ( ) to specify that the negotiation is repeated, or concurrency ( ), to specify that an agent proposes to more than one agent simultaneously. The logic also enables easy specification of other negotiation acts, such as. In general agents will distinguish levels of commitments among their obligations. These levels are easily expressed in the logic. An

5 agent might instance distinguish the following levels in his obligation to perm : From these we logically derive, instance:,. In general, violation of each level of commitment comes with its own penalty the agent. This penalty is part of the agents utility function. Here we only focus on the influence of violating commitments on the trust component of utility functions..2 Strategy At this point, strategy comes in. A strategy can be seen as a heuristic to optimize a utility function within the search space defined by the logic mulas that model the reasoning of an agent in a specific negotiation context. Generally the space is much to large to find the plan that leads to the optimal value of the utility function. Apart from all possible different effects of actions, trust will be a component in the utility function. Generally, agents want to maximize their trustworthiness, but also the trustworthiness of the agents they communicate with. But this might not be the case if the utility of perming non-trust gaining actions is very high. Concluding remarks In our model, an agent has the following two characteristic properties: he can decide to violate his commitments (or obligations), he is thus in this sense autonomous, norm violations he distinguishes between short term effects like paying penalties, long term effects like loosing reputation theree future opportunities, he is thus resource bounded, because he cannot simply calculate the expected utilities of all his actions decide accordingly. An agent s reputation is based on the degree of trust other agents have in his behavior, in particular the degree in which he fulfills his commitments. Whether agents trust other agents, whether they use this trust in making decisions, depends on the application at h. There are two extreme cases: cooperative agents have a high trust in other agents use this inmation when making decisions, competitive agents have a low trust in other agents do not use this inmation when making decisions. For these agents, we sketched a framework with the following ingredients: negotiation protocols that constrain the order choice in negotiation acts; a dynamic logic with iteration concurrency is used to reason according to the finite state machines that are commonly used to represent these; the dynamic framework also malizes the effects of actions such as states described below; qualitative decision represented by qualitative logics desires beliefs; commitments towards oneself represented by intention (= choice + commitment) towards others represented by obligation; making a commitment is a change of deontic state such as creating an obligation; there is a stage of commitment a level of commitment [16]; agent profiles represented by a trust factor; the default value of trust is set by the protocol, e.g. high violations set high trust; the value is adjusted when the agent violates (decrease) or fulfills (increase) his obligations. An interesting issue further research is the resolution of conflicts between desires obligations. Often an agent prefers a state which is bidden; how to act? In our system, this becomes a tradeof between loss in utility versus loss of trustworthiness, i.e. between short term long term effects. To resolve this kind of conflicts additional machinery has to be introduced in the logic, such as qualitative preferences between these two items, or quantitative measures. One proposal can be found in []. REFERENCES [1] M. Barbuceanu, Agents that work in harmony by knowing fulfilling their obligations, in Proceedings of the AAAI 98, pp , (1996). [2] P. Faratin, C. Sierra, N. Jennings, Negotiation decision functions autonomous agents, International journal of robotics autonomous systems, 2, , (1998). [] N.R. Jennings J.R. Campos, Towards a social level characterisation of socially responsible agents, IEEE proceedings on software engineering, 1, 11 2, (1997). [] A.J.I. Jones M. Sergot, Deontic logic in the representation of law: Towards a methodology, Artificial Intelligence Law, 1, 6, (1992). [] A.J.I. Jones M. Sergot, On the characterisation of law computer systems: The normative systems perspective, in Deontic Logic in Computer Science, John Wiley Sons, (199). [6] C. Jonker J. Treur, Formal analysis of models the dynamics of trust based on experiences, in Multi-Agent System Engineering, Proceedings of the 9th European Workshop on Modelling Autonomous Agents in a Multi-Agent World, MAAMAW 99. LNAI 167, (1999). [7] S. Kraus, K. Sycara, A. Evenchik, Reaching agreements through argumentation: a logical model implementation, Artificial Intelligence, (1998). [8] C.-J. Liau, A logic reasoning about action, preference commitment, in Proceedings of the ECAI 98, ed., H. Prade, (1998). [9] J. Meyer, W. van der Hoek, B. van Linder, A logical approach to the dynamics of commitments, Artificial Intelligence, (1999). [10] J.-J.Ch. Meyer, A different approach to deontic logic: deontic logic viewed as a variant of dynamic logic, Notre Dame Journal of Formal Logic, 29, , (198). [11] S. Paurobally J. Cunningham, Formal models negotiation using dynamic logic, in?, (2000). [12] H. Raiffa, The art science of negotiation, Harvard university press, Cambridge, Mass., [1] Rao Georgeff, Modeling rational agents within a bdi architecture, in Proceedings of the KR91, (1991). [1] Rao Georgeff, An abstract architecture rational agents, in Proceedings of the KR92, (1992). [1] T. Sholm V. Lesser, Issues in automated negotiation electronic commerce, in Proceedings ICMAS 9, (1999). [16] L. van der Torre Y. Tan, Rights, duties commitments between agents, in Proceedings of the IJCAI 99, pp , (1999). [17] R.J. Wieringa J.-J.Ch. Meyer, Actors, actions, initiative in normative system specificati on, Annals of Mathematics Artificial Intelligence, 7, 289 6, (199).