Multi-Strategy Selection Model for Automated Negotiation

Transcription

1 th Hawaii International Conference on System Science Multi-Strategy Selection Model for Automated Negotiation Mukun Cao School of Management, Xiamen University, Xiamen, , China Xiaopei Dai School of Management, Xiamen University, Xiamen, , China Abstract Automated negotiation has played an important role in supporting the dynamic trading based e- commerce Research in automated negotiation, especially for computer-computer negotiation pays little attention on the implementation related issues such as multi-strategy selection, which will be very useful for the computer-human negotiation The strategy selection is very important for negotiating agent to achieve better negotiation outcomes The lack of such study has slowed down the process of applying automated negotiation to real world problems To address the issue, this paper develops a multi-strategy selection model grounded on the integration of the time-dependent and behavior-dependent tactics, the multi-strategy selection theoretical model and algorithm is proposed to meet the following three goals: supporting multi-strategy selection, easy to be implemented and less resources consuming To demonstrate the effectiveness of the proposed model and algorithm, a prototype of the model is built, and a lot of experiments are conducted to demonstrate the effectiveness of the model 1 Introduction Nowadays, the dynamic trade based on negotiation has gradually become the mainstream of e-commerce [1-3] Negotiation is a joint decision-making process by parties with conflicting interests or preferences to reach an agreement or compromise The research on e- commerce oriented negotiation can be divided into three dimensions: human-human negotiation, computer-computer negotiation and computer-human negotiation The human-human negotiations are often inefficient and ineffective in reaching optimal outcomes due to cognitive biases, limited information processing capacity and capability, and socioemotional obstacles [4] This has motivated the design of Negotiation Support System (NSS) to facilitate human negotiators in reaching better outcomes using decision and communication support tools Owing to the rapid growth of global e-markets, there has been a significant interest in designing software agents that can serve as surrogates for human business decisionmakers to construct Automated Negotiation System (ANS), in which software agents are designed to autonomously act on behalf of the real-world parties ANS composed of computational agents from different individuals or organizations that are capable of reaching agreement through negotiation is becoming more and more important and pervasive The computer-computer negotiation and computerhuman negotiation are the two forms of the ANS However, currently the study of automated negotiation system primarily focus on an computer-computer negotiation context, while there is little research on computer-human negotiations[5]although there have been vast researches regarding the negotiation strategy and protocols for constructing a computer-computer negotiation system, few such systems can be found in the practical negotiation applications One important reason is that the human involvement decision-making is still required in the most present online negotiation situations As a result, most computer-computer ANS was developed as experimental platforms for validate the designed negotiation protocols or strategy algorithms In fact, with the growth of e-commerce and e- markets, there is an increasing potential for the use of software agents to negotiate business tasks with human negotiators [4] The computer-human negotiation plays more and more important role in the nowadays e- commerce oriented applications, especially in the B2C context, in which the agent act as business or products provider and human act as the consumer [6] Comparing with the traditional online sales mode, in which the customers can view the basic product or service information on the website, but often need to negotiate with human salespeople through a contact us link, the computer-human automated negotiation system can help the company to cut the salespeople s labor-cost for negotiation and increase the transaction efficiency to the maximum extent Owing to the /14 $ IEEE DOI /HICSS

2 randomness of the human s negotiation behavior, the computer-human context is a more complicate negotiation situation than computer-computer context Therefore, computer-human negotiation system needs much smarter software agents to support the automated negotiation with the human negotiators effectively In such cases, the ability to autonomously select appropriate strategy among the candidates to cope with the current negotiation situation is a very important perspective for evaluating the agent s intelligence level The selection of an appropriate initial strategy is a critical step in preparing for negotiation [7] Effective negotiators often make a conscious analysis of the negotiation situation and the opposing parties, and actively prepare initial strategies that match their judgment They also update their judgment as negotiation unfolds, information received during the negotiation frequently causes them to revise their beliefs of the negotiation situation and the other parties [8] Hence, effective negotiators may move back and forth among different strategies in discernible patterns In automated negotiation, people entrust the software agent to negotiate automatically on line, and normally hope the agent can try different strategies to get a better negotiation outcome People will make the final decision for accepting the negotiation result returned by the agent or not Despite the importance of strategy selection in automated negotiation, existing AI research has ignored many issues related to strategic choices [8], and hence has hindered the development of the real-world applications of the system [9]The main objective of this research is to construct a generic decision-making model based on an agent architecture, which can support multi-strategy selection during the automated negotiation The remainder of this paper is organized as follows Section 2 reviews related work on the negotiation strategy of the negotiating agent Section 3proposes the multi-strategy selection theoretical model in mathematical form, and presents the multi-strategy selection algorithm based on the theoretical model Section 4is the experimental evaluation for the model The paper is concluded in Section 5 with the summary of findings and suggestion for future research directions 2 Related work on negotiation strategy A negotiation strategy is a decision-making model, used by the participants to achieve their purposes [10] In order to be more successful, an agent needs to adapt to the behavior of its partners and changing environment Effective mechanism should allow an agent to learn about its partner during the current encounter from the previous offers in order to predict the partner s future behavior and adapt to it [11] There have been vast literatures about enhancing the agents with techniques and enabling them to predict their opponents negotiation behavior based on learning from previous negotiations, and thus achieves more profitable results and better resource utilization The main solutions that were adopted for the inclusion of a learning capability into a multi-agent system are the reinforcement learning with a special emphasize on Q- learning, Bayesian learning and case-based-reasoning [12]and evolutionary computation [11, 13] The main point of these negotiation methods is prediction, but it cannot be ignored that in most cases prediction is not reliable, in other words, the result of prediction always has more or less deviation Take the work of [12] as an example, they use feed forward artificial neural network technology to create negotiation model, which forecast the opponent s next price proposal based on its past three price proposals This prediction would be pretty effective and relatively accurate as the curve of the price proposal is regular and smooth, but near the inflexion point of the curve, the prediction would be very hard and unreliable, while the area near the inflexion point is the critical area of the negotiation According to time-dependent Boulware tactics proposed by Faratin [14], as Figure 1 shows, when the price proposal curve happens to turn down, that means that the negotiation is approaching the deadline, where is the most negotiation reach or not reach the agreement In addition, if the negotiation happens in computer-human context, predicting human s behavior would be even more difficult, because the human s offer is more random, and always do not comply with a fixed offer function or curve However, the development direction of the e- commerce in the future is the computer-human negotiation as we discussed before, we must confront and find solutions for this situation in advance Of course, we can also find some studies have relatively high accurate rate of the prediction, such as [15] proposed learning techniques based on MLP (Multi Layer Perception) and GR (Generalized Regression) neural networks (NNs) that are used mainly to detect, at an early stage, the cases where agreements are not achievable, supporting the decision of the agents to withdraw or not from the specific negotiation thread The method, however, needs at least 30 rounds of the opponent s history negotiation offers to train the NNs model, and the predictions of the opponent s future offer are generated after the round 30 of the negotiation Nevertheless, as show in our latter experiments, the most negotiations terminate within 30 rounds, whatever deal or not In other words, such intelligent methods consume much computational 251

3 resource to make the prediction, whereas we need a more economical way to resolve the problem On the other hand, the main idea of automated negotiation is enabling the software agent to learn the human being s method to negotiate automatically on behalf of its entrusting party However we people do not usually surmise the opponent s next offer in real world negotiation, while we always observe the opponent s behavior (including offer, words, action, expression and so on) to collect enough information first, and then make the next decision based on the previous judgments During this process, imitating the opponent s negotiation behavior is the most conventional method, just as Faratin and other scholars summarized [14] As a result, we consider in our work, the intelligent method for the agent to enhancing its capability of learning is not to predict the opponent s behavior, but to timely adjust its offer strategy along with the opponent s changing proposal That is the basic idea of the multi-strategy selection So, the question then is: how to realize the idea in negotiating agent systems? Based on Faratin s work on time-dependent and behavior-dependent negotiation tactics, Literature [14] considers that negotiators behavior are influenced by a mixture of two factors: imitation and time, they propose to construct separate predictions for estimating to what extent, which is indicated as value of weight, the agent uses time-dependent or behavior-dependent tactics, and mix them together using the weights obtained before to predict the negotiation partner s behavior from the current encounter Some other researchers have tried combing different negotiation tactics to make a novel negotiation strategy For example, Raquel et al design a negotiation metastrategy that combining trade-off and concession moves, demonstrating by experimental analysis that the combination of different negotiation tactics allows agents to improve the negotiation process and as a result, to obtain more satisfactory agreements [16] Enlightened from these studies, we found that, in the ongoing negotiation, there would be a way to design a method for multi-strategy selection through combining the behavior-dependent and time-dependent strategy to take into account both the opponent s negotiation history and the time factor of itself The timedependent and behavior-dependent tactics should be applied in combination with each other but not alone In conclusion, in our work, we want to design a new negotiation decision-making model containing the following three advantages First, the novel negotiation model can support agent autonomously selecting suitable one among the strategy candidates to deal with the changing negotiation situation Second, the novel negotiation model enables the agent has the ability to imitate the opponent s offer behavior, while following the time-dependent However, in our method, we do not want to let the agent to learn the opponent s concession pace, but to learn the concession rate made by the partner, which is different from most of the past research Third, the novel model should be easy to be implemented and consume less computational resources (ie, only requires a few negotiation rounds, comparing to the agent s deadline expiration round, to collect the data needed by the selection algorithm) This is the main reason why this technique turns out to be extremely useful 3 The Multi-Strategy Selection Model This section presents our method for strategy selection 31 The Negotiation Strategy Before we present our negotiation strategy selection model, some optional strategies should be prepared in advance The proposed negotiating agent s architecture and the BDI reasoning process model are general enough to handle different negotiation strategies and decision models As a representative work, Faratin et al [14] presented a model for a bilateral service-oriented negotiation that defines a range of strategies (functions that map a matrix of real numbers ranging from zero to one into another similar matrix) and three groups of concession tactics: (i) time dependent (functions of time), (ii) resource dependent (functions of limited resources), and (iii)behavior dependent or imitative (functions of the opponent s behavior) Time dependent tactics model the fact that the agent is likely to concede more rapidly as the deadline approaches, if an agent has a deadline by which an agreement must be in place The shape of the curve of concession, a function depending on time, is what differentiates tactics in this set They distinguished two families of functions: polynomial and exponential Due to the similarity with the human s negotiation behavior, we take the exponential function as an instance The function is actually a family of time dependent functions, which can be defined simply by varying the way choosing the value of its parameters As depicted in Figure 1, the function family is parameterized by that determines the convexity degree of the curve This family of functions represents an infinite number of possible tactics, one for each value of There are two extreme sets showing clearly different patterns of behaviour: Boulware and Conceder The Boulware tactics, discriminated by 252

4 , maintain the offered value until the time is almost exhausted, whereupon they concede up to the reservation value The Conceder tactics, discriminated by, lead the agent goes quickly to its reservation value As can be seen from figure 1, the exponential time dependent function simulates the human s negotiation behavior Take, and as three examples For, which is a Boulware tactic, the curve represents one conservative negotiation behavior, which concedes slowly at the beginning of the negotiation (see Fig1), and concedes quickly to the reservation price as the deadline approaches For, which is a Conceder tactic, the curve represents one radical negotiation behavior, which concedes quickly at the beginning of the negotiation, and gradually approaches to its reservation price in the later process The curve with represents the intermediate state between Boulware and Conceder As there are infinite proposal curves (corresponding to infinite values of ) included in the solution space in figure 1, theoretically speaking, the model covers the entire possible proposal curves the human being might choose during the process of the negotiation Figure 1: The exponential functions for the computation of Time is presented as relative to [14] The function can be expressed formally as follows: α j a (t) = exp (1 a min(t,t max a t max ) ) β a ln K j 0 α j a (t) 1, (1) where is the agent s name, denotes the negotiation issue, is time that is predominant factor used to decide which value to offer next, is a constant that represents the time by which agent must have completed the negotiation, and is a constant that when multiplied by the size of the interval, determines the value of issue to be offered in the first proposal by agent So we have and The behavior dependent or imitative tactic models the fact that the negotiator s counter offer depends on the behavior of the negotiation opponent For example, the negotiator concedes $5 just because the opponent conceded $5 in the last round This happens in the situations in which the agent is not under a great deal of pressure to reach an agreement, it may choose to use imitative tactics to protect itself from being exploited by other agents In this case, the tactics in this family differ in accordance with the aspect of their opponent's behavior they imitate, and to the degree the opponent's behavior are imitated We apply the time-dependent and behaviordependent tactic as the benchmark For the imitation, we will not simply make the agent imitate the opponent s concession, but imitate the opponent s concession rate, which is the ratio between the two neighboring concessions That will be explained in detail in the later discussion 32 The Selection Model In the subsection, we propose an innovative method for multi negotiation strategy selection, mainly inherits the concession tactics We believe that negotiation is a repetitive process, which final purpose is to help the negotiators to find a compromise solution quickly During this process, the negotiator must keep learning its counterpart s negotiation behaviors, and then adjusting its current strategy to a proper one at a proper time to respond the opponent s possible price changes With the opaque of both negotiators strategies and something that affect negotiation processes, we can only conjecture, imitate and adjust through the offer prices that we can see That is the main idea of the strategy selection Before we propose the formal model, there are two basic concepts need to be introduced first Definition1 A concession is the difference between the agent s two neighboring offer prices, which can be expressed formally from the seller s perspective as: t x s b x s b (2) Definition 2 The concession rate, denoted as, is the ratio between the two neighboring concessions, which can be expressed formally from the seller s perspective as: θ = x t s b x s b x s b t 2 x s b (3) where means the price offered by seller to buyer at time Although both the buyer and seller can change their strategy during the negotiation, in order to clearly describe the agent s changing process of the strategy, we assume the seller keeps its negotiation strategy 253

5 unchanged from the beginning to the end The buyer can change its negotiation strategy according to the seller s negotiation behavior in the process of the negotiation Complying with the contract net protocol, the seller initiates the negotiation process and offers first In the first three round of the bargaining, the seller agent and the buyer agent offer their own prices according to their initial negotiation strategy When the seller offers its third price, the buyer can calculate the seller s concession rate of the first three round by formula (3), and based on the value of the, begin to make the decision whether to change the strategy or not If the buyer decides to change the strategy, it will offer its third price complying with the new strategy In accordance with the same pattern, the buyer keeps on collecting seller s last three bid prices in subsequent each rounds of negotiation to get the seller s for strategy selection decision There are two situations for : and If, which means the seller agent keeps a steady concession rate, or the seller made same concessions between the last two neighbouring offers, the buyer agent will decide simply to keep the current strategy, rather than changing If, which means that comparing with the last round, the seller agent s concession of the current round has changed to some extent, increase ( ) or decrease ( ), the buyer will decide to change the strategy to respond to the opponent For, which means the seller agent accelerates concession, according to the time dependent tactic model, we can imagine that the seller agent is accelerating to approach its deadline In order to guarantee the negotiation can finally reach an agreement, the buyer agent has to adjust its strategy to cater to the seller We let the buyer learn (or imitate) the seller s concession rate, from which deduce the buyer s new strategy function For, which means the seller agent decelerates concession, according to the time dependent tactic model, this kind of situation takes place when the agent makes big concession at the beginning of the negotiation, and then gradually decreases concession to approach the reservation price, and finally terminates offer at the deadline Under this circumstance, the buyer agent will take as its concession rate, from which deduce the new strategy function The reason for the agent learns the instead of is because, which can enable the agent to develop more conceder strategy to cater to the seller s fast concession and reach an agreement quickly That is proved by the following experiments in Section 4 As designed in the time dependent tactic model, the parameter of solely determines the curve s shape of the negotiation strategy As different negotiation strategy corresponds to different, the multi-strategy selection is in essence to select appropriate among all the s The following discusses the deduction process for how to get the value of from the known concession rate of Firstly, the buyer agent calculates concession rate of the seller by formula (3) Secondly, in order to get the offer for current round of negotiation, the buyer imitates the seller s concession rate For, we have θ = x t b s x b s (4) Please note the difference between formulas (3) and (4): for (3), for (4) For, we have t x a b 1 θ = x t b s x b s x b s t 2 x b s x b s t 2 x b s From Faratin s work [14], we know that min a j +α a j (t)(max a j min b j ) [ j] = min a j + (1 α a j (t))(max a j min a j ) if V j a is decreasing if V j a is increasing, where is the vector of values for issue proposed by agent to agent at time, the range of the values acceptable to agent for issue is represented as the interval of ], and is a scoring function giving the score agent assigns to a value of issue in the range of its acceptable values Since only the price is considered as the single issue in the negotiation we concern, and only the buyer s offer price, which is monotonically increasing, is taken into account, we can simplify formula (6) as follows: t x (7) b s = min b + (1 α b (t))(max b min b ) The above formula means the agent s offer price at the current time is a point on the strategy curve, which is solely determined by in formula (10) As a result, the subsequent work is to solve, (5) (6) 254

6 and then proceeds to get the agent s offer price at time In the same way of, we can get and By substituting, and into formula (4) and (5), and then we obtain (θ +1)α b (t 1) θα b (t 2) α b (t) = ( 1 θ +1)α b (t 1) 1 θ α b (t 2) From formula (7), we obtain α b (t 1) = maxb x b s if θ >1 if θ <1 (8) (9) max b min b Similarly, we can get Then, in formula (11), all the items are known except Thus we can calculate it as a specific value of Integrating with formula (1) for the value of, we get A = exp (1 b min(t,t max b t max (10) From (10), we obtain the new value of, by which the buyer agent finds a new strategy curve to respond to the seller, as follows: (11) Then, we can get the buyer s new negotiation strategy function and curve, which will lead the agent s subsequent negotiation, as follows: b (12) x b s = min b + (1 exp (1 min(t,t max ) ) β ln K b b t max )(max b min b ), Where, is independent time variable and is the dependent offer price variable At the same time, we get the buyer s new price for the negotiation at time ) ) β ln K b ln( ln A lnk ) b β = ln(1 min(t,t b max) ) b t max t x b s = min b + (1 A)(max b min b ) 4 Experimental Evaluation of the Multistrategy Selection Model (13) In this section, we will do lots of experiments to evaluate our model of negotiation strategy selection 41 Environments and Tactics Environments are characterized by some attributes described as follows: the number of agents they contain, the issues that are being discussed, the deadlines by when agreements must be reached, and the expectations of the agents Since there are infinitely many potential environments, we selected a representative environment in which we can assess an agent's negotiation performance, with the aim to test our strategy selection algorithm In the same negotiation environment, if agent who takes the strategy selection algorithm can generate a higher success rate and obtain better negotiation results than who does not, then the effectiveness and correctness of our strategy selection algorithm can be proved To this end, we set one simplification, under which the experiment is limited to a bilateral negotiation between a single buyer and seller over the single issue of price The buyer is a negotiation agent with the strategy selection ability, whose initial price offer is 5, and reservation price is 80 The seller represents a human negotiator who negotiates along with a pre-set strategy with the initial price offer 115, and the reservation price 40 Given this situation, the experimental environment is uniquely defined by the following variable: (14) ie, the amount of time available to make an agreement ( ), the relative value of the initial offer ( ), and price interval of the buyer and seller, respectively In the following experiments, we fixed = Referring from Faratin s work [14], since high value of over constrain the true behavior of tactics, we set for both agents Then several groups of experiments will be conducted according to different range of, and More specifically for and, we define an environment that their values are randomly generated in the interval of [3,100] The reason the max trading time starts from 3 is because the agents must bargain three times at least to guarantee the designed selection algorithm can collect enough data to proceed Note that there are 3 different relationships between and, ie,, and, under which the different negotiation results will be generated and discussed in the following experiments Another simplification involves selecting a finite range of tactics since the model allows for an infinite 255

7 set (eg, the range of is infinite, which means there are infinitely many time-dependent tactics) For analytical tractability, we follow the Faratin s set, in which defines Boulware tactic, and defines Conceder tactic accordingly [14] At the very beginning, the seller chooses the value of randomly to set its strategy curve, and then fixes its strategy at this value of until the end of this negotiation Meanwhile, the buyer runs the strategy selection algorithm to choose an appropriate strategy among different strategy curves with different values of to response the seller s offer In addition, the experiments tacitly approve that the buyer takes Boulware tactic (ie, ) as its default initial strategy to deal with possible offer that the seller would make This can be explained from two sides On the one hand, if the seller takes Boulware tactic as its strategy, the Boulware tactic is intuitively a more secure choice for the buyer, as it can help the buyer to ensure not to lose much profit On the other hand, if the seller takes Conceder tactic as its strategy, according to the strategy selection algorithm, buyer will adjust its tactics from Boulware to Conceder tactic to make sure the negotiation can make a deal in the end, though the buyer takes Boulware tactic at the beginning of the negotiation As a result, the buyer takes the Boulware as its initial negotiation strategy is a reasonable choice This will be further verified by experiment in the following hypothesis 2 42 Experimental evaluation measurement To evaluate and compare the effectiveness of the algorithm, we propose two measures to evaluate our experiments: Success Rate (SR): Whether an automated negotiation system can enable most of the negotiation runs on it to reach agreement is an important evaluation factor for the system Therefore, we utilize success rate as one of the evaluation measure to test the validation of the multi-strategy selection algorithm for all the negotiation experiments We conduct times of experiments, of which times experiments reach a deal, and then the success rate is calculated as follows: (17) Distance to Win-Win Point (DWWP): The winwin point represents a solution simultaneously conforming to the indices of efficiency and equality So, the distance between the settled point and the winwin point in the settlement space can be used to evaluate whether the settlement is efficient and fair or not The smaller the distance to the win-win point is, the more efficient and fairer the settlement is As the experiment we are conducting is a bilateral negotiation between a buyer and a seller over the single issue of price, the feasible solution s interval is the intersection between the seller s minimum acceptable price and buyer s maximum acceptable price, ie, Then, the middle point ( ) of this interval is the win-win point of this sort of negotiation Then, we have the formula of the distance: (20) Xdistan ce = Xsettle point Xwin win point In summary, SR and DWWP describe the overall effect of the negotiation, while IU indicates the impact of the final negotiation result on the unilateral agent 43 Experimental hypotheses, procedure and discussions In this section, we conduct the experiments to compare the relevant evaluation measurements between strategy-selection and no-strategy-selection 431 Experimental hypotheses The hypothesizes for the experiments are as follows: Hypothesis 1: Agent adopts the strategy-selection algorithm can achieve a higher success rate than those who adopt no-strategy-selection algorithm Hypothesis 2: Under the situations that one side s negotiation strategy is unknown to its opponent, the other side takes Boulware tactic would be a better choice because it can get a better intrinsic utility, and its overall effect, which is indicated by the success rate (SR) and distance to win-win point (DWWP), of the negotiation is better as well 432 Experimental discussion Through the comparison experiments, we can get the following Table1 Consider hypothesis 1 first, the experiments results show that the adoption of strategy-selection can truly improve the success rate of the negotiation under the situation of other experiment environments are set to be consistent Especially, when the seller adopts Conceder strategy, if the buyer chooses strategyselection algorithm, we can see an obvious enhancement in success rate The experimental results support the hypothesis Moving onto hypothesis 2, it can be divided into two situations as follows: When buyer adopts no-selection strategy, which is ambiguous to the seller, as the data shows in Table1, compared with the situation that the seller adopts Conceder tactics (1<β 50), if the seller adopts Boulware tactics (0<β<1), the experiments can reach a 256

8 higher success rate (except the situation of, that is because the buyer is more eager to make a deal, so that bring about a higher success rate), and better distance to win-win point The experimental results fully support the hypothesis When the buyer adopts strategy selection mechanism to negotiate with the seller, according to the experimental data as shown in Table 1, we can find out that, comparing with the seller who adopts Boulware tactics, the seller who adopts Conceder tactics can get a higher success rate, but the distance to win-win point is worse This implies the conceder tactic can help the seller agent get more agreement for it is an aggressive strategy sacrificing the agent s utility as compensation When we take all the factors into account, in order to get a better negotiation outcome for both sides, the Boulware tactic is a better choice for the seller Buyer s Negotiation Strategy No-selection 0<β<1 Selection 0<β<1 Table 1: Comparison of the Experiment Results Seller s β t Success rate (SR) 0<β<1 (Boulware) 1<β 50 (Conceder) 0<β<1 (Boulware) 1<β 50 (Conceder) Mean of distance to win-win point (DWWP) 125% % % % % % % % % % % % The typical situations of the experiments are shown in figure 2, in which the blue lines denotes buyer's each selected strategy curve, and the red line denotes buyer s final strategy curve which leads buyer to reach an agreement with seller The black-cross point indicates the offers of both sides and the red-cross point indicates the final agreement point The figure is divided into three columns based on the relations between and The first line figures (a, b and c) depict the negotiation situations that the seller adopts Boulware tactic, while second line figures (d, e and f) depict the negotiation situations that the seller adopts Conceder tactic Taking the figure (a) and (d) as two examples, we can clearly see the buyer s strategy selection process The buyer s initial negotiation strategy curves are the nethermost one, which certainly cannot reach agreement with the seller because the expected agreement point is beneath the seller s reservation price The buyer negotiates with the seller along its initial strategy curve for the first two offers till it receives the seller s third offer The buyer then judges whether to change its negotiation strategy according to the selection model Then the negotiation proceeds to the process of changing strategy Every time the buyer changes its strategy, it can get a new offer against its opponent Finally they reach an agreement that both sides can accept in the area of the feasible solutions From the figures we can see that, without the strategy selection, the seller and buyer obviously cannot make a deal in most cases That is the reason why the selection model can improve the success rate of the negotiation Through the above analysis, the experimental results basically support hypothesis 2 The results suggest that, when we set the initial negotiation strategy for a developed automated negotiation system, the Boulware tactics should be considered firstly because it is a more safe strategy 5 Conclusions and Future Research Directions In this study, we present a negotiating agent decision-making model, which can support the multistrategy selection in automated negotiation context The proposed multi-strategy selection model gives the agent more autonomous ability to cope with dynamically changing negotiation situations, without any effect from the outer environment By so doing, the agent can select appropriate negotiation strategy properly and implement the decision making process autonomously, enable the agent has the ability to imitate the opponent s offer behavior, while following the time-dependent Besides, the novel model is easy to be implemented and consume less computational resources 257

9 However, there are a number of issues need to be further addressed The aim for studying the automated negotiation systems is to find a way to put the theoretical results into practical applications We intend to evolve the current computer-computer negotiation test bed, which is based on a BDI theory complied agent software development kit JADEX, into a computer-human negotiation experimental platform, on which we can conduct some computer-human negotiation experiments to test whether our multistrategy selection algorithm can handle the human partner s random negotiation proposal More importantly, this work will provide us with the critical test data and experiences for developing and applying the computer-human negotiation application system, which is anticipated to be widely used in the future B2C e-commerce Based on the current empirical studies on the performance of the multi-strategy selection mechanism, when the computer-human negotiation system is under design and development, at least two principles should be followed: (i) no matter what kind of roles the agent acts in a computer-human negotiation context (seller that represents an on-line company to provide products or services, or a human buyer), the strategy selection mechanism should be employed for a better negotiation result (ii) The agent employs the Boulware tactic as its initial negotiation strategy regardless of the human choice (Boulware or Conceder), or any other random proposal curves In conclusion, we believe the proposed model can fill the gap between the theoretical development and the practical implementation of the negotiation support model The work of this paper can help to build the foundation for developing a practical automated negotiation system (a) (b) (c) (d) (e) (f) Figure 2: typical negotiation situation in case of buyer taking selection strategy 258

10 ACKNOWLEDGEMENTS This work is supported by the National Natural Science Foundation of China (No ), and by the Fundamental Research Funds for the Central Universities (No ) REFERENCES [1] Luo, X, Miao, C, Jennings, NR, He, M, Shen, Z, and Zhang, M, "Kemnad: A Knowledge Engineering Methodology for Negotiating Agent Development", Computational Intelligence, 28(1), 2012, pp [2] Baarslag, T, Fujita, K, Gerding, EH, Hindriks, K, Ito, T, Jennings, NR, Jonker, C, Kraus, S, Lin, R, and Robu, V, "Evaluating Practical Negotiating Agents: Results and Analysis of the 2011 International Competition", Artificial Intelligence, 198(5), 2013, pp [3] De La Hoz, E, López-Carmona, MA, and Marsá- Maestre, I, "Trends in Multiagent Negotiation: From Bilateral Bargaining to Consensus Policies": Agreement Technologies, Springer, 2013, pp [4] Yinping Yang, SS, And Yunjie (Calvin) Xu, "Alternate Strategies for a Win-Win Seeking Agent in Agent Human Negotiations", Journal of Management Information Systems, 29(3), 2013, pp [5] Lin, R, and Kraus, S, "Can Automated Agents Proficiently Negotiate with Humans?", Communications of the ACM, 53(1), 2010, pp [6] Bosse, T, and Jonker, CM, "Human Vs Computer Behavior in Multi-Issue Negotiation", in (Editor, 'ed'^'eds'): Book Human Vs Computer Behavior in Multi-Issue Negotiation, IEEE, 2005, pp [7] Luo, X, Jennings, NR, and Shadbolt, N, "Acquiring User Tradeoff Strategies and Preferences for Negotiating Agents: A Default-Then-Adjust Method", International Journal of Human-Computer Studies, 64(4), 2006, pp [8] Lopes, F, Wooldridge, M, and Novais, AQ, "Negotiation among Autonomous Computational Agents: Principles, Analysis and Challenges", Artificial Intelligence Review, 29(1), 2008, pp 1-44 [9] Lin, K-J, "E-Commerce Technology: Back to a Prominent Future", Internet Computing, IEEE, 12(1), 2008, pp [10] Skylogiannis, T, Antoniou, G, Bassiliades, N, Governatori, G, and Bikakis, A, "Dr-Negotiate a System for Automated Agent Negotiation with Defeasible Logic- Based Strategies", Data & Knowledge Engineering, 63(2), 2007, pp [11] Brzostowski, J, and Kowalczyk, R, "Modelling Partner s Behaviour in Agent Negotiation": Ai 2005: Advances in Artificial Intelligence, Springer, 2005, pp [12] Oprea, M, "An Adaptive Negotiation Model for Agent- Based Electronic Commerce", Studies in informatics and Control, 11(3), 2002, pp [13] Pan, L, Luo, X, Meng, X, Miao, C, He, M, and Guo, X, "A TwoStage WinWin Multiattribute Negotiation Model: Optimization and Then Concession", Computational Intelligence, 2012 [14] Faratin, P, Sierra, C, and Jennings, NR, "Negotiation Decision Functions for Autonomous Agents", Robotics and Autonomous Systems, 24(3), 1998, pp [15] Roussaki, I, Papaioannou, I, and Anangostou, M, "Building Automated Negotiation Strategies Enhanced by Mlp and Gr Neural Networks for Opponent Agent Behaviour Prognosis": Computational and Ambient Intelligence, Springer, 2007, pp [16] Ros, R, and Sierra, C, "A Negotiation Meta Strategy Combining Trade-Off and Concession Moves", Autonomous Agents and Multi-Agent Systems, 12(2), 2006, pp