The Wisdom of Networked and Evolving Agents

Transcription

1 The Wisdom of Networked and Evolving Agents How mutual beneficial rules can be built and sustained without knowing the motives of others, and without sharing goals. Akira Namatame National Defense Academy,Japan Artificial Economics 06 (Aalborg)

2 Social Intelligence <Intelligence is valuable in social contexts>. Economic theory presupposes rationality that rely on the existence of intelligence, but why would such intelligence has evolved? Two hypotheses have been considered as explanations for this question. (1) The ecological hypothesis: The demands of the non-human environment were a key impetus to higher intelligence. (2) The social intelligence hypothesis: A key factor is interactions with other human beings. :The Red Queen effect (co-evolution) in biology was introduced to explain why species have been evolved to higher intelligence. 1 - Artificial Economics 06 (Aalborg)

3 Basic Issues [1] (1) Why do we need to focus on interactions? Global stability despite many local interactions It is remarkable that in almost all cases interactions play a vital role, yet many aggregate outcomes are not changed too seriously by them. (Philip W. Anderson: in his Nobel Prize lecture, 1977) (2) Endless and meaningless co-evolution? The Red Queen principle (arms race) The more things change, the more they stay the same <van Valen (evolutionary biologists), 1973> 2 - Artificial Economics 06 (Aalborg)

4 Flocking Behaviors What is the mechanism behind flocking behavior? 3 - Artificial Economics 06 (Aalborg)

5 Relationship Rather Than Rationality BOID Model: Craig W. Reynolds (1994) <Frocking Behavior consists of three relational rules> Cohesion: Head for the perceived center of mass of the neighbors. Separation: Don't get too close to any object. Alignment: Try to match the speed and direction of nearby neighbors. Cohesion Separation Alignment 4 - Artificial Economics 06 (Aalborg)

6 Basic Issues [2] Self-interest vs. Mutual Benefit Self-interested agents are often faced with the dilemma of acting in their own interest or pursuing a more cooperative action in social contexts. The prevailing orthodoxy in the evolutionary theory is that the better individual (or gene) is the unit of selection. :Group solidarity would work for a desirable outcome for mutual benefit, but evolutionary dynamics based on natural selection can work against it. A challenging task is to identify conditions under which agents are more coordinated than they are acting in their own interest. 5 - Artificial Economics 06 (Aalborg)

7 Madness of Crowds vs. Wisdom of Crowds (1) Researches on collectives of agents Social sciences: <Crowd psychology> herding, cascade, group think, Game theory: <the problem of a coordination failure> The existence of externalities lead to coordination failure and results in Pareto-inferior outcomes. Computer science: <the price of anarchy> Selfish behavior may not achieve full efficiency at the collective level. 6 - Artificial Economics 06 (Aalborg)

8 The Price of Anarchy (1) Coordination versus competition Equilibria of selfish agents are not efficient Pareto optimal is usually not Nash equilibrium Price of anarchy: Quantify inefficiency in terms of a global objective Price of Anarchy = objective function value at NEP optimal objective function value 7 - Artificial Economics 06 (Aalborg)

9 The Price of Anarchy (2) Agent A Agent B Pure strategy Nash equilibria include (H,H) and (M, M), but not (L,L) The efficient outcome is (L, L) H M L A modified coordination game H 5,5 M 0,0 L 12,0 0,0 2,2 0,0 0,12 0,0 10,10 Price of Anarchy=5(or 2)/10=1/2 (or1/5) -How can we improve the price of anarchy without the central authority? -What mechanisms one can use to improve the overall system performance? 8 - Artificial Economics 06 (Aalborg)

10 Madness of Crowds vs. Wisdom of Crowds (2) The Wisdom Of Crowds, Surowiecki, Random House (2004) A large collection of people are smarter than an elite few. (Example: Wikipedia) Surowiecki suggests new insights regarding how our social and economic activities should be organized. The wisdom of crowds emerges only under the right conditions, which are : diversity, : independence, : decentralization, and : aggregation. Network topology of interactions also play a vital role. 9 - Artificial Economics 06 (Aalborg)

11 Concept of Social Games Networked agents repeated play pair-wise games: Coordination game Nash demand game Rock-scissors-paper game Agents evolve their strategies Best-response strategy learning has been the focus of much research. However, this approach can yield undesirable outcomes. We seek a proper learning model leading agents to desirable outcomes Artificial Economics 06 (Aalborg)

12 Effects of Network Topology It is important to interact with the right peoples. It is not what you know, it s who you know that account. Payoff: Internal enforcement Lattice network AGENT Network topology: External enforcement Small-world network: Rewiring with probability p=0.5 Random network: Rewiring with probability p= Artificial Economics 06 (Aalborg)

13 Fixed Rule vs. Learnable Rule Learning: Memory-based strategy choice : Memory A t (strategy at time t) Is a short memory enough or a longer memory may be necessary in more complex games? : Reactionary strategy with history one: ( h t-1 ) A t : Finite history of k: { ( h t-k,, h t-2, h t-1 ) } A t Agents play the game repeatedly by evolving a rule rather than by a fixed rule such as TFT. bit past strategy Own Opp strategy at t # # # # Crossover 12 - Artificial Economics 06 (Aalborg)

14 A Coordinated Attack Problem Agent B S Agent A 1 (attack) S 2 (retreat) S 1 (attack) S 2 (retreat) Coordination Game The inferior strategy S 2 is likely to be selected. The basin of attraction of the retreat is 9 times as large as that of the attack. θ= Agents need to decide to attack or retreat. If they both attack, they win. However, one sided attack would result in losing the battle. (S 1, S 1 ): Pareto-dominates (S 2, S 2 ), (S 2, S 2 ): risk-dominates (S 1, S 1 ) S1 θ=2/3 S2 Pareto-dominance θ=0.1 risk-dominance From the point of view of social welfare this is a shame because agents are better off at the joint attack equilibrium Artificial Economics 06 (Aalborg)

15 Simulation Results: Coordination Game generation 200 generation 300 generation 2000 Lattice network average 0.99 average 1.0 average 0.98 Small-world network average 1.0 average 1.0 average 1.0 Random network average 0.0 average Artificial Economics 06 average (Aalborg) 0.0

16 What did Agents Learn through Co-evolution? bit past strategy Own Opp strategy at t # # # # The rules of 2,500 agents are aggregated into a few rules. S 1 : 0 S 2 : 1 Array location 1: Initial strategy 2: Choice when outcome is (0, 0) 3: Choice when outcome is (0, 1) 4: Choice when outcome is (1, 0) 5: Choice when outcome is (1, 1) random network lattice network small-world network 15 - Artificial Economics 06 (Aalborg)

17 The Phase Diagram of the Plays of Two Agents lattice network small-world network converges to 00 : Pareto-optimal outcome Initial strategy random network converges to 11 :risk-dominat outcome 16 - Artificial Economics 06 (Aalborg)

18 Evolutionary Bargaining: Sharing Game Agent A Agent B H = 7 M = 5 L = 3 The payoff matrix of a sharing game H = 7 0,0 M = 5 0,0 L = 3 7,3 0,0 5,5 5,3 3,7 3,5 3,3 There are three Nash equilibriums in pure strategy with one equitable (M, M), and two inequitable ones (H,L), (L,H) H Best reply: L Belief of agent «i» (Phan, 2005) M Best reply: M Best reply: H L 17 - Artificial Economics 06 (Aalborg)

19 Emergence of Self-reinforcing Classes (Axtell, 2001) Selective interaction: Agents can distinguish each other using tags. Classes formation: Agents behave fair (M) to agents within the same group but behave unfair between groups <Left-hand side> Both black and grey agents are fair within their group (Play M) <Right-hand side>black agents have beliefs in behavior of grey agents, and their best response is to play L (70%). Grey agents have beliefs in behavior of blacks. agents, and their best response is to play H, and therefore unfair outcome (H, L) is realized. H H M Within-group L M (Phan,2005) Between-group L 18 - Artificial Economics 06 (Aalborg)

20 Simulation Results: Sharing Game (i) The average payoff per agent : Highest payoff Average Payoff (ii) the ratio of each strategy S ( M 2 ) S 1( Rock ) H=7 M=5 L= 3 H = 7 0,0 M = 5 0,0 L = 3 7,3 0,0 5,5 5,3 3,7 3,5 3,3 All agents become to choose M and realize both efficient and fair outcome 19 - Artificial Economics 06 (Aalborg)

21 Repeated Games and the General Theory of Efficiency Economics recognizes three general ways to achieve efficiency: Competition: efficient resources allocation via markets Contracts/mechanism design: as a means to enforce efficient outcomes Repeated games: provide a formal framework to examine why selfinterested agents manage to coordinate in a long-term relationship 20 - Artificial Economics 06 (Aalborg)

22 <The Folk Theorem> The Folk Theorem in Repeated Games In a repeated interaction, any mutually beneficial outcome can be sustained in an equilibrium. <Criticism of The Folk Theorem> The theory of repeated games does not provide a criterion for equilibrium selection The majority of works on repeated games are symmetric games. The theory of repeated games is rather silent for asymmetric situations Artificial Economics 06 (Aalborg)

23 Generalized Rock-Scissors-Paper Games λ=2: Conventional R-S-P game λ>2: Dispersion games A Nash equilibrium strategy: (S 1 : 1/3, S 2 : 1/3, S 3 : 1/3) The expected payoff at Nash equilibrium : (λ+1)/3 Pareto-efficiency (average payoff): λ/2 Inequity at Pareto-efficiency : A lucky agent gets λ, and an unlucky agent gets nothing Artificial Economics 06 (Aalborg)

24 Characteristics of RSP Game Group solidarity would work for a Pareto-efficient outcome for mutual benefit, but evolutionary dynamics, which is based on natural selection, can work against it. The RSP game is not stable in the evolutionary dynamics. :However, with slight modification of the payoff of the original RSP game, the Nash equilibrium becomes a stable attractor Artificial Economics 06 (Aalborg)

25 Coupling Rules with Three Strategies Previous Strategy Next Strategy Own Opponent 0 0 # 0 1 # 0 2 # 1 0 # 1 1 # 1 2 # 2 0 # 2 1 # 2 2 # Coupled rule Strategy choice is driven by the outcomes of the joint actions Learnable coupled rule Agents update the coupling rule by changing value of #. 0:Rock, 1: Scissor, 2: Paper #: 0, 1 or Artificial Economics 06 (Aalborg)

26 Simulation Results (1): λ=2 max Error rate: 0% S 2 ( Scissor ) max min average (i) The average payoff per agent Error rate: 10% (ii) the ratio of each strategy max S 2 ( Scissor ) min S 1 ( Rock 25 - Artificial Economics 06 (Aalborg) )

27 What Did Agent Learn through Co-evolution? λ=2 (Error rate: 0%) The rules of 400 agents are aggregated into 67 types Learned coupling rules # # # # 0 # Nash equilibrium: 1 Pareto-efficiency: 1 The phase diagram of the plays between two agents Initial strategy 26 - Artificial Economics 06 (Aalborg)

28 Simulation Results(2): λ=10 : Highest payoff Nash equilibrium: 3.3 Pareto-efficiency: (i) The average payoff per agent Error rate: 0% Average Payoff 2.8 : Lowest Payoff (ii) the ratio of each strategy S 2( Scissor ) S 1( Rock S 3( Paper The strategy population is close to Nash equilibrium ) ) Error rate: 10% 4.0 S 1( Rock ) S 2( Scissor ) S 3( Paper ) 27 - Artificial Economics 06 (Aalborg)

29 What Did Agents Learn through Co-evolution? λ=10(error rate: 10%) The rules of 400 agents are aggregated into 8 rules with some common values Rule Type Number of Agents with the Same Rule Rule Type Rule Type Rule Type Rule Type 5 Rule Type 6 Rule Type 7 Rule Type Artificial Economics 06 (Aalborg)

30 Phase Diagram: Two Agents with the Same Rule If the two agents of having the same rule play, they absorb in the drawing cycle. draw win lose Initial State 11 draw win lose draw win lose 29 - Artificial Economics 06 (Aalborg)

31 Phase Diagram: Agents with the Different Rules If the two agents of having the different rules play, they absorb in the limiting cycle. draw Winning phase win Losing phase lose Initial State 11 draw win Limiting cycle lose draw win lose 30 - Artificial Economics 06 (Aalborg)

32 Emergence of Turn-taking Behavior By visiting all efficient outcomes alternatively under turn-taking behavior, the agents could achieve desirable outcomes in efficiency and fairness Artificial Economics 06 (Aalborg)

33 Concluding Remarks (1): The inverse problem of networked agents <The forward problem> Desired collectives : Analysis,Explanation, Understanding <The inverse problem> :Infer micro behavioral rules of agents : Design micro-rules through co-evolutionary learning Emergent Inverse problem behavior Forward problem Interacting agents with micro-motives 32 - Artificial Economics 06 (Aalborg)

34 Concluding Remarks (2): ABM vs. DE: What are the differences? Agent Based models (ABM) are widespread and many exciting applications, but lots of hype, not enough understanding of when ABM adds value. Replicator dynamics (DE) is the most thoroughly studied dynamic model for evolution with the more successful strategies being imitated more often. What are the differences between ABM and Differential Equations (DE) methods? 33 - Artificial Economics 06 (Aalborg)

35 Rule Rewards Learning Algorithm Meta-strategy Concluding Remarks (2): ABM vs. DE: What are the differences?(cont.) Best-response strategy learning has been the focus of much research. However, this approach can yield undesirable outcomes. Replicator dynamics converge to Nash equilibria which are not always desirable outcomes. Observations, Sensations World, State strategy <Learning relationship> -Agents should learn a better way to be coupled with others -ABM is the only way for the study of relationship 34 - Artificial Economics 06 (Aalborg)

36 Concluding Remarks [3]: Collective Norm Development Social norms are rules that are understood by members of a society, and that guide and constrain their behavior without the force of laws. Emergence and sustainability of social norms as core issues for social sciences. Social norms emerge out of social interaction and they are internalized in networked agents. Social norm Agent Internalization of social norm shared rule Agent preference 35 - Artificial Economics 06 (Aalborg)

37 Open Problems: Smart Agents Using ICT (ICT:Information/Communication Technology) Congestion control receive much attention What wisdom for avoiding congestion will be emerged in networked smart agents equipped with ICT? 36 - Artificial Economics 06 (Aalborg)

38 Open Problems(cont.): Smart Networks Design by Agents Smart Business Networks,Ubiquitous Web of Services, How mutual trust can be built and sustained without knowing the motives of others, and without sharing goals. Smart Networks Design with Genetics Approach 37 - Artificial Economics 06 (Aalborg)

39 New book: World Scientific, Artificial Economics 06 (Aalborg)

40 Thank you!! and Question time 39 - Artificial Economics 06 (Aalborg)