Game theory (Sections 17.5-17.6)
Game theory Game theory deals with systems of interacting agents where the outcome for an agent depends on the actions of all the other agents Applied in sociology, politics, economics, biology, and, of course, AI Agent design: determining the best strategy for a rational agent in a given game Mechanism design: how to set the rules of the game to ensure a desirable outcome
http://www.economist.com/node/21527025
http://www.wired.com/2015/09/facebook-doesnt-make-much-money-couldon-purpose/
Simultaneous single-move games Players must choose their actions at the same time, without knowing what the others will do Form of partial observability Normal form representation: Player 1 0,0 1,-1-1,1 Player 2-1,1 0,0 1,-1 1,-1-1,1 0,0 Payoff matrix (Player 1 s utility is listed first) Is this a zero-sum game?
Prisoner s dilemma Two criminals have been arrested and the police visit them separately If one player testifies against the other and the other refuses, the one who testified goes free and the one who refused gets a 10-year sentence If both players testify against each other, they each get a 5-year sentence If both refuse to testify, they each get a 1-year sentence Bob: Testify Bob: Refuse Alice: Testify Alice: Refuse -5,-5-10,0 0,-10-1,-1
Prisoner s dilemma Alice s reasoning: Suppose Bob testifies. Then I get 5 years if I testify and 10 years if I refuse. So I should testify. Suppose Bob refuses. Then I go free if I testify, and get 1 year if I refuse. So I should testify. Dominant strategy: A strategy whose outcome is better for the player regardless of the strategy chosen by the other player Bob: Testify Bob: Refuse Alice: Testify Alice: Refuse -5,-5-10,0 0,-10-1,-1
Prisoner s dilemma Nash equilibrium: A pair of strategies such that no player can get a bigger payoff by switching strategies, provided the other player sticks with the same strategy (Testify, testify) is a dominant strategy equilibrium Pareto optimal outcome: It is impossible to make one of the players better off without making another one worse off In a non-zero-sum game, a Nash equilibrium is not necessarily Pareto optimal! Bob: Testify Bob: Refuse Alice: Testify Alice: Refuse -5,-5-10,0 0,-10-1,-1
Recall: Multi-player, non-zerosum game 4,3,2 4,3,2 1,5,2 4,3,2 7,4,1 1,5,2 7,7,1
Prisoner s dilemma in real life Price war Arms race Steroid use Diner s dilemma Defect Cooperate Collective action in politics Defect Lose lose Win big lose big Cooperate Lose big win big Win win http://en.wikipedia.org/wiki/prisoner s_dilemma
Prisoner s Dilemma in cartoons http://rall.com/comic/cartoon-for-december-22-2008
Prisoner s Dilemma in cartoons https://xkcd.com/1016/
Prisoner s Dilemma in cartoons http://www.smbc-comics.com/comic/parenting-game-theory
Is there a way out of Prisoner s Dilemma? Iterated Prisoner s Dilemma If the number of rounds is fixed and known in advance, and if your strategy cannot affect your opponent s, the equilibrium strategy is still to defect But if the number of rounds is unknown, and if opponents can respond to each other, things become interesting! Tournaments of iterated Prisoner s Dilemma Video Axelrod s 1980 tournament LessWrong tournaments: 2011, 2014 Training agents by reinforcement learning
Stag hunt Hunter 1: Stag Hunter 1: Hare Hunter 2: Stag Hunter 2: Hare 2,2 1,0 0,1 1,1 Is there a dominant strategy for either player? Is there a Nash equilibrium? (Stag, stag) and (hare, hare) Model for cooperative activity
Prisoner s dilemma vs. stag hunt Prisoner dilemma Stag hunt Cooperate Defect Cooperate Defect Cooperate Win win Win big lose big Cooperate Win big win big Win lose Defect Lose big win big Lose lose Defect Lose win Win win Players can gain by defecting unilaterally Players lose by defecting unilaterally
Game of Chicken Player 1 Player 2 Chicken Straight Chicken Straight S C S -10, -10-1, 1 C 1, -1 0, 0 Is there a dominant strategy for either player? Is there a Nash equilibrium? (Straight, chicken) or (chicken, straight) Anti-coordination game: it is mutually beneficial for the two players to choose different strategies Model of escalated conflict in humans and animals (hawk-dove game) How are the players to decide what to do? Pre-commitment or threats Different roles: the hawk is the territory owner and the dove is the intruder, or vice versa http://en.wikipedia.org/wiki/game_of_chicken
Game of Chicken in the movies Rebel without a cause Footloose
Mixed strategy equilibria Player 1 Player 2 Chicken Straight Chicken Straight S C S -10, -10-1, 1 C 1, -1 0, 0 Mixed strategy: a player chooses between the moves according to a probability distribution Suppose each player chooses S with probability 1/10. Is that a Nash equilibrium? Consider payoffs to P1 while keeping P2 s strategy fixed The payoff of P1 choosing S is (1/10)( 10) + (9/10)1 = 1/10 The payoff of P1 choosing C is (1/10)( 1) + (9/10)0 = 1/10 Can P1 change their strategy to get a better payoff? Same reasoning applies to P2
Finding mixed strategy equilibria P1: Choose S with prob. p P1: Choose C with prob. 1-p P2: Choose S with prob. q P2: Choose C with prob. 1-q -10, -10-1, 1 1, -1 0, 0 Expected payoffs for P1 given P2 s strategy: P1 chooses S: q( 10) +(1 q)1 = 11q + 1 P1 chooses C: q( 1) + (1 q)0 = q In order for P2 s strategy to be part of a Nash equilibrium, P1 has to be indifferent between its two actions: 11q + 1 = q or q = 1/10 Similarly, p = 1/10
Existence of Nash equilibria Any game with a finite set of actions has at least one Nash equilibrium (which may be a mixedstrategy equilibrium) If a player has a dominant strategy, there exists a Nash equilibrium in which the player plays that strategy and the other player plays the best response to that strategy If both players have strictly dominant strategies, there exists a Nash equilibrium in which they play those strategies
Computing Nash equilibria For a two-player zero-sum game, simple linear programming problem For non-zero-sum games, the algorithm has worstcase running time that is exponential in the number of actions For more than two players, and for sequential games, things get pretty hairy
Nash equilibria and rational decisions If a game has a unique Nash equilibrium, it will be adopted if each player is rational and the payoff matrix is accurate doesn t make mistakes in execution is capable of computing the Nash equilibrium believes that a deviation in strategy on their part will not cause the other players to deviate there is common knowledge that all players meet these conditions http://en.wikipedia.org/wiki/nash_equilibrium
Continuous actions: Ultimatum game Alice and Bob are given a sum of money S to divide Alice picks A, the amount she wants to keep for herself Bob picks B, the smallest amount of money he is willing to accept If S A B, Alice gets A and Bob gets S A If S A < B, both players get nothing What is the Nash equilibrium? Alice offers Bob the smallest amount of money he will accept: S A = B Alice and Bob both want to keep the full amount: A = S, B = S (both players get nothing) How would humans behave in this game? If Bob perceives Alice s offer as unfair, Bob will be likely to refuse Is this rational? Maybe Bob gets some positive utility for punishing Alice? http://en.wikipedia.org/wiki/ultimatum_game
Sequential/repeated games: Chain store paradox A monopolist has branches in 20 towns and faces 20 competitors successively Out Competitor In How should the monopolist respond to competitors going in? (1, 5) Monopolist Aggressive Cooperative (0, 0) (2, 2) https://en.wikipedia.org/wiki/chainstore_paradox
Mechanism design (inverse game theory) Assuming that agents pick rational strategies, how should we design the game to achieve a socially desirable outcome? We have multiple agents and a center that collects their choices and determines the outcome
Auctions Goals Maximize revenue to the seller Efficiency: make sure the buyer who values the goods the most gets them Minimize transaction costs for buyer and sellers
Ascending-bid auction What s the optimal strategy for a buyer? Bid until the current bid value exceeds your private value Usually revenue-maximizing and efficient, unless the reserve price is set too low or too high Disadvantages Collusion Lack of competition Has high communication costs
Sealed-bid auction Each buyer makes a single bid and communicates it to the auctioneer, but not to the other bidders Simpler communication More complicated decision-making: the strategy of a buyer depends on what they believe about the other buyers Not necessarily efficient Sealed-bid second-price auction: the winner pays the price of the second-highest bid Let V be your private value and B be the highest bid by any other buyer If V > B, your optimal strategy is to bid above B in particular, bid V If V < B, your optimal strategy is to bid below B in particular, bid V Therefore, your dominant strategy is to bid V This is a truth revealing mechanism
Application: Real-time bidding for Internet ads http://tutorial.computational-advertising.org/final-slides.pdf
Application: Real-time bidding for Internet ads https://en.wikipedia.org/wiki/real-time_bidding
Dollar auction A dollar bill is auctioned off to the highest bidder, but the second-highest bidder has to pay the amount of his last bid Player 1 bids 1 cent Player 2 bids 2 cents Player 2 bids 98 cents Player 1 bids 99 cents If Player 2 passes, he loses 98 cents, if he bids $1, he might still come out even So Player 2 bids $1 Now, if Player 1 passes, he loses 99 cents, if he bids $1.01, he only loses 1 cent What went wrong? When figuring out the expected utility of a bid, a rational player should take into account the future course of the game What if Player 1 starts by bidding 99 cents?
Dollar auction A dollar bill is auctioned off to the highest bidder, but the second-highest bidder has to pay the amount of his last bid Dramatization: https://www.youtube.com/watch?v=pa-snscnadk
Regulatory mechanism design: Tragedy of the commons States want to set their policies for controlling emissions Each state can reduce their emissions at a cost of -10 or continue to pollute at a cost of -5 If a state decides to pollute, -1 is added to the utility of every other state What is the dominant strategy for each state? Continue to pollute Each state incurs cost of -5-49 = -54 If they all decided to deal with emissions, they would incur a cost of only -10 each Mechanism for fixing the problem: Tax each state by the total amount by which they reduce the global utility (externality cost) This way, continuing to pollute would now cost -54
Review: Game theory Normal form representation of a game Dominant strategies Nash equilibria Pareto optimal outcomes Pure strategies and mixed strategies Examples of games Prisoner s Dilemma, Stag Hunt, Chicken Mechanism design Auctions: ascending bid, sealed bid, sealed bid second-price, dollar auction Regulatory mechanism design