Game Theory 7 Repeated Games
REPEATED GAMES Review of lecture six Definitions of imperfect information Graphical convention for dynamic games Information sets General failure of backwards induction with imperfect information Subgames, information sets, and strategies with imperfect information Searching NE and SPNE (from the tree to the matrix and back to the tree) 2
REPEATED GAMES Examples of repeated interactions Many interactions in politics, economy, and society happen in repetitive way Colleagues, friends, businessmen have routine contacts in offices, houses, and markets Parties in democratic systems compete regularly in electoral contests Elected representatives have almost daily contacts with each other along the time of one or more mandates Governments have continuing relations in the international setting 3
REPEATED GAMES Repetitiveness and cooperation In these conditions phenomena such as reputation, threats, promises, conventions, agreements may have a role in the analysis The theory of repeated games tries to deal with these phenomena Its major aim is to see if the reciprocal lasting acquaintance of actors involved in interactions may rationally induce to cooperate, also in cases where a one shot interaction suggests them to defect 4
REPEATED GAMES Repeated games as supergames Repeated games are a particular kind of dynamic games Mostly they consist of dynamic games of imperfect information where the same moves of the players are repeated two or more times The basic game that is going to be repeated is called stage game The repetition of the stage game is also called supergame Given a game G, the same game repeated t times (t=2, 3, ) can be indicated G t 5
REPEATED GAMES The general idea The main aim of repeated games analysis has been to find out if in strategic settings among rational actors the promises of future rewards or the threats of future punishments may in certain present circumstances induce incentives for cooperative behavior In order this to happen it must be: (value of being unfair) NOW < (value of the reward of being fair (dis)value of the punishment of being unfair) LATER ON Moreover, promises of rewards and threats of punishment tomorrow must be credible 6
REPEATED GAMES Payoffs and strategies In all dynamic games (of imperfect information) the outcome of subsequent stages depends on that of former stages In a repeated game payoffs are given by the sum of payoffs taken by players at each stage of the game The strategies of a supergame are plans of action that take into account all repetitions of the game 7
Example: PD 2 payoffs REPEATED GAMES 8 A B B cooperate A 6,6 4,7 7,4 5,5 B c c c d d d A 4,7 2,8 5,5 3,6 B c c c d d d A 7,4 5,5 8,2 6,3 B c c c d d d A 5,5 3,6 6,3 4,4 B c c c d d d
REPEATED GAMES Solving PD 2 by backward induction The second stage is made by four subgames Eliminating the dominated strategies any subgames gives the result of mutual defection This result is anticipated by players who can eliminate cooperative strategies in the first stage The SPNE is (defect 2,defect 2 ) and the outcome is (4,4) 9
REPEATED GAMES Solving PD 2 in normal form 2 c (2) d (2) 1 C (2) 3,3 1,4 D (2) 4,1 2,2 1 c 2 d C 5,5 3,6 D 6,3 4,4 The upper figure is the usual one shot PD Given the structure of PD 2, at stage (1) players know that at stage (2) they are playing [(D (2),d (2) ] with payoffs (2,2) Then at stage (1) they anticipate that the total payoffs are that of the lower figure As the lower figure too has NE [(D (1),d (1) ] the SPNE of PD 2 is [(D (1) D (2), d (1) d (2) ) = (D 2,d 2 ) 10
REPEATED GAMES PD repeated a known number of times The result of repeated mutual defection of PD 2 can be generalized to PD t (t being any finite integer number known to the players) This is intuitively inferred from backwards induction At the last stage each player knows that the game is ending, and that the cooperative move can be sanctioned by the defective move of the opponent, with no possibility of reply Then each player s strategy has the move defect at the last information set Having pruned the last stage, the same reasoning applies to the last but one stage, and the same conclusion arises This reasoning can be reiterated t times, and the final result is that players anticipate to choose always the defect strategy of the stage game: The only resulting SPNE of PD t is (defect t, defect t ) 11
REPEATED GAMES Supergames repeated t times The same result can be extended to all supergames whose stage game has only one NE (as PD) In a supergame where the number of repetition is finite and known to all players the stage game has only one NE, so that the NE of various stages are independent of each other Only one SPNE exists, which is the repetition t times of the NE of the stage game 12
REPEATED GAMES More than one NE When the stage game has more than one NE, the equilibria of various stages may depend on each other Then it may happen that supergames have NE different from the repetition of the Nash equilibria of the stage game Here we will not discuss any further about this possibility Rather 13
REPEATED GAMES The trouble with cooperation We have seen that generally cooperation is difficult when players know when the game is going to an end That is because of the reasoning by backwards induction, starting from the last stage, when there is no future, and playing defect cannot be punished by following moves of the antagonist For example the PD t induces always defection, and the only SPNE is (D t,d t ) 14
A different situation What happens when people do not know whether the repeated game is ending or not? That is usual in the real world REPEATED GAMES In society: interactions among peers, or in hierarchical organizations In economy: among businessmen, or between firm and consumer In politics: among citizens and bureaucrats, or in party coalitions In that circumstance the last stage is eliminated, and backward induction cannot be applied Then an important cause of defection is ruled out, and opportunities for cooperation might arise 15
REPEATED GAMES If the future of interaction is not sure As in the case of uncertain choice we suppose that people may conceive of reasonable ideas about the continuation of the game The simplest way to do so is assuming that players know the probability of the continuation Moreover, to be more realistic, we assume that future payoffs are discounted by that same probability That means that if b is the payoff of the first round of the game, bp is the expected value at the first repetition, bp 2 the expected value at the second one, etc. Then the total expected benefit is 2 n b b p b p... b p... 16
A complementary way Temporal preferences, discounted payoffs and discount rate Patient or impatient players? Better an egg today or a chicken tomorrow? A discount rate models how highly a person values the future as compared to the present The discount rate at time t of player i: d t i, where 0< d t i,<1, tells us the present value (PV) of i s payoff to be made at t periods from the present (where t can refer to days, weeks, years, etc.) A person with a discount rate close to 0 only cares about today. A person with a discount rate equal to 1 would be just happy to receive a million dollars in the future as she would be to receive it today
A complementary way Of course we can combine the two approaches, i.e., a person can discount the future and be uncertain about the continuation of the game, i.e. z=d*p Then the present value of the total expected benefit simply becomes: a+az+az 2 + az 3 + + az n, where z=d*p, z 2 = d 2 *p 2
How to compute an infinite series Four main scenarios (that we will employ ) First: PV(present value) = a+az+az 2 + az 3 + + az n is equal to: PV=a/(1-z) Second: PV(present value) = az+ az 2 + az 3 + + az n is equal to: PV=az/(1-z) Third: PV(present value) = a+ az 2 + az 4 + + az n is equal to: PV=a/(1-z 2 ) Fourth: PV(present value) = az+ az 3 + az 5 + + az n+1 is equal to: PV=az/(1-z 2 ) For sake of simplicity let s focus for now just on p
REPEATED GAMES PD Let us consider the PD game indefinitely repeated A c B d C 4,4 1,5 D 5,1 2,2 and let us suppose that players follow these instructions: At first stage play cooperate Then play cooperate until the competitor does the same If the competitor, at any stage, defects play defect from then on This set of instructions is called grim trigger strategy (GTS) 20
REPEATED GAMES Grim trigger strategy (1) This is a strategy as it instructs about what to do at any stage that, in the case of repeated PD, coincide with information sets We want to see if the strategy profile (GTS,GTS) is a SPNE of the PD repeated an indefinite number of times The total utility of cooperation when playing this strategy is 4 + 4 p+ 4 p 2 +... + 4 p n +... = 4 å p t = 4 t=0 1- p The defecting player will receive instead 5+ 2 p+ 2 p 2 +...+ 2 p n +... = 5+ 2 å p t = 5+ 2 p 1- p t=0 21
REPEATED GAMES Grim trigger strategy (2) With some algebra it is easy to see that 4 2 p 5 p 1/ 3 1 p 1 p That means that a profile of grim trigger strategies is a NE of PD if and only if, at each stage, players expect that the continuation of the game is more probable than 1/3 (with these payoffs) This is also a SPNE because when the game is repeated indefinitely each subgame coincides with the entire game, so that it has the same NE 22
REPEATED GAMES More generally The opportunities of cooperation among rational actors grow with the probability that the interaction continues The more the shadow of the future impinges on our present actions the more we are induced to adopt a fair behavior in social relations This conclusion does not mean that people change their basic attitudes, but that the environmental conditions of interaction may give opportunities to gain by good social behavior 23
REPEATED GAMES Not so drastic strategies Grim trigger strategy in PD illustrates the capacity of players to resist the temptation of defecting today for the expectation of better payoffs tomorrow However it express a drastic attitude toward the opponent that does not describe appropriately the usual repeated interactions among social players. Besides it does not allow any mistake! This is why theorists have tried to formalize in the strategic framework more realistic models of repeated social interaction An idea of retaliation for defecting that is in a sense opposite to grim trigger strategy is embedded in the so called strategy of tit-for-tat 24
Tit-for-Tat A (Biblical?) Conditional strategy: here a player begins by cooperating and then matches the opponents previous-period play. With a tit-for-tat strategy, defection is immediately punished; this punishment is applied until a cooperative response is evoked
REPEATED GAMES Tit-for-Tat As PD is concerned, its instructions are the following: play C at the first stage of the game then play C if (C,C) or a (D,C) profile occur in the preceding stage, and play D otherwise Does the use of this strategy by both players generate a NE of mutual cooperation? If yes, under what conditions, as to the probability of continuation of the game? 26
Tit-for-Tat Let s call F the free-riding payoff, C the cooperation payoff, N the mutual defection payoff and L ( loser ) the payoff of the lone cooperator We have a PD, if F>C>N>L
A PD Player B Cooperate Defect Player A Cooperate C, C L, F Defect F, L N, N F > C > N > L
REPEATED GAMES Tit-for-Tat Playing cooperatively the payoff is as known 2 n t C C C p C p... C p... C p t 0 1 p Defecting from cooperation the total payoff is 2 n Np F N p N p... N p... F 1 p For (C,C) being a NE the first payoff must be greater than the second, and that brings about the condition, that is: p > (F-C)/(F-N) 29
REPEATED GAMES Tit-for-Tat Alternatively, a player could decide to start to defect at the first stage, then to cooperate at the next stage, then defect again, and so no. In this case the total payoff is F L p 1 p 1 p 2 3 n F L p F p L p... F p... 2 2 For (C,C) being a NE the first payoff must be greater than also the third payoff, and that brings about the second condition: p > (F-C)/(C-L) 30
REPEATED GAMES Tit-for-Tat For (C,C) being a NE the first payoff must be greater than the largest alternative payoffs Generally, we can say that, the probability to cooperate increases when: 1) p ; 2) C ; 3) N ; 4) L ; 5) F Linking the theory with the empirical facts 31
Endogenous solutions? Yes, we can! Elinor Ostrom (Nobel Prize 2009) in her book Governing the Commons discusses several examples of resolution of collective action problems Most of these involve taking advantage of features specific to the local context in order to set up systems of defection and punishment The most striking feature of Ostrom s range of cases is their immense variety: some success, some failures Despite this variety, we can identify several common features that make it easier to solve cooperator s dilemmas
Which factors matter 1. It is essential to have an identifiable and stable group of potential participants 2. The benefits of cooperation have to be large enough to make it worth paying the costs of monitoring and enforcing the rules of cooperation 3. It is very important that the members of the group can communicate with one another Why do those factors matter? Now you know why!
Parallel games Players do not only interact over time. Sometimes they play more games at the same moment Linking two (or more) games: parallel games The example of the Japanese firms. Why Japanese firms use (used) to subsidize the social activities of their workers outside of the working place? We have two games now. The work-game and the social-game. Does it matter?
Parallel games Let s suppose that (F-C)/(F-N) > (F-C)/(F-L) Then we just saw that in an iterated game, if p>(f- C)/(F-N), then a possibility of cooperation arises. But now we have two games played simultaneously! In this scenario, the conditional strategy is not only affected of what is happening in one single game, but also to what is happening in the other game A typical conditional strategy now requires to link the behaviour of one player to what the other player is doing in both games
Parallel games Let s assume that in the second game (the social one) coooperation is feasible, that is : p>(f-c)/(f-n), while in the first game (the working one) it is not so, that is: : p<(f-c)/(f-n) Let s call Bs the net benefits from cooperation in the second game (Bs = C/(1-p)-[F+pN/(1-p)]) and Ps the net costs of cooperation in the first game (Ps = [F+pN/(1- p)]-c/(1-p)) If Bs>Ps, then if we have connected the two games, cooperation in the second game allows to produce cooperation also in the first game However, if Bs<Ps, then connecting two games can destroy the cooperation also in the second game
Parallel games Let s see the relevance of what just discovered A well-known, and pretty useful, example: irrigation systems in Nepal Donor intervention through the Agency Managed Irrigation Systems: to increase agricoltural yields and crop intensities, several traditional irrigation system, usually managed by the farmer community by its own, have been replaced by new ones with a permanent structure made by concrete. Beyond being more modern, the new structure required also less maintenance-work from the farmer community to keep it clean and working
The results: The devil is in the details On average the replaced (and more modern) irrigation systems worked (much) more poorly than the traditional ones after just few years. Let s see why
The devil is in the details How external interventions may create disruptive asymmetries as an unintended effect of financial assistance Let us assume that there is an untapped mountain stream and that a group of farmers wants to divert such stream to their area. Let us further introduce a substantial asymmetry related to physical location on the irrigation canal This a typical situation in Nepal: there are lots of mountains after all!!!
The devil is in the details stream Head-enders live here End-enders live here
The devil is in the details In this situation, the canal enters from one side. When this happen, some plots will receive water before the last ones. Irrigators located at the head end of a system (the head-enders) have differential capabilities to capture water and may not fully recognize the costs others bear as a result of their actions In addition, farmers located at the head end of a system receive proportionately less of the benefits produced by keeping canals (located below them) in good working order than those located at the tail (the tail-enders). These asymmetries are the source of considerable conflict on many irrigation systems
The devil is in the details There is therefore a close interrelationship among the willingness to invest in maintenance, farmers expectations about obtaining water, their expectations about the contributions others will make and the tensions that can exist among head-end and tail-end farmers. How to deal with this situation? One way is to craft rules that actually create and link two different games: the game of water and the game of canal preservation Why connecting the two games should matter? Which kind of credible strategies this connection make possible?
The devil is in the details The connection of the two games allow the following conditional strategy: The head-enders commit not to take too much water in the first game, while the tail-enders commit to provide a disproportional amount of work for the preservation of the canal in the second game. What does this commitment credible? Keeping a traditional irrigation canal, usually built with mud, in good conditions require a large amount of work! That is, for both groups of farmers it is true that Bs>Ps as soon as the two games become interlinked Of course, farmers at the tail-end have more bargaining power in relationship to the farmers at the head-end if the amount or resources needed for preservation is (relatively) large
The devil is in the details The previous analysis provides a potential answer to the puzzle of why many effective farmerorganized systems (such as the ones in Nepal) collapse soon after their system have been modernized using funds provided by international donors Why?
The devil is in the details Project evaluations usually consider any reductions in the labor needed to maintain a system as a project benefit. Thus, investments in permanent structure and lining canals are justified because of the presumed increase in agricultural productivity and the reduction in annual maintenance costs However the possibility that greatly reducing the need for resources to maintain a system would substantially alter the bargaining power of head-enders versus tail-enders is not usually considered in project evaluations.
The devil is in the details What therefore usually happens is that the credibility of previous rules simply disappears, given that Bs<Ps at least for the head-end farmers. As a consequence The head-end farmers now grab all the water they can and make no investment in maintenance. The head-enders can ignore the contributions of the tail-enders to maintenance because for a few years the concrete structure will operate well without any maintenance. And as a consequence of these behaviors, now the tail-enders will do the same (i.e., even for them Bs<Ps). But not only that
The devil is in the details At some time in the future the productivity of the system will fall below what it was prior to outside help. The tail-enders may initiate violence against the headenders due to their perception that the water rights they had achieved with their hard labor had been taken from them The end result can easily be that a community which had been knitted together by their mutual dependence dissipates into a setting of considerable conflict and low overall productivity
The devil is in the details If the farmers were expected to pay back the costs of the investment made in physical capital (or to pay taxes to keep the system well maintained) tail-end farmers would again find themselves in a better bargaining relationship with head-enders Paradoxically, a very disruptive aspect of external assistance was that it was free to the farmers involved. Without any need for resources (of any sort) from tailenders, head-enders could ignore the interests of the less advantaged and take a larger share of benefits
REPEATED GAMES Conclusion Cooperation in the repeated PD game can evolve on a purely egoistic basis: players will employ an initially suboptimal policy (a dominated strategy) in the long run because such a policy trades smaller short-term gains for larger long-term ones Rationality involves time preferences in addition to expectations of what others will do, that is, consideration of the short and long term However the possibility of cooperation does not imply its inevitability, even if the game is repeated! The Folk Theorem 49