Repeated game

Repeated games may be broadly divided into two classes, finite and infinite, depending on how long the game is being played for.

• Finite games are those in which both players know that the game is being played a specific (and finite) number of rounds, and that the game ends for certain after that many rounds have been played. In general, finite games can be solved by backwards induction.
• Infinite games are those in which the game is being played an infinite number of times. A game with an infinite number of rounds is also equivalent (in terms of strategies to play) to a game in which the players in the game do not know for how many rounds the game is being played. Infinite games (or games that are being repeated an unknown number of times) cannot be solved by backwards induction as there is no "last round" to start the backwards induction from.

Even if the game being played in each round is identical, repeating that game a finite or an infinite number of times can, in general, lead to very different outcomes (equilibria), as well as very different optimal strategies.

The most widely studied repeated games are games that are repeated an infinite number of times. In iterated prisoner's dilemma games, it is found that the preferred strategy is not to play a Nash strategy of the stage game, but to cooperate and play a socially optimum strategy. An essential part of strategies in infinitely repeated game is punishing players who deviate from this cooperative strategy. The punishment may be playing a strategy which leads to reduced payoff to both players for the rest of the game (called a trigger strategy). A player may normally choose to act selfishly to increase their own reward rather than play the socially optimum strategy. However, if it is known that the other player is following a trigger strategy, then the player expects to receive reduced payoffs in the future if they deviate at this stage. An effective trigger strategy ensures that cooperating has more utility to the player than acting selfishly now and facing the other player's punishment in the future.

There are many results in theorems which deal with how to achieve and maintain a socially optimal equilibrium in repeated games. These results are collectively called "Folk Theorems". An important feature of a repeated game is the way in which a player's preferences may be modeled. There are many different ways in which a preference relation may be modeled in an infinitely repeated game, but two key ones are :

• Limit of means - If the game results in a path of outcomes ${\displaystyle x_{t}}$ and player i has the basic-game utility function ${\displaystyle u_{i}}$, player i's utility is:

${\displaystyle U_{i}=\lim _{T\to \infty }\inf {\frac {1}{T}}\sum _{t=0}^{T}u_{i}(x_{t})}$

• Discounting - If player i's valuation of the game diminishes with time depending on a discount factor ${\displaystyle \delta <1}$, then player i's utility is:

${\displaystyle U_{i}=\sum _{t\geq 0}\delta ^{t}u_{i}(x_{t})}$

For sufficiently patient players (e.g. those with high enough values of ${\displaystyle \delta }$), it can be proved that every strategy that has a payoff greater than the minmax payoff can be a Nash equilibrium - a very large set of strategies.

Example: Iterated prisoner's dilemma game with unique stage Nash Equilibrium (D, D).

Repeated games allow for the study of the interaction between immediate gains and long-term incentives. A finitely repeated game is a game in which the same one-shot stage game is played repeatedly over a number of discrete time periods, or rounds. Each time period is indexed by 0 < t ≤ T where T is the total number of periods. A player's final payoff is the sum of their payoffs from each round.[1]

In each period of a finite game, players execute a certain amount of action. These actions lead to a stage-game payoff for the players. The stage game can be denoted by {A, u} where A = A1 * A2 *...* An is the set of profiles and ui(a) is player i's stage-game payoff when profile a is played. The stage game is played in each period. Additionally, we assume that in each period t, the players have observed the history of play, or the sequence of action profiles, from the first period through period t-1. The payoff of the entire game is the sum of the stage-game payoffs in periods 1 through T. Sometimes, one should assume that all players discount the future, in which case we include a discount factor in the payoff specification.[2]

For those repeated games with a fixed and known number of time periods, if the stage game has a unique Nash equilibrium, then the repeated game has a unique subgame perfect Nash equilibrium strategy profile of playing the stage game equilibrium in each round. This can be deduced through backward induction. The unique stage game Nash equilibrium must be played in the last round regardless of what happened in earlier rounds. Knowing this, players have no incentive to deviate from the unique stage game Nash equilibrium in the second-to-last round, and so on this logic is applied back to the first round of the game.[3] This ‘unraveling’ of a game from its endpoint can be observed in the Chainstore paradox.

If the stage game has more than one Nash equilibrium, the repeated game may have multiple subgame perfect Nash equilibria. While a Nash equilibrium must be played in the last round, the presence of multiple equilibria introduces the possibility of reward and punishment strategies that can be used to support deviation from stage game Nash equilibria in earlier rounds.[3]

Finitely repeated games with an unknown or indeterminate number of time periods, on the other hand, are regarded as if they were an infinitely repeated game. It is not possible to apply backward induction to these games.

Example 1: Two-Stage Repeated Game with Multiple Nash Equilibria

Example 1 shows a two-stage repeated game with multiple pure strategy Nash equilibria. Because these equilibria differ markedly in terms of payoffs for Player 2, Player 1 can propose a strategy over multiple stages of the game that incorporates the possibility for punishment or reward for Player 2. For example, Player 1 might propose that they play (A, X) in the first round. If Player 2 complies in round one, Player 1 will reward them by playing the equilibrium (A, Z) in round two, yielding a total payoff over two rounds of (7, 9).

If Player 2 deviates to (A, Z) in round one instead of playing the agreed-upon (A, X), Player 1 can threaten to punish them by playing the (B, Y) equilibrium in round two. This latter situation yields payoff (5, 7), leaving both players worse off.

In this way, the threat of punishment in a future round incentivizes a collaborative, non-equilibrium strategy in the first round. Because the final round of any finitely repeated game, by its very nature, removes the threat of future punishment, the optimal strategy in the last round will always be one of the game's equilibria. It is the payoff differential between equilibria in the game represented in Example 1 that makes a punishment/reward strategy viable (for more on the influence of punishment and reward on game strategy, see 'Public Goods Game with Punishment and for Reward').

Example 2: Two-Stage Repeated Game with Unique Nash Equilibrium

Example 2 shows a two-stage repeated game with a unique Nash equilibrium. Because there is only one equilibrium here, there is no mechanism for either player to threaten punishment or promise reward in the game's second round. As such, the only strategy that can be supported as a subgame perfect Nash equilibrium is that of playing the game's unique Nash equilibrium strategy (D, N) every round. In this case, that means playing (D, N) each stage for two stages (n=2), but it would be true for any finite number of stages n.[4] To interpret: this result means that the very presence of a known, finite time horizon sabotages cooperation in every single round of the game. Cooperation in iterated games is only possible when the number of rounds is infinite or unknown.

To develop a picture of the entire set of equilibria in the repeated prisoners’ dilemma, consider the following stage game.

Sample Prisoner's Dilemma Normal Form

We can create a diamond-shaped graph by connecting points (4, 4), (−2, 6), (0, 0), and (6, −2).

Sample Diamond Graph

This diamond depicts the set of feasible stage-game payoffs and the possible repeated-game payoffs in terms of “average per period,” by multiplying the discounted sum payoff by (1 − 𝛅). For example, the point (4, 4) is noted in the picture with a solid circle, which refers to the players obtaining (4, 4) each period in the game (by playing (C, C) each period). The point (6, −2) arises if (D, C) is played each period.

We can prove that any point on the edges of the diamond can be supported as an equilibrium average per-period payoff as long as the players are patient enough. For instance, consider the point (5, 1) designated by an open circle at the right edge of the diamond. Suppose the players alternate between (C, C)=(4,4) and (D, C)=(-2,6) over time, starting with (C, C) in the first period. For player 1, this sequence of actions yields a discounted payoff of

${\displaystyle 4+6\delta +4\delta ^{2}+6\delta ^{3}+....=4[1+\delta ^{2}+\delta ^{4}+...]+6\delta [1+\delta ^{2}+\delta ^{4}+...]}$

Letting ${\displaystyle v=1+\delta ^{2}+\delta ^{4}+...}$, we have ${\displaystyle v=1+\delta ^{2}v}$. Solving, we get ${\displaystyle v={\frac {1}{1-\delta ^{2}}}}$. The expression simplifies to:

${\displaystyle {\frac {4}{1-\delta ^{2}}}+{\frac {6\delta }{1-\delta ^{2}}}}$

Multiplying by ${\displaystyle 1-\delta }$ puts this in terms of average per period:

${\displaystyle {\frac {4+6\delta }{1+\delta }}}$

Likewise, player 2’s per-period average is:

${\displaystyle {\frac {4-2\delta }{1+\delta }}}$

Note that if discounting factor 𝛅 is close to 1, then this average payoff vector is arbitrarily close to (5, 1). Thus the diamond represents the set of feasible average per-period payoffs that can arise in the repeated game.

In general, repeated games are easily solved using strategies provided by folk theorems. Complex repeated games can be solved using various techniques most of which rely heavily on linear algebra and the concepts expressed in fictitious play. It may be deducted that you can determine the characterization of equilibrium payoffs in infinitely repeated games. Through alternation between two payoffs, say a and f, the average payoff profile may be a weighted average between a and f.

Repeated games can include incomplete information. Repeated games with incomplete information were pioneered by Aumann and Maschler.[5] While it is easier to treat a situation where one player is informed and the other not, and when information received by each player is independent, it is possible to deal with zero-sum games with incomplete information on both sides and signals that are not independent.[6]

• Fudenberg, Drew; Tirole, Jean (1991). Game Theory. Cambridge: MIT Press. ISBN 0-262-06141-4.
• Mailath, G. & Samuelson, L. (2006). Repeated games and reputations: long-run relationships. New York: Oxford University Press. ISBN 0-19-530079-3.
• Osborne, Martin J.; Rubinstein, Ariel (1994). A Course in Game Theory. Cambridge: MIT Press. ISBN 0-262-15041-7.
• Sorin, Sylvain (2002). A First Course on Zero-Sum Repeated Games. Berlin: Springer. ISBN 3-540-43028-8.

• Game-Theoretic Solution to Poker Using Fictitious Play
• Game Theory notes on Repeated games
• on Repeated Games and the Chainstore Paradox