Market mechanisms and stochastic programming Kjetil K. Haugen and Stein W. Wallace Molde University College, Servicebox 8, N-6405 Molde, Norway E-mail: Kjetil.Haugen/Stein.W.Wallace@himolde.no 18.12.01 Abstract There are two types of random phenomena modelled in stochastic programs. One type is what we may term external or natural random variables, such as temperature or the roll of a dice. But in many other cases, random variables are used to reflect the behavior of other market participants. This is the case for such as price and demand of a product. Using simple game theoretic models, we demonstrate that stochastic programming may not be appropriate in these cases, as there is no feasible way to replace the market mechanism by a random variable, and arrive at the correct decision. Hence, this simple note is a warning against certain types of stochastic programming models. Stochastic programming is unproblematic in pure forms of monopoly and perfect competition, and also with respect to external random phenomena. But in oligopolies, the modeling may not be approapriate. 1 Introduction The community of stochastic optimization is growing. Many universities have developed courses, although mostly at graduate level, and textbooks have started to appear. Examples of textbooks solely devoted to stochastic programming are Kall and Wallace [4] and Birge and Louveaux [2], but also more standard texts have started to include the matter. Despite the importance of stochastic programming for properly capturing the effects of uncertainty in decision-making under uncertainty (see for example Wallace [8]) there are also traps to be aware of. This paper is concerned with one potential problem of stochastic programs the inability to capture certain market mechanisms. The conclusions are not going to be a return to 1
determinsitic models, but a warning about the limitations of stochastic programs. Consequently, the focus we have is on stochastic programming and market mechanisms. Especially, we will focus on game theory which treats market mechanisms in a eneral way, and try to compare stochastic programming with game theory. We shall also give some general guidelines with respect to which situations require special care when formulating and solving stochastic programming problems. 2 Markets As the introduction indicates, the notion of a market is important. Let us look at some standard economic textbooks and their market definitions to see that the term is far from well defined. In a typical microeconomic textbook, A. Schotter [6] defines a market the following way: By a market, we mean a place where agents can go and exchange one good for another at a fixed price An alternative to this fairly narrow definition is for instance found in Gravell and Rees [3] substantially more advanced book (also in microeconomics): A market exists whenever two or more individuals are prepared to enter into an exchange transaction, regardless of time or place A comparison of the two definitions shows that economists do not have a simple stringent market definition. This is confirmed by Tirole in The Theory of Industrial Organization [7]. He says: There is no simple recipe for defining a market, as is demonstrated by the many debates among economists and antitrust practitioners about the degree of monopoly power in specific industries. Consequently, economists define a market relative to the problem at hand. We shall do the same. The following definition will serve our purpose. A market exists whenever two or more individuals are prepared to make decisions or actions which will result in utility function consequences for at least one of them, regardless of time or place Here, the term exchange transaction is substituted with the term decisions or actions which will result in utility function consequences for at least one of them. Clearly, this definition is more general, and will cover the two other definitions, as trade between agents involving prices definitely can be viewed as making special decisions involving utility function consequences. Additionally, this definition covers other situations, not necessarily involving trade. 2
In subsequent paragraphs, we will examine some extremely simple stochastic optimization problems, and their correspondence with games. At this point it is important to point out what type of situations we are discussing. Random variables are used for many purposes in stochastic programming. We have purely random phenomena, such as the roll of a dice, or the temperature next Midsummer Night. Then, we have phenomena which to a smaller or larger extent are consequences of other peoples actions. Examples here can be the demand for a product, which may depend on the actions of our customers as well as those of our competitors, or the price for a product, depending much on the same actors. This paper concerns only the latter type of random phenomena those that to some extent are a result of actions by others. Or to put it a bit differently, cases where we use random variables to capture the effect of other peoples actions, as we are unable or unwilling to model their actual behavior. 3 A simple stochastic optimization problem Figure 1: A simple decision tree The decision tree in Figure 1 describes an extremely simple stochastic optimization problem. We have made it as simple as possible in order to prove a point. Here, a decision maker (called D 1 ) faces a choice between two possible decisions; d 1 or d 2. A choice of d 1 yields either 4 or 2 with equal probabilities. Alternatively, a choice of d 2 yields 3 or 10 with the same equal probabilities. Assuming that these numbers are measured correctly (utils), and that the decision maker agrees with the Expected Utility Maximization theory, the solution is straightforward, { 1 max 2 (4 + 2), 1 } (3 + 10) = 6.5 (1) 2 3
and the optimal decision is d 2. Let us now assume that the decision maker actually participates in a simple 2-player game, but that her knowledge of game theory is limited, so she has chosen [ to model the other decision maker s choices by the probability distribution 1 2, ] 1 2, as depicted in Figure 1. The real game, involving simultaneous actions by the two players is fully described by the strategic form in Figure 2 Figure 2: A simple 2-player game The strategic form of a two player game is merely a double utility (or Payoff as game theorists like to name it) matrix, containing the player outcomes from various actions. As Figure 2 indicates, Player 1, whom we have named D 1 in Figure 1, gets 4 or 2 by the choice of her action d 1 if Player 2 chooses his actions s 1 or s 2, respectively. Alternatively, Player 1 obtains 3 or 10 by her choice of action d 2 if Player 2 chooses his actions s 1 or s 2. The numbers above the diagonal in Figure 2 correspond to pay-offs for Player 2. The game-theoretic solutional concept most widely acknowledged is the Nash equilibrium. A Nash equilibrium in a 2-player game is, simply stated, a pair of actions such that a deviation from these actions would yield a utility reduction for the player making the decision. A simple way to find Nash equilibria is to construct the so-called best reply functions or mappings. A best reply function is a function giving optimal actions (maximal utility or maximal expected utility) for each player as a function of the other player s actions. A Nash equilibrium is then all intersections between best reply functions. These intersections would imply best reply for both players. This procedure is easily carried out in strategic formed 2-player games by maximizing columns for Player 1 (circles) and rows for Player 2 (squares). Any strategy pair with both a circle and a square is then a Nash equilibrium point, as depicted in Figure 3. The Nash equilibrium point, or the predicted game theoretic solution, is the strategy pair (d 1, s 1 ). If we compare this solution to the solution of the corresponding stochastic optimization problem (1) we immediately observe a 4
Figure 3: A Nash equilibrium point divergence. The stochastic programming approach provides a wrong answer! This is surely one important observation; missing to identify a market mechanism in formulating stochastic optimization problems is dangerous 1. Another important observation is the fact that we can obtain the correct answer by changing the probability distribution in Figure 1. Again, it is straightforward to realize that to reach a correct solution, a change in the probability distribution of Player 2 s choices from [ 1 2, 1 2] to [q, 1 q] where q satisfies: 4 q + 2 (1 q) > 3 q + 10 (1 q) or q > 8 9 (2) would guarantee the correct solution, that is, Player 1 chooses d 1 as her optimal decision. To a certain extent, this seems reassuring and sensible. Given that market interactions exist, we could still rely on solving stochastic optimization problems, if we were able to adjust the probability distributions to take such a market interaction into account. That is, the market mechanism in this case does not produce a price, which would be the normal economic theoretic situation, but a probability constraint. This type of example is by no means the authors invention. Many modern textbooks on game theory apply the method of moving from a stochastic optimization problem to the generalization, a game theoretic problem, as a pedagogical mean see for instance Rasmusen s recent edition [5]. The argument concerning the probability constraint on q is however the author s responsibility. 1 Surely, this reasoning relies on the simple assumption that the game theoretic formulation is more correct than the stochastic optimization formulation. Given this simple example, this is easy to accept. Practical, more complex situations, may however raise more serious problems in this respect 5
4 Another simple stochastic optimization problem The example in the previous paragraph indicates that the correct game theoretic solution may be obtained by a correct adjustment of probability distributions. One important question to answer is then whether this is a general conclusion, or if certain gaming situations may lead to the substantially more negative conclusion: the stochastic optimization formulation will never provide the correct answer. Unfortunately, as the example in Figure 4 will show, this is indeed the case. Figure 4: Another simple decision tree Figure 4 should not need further explanation apart from the fact that the probability distribution is made parametric [q, 1 q], and that a utility parameter ɛ > 0 is introduced. It is straightforward to realize that for any given q, variations in ɛ could produce any solution to the decision problem as indicated by Table 1. Figure 5 shows the corresponding game with best reply functions included. Table 1: Parametric solutions for the problem in Figure 4 ɛ > 1 q q ɛ < 1 q q ɛ = 1 q q d 1 d 2 d 1 d 2 d 1 d 2 Comparing Figure 5 with Figure 4 we observe that no Nash equilibrium exists. From a game theoretic point of view, this is surely not satisfying any game should at least have one solution. Without jumping too deeply into game theory, the concept of randomized strategies is introduced. A random- 6
Figure 5: Another strategic form game ized strategy is as the name indicates, a probabilistic strategy. That is, the players are allowed to choose between their discrete strategies d 1, d 2 and s 1, s 2 by probability distributions [p, 1 p] [q, 1 q] for Player 1 and 2, respectively. The mathematical implication of such an enhancement of the decision space is obvious; the solution space is changed from a non-convex to a convex space. Consequently, game theory is able to launch existence proofs in this type of games. However, as stated earlier, our concern is not game theory here, so let us jump to the conclusion 2 : A Nash equilibrium in randomized strategies exists in this game: ( ) 1 (p, q ) = 3, 1 (3) ɛ + 1 The practical meaning of this is that Player 1 (or D 1 ) should choose action d 1 with a probability of 1 3 and d 2 with a probability of 2 3. It should be obvious, that we are simply not able to mumble around with q and ɛ in the original stochastic optimization problem and obtain such a solution. Even if we introduce randomized strategies as a possibility in the stochastic optimization problem, which would yield the following optimization problem for D 1, (The decision maker chooses strategy d 1 with probability p and strategy d 2 with probability 1 p in Figure 4) H(q, ɛ) = max {qpɛ + (1 q)(1 p) q [0, 1]} (4) p [0,1] the situation would be unchanged. The optimization problem in equation (4) is for any given q (and ɛ) linear in p and the solution for p could either be 0,1 or 2 Any standard text book in game theory shows how randomized strategy Nash equilibria are computed see for instance Ken Binmore s Fun and Games [1] for an excellent introduction to the topic 7
any number in the interval [0,1] depending on the parameters. In no case will the stochastic program give us the unique probability distribution [ 1 3, 3] 2, which was the optimal game theoretic solution. Consequently, we have shown by an example that we in general can not guarantee that a game theoretic problem can be solved as a stochastic optimization problem. 5 Back to Markets In section 2, we discussed the market concept. The basic result there was the introduction of a fairly general market definition. Now, if this generalized market definition creates divergence between game- and stochastic optimization problems, any less general market definitions should hold the potential of similar divergence. Obviously, normal market structures involving price formation is what we think of here they are definitely less general. Such markets are in principle characterized by the same structural properties as our market definition implies; decisions on quantity and price may affect all competing agents utility functions. The major point here is to a certain extent simple. The basic functionality of these types of market mechanisms is to establish values. No value can be established without a market mechanism, and as almost all practical stochastic optimization problems involves maximizing some kind of expected profit or minimizing some kind of expected cost, it is obvious that market mechanisms are present. There are however, two noteworthy exceptions to the warning against stochastic programming; Monopoly and Perfect Competition 5.1 Monopoly Monopoly implies that we are the only seller in a market. Hence, there are no competitors to take into account when modeling. Note that this is a very strict assumption. If our product has a substitute, even if it is not a perfect one, we do not have a monopoly in a proper sense. Furthermore, for a monopoly to represent a simple case, the demand side must consist of only small players price-takers. If there are large players on the demand side, we are still monopolists, but the case is not a simple one. So, the simple case is when we are a monopoly and the demand side consists of only small price-taking participants. We can then assume that whatever price we set, the consumer will take it for granted, and then buy whatever he finds appropriate at that price. It may be that the demand is also affected by some external random variables (like temperature), but that is unproblematic in this context if we can predict what the demand will be for each possible value of the 8
random variables. The crucial point is that we have no other participants in the market making strategic decisions that we have to relate to. Of course, the above also apply if we are a monopsony, and all sellers are small price-takers. 5.2 Perfect Competition By perfect competition we here mean that all sellers and buyers are small and take the price for given. Hence, not a single player in the market can or will make strategic decisions. External random phenomena may affect the price, but all market participants simply observe the effects. In this case, planning is simple, at least as long as we understand the external random phenomena. But there are no problems of the type we observed in the example. We are too small to make strategic decisions, and we do not have to be worried that others make some that will affect us. 5.3 Oligopoly In economic theory, all market situations which can not be characterized as either monopoly or perfect competition are named oligopolies. As the above discussion should indicate, extreme care must be taken when formulating and solving stochastic optimization problems involving values which are a result of non-monopoly or non-perfectly competitive markets. Unfortunately, most markets are of this type; and as such, the correct guideline would be to use game theory instead of stochastic optimization. However, game theoretic progress, from a practical point of view, is very slow, and only very small and simplified problems can be analyzed. This applies to both numerical and closedform analytical approaches. Hence, stochastic programming has its obvious place. It would of course be very nice if we could characterize markets where the gaming-effect is insignificant and as such point out areas where stochastic programming is unproblematic from a theoretical point of view. Unfortunately, this theory does not exist, apart from the abovementioned cases of monopoly and perfect competition. 6 Conclusions Even though the former paragraphs seem to offer a fairly negative conclusion; stochastic programming is hardly ever appropriate, some important practical guidelines may be drawn. Firstly, the notion of two types of probabilities, the natural probabilities and the probabilities representing market interactions is an important partitioning in itself. Consequently, if a stochastic programming problem involves merely the first type of probabilities, it is straightforward to produce practical solutions and use them in decision-making. If, alternatively, 9
a stochastic programming problem contains probabilities within the second set, some more caution must be taken. One practical way of being considerate may be to introduce sensitivity analysis on these probabilities in order to secure robustness against possible market forces implied by these probabilities. (But as we have noted, in many cases there does not exist any probabilities leading to the correct decisions). Last, but not least, game theory may be applied on the side in order to try to repartition the second set of probabilities into several sub-sets with different market characteristics. Some intuition on the markets underlying these subsets may then be used as input in a more refined sensitivity analysis on probabilities. References [1] K. Binmore. Fun and Games. D. C. Heath and Company, Lexington, Massachusettes, Toronto, 1992. [2] John Birge and Francois Louveaux. Introduction to Stochastic Programming. Springer-Verlag, New York, NY, 1997. [3] H. Gravelle and R. Rees. Microeconomics. Longman, London and New York, 1981. [4] Peter Kall and Stein W. Wallace. Stochastic Programming. Wiley, Chichester, 1994. [5] E. Rasmusen. Games and Information. Blackwell, Oxford, UK, 2001. [6] A. Schotter. Microeconomics A Modern Approach. HarperCollins, New York, 1994. [7] J. Tirole. The Theory of Industrial Organization. MIT Press, Cambridge Mass., London, 1988. [8] Stein W. Wallace. Decision making under uncertainty: Is sensitivity analysis of any use? Operations Research, 48(1):20 25, 2000. 10