STOCHASTIC PROGRAMMING IN REVENUE MANAGEMENT

Transcription

1 STOCHASTIC PROGRAMMING IN REVENUE MANAGEMENT DISSERTATION Presented in Partial Fulfillment of the Requirements for The Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Lijian Chen THE OHIO STATE UNIVERSITY 2006 Dissertation Committee: Dr. Tito Homem-de-Mello Approved by Dr. Clark Mount-Campbell Dr. Mark McCord Dr. Theodore Allen Advisor Graduate Program in Industrial and Systems Engineering

2 c Copyright by Lijian Chen 2006 All Rights Reserved

3 ABSTRACT Airline revenue management aims to assign the right seat to the right customer with right prices at the right time. Due to the existence of large uncertainty in customer demand and the unavailability of perfect information, decisions must be made in advance. Also, such decisions are subject to constraints, such as seat availability, demand forecasts, and customer preferences. The objective of revenue management is to maximize the long term booking revenue. In this research, we studied two models in detail, the seat allocation model and the customer choice model based on preference orders. The seat allocation model is to decide the number of seats available for booking at class level by assuming the demands among booking classes are independent. The customer choice model is to assign seats at class level without forecasting demands individually. Both research topics in revenue management, the seat allocation optimization and customer choice optimization, are built by stochastic programming models. We present a multi-stage stochastic programming formulation to the seat allocation problem that extends the traditional probabilistic model proposed in the literature. Because of the lack of convexity properties, solving the multi-stage problem exactly may be difficult. In order to circumvent that obstacle, We use an approximation based on solving a sequence of two-stage stochastic programs with simple recourse. Our theoretical results show that the proposed approximation is robust, in the sense that solving more successive two-stage programs can only improve the expected revenue. We also discuss a heuristic method to choose the re-solving points. ii

4 Numerical results are presented to illustrate the effectiveness of the proposed approach. Besides the strong uncertainty in customer demand, the customer s preference can make a difference in total revenue too. In our research, we assume that customers make choices according to personal preferences, such as preferences for connected trips or direct trips. We realize the fact that different customers might possess similar preference and behave. It becomes the idea to construct the customer preference orders. Instead of forecasting the demands by classes in seat allocation model, our preference order model requires the distribution information by preference orders. Essentially, this model is also a stochastic programming model. With properly implementing the model recommendation, the numerical experiment indicates that the preference order method tends to generate no less operating revenue than independent demand methods by catching more valued customers. In this text, we also gave a detailed literature review in the Chapter 1 for most up-to-date airline revenue management progresses. Compared with existing research, we have made two major contributions. First, we proposed a network heuristic to improve the rolling horizon method with analytic justification; second, we proposed the first solvable network customer choice behavior model by mathematical programming. Those contributions open a way to incorporate the demands distributional information and the customer choice behavior into the real airline booking optimization process. iii

5 ACKNOWLEDGMENTS Guidance and advice from my advisor, Dr. Tito Homem-de-Mello, have influenced this text, and even my career path. I have been working with Dr. Homem-de-Mello since During the almost five years thereafter, I have learned a lot from him. Dr. Homem-de-Mello showed me the right way to conduct excellent research. He verified most of the theoretical conclusions and provided many helpful suggestions and most of which become part of my dissertation. Other faculty and students in Northwestern University also contributed in my research. Dr. Birge showed the applications of Monte Carlo simulation in financial engineering, and Dr. Nelson presented the most recent research progress in simulation. The students, Xuemei Shan, Zhen Liu, Liming Feng, Xiaodong Xu, Fang Liu, Jun Liu, Feng Yang, Liu Hong, Min Huang, exchanged ideas and comments which lead to valuable improvements. I really enjoyed the experience working in such a productive and creative environment. I would like to thank all my committee members, Dr. Clark Mount-Campbell, Dr. Mark McCord and Dr. Theodore Allen, for suggestions in my proposal. Dr. Mount- Campbell voluntarily took all academic and immigration paperwork in OSU. Dr. McCord provided detailed and constructive suggestions about methods for estimating transition probabilities in the preference order model. Also, my dear wife, Xituan, and my colleagues, Samrat Sondhi, Xiaoyan Jiang, Alyssa Quill, and etc. provided me great support at both home and work, the Department of Revenue Management, Storage USA, LLC, Extra Space Storage, LLC. iv

6 In preparing this text, I use TEX dissertation template developed by Miguel Lerma, Department of Mathematics, Northwestern University to follow the graduate school dissertation guidelines. All the numerical experiments are coded under C/Matlab/XpressMP. The dash-optimization customer support team s timely support is another critical input to our research. During the first four years of research, I was fully supported by the National Science Foundation under grant DMI from Any opinions, findings, and conclusions or recommendations expressed in this material are those of mine and do not necessarily reflect the views of the National Science Foundation. v

7 VITA September 27, Born - City of Qinhuangdao, China B.S., Industrial Engineering, Tianjin University M.S., Management Science, Tianjin University Researcher, McGill University present Graduate Research Associate, The Ohio State University Visiting Scholar, Northwestern University Assistant Manager, Revenue Management, GE Commercial Finance PUBLICATIONS Research Publication E. Qi and L. Chen, Integrated System and Strategy for Manufacturing Industry, Journal of Industrial Engineering and Engineering Management, 13(3):61-64, FIELDS OF STUDY Major Field: Industrial and Systems Engineering. vi

8 TABLE OF CONTENTS ABSTRACT ii ACKNOWLEDGMENTS iv VITA vi LIST OF TABLES ix LIST OF FIGURES x CHAPTER 1. INTRODUCTION Review on airline revenue management Research on origin and destination Model Research on the customer choice behavior models Our airline revenue management solution CHAPTER 2. MULTI-STAGE STOCHASTIC PROGRAMMING IN REVENUE MANAGEMENT Model description Allocation methods Re-solving SLP model Bid-price methods Numerical results Improving the partition Conclusions vii

9 CHAPTER 3. MODELING CUSTOMER CHOICE BY PREFERENCE ORDER Effect of the customer choice behavior in airline industry Our idea on modeling the network customer choice Model formulation Our heuristics Implement the policy Relationship with the independent demand model Numerical experiments Conclusions REFERENCES APPENDIX A. NUMERICAL EXPERIMENTS CODES A.1. The structure of the program viii

10 LIST OF TABLES 2.1 Simulation results for allocation policy Simulation results for bid price policy Simulated expected revenue for plan A and B Simulation results for allocation policy with new partition Simulation results for bid price policy with new partition Comparison between plan B and C in expected revenue Wait-and-see values for examples 1 and Experiment 2 fare level setting Experiment 2 setting, budget - budget sensitive; time - time sensitive; hybrid - hybrid type Model (3.11) simulation confidence interval summary SLP model simulation confidence interval summary Comparison results ix

11 LIST OF FIGURES 1.1 The rate level snapshot of coach class on Monday The rate level snapshot of coach class on Friday Illustration of customer choice The position of revenue management department Our total solution for airline revenue management Graph of the recourse function in a three-stage problem Example 1 for Numerical Experiment Example 2 for Numerical Experiment Problem Setting Our adaptive re-solving heuristic on identifying right solving point Graph of the function H(t) = j,k q j,ke[ξ jk (t)] for Examples 1 (left) and 2 (right) Rejection frequencies between plan A (left) and B (right) happened in Example Solving frequencies upon time points between plan A (left) and B (right) Small Example on Preference Orders Business Customers Preference Order x

12 3.3 Budget Sensitive Customers Preference Order The customer choice process inside a preference order Example for the computational issue Roadmap of modeling the customer choice Applying the non-zero allocation heuristically Applying heuristic The independent demand model s structure The comparable preference order model s structure Numerical Experiments Settings Preference orders in example Experiment to show the value of customer choice Structure of preference orders A.1 Brief program structure xi

13 CHAPTER 1 INTRODUCTION 1.1. Review on airline revenue management Revenue management involves the application of quantitative techniques to improve profits by controlling the prices and availabilities of various products that are produced with scarce resources. Perhaps the best known revenue management application occurs in the airline industry, where the products are tickets (for itineraries) and the resources are seats on flights. In view of many successful applications of revenue management in different areas, this topic has received considerable attention in the past few years both from practitioners and academics. The recent book by Talluri and van Ryzin (2004b) provides a comprehensive introduction to this field, see also references therein. Most people who have experience of booking airline tickets realize the price changes frequently as time goes. For example, suppose you want to book a round trip ticket from Memphis to Washington, DC on Feb 23 through Feb 28, coach class for one adult. The booking snapshots in Figure 1.1 and Figure 1.2 show a possible outcome of the request. The flight 2063 s coach class rate is $1043 on Monday and $1172 on Friday the same week. Such changes result from the internal operations, pricing, and seat allocation. Although pricing is an important activity in revenue management, the model for pricing is largely heuristic and empirical. Basically, the pricing method is based on an internal parameter called performance index. The company usually picks several sensitive parameters, such as occupancy percentage, competitor s rate, sales forecast, 1

14 Figure 1.1. The rate level snapshot of coach class on Monday and other related historical records, to calculate a number called the performance index. The performance index is a measure for market performance of certain product offered. Usually, higher score on performance implies better performance, and therefore, higher price and vise versa. Although adjusting prices by performance index is not a well recognized model for pricing, it has been widely used in various industries, such as self-storage, retail and airline industries. We acknowledge that the methods applied among individual companies might be different in formation and description. Adjusting prices by performance index might backfire the revenue. The most obvious impact is that it could confine the airplane occupancy at certain level. Also, 2

15 Figure 1.2. The rate level snapshot of coach class on Friday due to the time lag in data processing and validating, this model can not handle all the real time price change requests from the field. Another important operation is the seat allocation which could reflect the price change as well. The airline company makes the change on the number of available seats with a fixed price. In the seat allocation operation, there are usually classes for a 138 seats Boeing 737. All those classes have different prices and some small classes only have fewer than 5 seats listed. Upon the booking requests, the airline company makes decisions to open or shut down certain classes. This shapes a 3

16 way of dynamic pricing. In this dissertation, we focus on seat allocation method and assume the rate levels are largely fixed Research on origin and destination Model A common way to model the airline booking process is as a sequential decision problem over a fixed time period, in which one decides whether each request for a ticket should be accepted or rejected. A typical assumption is that one can separate demand for individual itinerary-class pairs; that is, each request is for a particular class on a particular itinerary, and yields a pre-specified fare. Typically, a class is determined by particular constraints associated with the ticket rather than the physical seat. For example, a certain class may require a 14-day advance purchase, or a Saturday night stay, etc. The existence of different classes reflects different customer behaviors. The classical example is that of customers traveling for leisure and those traveling on business. The former group typically books in advance and is more price-sensitive, whereas the latter behaves in the opposite way. Airline companies attempt to sell as many seats as possible to high-fare paying customers and at the same time avoid the potential loss resulting from unsold seats. In most cases, rejecting an early (and lower-fare) request saves the seat for a later (and higher-fare) booking, but at the same time this creates the risk of flying with empty seats. On the other hand, accepting early requests raises the percentage of occupancy but creates the risk of rejecting a future high-fare request because of the constraints on capacity. The airline booking problem was first addressed by Littlewood (1972), when he proposed what is now known as the Littlewood Rule. Roughly speaking, the rule proposed for a two-class model says that low-fare bookings should be accepted as long as their revenue value exceeds the expected revenue of future full fare bookings. 4

17 This basic idea was subsequently extended to multiple classes (see, e.g., Brumelle and McGill (1993)). Later, it was shown that, under certain conditions, it is optimal to accept a request only if its fare level is higher or equal to the difference between the expected total revenues from the current time to the end when respectively rejecting and accepting the request; see, for instance Subramanian, Stidham, and Lautenbacher (1999). This rule immediately leads to the question How to evaluate or approximate the expected total revenue from the current time until the end of booking? We will return to that later. Many of early models were built for single flights. While that environment allows for the derivation of optimal policies via dynamic programming even with the incorporation of extra features Subramanian et al. (1999), the drawback is clear in that the booking policy is only locally optimized and it cannot guarantee global optimality. To see this, consider the following example. A company operates among three cities, A, B, and C. A request pays $100 for trip from A to C (via B) while two other requests pay respectively $50 for a trip from A to B and $80 for a trip from trip B to C. Suppose there is only one seat available on each of the flights (i.e., AB and BC). If the company optimizes for each flight individually, then the A-to-C request will probably be accepted. However, if the two flights are viewed together, then it is clear that the A-to-B and B-to-C requests should accepted instead. For that reason, we focus on the discussion of the network-based problem. In airline revenue management terminology, this model is sometimes called origin-destination (OD) model or passenger-mix problem. This model was raised by Statistics and Forecasts Branch of Transport Canada to generate air traffic forecasts in Since its structure could be formulated into mathematical programming models (see Curry (1990)), OD model has become one of most important models in network airline revenue management. Williamson (1992) provided a detailed analysis on it. 5

18 In general, network models can only provide heuristics for the booking process, since determining the optimal action for each request in a network environment is impractical from a computational point of view. One type of heuristics is based on mathematical programs, where the decision variables are the number of seats to allocate to each class. In particular, methods based on linear programming techniques have been very popular in industry, for several reasons. First, linear programming is a well developed method in operations research; its properties have been thoroughly studied for decades. Secondly, commercial software packages for linear programming are widely available and have been proved efficient and reliable in practice. Finally, the dual information obtained from the linear program can be used to derive alternative booking policies, based on bid-prices; We will return to that in section 2.4. Glover, Glover, Lorenzo, and McMillan (1982) were perhaps the first to describe a network revenue management problem in airlines. By assuming that passenger demands are deterministic, they focused on the network aspects of the model (e.g., using network flow theory) rather than on the stochastic aspect of customer arrivals. Dror., Trudean, and Ladany (1988) proposed a similar network model, again with deterministic demand. The proposed improvements allow for cancelations, which often happens in the real booking process. Booking methods based on linear programming were thoroughly investigated by Williamson (1992). The basic models take stochastic demand into account only through expected values, thus yielding a deterministic program that can be easily solved. However, the drawback of such approach is obvious, as it ignores any distributional information about the demand. Furthermore, there is a method called nesting which arranges all the booking classes by their net contributions. A higher contribution implies a higher class. In a single leg setting, the net contributions are the fare levels; in a network environment, the net contributions are the shadow 6

19 prices for legs involved. Usually, higher class customers could reserve seats from all the allocations assigned to lower booking classes. Bertsimas and de Boer (2003) and van Ryzin and Vulcano (2003) give a detailed algorithm about nesting under uncertain demand. However, the nesting method encounters a modeling difficulty. Also, it is hard to evaluate revenue analytically rather than through simulation because the customer arrival sequence involved. We provided detailed analysis in the Section 2.5. Hence, in this dissertation, we did not simulate any nesting solutions. A common way to attempt to overcome that problem is to re-solve the LP several times during the booking horizon. While such an approach may seem intuitive, it turns out that re-solving can actually backfire indeed, Cooper (2002) shows a counter example where re-solving the LP model may lower the total expected value. An alternative way to incorporate demand distribution information into the model is by formulating a stochastic linear program (SLP). In the simplest case, an SLP contains two stages in the first stage decisions are made before the uncertainty is realized, and in the second stage (i.e. after uncertainty is realized) a correction is made. Such a correction is called recourse. Stochastic programs have been used in a wide range of settings; see, for instance, Birge and Louveaux (1997) for a comprehensive treatment. In the particular case of airline bookings, such models typically reduce to a subclass of recourse models called simple recourse, a formulation that is called probabilistic nonlinear program in the revenue management literature (see, e.g., Williamson (1992)). Higle and Sen (2001) propose a different stochastic programming model, based on leg-based seat allocations, which yields an alternative way to compute bid prices. Their experiments suggest that the bid price policy derived from the leg-based stochastic programming model tends to yield higher revenues than the bid price policy generated by the standard deterministic linear programming model, especially when the number of fare classes is not very large; we refer the reader to 7

20 that paper for details. Another stochastic optimization model is proposed by Cooper and Homem-de-Mello (2003), who consider a hybrid method where the second stage is actually the optimal value function of a dynamic program. A natural extension of a two-stage stochastic program is a multi-stage stochastic program (MSSP). Instead of making an initial decision and making adjustments at the end as in the two-stage case, the basic idea of MSSPs is to revise the decisions periodically over time, taking into account the realization of the uncertainty up to that point. Thanks to this flexibility, it is clear that multi-stage models produce better results than two-stage models; the price to pay, of course, is that such problems are harder to solve. We refer again to Birge and Louveaux (1997) for a detailed discussion. In this dissertation, we focus on various aspects of the multi-stage version of the simple recourse model discussed above (henceforth denoted SLP). We show that, while MSSP does provide a better policy than SLP in terms of expected total revenue, the loss of computational efficiency resulting from the structure of the problem is costly simple recourse models can be solved very efficiently with linear integer programming, whereas the multi-stage extension does not have convexity or concavity properties (even its continuous relaxation). Lack of convexity/concavity prevents the application of most techniques developed for multi-stage models (see, e.g., Birge (1985), Birge, Donohue, Holmes, and Svintsiski (2005), Gassmann (1990)). Given the difficulty to develop an exact algorithm for the underlying MSSP, we propose an approximation based on solving a sequence of two-stage simple recourse models. The main advantage of such an approach is that, as mentioned above, each two-stage problem can be solved very efficiently, so an approximating solution to the MSSP can be obtained reasonably quickly. The idea of solving two-stage problems sequentially is not new, and appears in the literature under names such as rolling horizon and rolling forward; see, for instance, Balasubramanian and Grossmann (2004), 8

21 Bertocchi, Dupacova, and Moriggia (2001), Kusy and Ziemba. (1986). The details on the implementation of the rolling horizon, however, vary in the above works. Our work is more closely related to Balasubramanian and Grossmann (2004) in that we consider shrinking horizons, i.e., each two-stage problem is solved over a period spanning from the current time until the end of the booking horizon. This is called the re-solving SLP approach. Although the rolling horizon approach has been proposed in the literature, to the best of our knowledge there have been no analytical results regarding the quality of the approximation. We provide some results of that nature, though we do not claim to give definitive answers. More specifically, we compare the policies obtained from the re-solving SLP approach with the policies from SLP model without re-solving and from MSSP. Our results indicate that solving the SLP sequentially cannot be worse (in terms of expected revenue of the resulting policy) than solving it only once a result that, although seemingly intuitive, does not hold when one uses the deterministic LP model Gupta and Cooper (2002). We also show that, for a given partition into stages, the policy from MSSP is better than the policy from re-solving SLP. However, the inclusion of just one extra re-solving point can make the re-solving approach better. The importance of this conclusion arises from the fact that, because of the sequential nature of the re-solving procedure, adding an extra re-solving point requires little extra computational effort; in comparison, including an extra stage in a multi-stage model makes the problem considerably bigger and therefore harder to solve. Motivated by the flexibility of the re-solving approach, we also study the issue of whether one can improve the results by carefully choosing the re-solving points instead of using equally sized intervals as it is usually done. Indeed, the structure of our problem allows us to do so, and we provide a heuristic algorithm to determine the 9

22 re-solving points. Our numerical results, run for two medium-sized networks, indicate that the procedure is effective Research on the customer choice behavior models In practice, customers are not completely segmented into classes. Consider the following live example in Figure 1.3. I was booking a ticket from Salt Lake City to Memphis for a business trip. Figure 1.3. Illustration of customer choice There were two possible choices, Delta 1231/5459 or Delta 1231/5461, that would allow me to catch a meeting in the afternoon. However, there is a big difference 10

23 between prices of all the possible choices. In order to control the budget, I would like to choose flight 1231/5459 to save some money. This illustrates the case where a customer may be willing to buy an expensive ticket for a particular class on a particular itinerary, but if a lower-priced ticket is available on the same flight he or she will likely take it. Such a shortcoming of the segmented-demand model has long been recognized by academicians and practitioners alike. However, that practice was considered relatively harmless until it was realized that this model deficiency was leading to progressive deterioration of revenues, even though an underlying optimization model was being used. To illustrate the point, suppose that there are two classes of tickets on a single flight and that customers choose to buy a low-fare ticket unless there is none available, in which case they purchase the high-fare one. Suppose also that the airline chooses how many seats to reserve for high-fare tickets based on past sales of high-fare tickets, while neglecting to account for the fact that availability of low-fare tickets will reduce sales for high-fare tickets. Then, if more low-fare tickets are made available, low-fare sales will increase and high-fare sales will decrease, resulting in lower future estimates of high-fare demand, and consequently leading to smaller availability of high-fare tickets and greater availability of low-fare tickets. The pattern continues, resulting in a progressive decrease of high-fare sales and revenues. This phenomenon, called spiral-down, was first demonstrated through simulation by Boyd, Kambour, and Tama (2001), and subsequently analyzed in detail by Cooper, Homem-de-Mello, and Kleywegt (2005). A few papers in the literature deal with models for the customer choice behavior in the airline setting. Belobaba (1989) models a heuristic called the expected marginal seat revenue (EMSR) to allocate seats to the different classes. The idea assumes that every customer has a non-homogenous probability distribution on choosing multiple tickets. Although this work is based on the independent demands assumption 11

24 across classes, the buying up-down model could be an excellent method for modeling customer choice behavior on discounted seats. Belobaba and Weatherford (1996) and Bodily and Weatherford (1995) build their heuristics on the modified EMSR model for solving customer choice in the single leg problem. Brumelle, McGill, Oum, Sawaki, and Tretheway (1990) provide the optimal condition for the single leg customer buying-up model. Talluri and van Ryzin (2004a) provide an exact analysis of a single leg model of RM under a general discrete choice model of demand. Despite all those works reveals the customer choice in single leg problems, those models have two common drawbacks: first, the operation in major airline is largely based on the network environment which is expected to be a large and multiple resources problem; second, all those models require the information of the next customer s behavior. Upon each request, we should know the probabilities for all the customer choices at booking class level. Both academia and industrial partners realize that a customer choice model for the network environment would be a great step forward. Currently, most customer choice models are usually built by dynamic programming. However, the problem scale becomes more considerably huge to solve. To simplify the model, researchers look for special structures and assumptions. For example, Zhang and Cooper (2004) analyze the customer choice among different departure times between the same city pair. They assume the customer will only make a choice among alternatives within the same booking classes. Their model suggests that the customer cares about the flight schedule most. They also developed the bounds and approximations for the dynamic programming model. van Ryzin and Liu (2004) discussed a choice based deterministic linear programming model which is to determine the amount of time to offer each possible subset of available products. The author propose a dynamic decomposition heuristic method to solve this problem. They applied simulation based 12

25 method to calculate virtual control offer set at certain time. This method avoids the computational obstacle in dynamic programming. At the same time, since it is a simulation based method, it might be hard to reach an analytic conclusion. In our research, we propose a new model for the seat allocation problem in a network environment that takes customer choices into account. A central concept to our work is that of the customer preference order to describe the gross customer behavior regarding the order of classes for which they will try to purchase tickets. If the customer s first choice is not available, she will either try her second choice or decide not to purchase anything. If her second choice is not available either, then again either she moves to her next choice or decides not to purchase anything, and so on. We model each customer s decision made at each step i.e., between trying the next choice or leaving the system as a Bernoulli random variable with known probability. Our basic assumption is that customers can be grouped into categories so that all customers in the same category have the same preference order and the corresponding probabilities reflect well their common behavior. Moreover, we assume that the booking requests for each category are mutually independent with known distribution. Based on the above assumptions, we propose a stochastic programming model to determine the number of seats to be allocated to each class within each preference order. The form of that model prevents the use of efficient solution techniques; hence, we propose an approximating model, whereby the random number of buyingup customers is replaced with its expected value. We show that the approximating problem is a two-stage stochastic integer program, which can then be efficiently solved. Despite its convenience in terms of allowing for solution techniques, our model does not produce an allocation policy that is ready to be implemented. Its drawback comes mainly from the fact that the allocations for each class are fragmented across 13

26 the various preference orders in which that class appears. To remedy that problem, we use a post-optimization heuristic called backup policy, which essentially combines the allocations of the various fragments for each class. Our numerical experiments suggest that the use of the backup policy considerably improves the revenue Our airline revenue management solution An airline revenue management department is a quantitative work related department. It takes models inputs from the planning and scheduling department, sales and marketing department, and pricing department. The overall model output should be a flexible seat allocation for the distribution department to use. All the operations are initialized by scheduling and the planning department and ended by distribution department. Figure 1.4. The position of revenue management department Our research focuses on the operation inside the revenue management department only. Clearly, our research has two parts, the seat allocation optimization and the customer preference order model. Although being discussed individually, they are 14

27 highly related and should be integrated in the revenue management process altogether. Applying the seat allocation model only would have to assume highly correlated demands which are not the case in the real business. On the other hand, the customer preference order model only functions as a way to incorporate the demands correlation. Therefore, our total solution for airline revenue management would be shown in Figure 1.5. Figure 1.5. Our total solution for airline revenue management All those modules would be inside the revenue management department and we believe this method would be compatible with all the other airline operations, such as planning and pricing. This property gives us advantage to move forward for business implementation. In the next several chapters, we would cover detailed discussion on both parts. 15

28 CHAPTER 2 MULTI-STAGE STOCHASTIC PROGRAMMING IN REVENUE MANAGEMENT 2.1. Model description Following the standard models in the literature, we consider a network of flights involving p booking classes of customers. This model can represent demand for a network of flights that depart within a particular day. Each customer requests one out of n possible itineraries, so we have r := np itinerary-fare class combinations. The booking process is realized over a time horizon of length τ. Let {N jk (t)} denote the point process generated by the arrivals of class-k customers who request itinerary j. Typical cases customary in the revenue management literature are (i) {N jk (t)} is a (possibly nonhomogeneous) Poisson process, and (ii) there is at most one unit of demand per time period. we assume that arrival processes corresponding to different pairs (j, k) are independent of each other. The demand for itinerary-class (j, k) over the whole horizon is denoted by ξ jk (i.e., ξ jk = N jk (τ)) and we denote by ξ the whole vector (ξ jk ) jk. The network is comprised of m leg-cabin combinations, with capacities c := (c 1,..., c m ), and is represented by an (m np)-matrix A (a i,jk ). The entry a i,jk {0, 1} indicates whether class k customers use leg i in itinerary j. Most policies we deal with are of allocation type. We denote by x jk the decision variable corresponding to the number of seats to be allocated to class k in itinerary j. Whenever a itinerary-class pair (j, k) is accepted, the revenue corresponding to the fare f jk accrues. In that case, of course, a seat is assigned to that customer, so 16

29 there will be fewer seats available to sell to future customers (we do not consider overbooking in our model). A customer s request is rejected if no seats are available for his itinerary-class, in which case no revenue is realized. The vectors of decision variables and fares are denoted respectively by x = (x jk ) and f = (f jk ) Allocation methods Allocation methods require solving an optimization problem to find initial allocations before the booking process starts. Suppose for the moment that we want to keep these initial allocations throughout the horizon. Then, since all costs and constraints are linear, we can formulate the following linear program: max f T x [LP] subject to Ax c, x ξ w.p.1 x 0. Despite its apparently simplicity, the above model is not practical, because of the constraint x ξ. Such a constraint implies that either x must be less than any possible realization of ξ which is too strong a restriction or that x is determined after the random vector ξ is realized. In our model, however as is the case with most practical problems where uncertainty is involved the decision must be made in advance. Therefore, formulation [LP] is not adequate. A simple attempt to overcome the problem with the constraint x ξ is to replace the random vector ξ with its expected value. The resulting deterministic linear 17

30 program (DLP) is then written as follows: max f T x [DLP] subject to Ax c, x E[ξ] x 0. Model [DLP] is well known in the revenue management literature, and indeed it is often used by airline companies thanks to its simplicity. Implementation of the resulting policy x is very simple accept at most x jk class k customers in itinerary j. Notice that the policy is well-defined even if the solution x is not integer. Notice also that the objective function of [DLP] is not the actual revenue resulting from the policy x, even if x is integer. A major drawback of formulation [DLP] is the fact that it completely ignores any information regarding the distribution of demand, except for the mean. It is well known that, in general, the introduction of information about the distributions of the underlying random variables leads to richer models although there are exceptions to this rule, see e.g. Williamson (1992) for a discussion in the context of nested allocations. This suggests that, in order to capture the randomness inherent to the problem, one should replace model [DLP] by something that explicitly models randomness. As seen above, formulation [LP] is not the answer either. As discussed extensively in Birge et al. (2005), rather than trying to find a solution for each specific scenario, one should compute a solution that is good for all the possible scenarios, 18

31 in terms of maximizing the expected total revenue. This leads to the formulation max f T E[min{x, ξ}] subject to Ax c, x Z +, where the min operation is interpreted componentwise (note that we have imposed an integrality constraint on x to ensure we get an integer allocation of seats; we will comment on that later). Equivalently, we have (see also Higle and Sen (2001)) max f T x + E[Q(x, ξ)] [SLP] subject to: Ax c, x Z +, where Q(x, ξ) = max { f T y x y ξ, y 0}. Notice that [SLP] is in the standard form of two-stage stochastic programs. In particular, it is a two-stage integer problem with simple recourse. A major advantage of such models is that, when ξ has a discrete distribution with finitely many scenarios, problem [SLP] can be easily solved because of its special structure. In principle, this may not be the case of our model, for example when the total demand for each itinerary-class pair (j, k) has Poisson distribution, which has infinite support. It is clear, however, that in that case all but a finite number of points in the distribution have negligible probability; thus, we can approximate the distribution of ξ jk by a 19

32 truncated Poisson. Thus, in what follows we assume that ξ takes on finitely many values, and that those values are integer. To describe the solution procedure, we need to introduce some notation. For each itinerary-class pair (j, k), let {d 1 jk,..., ds jk jk } denote the possible values taken by ξ jk, ordered from lowest to highest. Let {δ s jk }, s = 1,..., S jk be coefficients defined as δjk s := f jkp (ξ jk d s jk ). As discussed in Birge and Louveaux (1997) and Kall and Wallace (1994), problem [SLP] can then be re-written as max x,u,u 0 f T x j,k S jk s=1 δ s jku s jk (2.1) subject to: S jk Ax c u 0 jk + u s jk x jk = E[ξ jk ] s=1 u 0 jk d 1 jk E[ξ jk ] 0 u 1 x Z +. Notice that the decision variables of the linear integer program (2.1) are the vectors x = (x jk ), u 0 := (u 0 jk ) and u := (us jk ), s = 1,..., S jk. That is, problem (2.1) has O(Snp) variables and O(Snp + m) constraints (where S := max j,k S jk ), which is far smaller than the deterministic linear program corresponding to general two-stage programs with finitely many scenarios in that case, it is well known that the number of constraints and variables is exponential on the number of scenarios, see for instance Birge and Louveaux (1997). Thus, problem (2.1) can be solved by standard linear integer programming software. 20

33 It is worthwhile pointing out that the objective function of [SLP] does correspond to the actual expected revenue resulting from a feasible policy x though this is not true if the integrality constraint is relaxed. Note also that the solution obtained from [DLP] yields the same expected revenue as its truncated version. Moreover, it is easy to see that a truncated feasible solution to [DLP] is feasible for [SLP]. An immediate consequence of these facts is that the optimal policy calculated from [SLP] is never worse than that of the DLP model in terms of expected total revenue. We emphasize that this is true in the present context of simple allocation policies as mentioned earlier, it has been observed that such a property does not hold for nested policies Multi-stage models As discussed above, by taking uncertainty into account the SLP model yields a better policy than the deterministic model DLP. However, both models have a major drawback they are static, in the sense that they determine a policy at the beginning of the horizon and follow this policy throughout the booking period. It is plausible to think that, in a practical setting, one would revise the policy from time to time, taking into account the information about demand learned so far. This is precisely the idea of multi-stage stochastic programming (MSSP) models. We describe next a MSSP formulation for the booking problem under study. Suppose we divide the time horizon [0, τ] into H + 1 stages numbered 0, 1,..., H. The stages correspond to some partition 0 = t 0 < t 1 <... < t H 1 < t H = τ of the booking horizon, so that stage 0 corresponds to the beginning of the horizon and stage h (h = 1,..., H) consists of time interval (t h 1, t h ]. The decision variables at each stage are denoted x 0,..., x H, where x h = (x h jk ) j,k. Also, we associate with each stage h, h = 1,..., H, random variables ξ h jk representing the total demand for itinerary-class (j, k) between stages h 1 and h, that is, ξ h jk = N jk(t h ) N jk (t h 1 ). 21

34 We denote by ξ h the random vector (ξ h jk ) j,k. Notice that the decision vector x h at stage h is actually a function of x 0, x 1,..., x h 1 and ξ 1,..., ξ h. The resulting multi-stage model is written as follows: [ max f T x 0 + E ξ 1 Q1 (x 0, ξ 1 ) ] [MSSP] subject to Ax 0 c x 0 Z +. The function Q 1 is defined recursively as Q h (x 0,..., x h 1, ξ 1,..., ξ h ) = max x h f T x h f T [x h 1 ξ h ] + + E ξ h+1 [ Qh+1 (x 0,..., x h, ξ 1,..., ξ h+1 ) ] subject to h Ax h c A min{x m 1, ξ m } x h Z +, m=1 (h = 1,..., H 1), where [a] + := max{a, 0} and the max and min operations are interpreted componentwise. Notice that we use the notation E ξ h+1 [ Qh+1 (x 0,..., x h, ξ 1,..., ξ h+1 ) ] 22

35 to indicate the conditional expectation E [ Q h+1 (x 0,..., x h, ξ 1,..., ξ h+1 ) ξ 1,..., ξ h]. For the final stage H We have Q H (x 0,..., x H 1, ξ 1,..., ξ H ) = f T x H, where x H = [x H 1 ξ H ] +. Observe that in the above formulations we use the equality E[min{a, Y }] = a E[[a Y ] + ], where a is deterministic and Y is a random variable. From the discussion above, we see that the multi-stage stochastic program [MSSP] is, in principle, the best model for the problem under study, since it takes the stochastic and dynamic features of the problem into account. More generally, multi-stage stochastic programming models have proven to be very suitable for a variety of applications, from finance to energy planning see e.g., Birge and Louveaux (1997). Several methods have been developed for the case where all stages involve linear problems. One of the most well-known algorithms is an extension to the so-called L-shaped algorithm developed for two-stage models by Slyke and Wets (1969). Roughly speaking, the L-shaped method computes successive outer linearizations of the expected recourse function without violating the feasibility constraints. Each newly generated cut leads to a relaxed model, and cuts are added to the relaxed model until the optimal solution is reached. Birge (1985) extended this idea to multistage stochastic linear programs, see also Birge et al. (2005) and Gassmann (1990). One requirement for the above mentioned algorithms to work is that the expected recourse function be convex (or concave, for a maximization problem). Otherwise, the linearization may not constitute an upper envelope to the objective function, and the algorithm may not converge to the optimal solution. Moreover, these algorithms were devised for continuous problems research on multi-stage integer problems is 23

36 ongoing. Unfortunately, model [MSSP] described above does not fit that framework. In fact, as we show below, even the continuous relaxation of that problem does not have a concave expected recourse function. Proposition 1. The continuous relaxation of model [MSSP] is neither convex nor concave. Proof. It suffices to show a counter-example. To do so, consider a three stage programming problem. The second stage recourse function is Q 1 (x 0, ξ 1 ) = [ max f T x 1 f T [x 0 ξ 1 ] + + E ξ 2 Q2 (x 1, ξ 2 ) ] x 1 subject to Ax 1 c A min{x 0, ξ 1 } x 1 0, where Q 2 (x 1, ξ 2 ) = f T [x 1 ξ 2 ] +. Suppose now that the model corresponds to a single-leg, two-class problem, with the capacity equal to 9. There are two independent time periods. The demands for the two time periods are deterministic, respectively ˆξ 1 = (2, 6) and ˆξ 2 = (3, 7). Notice that it suffices to show that under this specific scenario the function Q 1 (, ˆξ) is neither convex nor concave, because we can assign probabilities to make this scenarios significant. The fares for each class are respectively f 1 = $500 and f 2 = $100. Figure 2.1 below shows the graph of the recourse function Q 1 (x 0, ˆξ 1 ) as a function of the vector x 0. Notice that the negative values in the graph are artificial and simply correspond to values of x 0 that are infeasible, i.e., x 0 1 +x 0 2 > 15. The lack of concavity or convexity is evident from the picture. In particular, we have Q 1 ([2, 5], ˆξ 1 ) =

37 and Q 1 ([2, 7], ˆξ 1 ) = 400. The average of these values is 1400/2 = 700, which is larger than 500, the value of Q 1 at the midpoint [2, 6] Figure 2.1. Graph of the recourse function in a three-stage problem The lack of concavity of the recourse function imposes difficulties to solve problem [MSSP]. Notice that this obstacle exists only for problems with three or more stages. Based on the latter property, in section 2.3 we will discuss an alternative approach to solve the allocation problem Re-solving SLP model A natural alternative to the multi-stage approach described in section is to revise the booking policy from time to time by re-solving a simpler model such as [SLP]. While intuitively it can be argued such an approach may yield policies that are inferior to the ones given by [MSSP] which by construction finds the optimal dynamic policy, we believe that the re-solving approach is worth studying, for several reasons. First, as discussed before, the multi-stage model [MSSP] lacks 25

38 the convexity structure that is an essential requirement for the use of efficient algorithms; consequently, solution of [MSSP] is likely to be computationally intensive, especially for large networks. Re-solving a problem such as [SLP], on the other hand, is reasonably fast, since as described earlier each instance of [SLP] is equivalent to a moderately sized linear integer program. Finally, it is clear that the complexity of [MSSP] increases rapidly as the number of stages grow, since the number of decision variables and constraints becomes larger; the complexity of re-solving, on the other hand, clearly grows linearly with the number of stages. We formalize now the re-solving approach. As in the case of the multi-stage model, we partition the booking time horizon [0, τ] into segments {0}, (0, t 1 ], (t 1, t 2 ],..., (t H 1 + 1, τ] with 0 = t 0 < t 1 <... < t H 1 < t H = τ, and define ξ h, h {1,..., H} as the demand during the time interval (t h 1, t h ]. By ˆξ h we denote a specific realization of the random variable ξ h. We initially (i.e. at time 0) solve the following problem: [ { }] H max f T E min x, ξ m m=1 [SLPR-0] subject to Ax c x Z +. 26

39 Let x 0 denote the optimal solution of the above problem. Then, at each time t h, h = 1,..., H 1 we solve [ { } ] H max f T E min x, ξ m ξ 1 = ˆξ 1,..., ξ h = ˆξ h m=h+1 [SLPR-h] subject to h Ax c A min{x m 1, ˆξ m } x Z + m=1 and denote the optimal solution by x h. Note that the above model makes use of the information up to time t h this is the reason why the constraints involve the realizations {ˆξ m } instead of the random variables {ξ m }. Also, as seen in section 2.2 each model [SLPR-h] is relatively easy to solve, since it can be formulated as a not very large linear integer program. Notice that the expectation in the objective function of [SLPR-h] is calculated with respect to the overall demand from period h + 1 on. The idea is that this would be the problem one would solve if it was assumed that no more re-solving would occur in the future. The idea of re-solving an optimization model to use the available information is not new. In the context of stochastic programming, this is sometimes called the rolling forward approach in the literature, see for instance Bertocchi et al. (2001) and Kusy and Ziemba. (1986). The specific aspects of each problem, however, lead to different ways to implement the rolling mechanism. For example, in Bertocchi et al. (2001) a two-stage model is initially solved where the realizations at the second stage correspond to all possible scenarios of the original problem. That is, such a problem can be enormous and must be solved via some sampling method (see 27

40 Shapiro and Ruszczyński (2004) for a compilation of results). In contrast, in our case the two-stage program [SLPR-0] deals only with the total demand H m=1 ξm. This reduces the number of scenarios drastically, especially when the distribution of H m=1 ξm can be determined directly. Such is the case, for example, when the demand for each itinerary-class (j, k) arrives according to a Poisson process in that case, H m=1 ξm has a Poisson distribution, which then can be truncated as discussed in section 2.2. This results in tractable two-stage simple recourse problems that can be solved exactly. The approach of considering two-stage problems with increasingly smaller horizons has been used by Balasubramanian and Grossmann (2004) in the context of a multiperiod scheduling problem. They call it the shrinking horizon framework. Their motivation is similar to ours to provide a practical scheme for a difficult multistage integer problem. Notice however that in our case we have the additional issue of lack of convexity/concavity, as discussed in section The idea of re-solving an optimization model to take advantage of the information accumulated so far has been a common practice in revenue management applications. While intuitive, such an approach may actually backfire if not used carefully. Indeed, Cooper (2002) shows a simple counter example where re-solving [DLP] leads to a worse policy (in terms of expected revenue) than what would be obtained if one had kept the original policy throughout. As we show next, this does not happen in case [SLP] is re-solved. Proposition 2. The policy obtained from using models [SLPR-h], h = 0,..., H 1, yields an expected revenue which is greater or equal to that given by the policy obtained from solving [SLPR-0] only. 28

41 Proof. It suffices to show that the first re-solving cannot worsen the expected revenue. Let x 0 denote the optimal solution of [SLPR-0]. Then, during the time interval (0, t 1 ], the booking allocation policy is x 0 regardless of whether we are resolving or not. The difference happens after time t 1. For the non-re-solving process, the policy to be used from time t + 1 on is x 1 := x 0 min{x 0, ˆξ 1 } because the nonre-solving process continues to apply the initial policy. The policy for the re-solving process is obtained by solving (SLPR-1), i.e., [ { } ] H max f T E min x, ξ m ξ 1 = ˆξ 1 m=2 subject to Ax c A min{x 0, ˆξ 1 } x Z +. Notice that Ax 1 = Ax 0 A min{x 0, ˆξ 1 } c A min{x 0, ˆξ 1 } for any possible realization of ξ 1. That is, x 1 is a feasible solution for the re-solving model, which means that the policy obtained by re-solving cannot be worse than x 1. Since the objective function of [SLPR-1] is the expected revenue from time t + 1 follows that re-solving can only improve upon the initial policy. on, it Proposition 2 shows that solving [SLPR-h] successively will keep improving the booking policy. Notice that the re-solving method is somewhat similar to the multistage model in the sense that both yield dynamic booking policies that take the information available so far into account. As mentioned earlier, it is intuitive that in general [MSSP] gives a better solution. The proposition below formalizes that result. 29

42 Proposition 3. Under the same partition setting, the booking policy from multistage model [MSSP] yields an expected revenue which is greater or equal to that given by the policy obtained from solving [SLPR-h] successively. Proof. Let Ω h be the set of all possible sample paths of (ξ 1,..., ξ h ). A feasible solution for problem [MSSP] has the form (2.2) x 0 H h=1 ˆξ 1,...,ˆξ h Ω h x h (ˆξ 1,..., ˆξ h ), where indicates the cartesian product and each component x h (ˆξ 1,..., ˆξ h ) satisfies (2.3) Ax h (ˆξ 1,..., ˆξ h h ) c A min{x m 1 (ˆξ 1,..., ˆξ m 1 ), ˆξ m }. m=1 On the other hand, consider model [SLPR-h] under a specific realization ξ 1,..., ξ h in the scenario tree, and denote its optimal solution by x h ( ξ 1,..., ξ h ). Consider the vector formed by the cartesian product of such solutions for all realizations and all stages, i.e., x 0 H h=1 ξ 1,..., ξ h Ω h x h ( ξ 1,..., ξ h ), It is clear that the resulting vector has the form (2.2). Moreover, since the [SLPRh] problem has the constraints Ax h c A h m=1 min{xm 1, ξ m }, it follows that x h ( ξ 1,..., ξ h ) satisfies (2.3). Therefore, the combined solution from the [SLPR-h] models is actually a feasible solution in [MSSP]. Despite the above result, the proposition and corollary below show that, under perfect information, re-solving is in fact optimal. Proposition 4. Under perfect information, the policies given by models [SLPR-0] and [MSSP] are equivalent, in the sense that they yield the same expected revenue. 30

43 Proof. First let us write problem [MSSP] in a single mathematical program. we need one decision variable for each stage h and each possible sample path (ˆξ 1,..., ˆξ h ) up to stage h, which we shall denote x h (ˆξ 1,..., ˆξ h ). Then, [MSSP] can be written as max H... f T min{x h 1 (ˆξ 1,..., ˆξ h 1 ), ˆξ h }p(ˆξ 1,..., ˆξ h ) ˆξ 1 ˆξ h h=1 [MSSP-exp] subject to Ax 0 c Ax 1 (ˆξ 1 ) c A min{x 0, ˆξ 1 } Ax 2 (ˆξ 1, ˆξ 2 ) c A min{x 0, ˆξ 1 } A min{x 1 (ˆξ 1 ), ˆξ 2 }. Ax H 1 (ˆξ 1,..., ˆξ H 1 H 1 ) c A min{x h 1 (ˆξ 1,..., ˆξ h 1 ), ˆξ h } h=1 ˆξ 1,...,H 1 x Z +, In the above, the notation ˆξh means the summation over all possible realizations of the random vector ξ h. Also, p(ˆξ 1,..., ˆξ h ) denotes the probability of the sample path (ˆξ 1,..., ˆξ h ). Suppose now there is perfect information, i.e. there is only one sample path, which we denote by ( ξ 1,..., ξ H ). Each decision x h is made with knowledge of the whole 31

44 vector ( ξ 1,..., ξ H ). Then, [MSSP-exp] is written as max H f T min{x h 1, ξ h } h=1 (2.4) subject to Ax 0 c Ax 1 c A min{x 0, ξ 1 } Ax 2 c A min{x 0, ξ 1 } A min{x 1, ξ 2 }. H 1 Ax H 1 c A min{x h 1, ξ h } x Z +. h=1 It is clear from the above formulation that, if x := ( x 0,..., x H 1 ) in an optimal solution for (2.4), then so is (min{ x 0, ξ 1 },..., min{ x H 1, ξ H }). That is, we can rewrite 32

45 (2.4) as max H f T x h 1 h=1 (2.5) subject to Ax 0 c (2.6) (2.7) Ax 1 c Ax 0 Ax 2 c Ax 0 Ax 1. (2.8) H 1 Ax H 1 c A h=1 x h 1 x h 1 ξ h h = 1,..., H x Z +. Since the region defined by each constraint (2.6)-(2.8) contains the region defined by the next inequality (recall that A has only non-negative entries), it follows that the above problem can be simplified to max f T H x h 1 h=1 (2.9) subject to H A x h 1 c h=1 x h 1 ξ h h = 1,..., H x Z +. 33

46 Consider now the problem max f T y (2.10) subject to Ay c y H ξ h h=1 y Z +. Notice that (2.10) is precisely problem [SLPR-0] under perfect information. Thus, to show the property stated in the proposition, it suffices to prove that the policies derived from (2.9) and (2.10) are equivalent. To do so, define C y := {x feasible in (2.9) : H h=1 xh 1 = y}. Let F be a mapping from {C y : y R np } into the feasible region of (2.10), defined as F (C y ) = y. Consider now the mapping G that represents the application of the policy obtained from [SLPR-0]. We can express G as a mapping from the feasible region of (2.10) into R H np, defined as follows. For each pair (j, k), let a jk H be the largest number such that y jk a jk ξ h=1 jk h. Then, we define G as G h 1 (y) jk := ξ h jk 1 h a jk y jk h 1 ξ m=1 jk m h = a jk a jk + 2 h H. Notice that from the above definition we have 0 G h 1 (y) jk ξ h jk for all h = 1,..., H. Moreover, H G h 1 (y) jk = h=1 a jk h=1 ξ h jk + y jk a jk m=1 ξ m jk + H h=a jk +2 0 = y jk. 34

47 It follows that G(y) := (G 0 (y),..., G H 1 (y)) is feasible for (2.9) and, in addition, H h=1 Gh 1 (y) = y. That is, G(y) C y. By viewing C y as an equivalence class, we see that G is actually the inverse of F, i.e., F is a one-to-one mapping. Now, since F preserves the objective function value, we conclude that the policies from (2.9) and (2.10) are equivalent. Propositions 2, 3 and 4 together yield the following result. Corollary 1. Under the same partition into stages and perfect information, the policies given by models [SLPR-h], h = 0,..., H 1 and [MSSP] are equivalent Bid-price methods It is well known that allocation policies suffer from a drawback resulting from the rigid segmentation of customers into classes. For instance, it could happen that all seats allocated to a high-fare class are sold, while some low-fare class seats might still be open for booking. Thus, all the requests for the high-fare class thereafter would be rejected even though there are seats available on the plane, which at best will be sold to a low-fare customer. This is obviously not an optimal booking policy. Some methods have been proposed in the literature to address this problem. One of the methods is called nesting, which roughly speaking means that high-fare customers can use seats that had been initially allocated to lower-fare classes. While this is intuitive in a single-leg setting, in a network environment it is not immediate how to implement such a mechanism, since there is no natural ranking of classes. There are some heuristic methods to construct a ranking, such as the ones proposed in Bertsimas and de Boer (2003) and de Boer., Freling, and Piersma., which yield a nested allocation policy. Another well-known approach is the bid price method. The basic idea of this method is to assign prices to resources, and then accept only the requests whose 35

48 contribution is at least the same as the total bid price of resources used by that request. In the context of airline booking, this means each leg has an incremental price. A booking request corresponds to seat occupation in one or more legs; the sum of the incremental prices for those legs is called bid price for this request. Then, the request is accepted only if its fare is greater or equal to that amount. Notice that this method automatically provides a form of nesting even in a network environment, since by construction it cannot happen that a low-fare customer is accepted while a high-fare request is rejected. A natural way to determine the bid prices is to consider the expected benefit of accepting a customer. Suppose that a request for class i (with fare f i ) arrives at time t, and let u = (u 1,..., u m ) denote the vector of unsold seats on each leg at that point. Let J t (u) be the expected total revenue from time t + to the end given the vector u of available seats. Then the bid price is J t (u) J t (u e i ), i.e., the expected revenue obtained if the request is rejected minus the expected revenue if the request is accepted (e i is the vector with 1 in component i and zeroes otherwise). If f i J t (u) J t (u e i ), the request should be accepted, otherwise rejected. The difficulty with this approach is that evaluation of J t (u) and J t (u e i ) can be timeconsuming. Bertsimas and Popescu (2003) propose using the value of a deterministic network linear programming problem in place of J t, which allows for fast calculation because of the structure of the problem. An alternative way to determine bid prices is through the dual variables of the allocation problems discussed in section 2.2. The argument for this approach is based on two premises: first, the cost-to-go function J t (u) can be approximated by the optimal value v(u) of a mathematical programming problem such as [DLP] or [SLP], with capacity vector equal to u; second, the difference v(u) v(u e i ) can be approximated by s T e i, where s is a sub-gradient of v( ) at u. Under proper assumptions 36

49 in particular, when v( ) is convex or concave it is known that s is given by the dual multipliers corresponding to the constraints Ax u. Thus, the approach works well when the two approximations referred to above are reasonable. Williamson (1992) studies the case where the bid prices are the dual variables of [DLP]. This method is quite simple and easy to use. However, as pointed out in Talluri and van Ryzin, it may behave poorly. As a remedy, Talluri and van Ryzin propose a method based on a randomized linear programming. Essentially, the idea of the method is to approximate the cost-to-go function J t (u) by the optimal value v P I (u) of problem [SLP] with capacity vector equal to u under perfect information. That is, v P I (u) := E[V (u, ξ)] where V (u, ξ) = max{f T x Ax u, 0 x ξ}. Since V is a concave function (jointly in (u, ξ)), it follows that the sub-differential set v P I (u) is given by E[V (u, ξ)] = E[ V (u, ξ)]. Therefore, one can estimate a sub-gradient of v P I ( ) by drawing a sample ξ 1,..., ξ N from the distribution of ξ and averaging the dual multipliers of the problems max{f T x Ax u, 0 x ξ i }, i = 1,..., N. This estimate is then taken as the bid price for the problem. One significant advantage of this approach over the bid prices from [DLP] is that it incorporates distributional information on future demands. An alternative method for calculating the bid price, also suggested in the literature, hinges on the two stage stochastic programming model [SLP]. That is, one approximates the cost-to-go function J t (u) by the optimal value v SLP (u) of problem [SLP] with capacity vector equal to u. Notice the distinction between this approach and the method of Talluri and van Ryzin v SLP (u) is the optimal value of a hereand-now problem, whereas v P I (u) corresponds to a wait-and-see approach. A related approach is described in Higle and Sen (2001), where the cost-to-go function J t (u) is approximated by a different here-and-now problem (a leg-based seat allocation formulation). 37

50 Although the here-and-now methods seem to be more promising in the sense that the true cost-to-go function J t (u) is of here-and-now nature they may still generate a worse policy, even comparing with the DLP method. Williamson (1992) performed extensive simulation experiments on bid price policies. Her results show that the bid price method from the deterministic model is not necessarily worse than that from the stochastic model as one might expect. It is difficult however to provide a theoretical comparison between the various methods. The reason is that the expected total revenue from bid price is difficult to be evaluated analytically, since the arrival order plays a role in the calculations. To illustrate this point, consider a single-leg problem with two classes, and let f 1 > f 2 be the respective revenues. Suppose we set the bid price at some value p < f 2. Then all the requests will be honored as long as the capacity allows. It follows that the revenue corresponding to a sample path where all class-2 customers arrive before class-1 customers will be different from that corresponding to a sample path where all class-1 customers arrive before class-2 customers (assuming the total demand exceeds capacity). Thus, typically one resorts to simulation in order to compare various bid-price policies. Despite this difficulty, Talluri and van Ryzin provide an extensive analysis on bid price method in Talluri and van Ryzin. In their work, they give two counter examples to show the suboptimality of the bid price method, but they also show that the bid price method is asymptotically optimal as the capacity and customer demand grow Re-solving bid prices It is clear from the structure of the bid-price policy that its form is too rigid depending on the values of the bid prices, entire classes may be rejected. In practice, the bid prices are re-calculated on a regular basis in order to take into account new information about the demand, thus providing a more flexible policy. When the bid 38

51 prices are obtained from dual multipliers of a mathematical program, this amounts to re-solving the problem with updated information, which is precisely the setting of section 2.3. In light of the results of section 2.3, a natural question that arises is whether the expected revenue under a bid price policy can be guaranteed to improve with a re-solving approach. Unfortunately, the answer is negative, even if the bid prices are calculated from the SLP model. We show below some small examples to illustrate this issue, and the numerical results in section 2.5 for larger scale problems corroborate our conclusions. The example setting is the following. Consider a one-leg model with two booking classes, the less price-sensitive customers (class 1) paying $100 and more pricesensitive customers (class 2) paying $60. The capacity is 4. The demands for those customers are denoted by ξ 1, ξ 2. Therefore the DLP model is max 100x x 2 subject to: x 1 + x x 1 E[ξ 1 ] 0 x 2 E[ξ 2 ]. For an arbitrary time t τ, let ξ t k denote the (random) number of arrivals of class k up to time t, and let ˆη t denote the actual number of sold seats up to time t. Therefore, 39

52 the re-solving DLP model is max 100x x 2 subject to: x 1 + x 2 4 ˆη t 0 x 1 E[ξ 1 ξ1] t 0 x 2 E[ξ 2 ξ2]. t The tables below show the possible outcomes of the corresponding bid price policy, according to the values of the various quantities involved in the above problems. Case 1 E[ξ 1 ] > 4 and E[ξ 1 ξ1] t > 4 ˆη t : Booking Methods Open classes through time t Open classes thereafter DLP without re-solving 1 1 DLP with re-solving 1 1 Case 2 E[ξ 1 ] 4 and E[ξ 1 ξ1] t > 4 ˆη t : Booking Methods Open classes through time t Open classes thereafter DLP without re-solving 1, 2 1, 2 DLP with re-solving 1, 2 1 Case 3 E[ξ 1 ] > 4 and E[ξ 1 ξ1] t 4 ˆη t : Booking Methods Open classes through time t Open classes thereafter DLP without re-solving 1 1 DLP with re-solving 1 1, 2 40

53 Case 4 E[ξ 1 ] 4 and E[ξ 1 ξ1] t 4 ˆη t : Booking Methods Open classes through time t Open classes thereafter DLP without re-solving 1, 2 1, 2 DLP with re-solving 1, 2 1, 2 Suppose now the demand for the whole horizon has the following distribution: ξ 1 = { 5 with probability with probability 1 2, ξ 2 = { 2 with probability with probability 1 2. Moreover, suppose that we can divide the time horizon into two periods such that in the first period there are two class-2 arrivals only. The capacity is c = 4. Notice that we have E[ξ 1 ] = 3 < 4 and ξ1 t = 0 w.p.1, so E[ξ 1 ξ1] t = 3. From the tables above we see that the initial policy determined by the bid prices is to accept all the requests. Then, after the first period, two seats are occupied, i.e., ˆη t = 2, and thus case 2 above applies. It follows that, when re-solving model [DLP] to obtain new bid prices, the policy becomes only accepting class-1 customers. It is not difficult to verify that the expected revenue for the second period is $ under non-re-solving policy, and $150 under the re-solving one. Since the expected revenue for the first period is the same for both policies, it follows that the re-solving policy behaves worse than the non-resolving one. Similar results are obtained for the case of bid prices generated from model [SLP] (under the same demand distribution as above), although the calculations are slightly more complicated. The solution of the SLP problem is (2, 2) T, which implies that the incremental price for the leg is $60. Thus, in the first time period two seats are allocated to the class-2 customers, so the remaining capacity for the second period is 2. When re-solving SLP model again, we get the new allocation at (2, 0) T which implies the incremental price for the leg is $100. That is, the re-solving SLP method 41

54 changes policy from accepting all customers into accepting only class-1 customers. It follows that the expected revenue for the second period is the same as in the DLP case $ for non-re-solving, $150 for re-solving. This shows that re-solving can be worse under the bid-price policy, even if it is generated from the SLP model. Again, the results in section 2.5 corroborate these conclusions Numerical results In this section we describe the results from numerical experiments performed with the policies discussed above. Although our data set was randomly generated, we tried to mimic real data as much as possible. To do so we imposed the following features, which according to Weatherford, Bodily, and Pfeifer (1993) (1993), are characteristic of actual booking processes. They are (1) uncertain number of potential customers; (2) uncertain mix of high- and low-fare customers; (3) uncertain order of arrivals; and (4) high-fare customers tend to arrive after the low-fare ones. The first example is a 10-leg network described in Figure 2.2 below. We consider all flights to/from the hub from/to each city, as well as the flights between two cities connecting at the hub. Therefore, there are 30 possible itineraries in the network. There are two booking classes for each flight, with the proportion of 1:3 between high and low fare classes in terms of total requests. Following Weatherford et al. Figure 2.2. Example 1 for Numerical Experiment 42

55 (1993) (1993), we model the booking process by a doubly stochastic non-homogeneous Poisson process (NHPP), where the arrival intensity at time t has gamma distribution. More specifically, for each itinerary j let λ j1 (t) and λ j2 (t) be the arrival intensity of respectively high-fare and low-fare customers at time t. Denote by α j > 0 the total expected number of requests for itinerary j over the booking horizon (i.e., for both classes together). Let G j be a random variable with gamma distribution with shape parameter α j and scale parameter β = 1 (that is, the density function of G j is f j (x) = (x/β ) α j 1 e x/β β Γ(α j ), x 0). We define λ jk (t), k = 1, 2, as where λ jk (t) = β jk (t) G j ψ k β jk (t) = 1 ( ) ajk 1 ( t 1 t ) bjk 1 Γ(a jk + b jk ) τ τ τ Γ(a jk )Γ(b jk ). The parameters ψ 1, ψ 2 are set with the goal of reflecting the proportion of arrivals for high- and low-fare customers. We take this proportion to be 1:3 in all itineraries, so we set ψ 1 = 0.25 and ψ 2 = Notice that, for each t, λ jk (t) has gamma distribution with shape parameter α j and scale parameter β jk (t)ψ k. In particular, E[λ jk (t)] = α j β jk (t)ψ k and hence the total expected number of arrivals for itinerary j is τ 0 E[λ j1(t)]+e[λ j2 (t)]dt = α j ψ 1 +α j ψ 2 = α j, which is consistent with our definition of α j. The parameters β jk (t) are selected to reflect the arrival patterns of different classes. High-fare customers tend to arrive close to the end of the booking horizon, whereas low-fare customers usually appear early in the booking process. To model that, we set a j1 > b j1 (high-fare customers), and a j2 < b j2 (low-fare customers). In our example we used a jk, b jk {2, 6} for all j, k. 43

56 From Figure 2.2 we see that there are 10 one-leg and 20 two-leg itineraries. For two-leg itineraries, we set the total expected number of requests equal to 100, that is, α j = 100. The high and low fares are respectively f j1 = $500 and f j2 = $100. For one-leg itineraries, we set the total expected number of requests equal to 40, that is, α j = 40, with the high and low fares set as f j1 = $300 and f j2 = $80. All legs in the network have capacity equal to 400, and the booking horizon has length τ = 1000 time units. The second example is depicted in Figure 2.3. Again, we consider two classes for each itinerary. Notice that there are 10 one-leg, 12 two-leg, and 8 three-leg itineraries. The expected number of requests are 60, 150, and 100 respectively. The fare levels for different type of itineraries are set as ($300, $80), ($500, $100), and ($700, $200). The parameters a jk, b jk, ψ 1 and ψ 2 are the same as in the first example, as well as the horizon length. The leg capacities are slightly different from the example 1. For non-hub legs, such as leg 1, 2, 3, 4, 7, 8, 9, 10, the capacity is 400 seats. Leg 5, 6 are hub legs whose capacity is 1000 seats. Figure 2.3. Example 2 for Numerical Experiment For each of the problems we implemented four basic policies: DLP, SLP, DLPbased bid price and SLP-based bid price. For each of the policies, we considered the effect of solving it only once as well as twice and five times over the booking horizon. The linear (integer) programs required to determine the policies were solved using the software package XPressMP T M from Dash Optimization (under the Academic 44

57 Partnership Program). The results in Tables 2.1 and 2.2 show the average revenue for each policy and each example, computed over 1000 replications. Next to each number we display a 95% confidence interval for the corresponding expected revenue. The results confirm the findings reported in sections 2.3 and 2.4 that the allocation policy based on model [SLP] is more robust, in the sense that re-solving can only improve the revenue. With bid prices or DLP-based allocation this is not necessarily the case, as discussed in sections 2.3 and % C.I.($) LP LP twice LP five times Example 1 [401574,402386] [404094,44946] [409272,410208] Example 2 [583078,584182] [584050,585170] [591833,592941] E.R. ($), C.I. SLP SLP twice SLP five times Example 1 [414812,416008] [417425,418599] [418668,419856] Example 2 [594894,596346] [596560,597956] [602482,603694] Table 2.1. Simulation results for allocation policy 95% C.I.($) LP LP twice LP five times Example 1 [346723,348657] [378647,380477] [390855,392681] Example 2 [282295,283685] [350111,351577] [390031,391377] 95% C.I.($) SLP SLP twice SLP five times Example 1 [347058,348922] [372819,374701] [413416,415204] Example 2 [294254,295666] [315813,317215] [331297,332747] Table 2.2. Simulation results for bid price policy 2.6. Improving the partition As discussed earlier, although the re-solving approach provides a computationally attractive alternative to the multi-stage problem [MSSP], the resulting solution is always sub-optimal (cf. Proposition 3). However, we must stress that such a result is valid under the same partition into stages. Indeed, the flexibility of the re-solving approach allows for the inclusion of additional re-solving points without much burden in other words, the complexity grows linearly with the number of stages, which 45

58 in general is not true for the multi-stage model. It is natural then to compare the MSSP model and the re-solving SLP model with a refined partition. Let us look at the following example. Consider a single-leg problem with two independent booking classes, 1 and 2, with f 1 = $300, f 2 = $200. The capacity is equal to 15, and the booking time horizon has three time periods, 1, 2, 3. During period 1, the distribution of demand for classes 1 and 2 is ξ 1 1 = { 0 with probability 1 2, ξ 1 2 = 0 with probability one. 1 with probability 1 2 Likewise, the distribution of demand during period 2 is ξ 2 1 = { 3 with probability with probability 1 2, ξ 2 2 = { 3 with probability with probability 1 2 and ξ 3 1 = { 5 with probability with probability 1 2, ξ 3 2 = { 4 with probability with probability 1 2 during period 3. Because of the limited scale of this problem, the multi-stage model can be solved by enumeration. Suppose we solve a three-stage problem with the second and third stages defined respectively as time intervals (0, 1] and (1, 3]. It is easy to check that the optimal solution from this model is x 0 = (15, 0) T, x 1 1 = (10, 5) T (when ξ1 1 = 0 happens) and x 1 1 = (10, 4) T (when ξ 1 1 = 1 happens). The expected total revenue is $3900. For the re-solving SLP approach, x 0 = (9, 6) T is the first stage decision. Although this solution does not coincide with that from the MSSP model, it turns out that, once we re-solve at time 1, we obtain the same expected revenue of $3900 resulting from [MSSP]. When we re-solve again at time 2, the expected total revenue becomes 46

59 $4000, which is $100, or 2.56%, higher than that of MSSP model. For a large network, the improvement would be significant. This example suggests that applying the resolving procedure can be more beneficial (in terms of expected revenue) than solving a more complicated multi-stage model. In addition to including additional re-solving points, one may also consider where to place such points. Although the standard practice appears to be to re-solve at equally sized intervals, there is no reason this must be done so in fact, one may benefit from a better choice of re-solving points. To illustrate, consider a situation where re-solving is applied once, at some time t. That is, we have an initial allocation x 0 and a revised allocation x 1, which is obtained from the problem solved at time t. Using the notation defined earlier, let ξ 1 (t) and ξ 2 (t) be the vectors of total number of requests during intervals (0, t] (stage 1) and (t + 1, τ] (stage 2) respectively. Notice that, under this policy, the expected revenue from time t on is given by f T E [min{x 1, ξ 2 (t)}]. Thus, the improvement from re-solving is given by (2.11) (2.11) f T E [ min{x 1, ξ 2 (t)} ] f T E [ min{(x 0 min{x 0, ξ 1 (t)}), ξ 2 (t)} ] 0. The second term above discounts the revenue resulting from keeping policy x 0 from time t on. The term min{x 0, ξ 1 (t)} gives the number of sold seats up to time t, so x 0 minus that quantity is the number of available seats at time t under the initial policy. By maximizing (2.11) over t, we obtain the greatest benefit from re-solving. Let us consider initially a single leg model. Assume we have the problem on time horizon t = 0,..., T. Recall that the two stage model we solve to decide the initial allocation is: 47

60 Figure 2.4. Problem Setting maxf T E ξ1 (t)+ξ 2 (t) min(ξ 1 (t) + ξ 2 (t), x) x Z + (2.12) Subject to: Ax c Its solution is x 1=argmax(SLP). If the re-solving point is t, and the remaining demand is ξ 2 (t), as a result, the new two stage stochastic programming should be, maxf T E ξ2 (t) min(ξ 2 (t), x) x Z + (2.13) Subject to: Ax c A(x 1 min(x 1, ˆξ 1 (t))) Its solution is x 2=argmax(SLP-R) which is a function of x 1 and ˆξ 1 (t). Notice, x 1 does not depend on particular realizations of ξ 1 (t), ξ 2 (t). The difference in expected revenue happens after time t. Under the scenario of ˆξ 1 (t), ˆξ 2 (t), the revenue coming from non-re-solving process is, f T min(ˆξ 1 (t) + ˆξ 2 (t), x 1) = f T min(ˆξ 1 (t), x 1) + f T min(ˆξ 2 (t), x 1 min(ˆξ 1 (t), x 1)). Hence, the expected revenue is, (2.14) E ξ1 (t)(f T min(ξ 1 (t), x 1)) + E ξ1 (t)e ξ2 (t)(f T min(ξ 2 (t), x 1 min(ξ 1 (t), x 1))) Likewise, for re-solving problem, the expected value is, (2.15) E ξ1 (t)(f T min(ξ 1 (t), x 1)) + E ξ1 (t)e ξ2 (t)(f T min(ξ 2 (t), x 2)) 48

61 We have already discussed that re-solving always lifts the expected revenue. Hence, the difference by subtracting non-re-solving revenue from re-solving revenue is always positive, that is, E ξ1 (t)e ξ2 (t)[f T min(ξ 2 (t), x 2(x 1, ξ 1 (t))) f T min(ξ 2 (t), x 1 min(ξ 1 (t), x 1))] 0 Intuitively, if we can re-solve the problem frequently, the expected revenue tends to be higher. However, this idea is not true. Recall the settings for previous examples. Our booking horizon is from time 0 to We have two placements for solving time points, placement A at (0, 1, 2, 3, 4, 5) and placement B at (0, 500). The numerical result for the placement A does not reflect any change after solving 6 times. On the contrary, the placement B, which indicates only 2 times solving, shows a clear improvement on the expected total revenue. Therefore, we need more time points for solving only if we can catch more uncertainty. For the placement A, the problem is that the solving points are too close to catch change. Another reason that we can not do this in airline industry because re-solving the entire network frequently implies huge cost in manpower and equipment capacity. Pulling out data from fields could also be time consuming. Therefore, we have to re-solve the problem only when it is necessary. By the discovery that solving more frequently does not imply higher expected total revenue, we believe that the best way to improve the seat allocation model is to decide the best solving points. Good solving point would lead to an improvement on expected total revenue. In our research, we realized that revenue management is trying to minimize two types of risk. One type of risk is to reject valued customer with seats flying empty. Or, the another type of risk is to accept discount customers early and have to reject valued customers later. Therefore, the time points when rejection decisions are made would have an impact on total expected revenue. Re-solving the model at this point and 49

62 updating seat allocation mean a second thought on rejecting decisions. It could cut the risk of rejecting valued customer and has a direct impact on total revenue. Also, updated seat allocation would minimize the risk of over-accepting discount customers because it is an optimal seat allocation regarding the remaining demand. Our heuristic implementation can then be as follows (observe the notation with hats whenever we refer to realizations of the corresponding random variables): (1) Let t 0 = t 0 (ˆξ) be the first time (along the sample path ˆξ) at which we have ˆξ 1(t) i > x i 1 for some i; (2) Take time t 0 as a candidate for re-solving process. Other business variables, such as re-solving cost and computational capability, should also be taken into account in locating the re-solving point. (3) Solve the model and update the seat allocation on selected t 0. In our heuristic, the rejection could only been made when both the original and the newly calculated seat allocations suggest so. We showed the our heuristics in Figure (2.5). The above analysis procedure is intuitive in the single leg case, since there is a natural ordering of the classes by fare. However, as mentioned earlier the situation is more complicated when dealing with a network environment. For example, consider a situation where the fare for a certain itinerary-class pair that uses legs 1 and 2 is $130, whereas the fare for another itinerary-class pair that uses leg 1 only is $100, and the fare for another itinerary-class pair that uses leg 2 only is $70. Intuitively, if one expects to see more arrivals of the latter classes, then the second class should be preferred over the first one when making decisions about leg 1. One way to quantify this is through the net contribution of each class. Suppose for example that the bid price associated with each leg is $40. Then, the net contribution of the first itinerary-class is $130 $80 = $50, whereas the net contribution of the second one is 50

63 Figure 2.5. Our adaptive re-solving heuristic on identifying right solving point. $100 $40 = $60. That is, the second class is more profitable even though its fare is smaller. Therefore, for network model, we have three plans to improve the partition Plan A If there is a customer who requests a ticket from a closed booking class, she/he might be rejected by seat allocation policy. However, it leads to two kinds of risk. First, we might lose the sale. The second, if we honor this request, we might have to reject a valued customer in the future. To minimize the risk, we suggest suspend the rejection and re-solve at the time point. We will act by the new policy generating by re-solving SLP. This becomes the plan A. It is obvious that the plan A is not practical in the real business because it may lead to huge number of re-solvings which might slow down the booking system. Also, 51

64 the cost related with re-solving might also be significant. In order to cut the re-solving cost, we have another plan based on plan A Plan B In plan B, we do not respond to all the potential rejections. Instead, we only re-solve the SLP model when any rejection happens on selected classes, say business class or higher. For any potential rejection on coach or lower classes, we will let it go. We believe the risk of losing sale on low-fare class would not hurt the gross revenue in the same level as the risk of losing high-fare customers Plan C Some company would like to solve the model at fixed time points regardless the demand realization already happened to make their life easier. In order to find an appropriate re-solving point taking the above considerations into account, we propose the following heuristic method. Consider initially an arbitrary class k in a single-leg problem, and let us write the demands ξk 1 = ξ1 k (t) and ξ2 k = ξ2 k (t) as functions of the re-solving epoch t. The idea to improve the partition is to find a point t k such that E[ξk 1(t)] and E[ξ2 k (t)] are relatively close to each other. Clearly, all those t k s will not be necessarily the same. Therefore, we must develop a process to determine a single re-solving time t. In a network environment, the improvement in total revenue is decided partly by changes in actual arrivals and 52

65 partly by their fare levels. At time 0 we solve the problem (2.16) max f T min{x, E[ξ]} subject to n x i c, i=1 x Z +, and the solution gives the initial allocation x 0. Assume that the classes are ordered such that f 1 f 2... f n. Then, we can easily solve (2.16). The solution is x 0 = E[ξ 1 ]. E[ξ k ] c k i=1 E[ξ i], 0. 0 where k is the smallest index such that k+1 i=1 E[ξ i] > c. For simplicity, assume that k i=1 E[ξ i] = c, so x 0 k+1 = 0. 53

66 Now consider the re-solving problem at time t. Let ξ 1 (t) and ξ 2 (t) have the same meaning as before, so ξ 1 (t) + ξ 2 (t) = ξ. Then, the re-solving problem is subject to max f T min{x, E[ξ 2 (t)]} n x i c i=1 n min{x 0 i, ˆξ i 1 (t)}, i=1 x Z +. From the value of x 0 calculated above, we can rewrite the problem as (2.17) max f T min{x, E[ξ ξ 1 (t)]} subject to n x i c i=1 k min{e[ξ i ], ˆξ i 1 (t)}, i=1 x Z +. Again this is easy to solve, and we get E[ξ 1 ξ1(t)] 1. E[ξ l ξl 1(t)] x 1 (t) = c l i=1 E[ξ i ξi 1 (t)],

67 where l is the smallest index such that l+1 E[ξ i ξi 1 (t)] > c i=1 k min{e[ξ i ], ˆξ i 1 (t)}. i=1 Again, for simplicity let us discard the residual, i.e., assume that x 1 l+1 (t) = 0. Note that the assumption that c = k i=1 E[ξ i] means that the above condition can be written as (2.18) l+1 E[ξ i ξi 1 (t)] > i=1 k E[ξ i ] min{e[ξ i ], ˆξ i 1 (t)} = i=1 k i=1 [ E[ξ i ] ˆξ 1 i (t)] +. The expected objective function value of problem (2.17) is given by ν 1 (t) = E [ f T min{x 1 (t), E[ξ ξ 1 (t)]} ] = l f i E[ξ i ξi 1 (t)]. i=1 Next, we calculate the value of re-solving. If we do not re-solve, the policy at time t is given by x(t) = x 0 min{x 0, ξ 1 (t)}, which given the value of x 0 calculated above can be written as x(t) = [ E[ξ 1 ] ˆξ ] + 1(t) 1. [E[ξ k ] ˆξ ] + 1k (t) As a side remark, notice that condition (2.18) implies that n x 1 i (t) i=1 n x i (t). i=1 The value of re-solving is given by the improvement in the objective value of problem (2.17) when using the optimal solution x 1 (t) instead of keeping x(t). The 55

68 objective value at x(t) is given by ν(t) = E [ f T min{ x(t), E[ξ ξ 1 (t)]} ] = k i=1 [ { [E[ξi f i E min ] ξi 1 (t) ] }] +, E[ξi ξi 1 (t)]. In order to compare ν 1 (t) and ν(t), let suppose that l = k in the respective expressions. Define (t) := ν 1 (t) ν(t). Then, we have (2.19) (t) = k i=1 [ { [E[ξi f i {E[ξ i ξi 1 (t)] E min ] ξi 1 (t) ] }]} +, E[ξi ξi 1 (t)] To alleviate the notation, define µ i := E[ξ i ], and let Z i := ξ 1 i (t) (so EZ i µ i ). Note that the second term in (2.19) can be rewritten as (2.20) E [ min { [µ i Z i ] +, µ i EZ i }] = E [min {µi min{µ i, Z i }, µ i EZ i }] = µ i E [max {min{µ i, Z i }, EZ i }], so by substituting the above into the expression for (t) we have (2.21) (t) = = = k f i (E [max {min{µ i, Z i }, EZ i }] EZ i ) i=1 k f i E [max {min{µ i, Z i }, EZ i } EZ i ] i=1 k f i E [ [min{µ i, Z i } EZ i ] +] i=1 56

69 Note that we can write the expectation inside the sum as (2.22) E [ [min{µ i, Z i } EZ i ] +] = E [ [min{µ i, Z i } EZ i ] + ] I {Zi >µ i } + E [ [min{µ i, Z i } EZ i ] + ] I {Zi µ i } = E [ [ (µ i EZ i )I {Zi >µ i }] + E [Zi EZ i ] + ] I {Zi µ i } = (µ i EZ i )P (Z i > µ i ) + E [ [Z i EZ i ] + I {Zi µ i }]. Recall that the goal is to find the re-solving point t such that t maximizes (t). The second term in (2.22) is difficult to evaluate, so we will modify our objective as to find t that maximizes (2.23) (µ i EZ i )P (Z i > µ i ). Using the Markov inequality P (Z i > µ i ) EZ i /µ i and substituting this bound for P (Z i > µ i ), we obtain the problem max EZ i (µ i EZ i )EZ i /µ i. It is easy to check that the solution to this problem is EZ i = µ i /2. That is, we want to find t such that E[ξ 1 (t)] = E[ξ i ]/2. This suggests that we choose t such that f k E[ξk(t)] 1 k k f k E[ξ 2 k(t)] or, equivalently, f k E[ξk(t)] 1 1 f k E[ξ k ], 2 k where as before ξ k := ξk 1 + ξ2 k denotes the class-k demand for the whole horizon. k 57

70 The above idea can be generalized to multiple re-solving points. In a single leg problem, suppose we decide we want to re-solve R times. Then the time we pick for the r th re-solving point is t such that (2.24) f k E[ξ k (t)] k r R + 1 f k E[ξ k ]. k We omit the superscript 1 from ξ k (t), which represents class-k demand up to time t. Notice that ξ k = ξ k (τ). One way to pick t such that (2.24) holds is to solve the one-dimensional problem min t k f ke[ξ k (t)] r/(r + 1) k f ke[ξ k ]. It is clear that, in a single-leg environment, the net contribution is actually the fare level. For networks, one can apply heuristic procedures to rank the classes based on the net contribution. Such procedures are proposed in Bertsimas and de Boer (2003) and de Boer. et al., for example. Borrowing from their ideas, we apply the follow steps to aggregate the demand vector into a one-dimensional quantity. Let f jk be the fare level for certain itinerary-class pair (j, k) and let S jk denote the set of legs which are used for that itinerary. We use the following algorithm to determine R re-solving points: (1) Solve model [SLP] at time 0. Let p l denote the bid price for leg l, obtained from the dual variables. (2) Let q jk be the net contribution of itinerary-class (j, k), calculated as f jk l S jk p l. (3) For each r, r = 1,..., R, the r th re-solving point t r is a t that satisfies q jk E[ξ jk (t)] j,k r R + 1 q jk E[ξ jk ], j,k where as before ξ jk (t) is the class-k demand for itinerary j up to time t. 58

71 We applied the above procedure to the two examples discussed in section 2.5. The demand distributions used in the examples allow us to calculate E[ξ jk (t)] exactly. It turns out that E[ξ jk (t)] = α j ψ k B jk (t/τ), where B j1 (s) := 6s 7 + 7s 6 B j2 (s) := 6s s 6 84s s 4 70s s 2 for all j. Notice that, as discussed in section 2.5, the expected number of arrivals of itinerary-class (j, k) over the whole horizon is E[ξ jk ] = E[ξ jk (τ)] = α j ψ k B jk (1) = α j ψ k. Figure shows the plot of the function H(t) := j,k q jke[ξ jk (t)] for both examples. It is clear that re-solving occurs more often as the slope of H gets larger so equally sized intervals are appropriate only when H is linear, which is not the case in these examples. Figure 2.6. Graph of the function H(t) = j,k q j,ke[ξ jk (t)] for Examples 1 (left) and 2 (right). In order to provide a comparison, we consider the same number of re-solving points as before, i.e., one and four. For one time re-solving, the algorithm changed the re-solving point from time 500 (which corresponds to equally sized intervals) to time 757 in Example 1, and to time 735 in Example 2. For four times re-solving, 59

72 the re-solving points were changed from (200, 400, 600, 800) to (587, 712, 797, 875) in Example 1 and to (504, 682, 782, 867) in Example 2. Notice that the algorithm suggests re-solving more often as the end of the horizon approaches, which reflects the fact that high-fare customers tend to book later than low-fare ones Simulation results The difference between plan A and plan B is the frequency of re-solving activities. In Figure 2.7, we have the rejection frequency in plan A and plan B happened in example 1. Since plan A responds to any rejection while plan B only respond to rejections on business or higher classes, there is clear difference for example 1 between both plans. Besides, we also compared the solving frequency upon time points. In Figure 2.8, we showed the frequency difference between plan A and B for individual time points in example 1. Figure 2.7. Rejection frequencies between plan A (left) and B (right) happened in Example 1 95% C.I. Plan A Plan B Example 1 [429707,431939] [425060,426070] Example 2 [659670, ] [655270,657862] Table 2.3. Simulated expected revenue for plan A and B The results indicate that re-solving upon high-fare rejection, or plan B, could control the total re-solving activities significantly. The reason is due to the rejections 60

73 Figure 2.8. Solving frequencies upon time points between plan A (left) and B (right) on low-fare customers might happen at the early stage of booking. As we stated before, the marginal revenue of coach customers is far less than that of the business or higher customers. Table 2.3 shows that the simulated expected total revenue for plan A is higher than plan B which is reasonable due to more computational efforts for all the potential rejections. However, if we take the re-solving cost, which could be high in network model, into the balance, the company might prefer plan B to A. Tables 2.4 and 2.5 below display the results for plan C. As before, the numbers correspond to a 95% confidence interval. Comparing with Tables 2.1 and 2.2, we see that a well picked partition definitely yields better allocation policies. With bid-price policies the results are mixed, which reflects the fact the procedure laid out in this section was devised to improve the revenues from the allocation model. 95% C.I. SLP SLP twice SLP five times Example 1 [414812,416008] [418732,419354] [421281,422507] Example 2 [594894,596346] [600410,601902] [604261,605457] Table 2.4. Simulation results for allocation policy with new partition 95% C.I. SLP SLP twice SLP five times Example 1 [347058,348922] [357083,358997] [378527,380337] Example 2 [294254,295666] [311325,312751] [329872,331260] Table 2.5. Simulation results for bid price policy with new partition 61

74 So far, plan B and C look promising to us. If you want to solve fixed times prior to booking, we can use plan C to set the time points for re-solving. Plan B gives us an adaptive process to improve the model with a random number of re-solving activities. The remaining question is, if the average solving workload from plan B is close to that of the plan C, which plan is expected to yield more revenue? To answer the question, we found that the average solving times from plan B for example 1 is 5.87 times, and 5.24 times for example 2. To conduct a fair comparison, we solve plan C 5 times and 6 times respectively on both examples. The numerical results are listed in Table 2.6, which indicates that the adaptive method tends to generate more revenue. Hence, if there is no control requirement on re-solving times for every sample path but at the average level, we should prefer plan B as our final plan. 95% C.I. Plan C 5 times Plan C 6 times Plan B Example 1 [421281,422507] [421890,424125] [425060,426070] Example 2 [604261,605457] [606786,609247] [610270,612862] Table 2.6. Comparison between plan B and C in expected revenue In order to assess the quality of these results we computed the wait-and-see solutions. These solutions determine the actions that would be taken if all uncertainty was known in advance. Although this does not yield a practical policy, it does provide an upper bound for the expected revenue. Table 2.7 displays the expected revenue and the 95% confidence intervals for the wait-and-see value. 95% C.I. Wait-and-see value Example ,[433310,434450] Example ,[669170,670690] Table 2.7. Wait-and-see values for examples 1 and 2 In both cases, the gap between the plan B wait-and-see values and the values on the third column of Table 2.4 is only about 2.0% for both examples. This suggests that 62

75 the re-solving approach with improved partition can be an attractive low-cost approach for the models we have studied in our research Conclusions We have discussed the airline booking process based on the origin-destination model. More specifically, we have presented a multi-stage stochastic programming formulation to the seat allocation problem, which extends the traditional two-stage model proposed in the literature. Our study suggests that solving this multi-stage problem exactly may be difficult, because of the lack of convexity properties. In order to circumvent that obstacle, we have used an approximation based on solving a sequence of two-stage stochastic linear integer programs (SLPs) with simple recourse. Our analysis suggests that the proposed approach is robust, in the sense that solving successive SLPs can only improve the expected revenue i.e., it is never better not to re-solve. While intuitive, to our knowledge such a property had not been shown in the literature. Moreover, this gives an advantage of SLPs over the standard deterministic linear program formulation, for which it is known that resolving can actually backfire. As it turns out, the same phenomenon happens with some bid-price policies, and we have presented some examples where re-solving worsens the expected revenue. We have also shown that, under perfect information, the multi-stage model and the re-solving approach for the seat allocation problem coincide; this may have some algorithmic implications (e.g., by applying the progressive hedging algorithm of Rockafellar and Wets. (1991)), which is a topic for further research. Note that theoretical comparisons involving bid-prices policies are harder to obtain, since those policies highly depend on the arriving order which might lead to a substantial differences on revenue. 63

76 The flexibility of the re-solving approach has allowed us to propose a heuristic method whereby the re-solving points are chosen with the goal of maximizing the incremental expected revenue; our numerical results, run for two medium-sized models (each with a different network structure) suggests that the approach is effective. We must remark that we have not included in our models some of the recent developments proposed in the literature, such as algorithms that include nesting of classes (see, e.g., de Boer, Freling, and Piersma (2002), van Ryzin and Vulcano (2003)) and consumer-choice modeling techniques (see, e.g., G. Gallego and Dubey. (2004), van Ryzin and Liu (2004)). There are two basic reasons for our decision: first, incorporation of the above features leads to different models which lie outside of the scope of our research for example, the problems in de Boer et al. (2002), van Ryzin and Vulcano (2003) are non-convex problems that are solved with simulation-based methods, whereas the linear programs in G. Gallego and Dubey. (2004), van Ryzin and Liu (2004) are of different nature than the ones discussed here. Second, our conversations with people in the airline industry have shown to us that the basic origin-destination model, particularly the deterministic linear programming formulation, is widely used in practice; thus, our goal is to provide the practitioners an easily implementable algorithm that can improve upon what is currently in use. Again, our theoretical and numerical results show that the methods proposed here have the potential to accomplish that goal. Nevertheless, we plan to study the applications of those methods under other settings, particularly when consumer choice is incorporated into the model. Research on that topic is in the next chapter. 64

77 CHAPTER 3 MODELING CUSTOMER CHOICE BY PREFERENCE ORDER 3.1. Effect of the customer choice behavior in airline industry In Chapter 2, we discussed the seat allocation method intensively with the assumption of independent demands among booking classes. Inside the organization, the marketing department provides the demand forecast as one of the model inputs. Typically, the major airline companies use the historical booking data as the primary data source which is organized by booking classes. As a result, the demand forecasting is conducted individually by booking classes. This is debatable in the practice of the airline revenue management, since in reality customers do not start the booking process with a specific class in mind but rather choose among available alternatives. Ignoring such behavior can lead to undesired consequences. For example, Boyd et al. (2001) and Cooper, Homem-de-Mello, and Kleywegt (2004) have shown a phenomenon called spiral-down which would lead to a progressive decrease of high-fare sales and revenues. Besides that, individual demands require clear customer stereotypes. All business customers should never purchase a coach or first class ticket even if those choices seems more promising to them. Likewise, coach class customers should always be price-sensitive, and have flexible schedules. Unfortunately, in the real booking process, independent demands assumption would not sustain. We have to take another critical input into account, the customer choice behavior. For each individual customer, there would be multiple different choices. There are many factors would make a difference, such as the departure time, the arrival time, 65

78 price, service during the trip, number of connections, etc. Consider the following case, whenever a business traveler on a business trip will prefer a coach ticket to business class ticket if both classes are open. Similar thing happens if a wealthy leisure customer tends to book a first class ticket for the excellent service during the trip. If the most wanted tickets are sold out, some customers would go back for other choices. That is, a rejection on the coach class might end up with a business class ticket for the same trip. Therefore, there are no clear stereotypes for business customers, coach customers, or wealthy customers and demands for different classes are correlated rather than independent. This is a serious matter in airline revenue management because the existence of customer choice could jeopardize the independent demand assumption, thereby the validity of the traditional OD seat allocation model. We would like to refer readers to Chapter 1 for detailed literature review on customer choice models. Both industrial partners and academic researchers realize the effect of discrete customer choice. As we stated in Chapter 1, many researchers model the customer choice by Markov Decision Process (MDP). For example, Talluri and van Ryzin (2004a) model the customer choice in a single leg problem as (3.1), (3.2), and (3.3). Basic problem settings are listed in the following, t = 1,..., T time periods x number of seats remaining (3.1) N = {1, 2,..., n} set of fare products λ = P {customer arrives in period} r j = revenue of fare product j, r 1,..., r n 0 66

79 The decision variable is S t which is the offer set at each time t = 1,..., T (3.2) S t N set of open fare products at time t P j (S t ) = P {customer chooses j S t } Built by MDP process, the Bellman equation is, V t (x) = max S t N { j S t λp j (S t )(r j + V t 1 (x 1)) + (λp 0 (S t ) + 1 λ)v t 1 (x)} (3.3) = max S t N { j S t λp j (S t )(r j V t 1 (x))} + V t 1 (x) Where V t 1 (x) = V t 1 (x) V t 1 (x 1) The solution is S t N for all the t = 1,..., T which is recognized as the optimal booking policy. However, this model has limited value on business implementations because of the computational obstacle. All the major airlines operate on nationwide networks. The customer choice model should be applicable to operating networks. Under this setting, there would be huge amount of S t at given time t. Further, the MDP model suffers the curse of dimension which is a long term flaw for them. Even if we would be able to find all S t, we could not solve it under current computational capability. Therefore, in order to incorporate the customer choice behavior, a solvable network based model is desired Our idea on modeling the network customer choice Before moving forward to model description, we illustrate the problem by the following example. Suppose you, currently in working in Columbus OH, would attend a team meeting with a brief lunch reception at Washington DC on next Monday. You favorite online travel agent gives all the possible choices in Figure

80 Figure 3.1. Small Example on Preference Orders The team meeting should be held at 1:00pm. Before the meeting, there might be a lunch reception at 12:00pm. Instead of assuming independence on demands and forecasting demands by booking classes, we categorize customers into different groups by their preferences. For instance, we have the budget-sensitive customer group, the time-sensitive customer group, etc. Inside each group, customers share the same behavior when facing multiple choices. For each group of customers, we arrange all the possible booking classes from the most wanted ticket to the least wanted ticket. The shift from one class to another will be associated with a probability because not all the customers would keep on trying to get a ticket. Some customers would leave for a less expensive way to be there, say the ground transportation. So, I believe you would like to purchase tickets in the order of Figure 3.2. The probabilities of leaving Figure 3.2. Business Customers Preference Order the booking process are 0.9 and 0.3. Likewise, if you are a leisure traveler with limited 68