INTEGER PROGRAMMING BASED SEARCH

Transcription

1 INTEGER PROGRAMMING BASED SEARCH A Thesis Presented to The Academic Faculty by Michael R Hewitt In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Industrial and Systems Engineering Georgia Institute of Technology December 2009

2 INTEGER PROGRAMMING BASED SEARCH Approved by: Professor Martin Savelsbergh, Advisor School of Industrial and Systems Engineering Georgia Institute of Technology Professor Alan Erera, Co-Advisor School of Industrial and Systems Engineering Georgia Institute of Technology Professor George Nemhauser,Co-Advisor School of Industrial and Systems Engineering Georgia Institute of Technology Professor Ozlem Ergun School of Industrial and Systems Engineering Georgia Institute of Technology Professor Mark Ferguson College of Management Georgia Institute of Technology Date Approved: August 2009

3 ACKNOWLEDGEMENTS I would like to thank my advisors Alan Erera, George Nemhauser and Martin Savelsbergh for the unique perspective and expertise they each brought to this thesis and my graduate studies. I valued both greatly. I could not have asked for more from doing research with the three of them; it simply has been a lot of fun. I would especially like to thank Martin Savelsbergh for his involvement and patience with my maturation as a researcher and writer. I would like to thank the industry partners Yellow Roadway Corporation, Saia. Inc and Exxon Mobil Corporation, who have sponsored my research at one point or another. Each of them reminded and reassured me of the practical importance of the work I was doing. Finally, I d like to thank my wife and daughter for letting me work so much but not all the time. Don t worry. Tenure is only 6 years away! iii

4 TABLE OF CONTENTS ACKNOWLEDGEMENTS iii LIST OF TABLES vi LIST OF FIGURES SUMMARY vii viii I INTRODUCTION II IP-BASED NEIGHBORHOOD SEARCH Literature Formulations Solution Approach Neighborhood Search Lower Bounds Initial Feasible Solution Computational Results Instance Generation Calibration Upper Bound Lower Bound III MODELING IP-BASED SEARCH Modeling the Search for Optimal Solutions Solving Instance of MP,MP = Multi-Commodity Fixed-Charge Network Flow Computational Results Primal Side Dual Side Comparison with IP Search IV IP-BASED SEARCH IN PRACTICE Background Literature Review iv

5 4.3 Enhanced Load Planning Modeling Freight Routing Load Plan Design Integer Program Variations on Traditional Load Plan Design In-tree Reoptimization Heuristic Computational Results Solving In-tree IPs Traditional Load Plan Improvements Variations on the Traditional Load Plan V CONCLUSIONS AND FUTURE RESEARCH REFERENCES v

6 LIST OF TABLES 1 Comparison with Tabu Search and Path Relinking Primal-side Comparison with CPLEX IP Search Given More Time Primal Side Comparison with CPLEX - Metaheuristic Instances - Single Path 34 5 Selection Method Contribution Primal and Dual Comparison with default CPLEX Results by Number Commodities in an Instance Primal Comparison with CPLEX and Local Branching Primal Side Comparison of 4 and 6 Processors Primal and Dual Comparison with IP Search Optimality Gap Reductions Number IIP d Solved to within.1% % Savings in thirty minutes Load Plan Savings For Each Week Load Plan Savings For Each Week Cost Component Comparison Load Plan Savings For Load Plan Variants Load Plan Variants Cost Component Comparison Allowing Different Paths on Different Days-Apr Allowing Different Paths on Different Days - Mar % Terminals That Load a Single Destination on Multiple Directs - Apr % Terminals That Load a Single Destination on Multiple Directs - Mar vi

7 LIST OF FIGURES 1 Data sharing between processes Primal-side Performance Over Time # Shipments by Service Standard Example Network Freight routing decisions yield the following trailer movements when empty costs are not considered Freight routing decisions made in conjunction with empty repositioning decisions yield higher loaded costs, but lower total costs Load plan on Monday does not consolidate freight Load plan on Tuesday-Friday consolidates freight through BB terminal b Holding Freight for Consolidation LN for a four terminal network, and load plan as a directed in-tree into d Time-space network depicting routing choices into d given a load plan Savings over time vii

8 SUMMARY When integer programming (IP) models are used in operational situations there is a need to consider the tradeoff between the conflicting goals of solution quality and solution time, since for many problems solving realistic-size instances to a tight tolerance is still beyond the capability of state-of-the-art solvers. However, by appropriately defining small instances, good primal solutions frequently can be found quickly. We explore this approach in this thesis by studying the design of algorithms that produce solutions to an integer program by solving restrictions of the problem via integer programming technology. We refer to this type of algorithm as IP-based search. This approach is also taken, for example, within LP-based branch-and-bound algorithms using techniques such as Local Branching and Relaxation Induced Neighborhood Search (RINS). These techniques use information from the LP solution and incumbent solution to define a small IP, which is then optimized. These techniques can be applied to any integer program and are available in commercial solvers such as CPLEX. We develop new IP-based search approaches for specific problems that exploit problem structure and an approach that can be easily applied to general integer programs. Finally, we leverage some of the strengths of IP-based search to develop new and more accurate models of a network design problem faced by freight transportation carriers. In the first part of the thesis we present a heuristic for the classical Multi-Commodity Fixed Charge Network Flow (MCFCNF) model that exploits problem structure to produce high quality solutions quickly. The solution approach combines mathematical programming and heuristic search techniques. To obtain high-quality solutions it relies on neighborhood search with neighborhoods that involve solving carefully chosen integer programs derived from the arc-based formulation of MCFCNF. To obtain lower bounds, the linear programming relaxation of the path-based formulation is used and strengthened with cuts discovered during the neighborhood search. Computational experiments demonstrate that viii

9 the proposed approach outperforms both best-known meta-heuristics and a state-of-the-art MIP solver. In the second part of the thesis we present an IP-based search algorithm for mixed integer programs that does not depend on problem structure. We formalize IP-based search as solving a restriction of the original problem and then develop an extended formulation to model the choice of restriction to solve. We propose a parallelized branch-and-price scheme for solving the extended formulation that is designed to produce high quality solutions quickly. We illustrate the application of the algorithm on the MCFCNF and computational experiments indicate it is competitive both with a state-of-the-art MIP solver and the structure-based heuristic presented in the previous part, even though it is a more general algorithm. Lastly, the thesis addresses the applicability of IP-based search to a real-world problem; namely the Service Network Design problem faced by Less-Than-Truckload (LTL) freight transportation carriers. In this part we present advances both in modeling and algorithm design. The developed models more accurately capture key operations of today s carriers: decisions for loaded and empty trailer movements are considered simultaneously, and a time discretization is used that can appropriately model the timing of freight consolidation opportunities. Along with providing decision support for traditional service network plans used by LTL carriers, the models also enable the development of plans that allow more flexibility, such as allowing certain freight routes to vary by weekday. Given the additional detail within the proposed models, very large problem instances result when they are applied to large-scale LTL networks. Yet computational experiments using data from a large U.S. carrier demonstrate that the proposed modeling and IP-based search approach has the potential to generate significant cost savings. ix

10 CHAPTER I INTRODUCTION For many optimization problems, no algorithms guaranteed to find an optimal solution for large instances in a reasonable amount of time are known or are likely to be found. Yet these problems are of great practical interest. For example, the traveling salesman problem can be used to determine both the route of a small package delivery driver and the order in which holes should be drilled in a circuit board. Models from the class of network design problems, the primary focus of this thesis, capture the keys decisions that many transportation companies must make in order to serve clients at low cost. There are many different ways to search for an optimal or high-quality solution to a difficult discrete optimization problem. Branch-and-bound [24] recursively partitions the set of feasible solutions into regions, searching one exhaustively when it is deemed easy to do so. Typically, these regions are modeled as restrictions of the original problem and are simple in nature, such as fixing the value of a binary variable to 0 or 1. Many techniques have been developed to select the next region to partition, and bounding arguments are used to determine those that cannot contain an optimal solution and thus need not be searched. Although much work has been done to speed up branch-and-bound-based algorithms, particularly in the area of generating strong bounds during branch-and-cut, it is still an exponential time algorithm. With the global nature of today s economy and an increased emphasis on time-definite transportation services, real-life network design instances are very large and thus nearly impossible to solve to optimality. While the strategy employed by branch-and-bound guarantees that an optimal solution will be found, there is no guarantee and little likelihood of finding a high quality solution quickly. Neighborhood or local search heuristics [1] present an alternative approach to searching for solutions. These techniques sacrifice the guarantee of finding an optimal solution to find 1

11 high quality solutions quickly. Instead of producing solutions by searching systematicallydefined regions of the feasible set, these heuristics iteratively define and search a neighborhood of a known solution for an improving solution. Thus, critical decisions when designing a local search heuristic are the size of the neighborhood to be searched at an iteration and whether its structure enables the neighborhood to be searched efficiently. Should it be small and easy to search? If so, the heuristic can perform many searches in a fixed period of time but risks becoming stuck at a bad local optimum. To avoid this phenomenon, many have studied the design of meta-heuristics [6], which are high-level strategies used to guide a more problem-specific search heuristic. By structuring the sequence of small neighborhoods searched, meta-heuristics have been quite successful at avoiding becoming stuck at bad local optima. Although they are often successful at producing high-quality solutions, local search heuristics are typically unable to produce a bound on the optimal value of the problem. Thus the only measure of solution quality is a comparison with solutions produced by alternative approaches. In this thesis we study the design of algorithms that produce solutions to an integer program (IP) by solving restrictions of the problem via integer programming technology. We refer to this type of algorithm as IP-based search. In addition to the simple variable fixing restriction structure used in branch-and-bound, we also consider more general structures. To guide our search we use different mechanisms both for creating restrictions and choosing the one we solve next. We consider modeling a problem with a formulation that captures both the problem and the choice of restriction to solve to produce its optimal solution. Then, by searching for solutions to this extended formulation with a branch-and-bound-based algorithm, we ensure that we can produce dual bounds and have an exact algorithm. In the spirit of local search, we also consider building restrictions that represent a neighborhood of a known solution. In contrast to meta-heuristics, we are trying to avoid getting stuck at a bad local optimum by trading a large number of searches for neighborhoods that are large enough to contain a locally optimal solution of high-quality. While we use neighborhoods similar (exponential) in size to Very Large Neighborhood Search [2], by not confining ourselves to those that can be searched efficiently, they are theoretically much harder to search. 2

12 However, by coupling intelligent neighborhood selection mechanisms with the power of today s commercial IP solvers, in practice these neighborhoods can be searched quickly. The idea of modeling the neighborhood of a solution as the set of feasible solutions to an integer program and then searching that neighborhood with integer programming technology has recently garnered a great deal of attention in the literature. For general integer programs, both Relaxation Induced Neighborhood Search (RINS, [12]) which creates variable fixing restrictions by combining information from a feasible solution and a solution to a linear program (LP), and Local Branching [14] which creates restrictions that restrict the number of binary variables whose value may differ from their value in a known solution have been very successful. For a structured problem, the heuristic presented in [13] for the Distance- Constrained Capacitated Vehicle Routing Problem is interesting both for its success and the fact that it solves integer programs that are not restrictions of the problem being solved. Local search and exact optimization approaches often differ both in their emphasis and the techniques they use. While exact optimization algorithms produce dual bounds and proofs of optimality, they often do so at the expense of finding high quality solutions early in the search process. On the other end of the spectrum, local search heuristics often produce good solutions quickly but no dual bound. However, dual bounds, proofs of optimality and finding high quality solutions quickly are all desirable algorithmic properties. In addition, it is natural when designing an algorithm to consider what techniques can be borrowed from approaches developed by both the heuristic and mathematical programming communities. One goal of this research is to study the design of IP-based search algorithms that combine ideas from these communities both to guide the search for good solutions and produce dual bounds. In studying IP-based search we first consider two questions: 1. Can we design IP-based search algorithms that quickly produce good solutions? On problems of both academic and real-world interest we show that we can, and that the solutions produced are often near-optimal. In addition, the solutions produced after a short period of time by these IP-based search algorithms are often much better than those produced by both state-of-the art commercial optimization solvers and meta-heuristics, even when these other algorithms are given significantly 3

13 more time. We also see that IP-based search algorithms are particularly effective for extremely large problem instances. While branch-and-bound begins its search at the root node of a tree with no extra restrictions placed on the variables of an IP, one can think of IP-based search as pruning nodes at an intermediate depth wherein many variables have their value fixed. These fixed variables obviate the need to load a full instance into memory, which in itself can cause problems. In addition, by fixing variables cleverly, we may be left with restrictions of the IP that have structure that can be exploited. We also see that by defining the search procedure as solving an integer program, IP-based search algorithms are flexible since considering different problems from a specific class only requires redefining the integer program. 2. Can we design an IP-based search algorithm which provides a measure of the quality of the solution produced? We focus on problems that have both a compact and extended formulation and actively use the two. A well-known fact from column generation is that one can couple the solution of the linear programming (LP) relaxation of an extended formulation with lagrangean techniques to generate a dual bound that is no worse and often stronger than the dual bound provided by the compact formulation. In addition, this fact provides a mechanism for producing a dual bound for problems whose size prohibits solving the LP relaxation of the compact formulation. We address these questions in the context of network design problems by developing IPbased search algorithms for the Multi-commodity Fixed Charge Network Flow (MCFCNF) problem and the load plan design problem encountered in the Less-Than-Truckload (LTL) freight transportation industry. While MCFCNF allows us to study the effectiveness of IP-based search in a more academic setting, the latter problem allows us to study its applicability to a real-life transportation problem that includes complicating side constraints. MCFCNF is a classic optimization problem that lies at the heart of many transportation problems. In MCFCNF, a set of commodities must be routed through a directed network, each of which has an origin and a destination. The MCFCNF models both the routing of 4

14 commodities and the design of the network itself by charging a fixed cost when an arc is used. While there are many variants of MCFCNF, we focus on instances with capacitated arcs, but consider both the case where the demand of a single commodity must follow a single path and the case where it may be split across multiple paths. Regarding cost structure, we associate a fixed cost with using an arc and a variable cost that depends on the quantity routed along the arc. The objective is to minimize the total cost. We first present an IP-based neighborhood search heuristic that solves restrictions derived from the arc-based formulation of MCFCNF wherein a subset of variables is fixed to their value in the best known solution. Like Variable Neighborhood Search [23], the heuristic first chooses a class of variables to fix and then uses ideas ranging from studying the structure of good solutions found so far to deriving information from a solution to the LP relaxation of the extended (path-based) formulation of MCFCNF to determine which variables of that class to fix. One mechanism for assessing the quality of an integer MCFCNF solution is to combine the LP relaxation of the path-based formulation with lagrangean techniques to produce a dual bound on the optimal IP value. However, for MCFCNF, the LP relaxation alone is known to produce a weak dual bound and valid inequalities are critical to producing a meaningful lower bound. Another innovative aspect of the heuristic is that the LP relaxation is strengthened by cuts which are found during the solution of the small IPs. Computational results indicate that IP-based search can quickly produce high-quality solutions to the MCFCNF. For instances of various sizes, the quality of the solution produced by IP-based search in only 15 minutes is often significantly better than the best solution found by the state-of-the-art commercial solver CPLEX or a best-known meta-heuristic, even when those approaches are given much more time. Next, after observing the success of an IP-based search heuristic that is designed for a specific problem class and uses a specific type of restriction, we present an approach that generalizes some of the ideas and techniques used. In particular, we do not require that restrictions are formed by fixing variables, but instead by a set of constraints added to the original problem. We also present methods for creating and selecting restrictions that are 5

15 systematic but not problem-specific; thus the resulting approach is a general integer programming algorithm. One of the novel aspects of the approach is that it creates restrictions by dynamically adding variables to the problem formulation via column generation. While the heuristic presented in Chapter 2 uses information from an extended formulation as one of many guides in the process of creating IPs to solve in its search for primal solutions, here we explicitly formalize the creation of restrictions by solving an optimization (pricing) problem. Given that we have a potentially huge number of restrictions, we are still left with the questions of which one we should solve next and whether we can guarantee that we find the optimal solution without solving them all. Fortunately, branch-and-price [5] gives us an effective and well-understood framework for making these decisions since it is specifically designed to solve integer programs that are modeled with an extended formulation that has more variables than can be considered explicitly. Although our algorithm falls into the category of a branch-and-price approach, it differs significantly from those in the literature. An extended formulation is typically chosen because it provides a stronger linear relaxation than the usual (compact) formulation for a problem. Therefore, an extended formulation is typically used to strengthen the dual bound produced throughout the course of a linear programming based branch-and-bound method. Our extended formulation is designed to facilitate creating restrictions of the compact formulation of the problem that are small enough to be solved quickly. Thus, the extended formulation improves our ability to produce primal solutions. In addition, an extended formulation is typically structurally different from the compact formulation since it involves modeling a decision with different objects (i.e. choosing a path for routing a commodity as opposed to choosing a set of arcs). As a result, effective valid inequalities for the compact formulation may not be applicable to the extended formulation. By simply adding variables to the compact formulation, our extended formulation allows the use of all inequalities that are valid for the compact formulation. While branch-and-price schemes traditionally only use LP bounds to prune nodes in the search tree, recent procedures such as [26] employ the idea that when deep enough in the branch-and-bound tree wherein many variables are fixed, it is best to switch back to 6

16 the compact formulation of the problem and let the IP solver prune the node. In addition to improving the dual bound by pruning a node, switching to the compact formulation and solving the IP also aids in the search for good primal solutions. With our extended formulation, we can easily create and solve a restriction of the compact formulation to produce a feasible IP solution at every node of the branch-and-bound tree such that one exists. We also show that our extended formulation provides easy and general ways to create integer programs that represent neighborhoods of the best known solution. To illustrate the application of the approach, we return to the single-path variant of the MCFCNF. The computational experiments show that the approach can quickly produce high quality primal solutions and tight dual bounds. For instances of various sizes, the approach often produces in 15 minutes a primal solution which is both near-optimal and better than what CPLEX can produce in 6 hours and a dual bound that is comparable to what CPLEX can produce in 6 hours. In addition, the dual bound produced by our approach in 15 minutes often provides a proof that our solution is optimal (or near-optimal). Lastly, we address the effectiveness of IP-based search on a real-life network design application, the load plan design problem in the Less-Than-Truckload (LTL) freight transportation industry. In addition to presenting a new IP-based search algorithm for producing cost-effective load plans, we develop new models to accurately capture how carrier s currently operate and to study the cost effectiveness of new business practices. In particular, we develop a model of LTL operations that differs significantly from existing load plan design research in three ways; it models time at a level that is appropriate to tight service standards, captures the interplay between empty and loaded trailer routing decisions and can support freight routing decisions that vary by day. The first two of these advances improve our ability to design cost-effective load plans that carriers find implementable, while the last allows us to study a new mode of operation for LTL carriers that has significant potential for cost savings. The transportation industry is one of the largest in the U.S. with $1.4 trillion (10.6% of total GDP) spent on transportation-related goods and services in In addition, the transportation industry represents 10% of the total U.S. labor force. The trucking industry 7

17 now accounts for 6% of GDP. The trucking industry has two sectors: Full Truckload (FTL or TL) and Less-Than-Truckload (LTL) with TL accounting for the major share of revenues (about 70% of the market is TL). While a truckload carrier transports freight directly from origin to destination without the freight being handled en route, the size of LTL shipments (typically between 100 and 10,000 lbs) render it too costly to transport each customers shipment directly from origin to destination. As a result, LTL carriers collect freight from various shippers and consolidate that freight in order to build nearly full trailers. The LTL industry has changed significantly over the last 20 years with many regional carriers consolidating into super-regional or national carriers. In addition, UPS and FedEx have entered the marketplace, increasing competition in both price and service. While historically a shipment was quoted a service standard from origin to destination of 5 business days, today s LTL customers often expect service standards of 1,2 and 3 days. To accommodate increased competition on price, LTL carriers must find new ways to increase the utilization of their current network infrastructure in spite of these tighter service constraints. The utilization of a carrier s network is driven by the load plan which determines shipment consolidation opportunities by prescribing how freight is routed from origin to destination and where it is handled. The concept unique to load plan design is that of a direct, which specifies where freight is handled. For example, saying a shipment takes the direct Atlanta Detroit means the shipment is loaded into a trailer in the Atlanta terminal which is not opened (and hence the freight not handled) again until it reaches the Detroit terminal. The local search heuristic presented by [28] (the engine within the commercial software used by nearly every LTL carrier at one point in time) leverages this property in its search for a better load plan by performing a sequence of add and drop direct operations and re-routing freight flows after each operation. However, the heuristic does not explicitly model daily freight volume fluctuations, service standards or the timing of consolidation opportunities and does not route loaded and empty trailers simultaneously. The path a shipment follows in an LTL network consists of a sequence of directs. The load plan prescribes how a shipment is routed through the carrier s network of directs by 8

18 specifying the unique direct to take given the shipment s current terminal location and ultimate destination terminal. For example, the load plan may give the following instruction: all freight in Jackson, TN destined for Atlanta, GA loads direct to Nashville, TN. Thus, the load plan dictates that freight follows a unique path within the network of directs and that the directs into a destination terminal must form a directed in-tree. We present an IP-based search heuristic that is motivated by this structural property in that it searches for a better load plan by solving an integer program to optimize the routing of all freight destined for a given terminal while holding fixed freight destined for all other terminals. While the work presented in [21] also bases the search for load plans on this in-tree structure, it neither models time as precisely as we do nor does it route loaded and empty trailers simultaneously. In addition, we present valid inequalities and preprocessing techniques to speed up the solution of these integer programs and heuristics to control their size without forgoing consolidation opportunities. Computational results indicate that IP-based search is very effective for the load plan design problem. To test the heuristic designed we use the existing load plan of a national carrier as a baseline and historical freight volumes as a data set. Although our detailed model of time yields networks with nearly 6,000 nodes and 500,000 arcs and full problem instances with over 1,000,000 integer variables and 2,000,000 constraints, we see that IP-based search can still produce load plans with proposed weekly savings of 3% in transportation and handling costs. To respond to the challenges presented by increased competition in both price and service, carriers are interested not only in better load plans but also in new modes of operation. One new mode of operation, heretofore not considered in the literature, is the use of predictable daily freight volume variations in the load plan design process to build plans that vary by weekday. The heuristic we present can easily accommodate this variation of the traditional load plan and computational results indicate that by allowing routing decisions to vary by day we can save nearly 6% in weekly transportation and handling costs. This thesis makes contributions both to algorithm design by illustrating the effectiveness 9

19 of integer programming based search and the study of network design for freight transportation carriers by developing new models of LTL operations. Our main algorithmic contributions show that: IP-based search algorithms can quickly produce high-quality solutions for network design problems of both theoretical and real-world interest; IP-based search algorithms are particularly well-suited to large problem instances and classes of problems that contain many variants; we can use an extended formulation of a problem both to guide the search for good primal solutions and generate dual bounds; and we can design an exact IP-based search algorithm that will converge to the optimal solution without sacrificing solution quality in the early stages of the algorithm. The main contributions to the study of network design for freight transportation carriers fall into two categories; studying the potential of new modes of operation and developing models which more accurately capture operational-level concerns. In particular, for the load plan design problem we show that: significant savings can be achieved by allowing freight routing decisions to vary by day; one can model time at the level of detail required in the presence of tight service standards without sacrificing the quality of load plan produced; and by integrating loaded and empty trailer routing decisions we can produce load plans which are both more cost effective and realistic than current practice. 10

20 CHAPTER II IP-BASED NEIGHBORHOOD SEARCH We begin our study by developing an IP-based local search heuristic for a specific problem (or class of problem); namely the Multi-commodity Fixed Charge Network Flow (MCFCNF) problem. The MCFCNF problem is a classic discrete optimization problem in which a set of commodities has to be routed through a directed network. Each commodity has an origin, a destination, and a quantity. Each network arc has a capacity. There is a fixed cost associated with using an arc and a variable cost that depends on the quantity routed along the arc. The objective is to minimize the total cost. Two versions of the problem are considered: commodities have to be routed along a single path and commodities can be routed along multiple paths. Many real-life instances of MCFCNF (or of instances of models that contain MCFCNF as a substructure) are very large (see for example [33]) - so much so that sometimes even the linear programming relaxation of the natural arc-based integer programming formulation can be intractable. In such situations, we can use a pathbased integer programming formulation, but that necessitates the use of column generation techniques. Furthermore, the linear programming relaxation of the arc-formulation and the path-formulation have the same optimal value which is known to be weak. Hence, it may not just be difficult to find feasible solutions, it may also be challenging to determine the quality of these solutions. As a result, even though much research has been devoted to MCFCNF, exact methods are only capable of handling small instances, far smaller than many realisticsized instance. Fortunately, in today s dynamic business environment getting high-quality solutions in a short amount of time is usually more important than getting provably optimal solutions. Hence the focus of this chapter is to develop a solution approach for MCFCNF that produces provably high-quality solutions quickly. Our solution approach relies heavily on linear and integer programming to take advantage of the power of commercially available linear and integer programming solvers. 11

21 Algorithmically, we combine mathematical programming techniques with heuristic search techniques. More specifically, we develop a primal local search algorithm that utilizes neighborhoods that are not searchable in polynomial-time and instead are searched by an integer programming solver, a scheme to generate dual bounds that involves strengthening the linear programming relaxation via cuts discovered while solving these integer programs. Our approach tightly integrates the use of the arc-based formulation of MCFCNF and the path-based formulation of MCFCNF. It also incorporates randomization to diversify the search and learning to intensify the search. The resulting solution approach is very effective. For instances with 500 nodes, with 2000, 2500 and 3000 arcs, and with 50, 100, 150, and 200 commodities, we compared the quality of the solution produced by our solution approach with the best solution found by CPLEX after 15 minutes of computation and after 12 hours of computation. On average, the solution we found in less than 15 minutes is 35% better than CPLEX best solution after 15 minutes and 20% better than CPLEX best solution after 12 hours. Furthermore, we find a better solution than CPLEX best solution after 15 minutes within 1 minute, and CPLEX best solution after 12 hours within 3 minutes. On these instances the approach produces dual bounds that are 25% stronger than the LP relaxation. We also compared the quality of the solutions produced by our solution approach with the quality of the solutions produced by a recent implementation of the tabu search algorithm of [15]. For nearly all instances in their test set, our solution is better than the solution of the tabu search algorithm and this solution is found much faster. The key characteristics of our solution approach for MCFCNF, which differentiate it from existing heuristic approaches, are: it uses exact methods to find improving solutions, it generates both a primal solution and a dual bound at each iteration, and it uses both the arc and path formulations of MCFCNF to guide the search. 12

22 2.1 Literature Metaheuristics have been developed that find good primal solutions to instances of MCFCNF. A tabu search algorithm using pivot-like moves in the space of path-flow variables is proposed in [10]. The scheme has been parallelized in [8]. A tabu search algorithm using cycles that allow the re-routing of multiple commodities is given in [15]. This cycle-based neighborhood is incorporated within a path-relinking algorithm in [16]. Our heuristic solves carefully chosen integer programs to improve an existing solution. As such, it considers exponential-sized neighborhoods similar to very large-scale neighborhood search (VLSN, [2]). However, in contrast to VLSN, no polynomial-time algorithm exists for searching these neighborhoods. General integer programming heuristics such as local branching [14] and relaxation-induced neighborhood search [12] also are integerprogramming based local search algorithms, but they are different from ours. The dual side of the problem is studied in [7], using various lagrangean relaxations and solution methods. Although lagrangean-based heuristics, such as the one proposed in [19] generate dual bounds, these bounds are no better than the value of the LP Relaxation. To the best of our knowledge, no heuristic produces dual bounds that are stronger than the LP relaxation. Combining exact and heuristic search techniques, a key characteristic of our approach, has received quite a bit of attention in recent years, see for example [13], [36], [3], and [35]. 2.2 Formulations Before describing the main ideas of the proposed solution approach, we present the arcbased formulation and the path-based formulation for MCFCNF. Let D = (N,A) be a network with node set N and directed arc set A. Let K denote the set of commodities, each of which has a single source s(k), a single sink t(k), and a quantity d k that must be routed from source to sink. Let f ij denote the fixed cost for using arc (i,j), c ij denote the variable cost for routing one unit of flow along arc (i,j) and u ij denote the capacity of arc (i,j). We use variables x k ij to indicate the fraction of commodity k routed along arc (i,j) and binary variables y ij to indicate whether arc (i,j) is used or not. The arc-based formulation 13

23 of MCFCNF (Arc-MCFCNF) is: min k K c ij (d k x k ij) + f ij y ij (i,j) A (i,j) A subject to x k ij x k j,i = δi k i N, k K, (1) j:(i,j) A j:(j,i) A d k x k ij u ij y ij (i,j) A, (2) k K y ij {0,1} (i,j) A. (3) If a commodity s demand must follow a single path we have x k ij {0,1} k K, (i,j) A, (4) otherwise, we have 0 x k ij 1 k K, (i,j) A. (5) The objective is to minimize the sum of fixed and variable costs. Constraints (1) ensure flow balance, where δ k i indicates whether node i is a source (δ k i = 1), a sink (δk i = 1) or an intermediate node (δi k = 0) for commodity k. Constraints (2) are the coupling constraints that ensure that an arc is used if and only if its fixed charge is paid and that the total flow on the arc does not exceed its capacity. It is well-known that Arc-MCFCNF has a weak LP relaxation and can be strengthened by disaggregating the coupling constraints to x k ij y ij k K, (i,j) A. (6) The resulting formulation has a tighter LP relaxation, but comes at the expense of many more constraints. Next, we consider the path-based formulation of MCFCNF. Let variable x k p denote the fraction of commodity k that uses path p and let P(k) denote the set of feasible paths for commodity k. We will consider instances where P(k) is not known in full. Hence let P(k) P(k) denote a subset of all feasible paths for commodity k. The path-based formulation of MCFCNF (Path-MCFCNF) is: min ( c ij )d k x k p + f ij y ij k K p P(k) (i,j) p (i,j) A 14

24 subject to x k p = 1 k K, (7) p P(k) k K p P(k):(i,j) p d k x k p u ijy ij (i,j) A, (8) y ij {0,1} (i,j) A. (9) We again have either x k p {0,1} or 0 x k p 1 depending on whether a commodity s demand may be split across multiple paths. As before, the objective is to minimize the sum of fixed and variable costs. Constraints (7) ensure that every commodity is routed through the network and the coupling constraints (8) ensure that fixed charges are paid and that arc capacities are respected. Here too we can disaggregate the coupling constraints: p P(k):(i,j) p 2.3 Solution Approach x k p y ij k K, (i,j) A. (10) At the heart of the primal side of our solution approach is a neighborhood search procedure. Consider the arc-based formulation of MCFCNF. A subset of variables V can be used to define a neighborhood of the current solution by fixing the values of variables v V to their value in the current solution. By selecting a suitably small subset of the variables a tractable integer program can be defined and solved using an IP solver. Hopefully, a better solution to the whole problem is obtained. The process can be repeated multiple times by choosing different subsets of variables. A pseudo-code describing this process is given in Algorithm 1. Algorithm 1 Neighborhood Search while the search time has not exceeded a prespecified limit T do Choose a subset of variables V Solve the IP defined by variables in V if an improved solution is found then Update the global solution end if end while 15

25 Note that the neighborhood in Algorithm 1 is searched using an integer programming solver. The key to making this neighborhood search scheme work is in the choice of the subsets of variables V. This approach can be used on extremely large instances because the algorithm never requires the full instance to be in memory. To evaluate the quality of the solution produced by the neighborhood search we find lower bounds on the value of the primal solution using a path-based formulation. While the number of variables in the path-based formulation is huge, variables can be considered implicitly rather than explicitly. The LP relaxation is solved over a subset of the variables and a pricing problem is solved to determine whether there is a need to expand the set of variables or not. The pricing problem is a shortest path problem and relies on the dual values associated with the constraints of the path-based formulation. A well-known observation related to this column generation process is that a valid lower bound can be obtained at every pricing iteration. Specifically, let z LP be the value of the solution to the LP and let c k p be the reduced cost of an optimal path p for the pricing problem for commodity k. Then z LP + k K ck p is a lower bound for the value of the LP when all columns are considered. Note that any solution of Arc-MCFCNF can be converted to a solution of Path-MCFCNF and vice versa. As a result, a solution to the IP based on a subset of arc variables V can be converted to a set of variables for Path-MCFCNF. It is highly likely that several of these variables do not appear in the path formulation yet. In this way, the neighborhood search generates variables for Path-MCFCNF. Similarly, a solution to Path-MCFCNF can be converted to a solution of Arc-MCFCNF and can thus be used to guide the choice of subset V in the neighborhood search Neighborhood Search During the course of the algorithm, we solve smaller integer programs defined by a subset of arcs in the network or a subset of commodities in the instance. We define a SOLV ER to be one of the two small integer programs together with a selection method for the necessary subset. A high-level implementation of a solver is given in Algorithm 2. A highlevel implementation of the approach is given in Algorithm 3. We currently use a stopping 16

26 Algorithm 2 Solver Template Require: A feasible solution F to the full MCFCNF instance Require: The set of all feasible paths P found so far Select Subset Solve Subproblem Construct Feasible Solution F to full MCFCNF instance Determine new paths P found Determine new cuts C found in course of solving subproblem return (F,P,C ) criterion of whether the search time has exceeded a prespecified limit T. Given that we have Algorithm 3 Neighborhood Search Find an initial feasible solution Set best solution = initial feasible solution Set PATHS = paths from initial feasible solution Set SOLV ERS = set of solvers we wish to consider using Set CUTS = while not done do Solve path formulation LP only considering PATHS but with CUTS and let z LP be the value of the optimal solution for all k do Solve pricing problem for commodity k to get path p k with reduced cost c pk if c pk < 0 then Add p k to PATHS Set z LP = z LP + c p k end if end for Select solver S from SOLV ERS if solution found by S better than best solution then Set best solution = solution found by S end if Add new paths p from S to PATHS Take new cuts (π,π o ) from S, lift and add to CUTS end while multiple solvers to choose from, we need a scheme for choosing a solver at each iteration. We use a scheme similar to the roulette wheel approach of [34] (see also [27]), i.e., we randomly pick solvers with the probability of choosing a solver favoring ones that have recently given improvement. Specifically, if C t is the cost of the feasible solution F t provided to solver S at iteration t and C t is the cost of the feasible solution F t found by S, then we calculate the improvement for solver S in that iteration as D S t = C t C t and in iteration T, we give 17

27 S the score E S = T 1 t=1 1 2 T tds t. The larger the value E S is, the more effective solver S has been in improving solutions in recent iterations. Assuming m solvers, ordered such that E Si E Si+1 for i = 1,...,m 1, we assign selection probabilities π i = m (i 1) m i = 1,...,m. j=1 m (j 1), We do not use a simple proportional scheme so that solver S i can be chosen even when E Si = 0, and to ensure that the selection is not biased too greatly in favor of solvers that have been effective in the last iterations Arc Subset Solves Given a subset of arcs A A, the IP defined by A is Arc-MCFCNF with A replacing A. Our schemes for choosing the set A are motivated by the simple idea that if we knew a set of arcs, A = {(i,j) A y ij = 1 for some optimal solution (x,y )}, we could find the optimal solution to the complete instance by solving the arc subset IP for A. Hence our schemes use various measures to try and guess which arcs exist in an optimal solution. Observe that if A contains the arcs associated with the current best solution, then we can seed the arc subset IP with that solution and ensure that we only find solutions at least as good as the current best. Therefore, our schemes for choosing a subset A start from the subset of arcs associated with the current best solution. Also, our schemes do not choose arcs directly but choose paths, thus ensuring that every arc chosen exists in some feasible solution. In fact, the schemes choose paths from among the paths that are part of the path-based formulation that is used to compute a dual bound. The schemes we next describe are used to generate a score for each path. We then randomly choose paths until A = N A, a preset threshold on A, with the probability of choosing a path assigned so as to bias the selection towards paths with higher scores. Scheme 1. This scheme is driven by the idea that paths that frequently appear in good solutions found during the course of the search may be in the optimal solution. Hence the 18

28 greater the number of times path p appears in an improving solution, the higher the score it is assigned. Scheme 2. Recall that at each iteration we solve the LP relaxation of the path-based formulation with the set of paths P AT HS. This scheme is motivated by the idea that the solution to the LP relaxation may guide us to paths that appear in the optimal IP solution. Let x k,lp p represent the value of variable x k p in the most recent solution to the LP relaxation of our path-based formulation. The higher the value of x k,lp p, the higher the score given to path p. Scheme 3. Recall that at each iteration we price new paths for each commodity for the LP-relaxation of the path-based formulation. This scheme is motivated by the idea that these paths are good candidates for the optimal IP solution. We also consider the reduced cost, c p, when assigning a score to a path, giving a higher score to a path that has lower reduced cost Commodity Subset Solves Given the current solution x and a subset of commodities J, let A = {(i,j) A x k ij = 0 k J}, i.e., the set of arcs that is not used by commodities not in J. Let f ij (i,j) A f ij = 0 (i,j) A and ũ ij = u ij k J dk x k ij. Then we solve an Arc-MCFCNF only considering commodities in J and with f ij instead of f ij and ũ ij instead of u ij. That is, if there exists a commodity not in J that already uses an arc, we allow the commodities in J to use that arc for free (in terms of the associated fixed cost charge), but we make sure that the available capacity on the arc reflects the flow already on it. Whereas we motivated our arc selection methods by the idea of guessing the optimal set of arcs, we motivate many of our commodity selection methods by the idea that when fixed charges (f ij ) are high, the optimal solution is likely to include as few arcs as possible. Hence we want to choose a subset of commodities J that are likely to share arcs. 19

29 Constructing a feasible solution to the full MCFCNF instance starting from the solution to the IP requires updating the paths used by the commodities in J and the appropriate y ij variables for (i,j) A. Note that since we do not have variables y ij for (i,j) A \ A our subproblem cannot turn off an arc used by a commodity not in J. We can again seed the IP with the paths used by the commodities in J in the current best solution. The first three schemes we describe below are used to select a subset of commodities, J. The last three schemes are similar to the path schemes described above in that they assign a score to each commodity and then randomly choose commodities with a bias towards those with better scores. Unless otherwise noted, we always select a subset of commodities, J of fixed size, N K. Scheme 1. Again, recall that at each iteration we solve the LP relaxation of the path-based formulation with the set of paths P AT HS. This scheme selects commodities whose paths in the current best solution contain arcs (i,j) for which the reduced cost of variable y ij, denoted by f ij, are far from 0. In an LP solution complementary slackness implies y ij = 1 only if f ij = 0. Thus the motivation for this scheme is to re-route commodities away from arcs that are far away from the complementary slackness condition. Scheme 2. Suppose in the current best solution ( x,ȳ), there is a node v with three commodities C1,C2,C3 entering on a single arc, but leaving on three different arcs The motivation for this scheme is that when the arcs leaving node v are not fully used, these three commodities are good candidates for re-optimization. (Of course, a similar situation occurs when we have multiple commodities entering on different incoming arcs, but leaving on a single outgoing arc.) Assume we have a pre-defined utilization threshold UTIL THRESHOLD, where we calculate the utilization of an arc for the current best solution ( x,ȳ) as UTIL ij = d k x k K k ij /u ij. Then we can score each node based on the number of under-utilized inbound and outbound arcs in the current best solution. Given those scores, we generate probabilities for selecting a node v. After randomly selecting a node v, we choose the commodities k such that in solution ( x,ȳ), w such that x k v,w > 0 or x k w,v > 0. Note this scheme may actually choose fewer (or more) than N K. 20

30 Scheme 3. Since we want to find sets of commodities that are likely to share arcs, we search for commodities whose paths in the current best solution are close together. Given a randomly chosen node v, we perform breadth-first search with v as root. As we discover nodes visited by commodities k in our current best solution, we select the commodities. We continue the search until we have chosen N K commodities. Scheme 4. Suppose the network is very large and there are two commodities k and l that have only one feasible path from their source to their sink, say p k and p l. In that case, it is unlikely that these paths have a common arc. Hence re-optimizing commodities k and l together is not likely to lead to an improving solution. Conversely, if k and l have many paths then they are likely to share arcs and thus are good candidates to optimize together. While we do not know the set of all feasible paths for each commodity, during the course of the algorithm we do generate a set of feasible paths P(k) for each commodity k. Hence, the higher the number P(k), the higher the score assigned to commodity k. Scheme 5. So far the selection schemes have been geared towards improving the current best solution. The purpose of this scheme is to provide diversification for our arc subset solvers. If there is a commodity k for which we have only generated a single path so far, then an arc subset solver will have little flexibility for re-routing it. Therefore, the lower the number P(k), the higher the score assigned to commodity k. Scheme 6. The final scheme is specifically designed for instances of MCFCNF in which a commodity must be routed along a single path. The scheme selects commodities whose demand is split across many paths in the most recent solution to the LP relaxation of the path-based formulation. Again, define x k,lp p to be the value of variable x k p in the solution of the LP relaxation to the path-based formulation and let P (k) = {p P(k) : 0 < x k,lp p < 1}. Then the higher the number P (k), the higher the score assigned to commodity k. In a sense, we are repairing commodities that violate the single path constraint in the solution to the LP relaxation of the path-based formulation. 21

31 2.3.2 Lower Bounds At each iteration our solution approach produces both an upper and a lower bound. The lower bound is provided by the value of the LP relaxation of the path-based formulation. However, since the pricing problem is a shortest path problem, the bound is no better than the LP relaxation of the arc-based formulation, which is known to be weak. We attempt to strengthen the bound by adding valid inequalities before solving the path-based formulation. We first note that we can translate inequalities between the arc and path formulations using the transformation x k ij = p P(k):(i,j) p xk p, where P(k) is the set of paths generated for commodity k thus far. Since we want valid inequalities that are violated by the optimal solution to the LP relaxation of the path-based formulation given the sets P(k), a desirable property of a valid inequality (π,π 0 ) is to have π k ij > 0 when xk ij > 0 in the optimal LP solution. We estimate the likelihood of x k ij being greater than 0 in the optimal LP solution by considering the ratio r k ij = {p P(k) : (i,j) p} P(k) which calculates the percentage of paths we have generated (and hence will consider in the LP) for commodity k that use arc (i,j). We have focused on two techniques for strengthening the path-formulation LP Judiciously adding disaggregate coupling inequalities, and Lifting cuts found while solving small IPs. We could further strengthen the path-formulation bound by generating valid inequalities after solving the LP relaxation which the current fractional solution does not satisfy. However, identifying the separating inequality and re-solving the LP relaxation would would take time away from the search for primal solutions. This trade-off could be avoided in a parallel implementation by having separate processes work on the lower and upper bounds Disaggregate Inequalities It is well-known that the disaggregate coupling inequalities (10) strengthen the LP bound, yet due to the great number of them it is typically not possible to add them all. In addition, 22

32 while adding the disaggregate coupling inequalities strengthens the LP bound they may also increase the solve time for the LP relaxation. Of course adding the inequality (10) for commodity k and arc (i,j) only strengthens the LP bound if in fact x k ij = p P(k):(i,j) p x k p > 0 in the optimal LP solution. Hence we determine the arcs (i,j) and commodities k for which the disaggregated coupling constraints will be added using the ratio rij k. A threshold value T [0,1] is specified and the inequality is added for commodity k and arc (i,j) only when r k ij T Lifted Cover Inequalities We have investigated re-using cuts (π,π 0 ) found while solving small IPs during the course of the algorithm. These cuts are lifted to make them globally valid and to strengthen them. As the inequalities are found while solving small IPs during the course of the algorithm, the computational effort is only in the lifting process. After lifting, these inequalities are added to the path formulation to strengthen the bounds its LP relaxation provides. We have focused our efforts on the single-path variant of MCFCNF and investigated lifting cover inequalities based on arc capacities. A cover with respect to an arc (i,j) is a set C K such that k C dk > u ij. Since the variables x k ij are binary for the single-path variant we have the cover inequality k C xk i,j C 1. We use the procedure suggested by [18] for separating cover inequalities. Note that since the cover inequality is generated during the solution of a commodity subset IP some variables are fixed to either 0 or 1. As suggested by [18], we first uplift variables fixed to 0 and then down-lift those fixed to 1. It is well known that the strength of the lifted cover inequality depends on the order in which the variables fixed to 0 (or 1) are lifted. Therefore we first lift variables x k ij whose value is likely to be positive in the optimal LP solution. Hence, the lifting order is quite simple: lift in decreasing order of rij k. The complete procedure is outlined in Algorithm 4. Currently, we limit ourselves to cover inequalities that are violated in the small IP. 23

33 Algorithm 4 Cover Inequality Lifting Require: Cover Inequality (π,π 0 ) for arc (i,j) from commodity subset IP Require: Set of commodities K 0 such that when generating (π,π 0 ), x k ij fixed to 0 Require: Set of commodities K 1 such that when generating (π,π 0 ), x k ij fixed to 1 Calculate rij k {p P(k):(i,j) p} =, k K P(k) 0 K 1 Uplift variables x k ij,k K0 in descending order of rij k Downlift variables x k ij,k K1 in descending order of rij k return new cut (π,π 0 ) Given that we are especially interested in strengthening the LP relaxation of the pathbased formulation, this methodology can be extended to search for cover inequalities that are not violated in the small IP, but are likely to be violated when all commodities are considered. Since our lifting order depends on the sets P(k) which may change at each iteration, we could re-lift the cover inequalities as the path sets change and get different valid inequalities. However, we do not re-lift because of our time-constrained setting and the computational effort lifting requires Initial Feasible Solution Our scheme for generating an initial feasible solution has two phases. In Phase I we find a path for each commodity by solving a shortest path problem where arc costs reflect a linearized fixed charge and are updated using a slope-scaling approach. Heuristics based on updating the linearization of the fixed charge are given in [4], [11], and [22]. The solution to the LP relaxation of Arc-MCFCNF has y ij = 1 u ij k dk x k ij, hence we have f ij y ij = f ij u ij k dk x k ij = k f ij dk u ij x k ij. This suggests that we can use a variable cost c ij = c ij d k + f ij d k u ij for each arc (i,j) and then solve a shortest path problem using these arc costs for each commodity k. Instead, we recognize that by assigning paths to commodities sequentially we remove the capacity of some arcs. Hence we update the variable cost on an arc (i,j) to reflect how much of its capacity remains. The details are given in Algorithm 5. If we have a feasible solution after Phase I, we stop. Since we do not explicitly enforce the capacity constraints on arcs during Phase I, we may not end with a feasible solution. In this case, we go on to Phase II, where for each arc whose capacity is exceeded, we find a set 24

34 of commodities to re-route away from the arc in an attempt to fix the infeasibility. The details are given in Algorithm 6. Note that this method cannot guarantee that a feasible solution is found when one exists, nor can it detect whether an instance is infeasible. Algorithm 5 Phase I algorithm Sort commodities in descending order of demand d k Set ũ ij = u ij for k = 1 to K do Set c ij = c ij + f ij dk ũ ij Solve a shortest path problem for commodity k w.r.t. arc costs c ij ; let x k ij indicate when arc (i,j) used by k Set ũ ij = max(1,ũ ij x k ij dk ) end for Algorithm 6 Phase II algorithm while (i,j) : k xk ij dk > u ij do Randomly choose such an (i,j) Sort commodities using the arc (x k ij = 1) in ascending order of demand dk Find the smallest l such that K k=l xk ij dk u ij Solve our commodity subset IP for those first l 1 commodities end while 2.4 Computational Results There are many questions to answer regarding the primal and dual-side performance of our approach, which we refer to as IP Search. Specifically: Primal-Side How competitive is IP Search with existing metaheuristics and commercial IP solvers? Is there value in using both commodity subset and arc subset IPs? Do the solvers contribute equally to finding good solutions? Dual-Side Does IP Search produce a stronger dual bound than the optimal solution to the LP relaxation of the arc-based formulation? 25

35 Is the dual bound produced by IP Search competitive with the root node bound produced by a commercial IP solver using an arc-based formulation? Do the disaggregate inequalities strengthen the dual bound? Do the lifted cover inequalities strengthen the dual bound? IP Search is implemented in C++ and uses CPLEX 9.1 to solve the subset IPs. Other than those taken from the literature, all the results are obtained from experiments performed on a machine with 8 Intel Xeon CPUs running at 2.66 GHz and 8 GB RAM. Whenever CPLEX 9.1 is used as a benchmark, it was run with an emphasis on integer feasibility, with the disaggregate constraints x k ij y ij added as user cuts, and with an optimality tolerance set to 5%. All computation times reported are in seconds Instance Generation A network generator was used to create instances for experimentation. The generator creates instances without parallel arcs. The inputs to the generator are: The number of nodes, arcs, and commodities in the network A range [c l,c u ] for arc variable costs A range [b l,b u ] for commodity quantities A range [1,k u ] of how many commodities can fit on an arc A ratio R of arc fixed cost to variable cost The generator creates an arc (i,j) by randomly picking nodes i and j. The variable cost c ij for arc (i,j) is uniformly drawn from [c l,c u ]. The fixed cost f ij for arc (i,j) is uniformly drawn from [Rc l,rc u ]. The capacity u ij for an arc is set to kb, where k is uniformly drawn from [1,k u ] and b is uniformly drawn from [b l,b u ]. Commodities k are created by randomly picking pairs of nodes (o(k), d(k)) such that a path exists between the two nodes. The quantity b k for commodity k is uniformly drawn from [b l,b u ]. Finally all commodity and arc data is rounded down to integer values. 26

36 We use the same instance classifications as [15]. An instance is classified as F if the ratio of fixed to variable cost for an arc is high and V otherwise. An instance is classified as T if the approximate number of commodities that can be accommodated on an arc is low and L otherwise. Hence we have 4 classes of instances (F, T),(F, L),(V, T),(V, L) Calibration Since we solve IPs as subroutines in IP Search, their size is important. If the IP is too large we may not find a good improving solution in the time alloted to the IP solver. On the other hand, if the IP is too small it simply may not yield a large enough neighborhood to contain a good improving solution. We determine the arc and commodity subset sizes for a specific instance size by measuring the performance of the heuristic as the subset size grows. For example, to determine the number of commodities in our subset when running IP Search on instances with 500 nodes, 2500 arcs and 150 commodities we randomly generate 5 instances of that size. We then run the heuristic on each of the 5 instances multiple times, varying the number of commodities selected. In the course of our calibration experiments we observed that choosing 20% of the total number of commodities in the instance is a reasonable rule-of-thumb. It is not surprising that the size of the commodity subset grows with the number of commodities in the instance. The more commodities outside of the subset, the more likely an arc (i,j) is used by a commodity outside of the subset, which implies there will be no fixed charge for arc (i,j). Hence the larger the number of commodities in the instance the more the commodity subset IPs become multi-commodity flow problems without fixed charges Upper Bound Comparison to Tabu Cycle and Path Relink To examine how IP Search compares with existing metaheuristics, we compared it to a cycle-based tabu-search algorithm [15] and a cycle-based path-relinking approach [16] on the instances used in [16]; except for the 6 instances identified as easy (solved in a few seconds by a commercial IP solver). Note that the comparison is only for the variant in which commodities can be routed along multiple paths (the variant handled by the 27

37 metaheuristics). The results are shown in Table 1, where we report the instance, the value of the solution found by CPLEX in 12 hours (CPLEX), the value of the solution obtained by the tabu search algorithm (Tabu Cycle), the value of the solution obtained by the path relinking algorithm (Path Relink), the value of the solution obtained by IP Search (IP Search), the percentage difference between the value of the solution found by IP Search and the value of the solution found by CPLEX in 12 hours (CPLEX gap), the percentage difference between the value of the solution found by IP Search and the best of the values of the solutions found by the tabu search and path relinking algorithms (Best gap), the optimality gap (Opt gap), and for each of the different algorithms the time to reach the best solution (TTB). When we report a percentage difference (or gap) between IP Search and X, it is computed as 100 IP Search X IP Search. To compute the the optimality gap, we use the dual bound produced by CPLEX in 12 hours. Note that we imposed a time limit of 15 minutes (900 seconds) on IP Search. We first observe that IP Search finds a better solution than the metaheuristics in all but 2 of the 37 instances. Since different hardware platforms were used to produce the metaheuristic and IP Search results a comparison of computing times cannot be exact. However, the time to the best solution for instance (30,700,400,F,L) indicates that the computing time is orders of magnitudes longer for the metaheuristics than for IP Search. For the instance (100,400,10,F,T) the percentage difference between the value of the best of the metaheuristics solutions and the value of the solution produced by IP Search is less than 1. Hence, we can comfortably conclude that IP Search is superior to these metaheuristics. We next observe that IP Search is producing solutions that are very competitive with those produced by CPLEX in 12 hours. Even though the instances are relatively small, we see that CPLEX often needs more than 5 hours to find its best solution. Finally, we observe that the optimality gap for the solution produced by IP Search is small, within 5% for 24 of the 37 instances and only 3.96% on average. 28

38 Table 1: Comparison with Tabu Search and Path Relinking 29 Solution Value TTB CPLEX Tabu Path IP CPLEX Best Opt CPLEX Tabu Path IP Problem Cycle Relink Search Gap Gap Gap Cycle Relink Search 100,400,10,V,L 28,423 28,786 28,485 28, ,400,10,F,L 23,949 24,022 24,022 23, , ,400,10,F,T 63,764 67,184 65,278 65, , ,400,30,V,T 384, , , , ,290 1, ,400,30,F,L 49,018 51,831 51,325 49, , ,400,30,F,T 138, , , , ,001 1, ,230,40,V,L 423, , , , ,230,40,V,T 371, , , , ,230,40,F,T 643, , , , ,230,200,V,L 94, , ,404 95, ,118 2,585 2, ,230,200,F,L 137, , , , ,079 3,142 2, ,230,200,V,T 97, , ,689 99, ,097 2,730 2, ,230,200,F,T 136, , , , ,397 3,634 3, ,300,40,V,L 429, , , , ,300,40,F,L 586, , , , ,300,40,V,T 464, , , , ,300,40,F,T 604, , , , ,300,200,V,L 74,929 81,367 78,184 75, ,364 4,086 3, ,300,200,F,L 117, , , , ,220 4,210 4, ,300,200,V,T 74,991 80,344 78,866 76, ,490 4,204 3, ,300,200,F,T 108, , , , ,585 4,855 3, ,520,100,V,L 53,958 56,603 54,904 54, ,261 1, ,520,100,F,L 94, , ,054 94, ,409 2,684 1, ,520,100,V,T 52,048 54,454 53,017 52, ,260 2,716 1, ,520,100,F,T 98, , ,130 98, ,302 2,892 1, ,520,400,V,L 113, , , , ,436 55,771 27, ,520,400,F,L 152, , , , ,624 40,070 36, ,520,400,V,T 114, , , , ,588 4,678 23, ,520,400,F,T 154, , , , ,127 49,886 52, ,700,100,V,L 47,603 50,041 48,723 47, ,959 1, ,700,100,F,L 59,995 64,581 63,091 60, ,900 3,182 1, ,700,100,V,T 45,879 48,176 47,209 46, ,111 3,746 1, ,700,100,F,T 54,904 57,628 56,575 55, ,396 3,547 1, ,700,400,V,L 98, , ,116 98, ,901 38,857 22, ,700,400,F,L 140, , , , ,231 68,214 75, ,700,400,V,T 96, , ,212 96, ,661 51,764 24, ,700,400,F,T 131, , , , ,464 79,053 44, Average

39 Comparison to Commercial IP Solver Next we focus on the performance of IP Search on large instances for the variant in which each commodity has to be routed along a single path by comparing the value of the solution produced by IP Search to the value of the solution produced by CPLEX in 15 minutes (CPLEX-15M) and in 12 hours (CPLEX-12H). We generated 24 instances of the classes (F,T) and (F,L), which we refer to as the GT instances. The results are given in Table 2. An X in a column indicates that no feasible solution was found. As before, the value reported for IP Search is the value of the solution produced within 15 minutes. We observe that in every instance IP Search finds a better solution in 15 minutes than CPLEX does in 12 hours. We also see that the improvement over the solution found by CPLEX in 15 minutes is significant, often greater than 30%. Even the improvement over the solution found by CPLEX in 12 hours is impressive, often greater than 20%. Unfortunately, little can be said with confidence regarding the true optimality gap of the solutions produced by IP Search since the dual bounds produced by CPLEX change very little over the course of the execution and are likely to be weak. In fact, for many of the loosely capacitated instances, CPLEX did not find a significantly better primal solution in 12 hours than it did in 15 minutes. This highlights the difficulty that an LP-based branch-and-bound algorithm can have in finding good primal solutions when the dual bounds are weak. We have also observed that IP search outperforms CPLEX in very little time. In fact, for the GT instances, IP search needed on average less than 1 minute to produce a solution that is better than the best solution produced by CPLEX in 15 minutes and less than 3 minutes to produce a solution that is better than the best solution produced by CPLEX in 12 hours. The time limit of 15 minutes on IP Search is, of course, self-imposed. Table 3 reports the results for IP Search after 15, 30, and 60 minutes as well as the percentage improvement over the initial solution found. We see that the greatest percentage improvement occurs in the first 15 minutes, but that IP Search continues to find improving solutions when given more time. The column labeled Times Phase II gives the number of times Phase II of the scheme for finding an initial feasible solution is executed. Not surprisingly, Phase II is only 30

40 Table 2: Primal-side Comparison with CPLEX CPLEX-15M CPLEX-12H IP Search CPLEX-15M CPLEX-12H Opt Problem Gap Gap Gap 500,2000,50,F,L 5,301,081 5,301,081 3,910, ,2000,50,F,T X 7,927,065 5,249,040 N/A ,2000,100,F,L 8,944,724 8,299,799 6,764, ,2000,100,F,T 10,199,000 8,306,181 7,718, ,2000,150,F,L 10,996,000 10,080,000 8,618, ,2000,150,F,T 12,115,000 10,770,000 9,448, ,2000,200,F,L 13,808,000 12,824,000 10,333, ,2000,200,F,T X X 12,425,600 N/A N/A ,2500,50,F,L 4,611,275 4,611,275 3,841, ,2500,50,F,T 5,779,926 5,084,529 4,666, ,2500,100,F,L 9,351,042 9,251,042 6,875, ,2500,100,F,T 9,724,997 7,995,284 7,235, ,2500,150,F,L 13,660,000 12,497,000 9,730, ,2500,150,F,T 11,385,000 10,683,000 7,934, ,2500,200,F,L 15,539,000 13,468,000 11,261, ,2500,200,F,T 18,906,000 14,948,000 12,825, ,3000,50,F,L 5,098,318 5,098,318 3,596, ,3000,50,F,T 5,615,096 4,866,768 4,504, ,3000,100,F,L 8,721,798 8,721,798 6,577, ,3000,100,F,T 10,119,000 8,330,109 7,517, ,3000,150,F,L 12,628,000 12,623,000 9,214, ,3000,150,F,T 12,615,000 10,147,000 9,186, ,3000,200,F,L 15,039,000 13,441,000 10,853, ,3000,200,F,T 17,883,000 13,674,000 11,578, Average executed for tightly-capacitated instances. In fact, for the majority of instances, Phase II is not needed at all. 31

41 32 Table 3: IP Search Given More Time Times Init. Feas. 15 Minute 30 Minute 60 Minute Imp. After Imp. After Imp. After Problem Phase II Soln Soln Soln Soln 15 M 30 M 60M 500,2000,50,F,L 0 6,175,840 3,910,120 3,907,440 3,823, ,2000,50,F,T 5 6,314,980 5,249,040 5,112,490 4,949, ,2000,100,F,L 0 10,244,000 6,764,310 6,655,370 6,453, ,2000,100,F,T 4 10,125,700 7,718,750 7,632,220 7,619, ,2000,150,F,L 0 13,317,800 8,618,060 8,279,270 8,081, ,2000,150,F,T 0 14,557,800 9,448,890 9,250,760 8,807, ,2000,200,F,L 0 17,411,800 10,333,200 10,138,900 9,828, ,2000,200,F,T 0 17,543,000 12,425,600 12,117,800 11,893, ,2500,50,F,L 0 5,712,380 3,841,350 3,780,160 3,612, ,2500,50,F,T 1 5,898,160 4,666,740 4,661,160 4,600, ,2500,100,F,L 0 10,640,700 6,875,420 6,621,140 6,400, ,2500,100,F,T 1 10,077,400 7,235,520 7,065,020 6,953, ,2500,150,F,L 0 14,834,200 9,730,100 9,475,760 9,089, ,2500,150,F,T 0 13,365,000 7,934,360 7,673,410 7,571, ,2500,200,F,L 0 17,446,900 11,261,300 10,623,700 10,099, ,2500,200,F,T 1 18,628,100 12,825,300 11,843,200 11,452, ,3000,50,F,L 0 5,687,990 3,596,980 3,519,400 3,457, ,3000,50,F,T 0 5,553,000 4,504,260 4,405,120 4,262, ,3000,100,F,L 0 10,383,600 6,577,980 6,202,380 6,015, ,3000,100,F,T 2 10,350,000 7,517,970 7,444,950 7,186, ,3000,150,F,L 0 13,707,300 9,214,960 9,126,970 8,919, ,3000,150,F,T 0 12,958,300 9,186,840 8,898,880 8,709, ,3000,200,F,L 0 16,869,900 10,853,400 10,419,400 10,040, ,3000,200,F,T 1 17,071,600 11,578,000 10,750,200 10,390, Average

42 We observed in Table 2 that IP Search significantly outperforms CPLEX on large instances when each commodity has to be routed along a single path. However the dual bounds produced by CPLEX were too weak to get a sense of the real quality of the solutions produced by IP Search. Next we investigate what happens when we have smaller instances and require a commodity to be routed along a single path. The results are given in Table 4. As before, IP Search is limited to 15 minutes. When CPLEX obtains a better solution than IP Search, we report how long it took CPLEX to find the first solution that is better than the solution found by IP Search (CPLEX Time Beat). We observe that in only 2 of the 37 instances does CPLEX find a better solution in 15 minutes than IP Search. In those 2 instances the difference in solution value between IP Search and CPLEX is less than.15%. On the other hand, IP Search often finds solutions that are 5% better than what CPLEX finds in 15 minutes. The optimality gap information reveals that the solutions produced by IP Search are of high quality, on average within 4% of optimality. When CPLEX does find a better solution, it often requires more than 3 hours to do so. We also see that IP Search produces solutions within 15 minutes that are competitive with what CPLEX produces in 12 hours. 33

43 34 Table 4: Primal Side Comparison with CPLEX - Metaheuristic Instances - Single Path Problem CPLEX-15M CPLEX-12H IP Search CPLEX-15M CPLEX-12H Opt CPLEX Gap Gap Gap Time Beat 100,400,10,V,L 31,785 31,785 31, ,400,10,F,L 24,104 24,104 24, ,400,10,F,T 328, , , ,400,30,V,T 750, , , ,400,30,F,L 55,665 50,391 52, , ,400,30,F,T 188, , , ,230,40,V,L 423, , , ,230,40,V,T 398, , , ,230,40,F,T 668, , , ,230,200,V,L 102,118 95,126 96, ,595 20,230,200,F,L 160, , , ,230,200,V,T 107, , ,469 20,230,200,F,T 157, , ,300,40,V,L 430, , , ,300,40,F,L 597, , , ,300,40,V,T 501, , , ,300,40,F,T 643, , , ,300,200,V,L 81, , ,300,200,F,L 128, , ,300,200,V,T 78, , ,403 20,300,200,F,T 120, , ,911 30,520,100,V,L 54,394 54,387 54, ,520,100,F,L 103, , ,520,100,V,T 53,812 53,812 53, ,520,100,F,T 102, , ,677 30,520,400,V,L 132, , ,520,400,F,L 164, , ,520,400,V,T 132, , ,520,400,F,T 166, , ,897 30,700,100,V,L 47,883 47,883 47, ,700,100,F,L 63, , ,109 30,700,100,V,T 47,864 47,670 47, ,236 30,700,100,F,T 57, , ,019 30,700,400,V,L 103, , ,599 30,700,400,F,L 677, , ,700,400,V,T 100, , ,700,400,F,T 142, , ,605 Average ,559.07

44 Value of Subset Solvers Given the success of IP search it is natural to ask which of its components are responsible. Since IP search utilizes two different neighborhoods, one defined by a subset of arcs and the other by a subset of commodities, the first question is what if we only used one of them? To answer this question we ran IP search on the GT instances two more times; the first time only using arc subset IPs and the second time only using commodity subset IPs. The percentage difference, or gap, information for only using subset type X is calculated as 100 X Arc & Commodity. We observed that on average, only using arc subset IPs lead to a Arc & Commodity gap of 6.67% whereas only using commodity subset IPs lead to a gap of 2.73%. Hence, while using both types of neighborhoods typically yields the best solution, each neighborhood is effective by itself. To gain a greater understanding of the efficacy of each selection scheme, we examine in detail the contribution of each on finding improving solutions. Table 5 gives the breakdown by scheme, averaged over the GT instances, after running IP Search for 60 minutes. We report the percentage of iterations a scheme was executed (% Called) and the percentage improvement that can be attributed to a scheme (% Improvement). The schemes are denoted by A-x for arc subset selection scheme x and C-x for commodity subset selection scheme x. We observe that the vast majority of solution improvement is due to the commodity Table 5: Selection Method Contribution Scheme % Called % Total Improvement A A A C C C C C C subset IPs. This may simply be due to the biased random selection mechanism repeatedly selecting the commodity subset solvers because of their effectiveness. Commodity selection scheme 5 results in the largest percentage improvement. Recall that this is the scheme that selects commodities for which few paths have been discovered and was designed to aid in 35

45 diversification for the arc subset solvers. These results indicate the scheme is effective in finding improving solutions itself. Recall we used a biased random selection method to determine which solver to execute at an iteration. To further evaluate the value of the biased random selection, we compare it to a simple round-robin approach for selecting solvers. Our simple round-robin scheme executes the solvers in the following order: A-1, C-1, C-2, A-2, C-3, C-4, A-3, C-5, and C-6. We executed IP search using the round-robin scheme on the GT instances. Using a gap value calculated as 100 Round Robin Biased Random, we observed that the average Round Robin gap is 2.46%, indicating that biased random selection does better than round-robin, but not significantly better Lower Bound The next question we address is the strength of the lower bound produced by IP Search. Since we want to evaluate the value of the lifted cover inequalities, we only consider the single-path variant of MCFCNF. For the GT instances we compared the value of the optimal solution to the LP-relaxation (LPR), the value of the root node bound (Root Node) produced by CPLEX,the lower bound IP Search produces and the percentage differences between the bound IP Search produces and the other two bounds (LP Gap and Root Gap). The percentage differences are calculated as 100IP Search LPR and IP Search 100Root Node IP Search. Note that this means a positive percentage difference with the Root Node value of the LP Relaxation indicates that IP Search has given a stronger lower bound and a positive percentage difference with the value of the root node LP indicates that CPLEX has a stronger root node bound. Since the disaggregate coupling constraints are added as user cuts, they are not present in the LP relaxation, but are reflected in the root node bound in addition to any other cuts generated by CPLEX. We observed that on average, LP Gap is 25% and hence IP Search produces lower bounds which are much stronger than the value of the LP Relaxation. Furthermore, we have observed that the difference is larger for loosely capacitated instances. On the other hand, we observed that Root Gap is 23%. Hence, we see that by adding cuts, CPLEX 36

46 produces a root node bound which is stronger than the bound produced by IP Search. Since both disaggregate coupling constraints and lifted cover inequalities are used to strengthen the dual bound IP Search produces, we next analyzed their respective contributions. For the GT instances, we compare LPR, the bound produced by IP Search when only considering the disaggregated coupling constraints, Disaggr, and the bound produced by IP Search when considering the disaggregated coupling constraints and the lifted cover inequalities, Disaggr + Cover. We then calculated gaps similar to the LP Gap described above. We observed that the Disaggr Gap is 21%, indicating that by only adding the disaggregated coupling constraints we are able to derive a much stronger lower bound than LPR. Furthermore, we observed that the Disaggr +Cover Gap is 25% indicating that the lifting of inequalities found during the solution of subset IPs does contribute to strengthening the lower bound. We also observed that with a threshold value T =.25 we typically only add 1% of the possible disaggregate coupling constraints. In addition, we observed that the lifted cover inequalities are most effective for tightly-capacitated instances. While we are pleased by the success of our approach and in particular its ability to handle both large instances of MCFCNF and different problem variants, we also recognize that it is heavily dependent on the class of problem studied. Although the concept of using information from both the compact and extended formulation of a problem may be re-used in the design of heuristics for other problems, many of our schemes for choosing variable subsets are derived from our understanding of the structure of good solutions to instances of the MCFCNF. Thus we next try and understand to what extent we can develop schemes for choosing variable subsets that are not problem specific as well as whether we should limit ourselves to integer programs created by fixing variables. 37

47 CHAPTER III MODELING IP-BASED SEARCH When integer programming (IP) models are used in operational situations there is a need to consider the tradeoff between the conflicting goals of solution quality and solution time, since for many problems solving realistic-size instances to a tight tolerance is still beyond the capability of state-of-the-art solvers. However, by appropriately defining small instances, good primal solutions frequently can be found quickly. This approach is taken, for example, within LP-based branch-and-bound algorithms using techniques such as local branching ([14]) and RINS ([12]). These techniques use information from the LP solution and incumbent solution to define a small IP, which is then optimized. These techniques can be applied to any integer program and are available in commercial solvers such as CPLEX. Another approach to defining small instances is to use problem structure as in Chapter 2 where small IPs are chosen according to the attributes of previous solutions, such as the arcs used by various commodities. Combining exact and heuristic search techniques by solving small IPs has received quite a lot of attention recently, see, for example, [13], [36], [3], and [35]. Still another heuristic approach is to use structure to define neighborhoods that can be searched in polynomial time such as the very large scale neighborhood search approach of [2]. A key difference between the methods that are embedded in an LP-based tree search algorithm (local branching and RINS) and the others is that they are connected with a dual bounding procedure so that optimality or weaker tolerance gaps can be proved. In this chapter we introduce a new approach to finding good solutions quickly that is capable of proving optimality as well. It is different from techniques such as local branching and RINS since it uses problem structure to define the small IPs to be solved. It is different 38

48 from other IP-based local search methods since the IPs to be solved are determined by a column generation scheme. The embedding of this column generation scheme into a branch-and-price algorithm gives the dual bounds that provide the capability of proving optimality. Our extended formulation, which requires column generation, is very different from typical column generation formulations that employ structurally different objects from the compact formulation, for example, paths rather than arcs. Our extended formulation keeps the original variables from the compact formulation and augments them with an exponential number of additional variables that are used to define problem restrictions to obtain small IPs. By preserving the original compact formulation, we are able to enrich it by preprocessing, cutting planes or any other techniques normally used in a branch-and-cut framework. The computational results demonstrate that the approach achieves its goals. For the instances used in the computational study, the approach often produces in 15 minutes a proven near-optimal primal solution. More specifically, the primal solution found in 15 minutes is often better than the one CPLEX produces in 6 hours, and the dual bound is usually close to the one CPLEX produces in 6 hours. The remainder of this chapter is organized as follows. In Section 3.1 we present our extended formulation which models the search for optimal solutions to an IP. In Section 3.2 we present a parallelized branch-and-price approach for solving the model. In Section 3.3 we discuss how the approach is applied to the MCFCNF and in Section 3.4 we present the computational results of applying the approach to MCFCNF. 3.1 Modeling the Search for Optimal Solutions For convenience, we present a model for binary mixed integer programs, but it can be used for general integer programs as well. Consider problem P given by max cx + dy s.t. Ax + By = b (11) x real, y binary. 39

49 Let V P denote the optimal value of P, S P = {(x,y) Ax + By = b,x real,y binary} be the set of feasible solutions to P, LP denote the linear relaxation of P and V LP its optimal value. For a given integer matrix N and a given integer vector q, both of appropriate dimension, we define restriction P N (q) of P as: max cx + dy s.t. Ax + By = b Ny q (12) x real, y binary with optimal value VN (q). We suppose that this restriction can be solved much faster than P. If q is a binary vector and N is a matrix with components n ij {0, ±1} then the constraints Ny q will be of the form of fixing components of y or simple clique constraints or bounding constraints like y i y j, all of which speed up a branch-and-bound algorithm. We define R = {r r = Ny for some (x,y) S P } to be the set of vectors associated with feasible solutions to problem P and for this chapter assume that N is chosen such that R contains only binary vectors. Clearly, we have V P V N (r) r R and V P = V N (r ) with r = Ny P for an optimal solution (x P,y P ) to P. Thus, a strategy for finding an optimal solution to P is searching over the set R and solving restrictions P N (q), which will be feasible if there is an r R such that q r. A major advantage of such a strategy is that it produces a feasible solution to P each time such a restriction P N (q) is solved. If we restrict ourselves to solving restrictions P N (r) for vectors r R, then we can strengthen the restriction P N (r) by replacing the constraints Ny r with Ny = r. However, we will see in Section 3.2 that it can be beneficial to consider vectors q that are not necessarily in R and hence we use the formulation P N (q) given by (12). Ideally, we would only solve restrictions P N (q) whose optimal value VN (q) is close to the optimal value VP. Consequently, we would need an oracle that considers all vectors r R, but returns only those with VN (r) V P. The role of this oracle is thus similar to the role of the pricing problem in column generation: consider all columns, but return only columns 40

50 with positive reduced costs (for a maximization problem). Therefore, we next assume that we know the entire set R and build a model that extends the formulation of P to both choose a vector r from R and solve the resulting restriction P N (r). Specifically, we define the problem MP: max cx + dy s.t. Ax + By = b Ny Rz 0 (13) 1z = 1 x real, y, z binary, where the binary variables z in MP represent the choice of vector r for which the restriction P N (r) should be solved. Let V MP denote the optimal value of MP, MLP its linear relaxation and V MLP its optimal value. In addition, by replacing Ny Rz 0 with Ny Rz = 0, we obtain problem MP = with linear relaxation MLP = and optimal linear relaxation value VMLP =. We first observe that MP and MP = are equivalent reformulations of P and that their linear relaxations are no weaker than LP. Proposition V MP = = V MP = V P and V MLP = V MLP V LP. Proof As MP contains the constraint set Ny Rz as well as the variables and constraints defining P we must have VMP V P. Since MP = is defined with Ny = Rz instead of Ny Rz we have VMP = VMP. Similar reasoning yields V MLP = VMLP V LP. However, for an optimal solution (x P,y P ) to P and r = NyP we know that r R given the definition of R. Thus (x P,y P,e r ) is a feasible solution with objective function value V P for both MP and MP = and we have V MP = = V MP = V P. Although the primary purpose of MP and MP = is to aid the search for high-quality primal solutions, we next see that MLP may in fact provide a tighter bound on V P than LP, i.e. 41

51 we can have VLP > V MLP. Consider the following integer program: VP = maximize 10y 3 + y 4 s.t. y 1 + y 2 = 1 10y 1 10y 3 10y 2 9y 4 y 1,y 2,y 3,y 4 {0,1} and the extended formulation M P associated with the restriction created by adding to P the constraints y 3 r 3,y 4 r 4. The only feasible solution to P is y 1 = y 3 = 1,y 2 = y 4 = 0. Therefore, we have VP = 10, the only element of R is r with r 3 = 1, r 4 = 0 and every (including the optimal) solution to MLP must have z r = 1. Thus, the constraint set Ny Rz 0 reduces to y 3 r 3 = 1, which is redundant, and y 4 r 4 = 0, which fixes the variable y 4 = 0. As a result, the optimal solution to MLP is integral, namely it is y 1 = y 3 = 1 and we have V MLP = 10. On the other hand, the solution to LP is y 2 =.9,y 4 = 1,y 1 = y 3 =.1 and V LP = = 2. Finally, we see that defining MP = with Ny = Rz instead of Ny Rz may yield an even tighter linear relaxation, i.e. we can have VMLP > V MLP,=. Consider the following integer program: V P = maximize 10y 2 s.t. y 1 = 1 y 1 10y 2 y 1,y 2 {0,1}. and the extended formulations MP (MP = ) associated with the restriction created by adding to P the constraints y 2 r 2 (y 2 = r 2 ). Again, P has a single feasible solution, namely y 1 = y 2 = 1 with V P = 10 and R consists of a single vector r, with r 2 = 1. Therefore, every (including the optimal) solution to MLP and MLP = must have z r = 1. For MLP, the constraint set Ny Rz 0 reduces to y 2 r a1 = 1 which is redundant and we have V MLP, = V LP = 10.1 since the optimal solution to LP is y 1 = 1,y 2 =.1. For MLP =, the constraint set Ny Rz = 0 reduces to y 2 = r a1 = 1 which is not redundant and induces the optimal solution to P. 42

52 Since instances of both MLP and MLP = have to be solved by column generation, the basis of our approach for solving instances of MP and MP = is a branch-and-price algorithm. We discuss the branch-and-price algorithm and procedures for speeding up the search for high-quality solutions in the next section. Although we can simply add steps to the branchand-price scheme to perform these procedures, we leverage the fact that solving P N (q) for q r for some r R can be done independently from the execution of the branch-andprie scheme and develop a more powerful framework specifically geared towards parallel computer architectures. 3.2 Solving Instance of MP, MP = We have presented two reformulations of P, MP and MP =, each of which are candidates for a branch-and-price algorithm. While MLP = may provide a tighter bound on VP than MLP it may be a more difficult linear program to solve. For example, we will see that when solving MLP for MCFCNF, the non-negative dual variables associated with Ny Rz 0 yield a relatively easy pricing problem, whereas when solving MLP = for MCFCNF, the unconstrained dual variables associated with N y Rz = 0 yield a relatively hard pricing problem. Thus we restrict the presentation of our approach to MP, although all the steps can be easily adapted to MP =. A branch-and-price algorithm consists of the following components: a restricted master problem, a pricing problem, and a branching scheme. We next illustrate how we define each of these components. Let R R be a set of known restrictions of P. The restricted master problem RMP is defined as: max cx + dy s.t. Ax + By = b Ny Rz 0 (π) (14) 1z = 1 (α) x real, y, z binary. with dual variables π 0 and α unrestricted. Let V RMP denote the optimal value of RMP, RMLP the linear relaxation of RMP and V RMLP its optimal value. 43

53 To ensure that we solve MP and in turn find an optimal solution (x P,y P ) to P, we need a mechanism for generating columns to add to R which will eventually produce r = Ny P. By searching for a column r R \ R with positive reduced cost, i.e. with πr α > 0, the pricing problem G(π, α) associated with RM LP will do exactly that. Specifically, G(π, α) is: max s.t. πr α r R r binary. (15) Observe that since our pricing problem searches for variables that are not in the formulation of P, we can strengthen RMLP with inequalities fx + gy γ that are valid for P without affecting the structure of the pricing problem. This column generation process also yields a dual bound. Specifically, if r represents the optimal solution to the pricing problem and u πr α, then we have V RMLP + u V MLP V MP = V P. The last component to define is a branching scheme to handle the situation where the column generation process has solved MLP but the optimal solution (x RMLP,y RMLP,z RMLP ) to RMLP is fractional. We consider two cases: 1. zrmlp is binary but (x RMLP,y RMLP ) is fractional. For this case we create the branches z r = 0 and z r = 1 for the vector r for which z r,rmlp = 1. However, we immediately evaluate and prune the branch with z r = 1 by solving P N (r). Note that branching on the z variable is equivalent to removing columns from R so that if we have already solved P N (r 1 ) we can add constraints to (15) to exclude r 1. In fact, since we know that r 2 r 1 VN (r 2) VN (r 1), we add constraints to exclude both r 1 and elements from R whose restrictions cannot yield a better solution than the optimal solution to P N (r 1 ). In particular, we define R D = {r R P N (r) already solved } and for each r i R D we add to (15) the constraint j:rj i=0 r j 1 since we are only interested in vectors r that contain an element j r i. 2. z RMLP is fractional. For this case, we choose a row i with 0 < ( Rz RMLP ) i < 1 and enforce (Rz) i 0 on one branch and (Rz) i 1 on the other. Note that we can satisfy (Rz) i 0 ( 1) in the pricing problem simply by setting r i = 0 (= 1). 44

54 Algorithm 7 Branch-and-price (BP) set R =, R D = while MP has not been solved do select an unevaluated node while RMLP has not been solved do solve RMLP only considering columns of R \ R D valid for this node solve the pricing problem to find an improving vector r to add to R end while branching if necessary end while We summarize the branch-and-price scheme (BP) in Algorithm 7. This branch-and-price scheme will, when given sufficient time, produce an optimal solution. However, its most important feature is that it automatically generates restrictions r R for which P N (r) may produce a high-quality solution to P. We next present the Primal Solution Construction (PC) algorithm defined in Algorithm 8 and the Primal Solution Improvement (PI) local search heuristic defined in Algorithm 9 to show how this feature can be exploited effectively to generate high-quality feasible solutions. Given a set of restriction vectors R PC, PC produces feasible solutions by choosing an r R PC and solving P N (r). When PC is given the value v BEST of the current best feasible solution, we strengthen P N (r) by adding the constraint cx+dy v BEST to reflect that only primal solutions with objective function value higher than the current best are of interest. Algorithm 8 Primal Solution Construction (PC) Require: RPC,RD PC,(x RMLP,y RMLP,z RMLP ) and best-known primal solution value v BEST if exists if r R PC \ RD PC such that z r,rmlp > 0 then set r to such an r with the highest value of zr,rmlp else if R PC \ RD PC then set r to a randomly chosen vector from R PC \ RD PC else convert r = Rz RMLP to an integral vector and set r = r end if Solve P N (r) with the constraint cx + dy v BEST and add r to RD PC return RD PC and new best solution if found We can also create an r such that solving P N ( r) represents searching a neighborhood of the current best solution (x BEST P,yP BEST ). Observe that r r BEST = NyP BEST implies 45

55 VN ( r) V N (rbest ). This suggests solving P N ( r) with an r > r BEST. The PI algorithm does so, and uses a variety of schemes for constructing r. We choose which scheme to use in each call to PI with a method similar to the biased roulette wheel approach used in Chapter 2. Scheme augment-best: To obtain a r > r BEST, we start from r = r BEST and then choose a subset of indices i with r BEST i = 0 and set r i = 1. We use two metrics, both based on the solution to RMLP, to choose the indices i for which we set r i = 1. The first metric is the dual variable π i associated with (Ny Rz 0) i and the second is the value ( R PI z RMLP ) i. The higher these values are, the more likely it is that we choose index i. We refer to solutions created in this way as belonging to the augment-best neighborhood. The next two schemes use the concept of path-relinking, i.e. by combining the structural information from two good solutions we may produce a restriction that yields an even better solution. Scheme best-relink-other-r: The first path-relinking mechanism uses vectors generated by the column generation process. Specifically, we choose a random r R PI and set r i = max(r i,ri BEST ). We refer to this neighborhood as the best-relink-other-r neighborhood. Scheme best-relink-other-solution: The next path-relinking mechanism uses vectors induced by other solutions. Specifically, we set r i = max(r i,ri BEST ) where r = NyP 1 for a solution (x 1 P,y1 P ) that was found either earlier in the execution of the current process or potentially by another process. We refer to this neighborhood as the best-relink-other-solution neighborhood. With these local search ideas care has to be taken that we do not create a vector r with too many indices i with r i = 1, as the resulting restriction may be too loosely constrained to be solved quickly. In our current implementation we ensure that all r satisfy r 1 γ where γ is an algorithm parameter. Algorithm 9 Primal Solution Improvement (PI) Require: RPI,(x RMLP,y RMLP,z RMLP ),π,(xbest P Choose scheme for creating r Solve P N ( r) return new solution if found,y BEST P ) 46

56 As noted, instead of inserting calls to PC and PI into the BP algorithm, we execute them in parallel and define Branch-and-Price Guided Search (BPGS) as the parallel execution of BP (Algorithm 7), PC (Algorithm 8) and PI (Algorithm 9). By executing PC in parallel to BP we can solve P N (r) for r s created by the column generation process and price out new r s at the same time. We present in Algorithms 10,11 and 12 the parallelized versions of BP (PBP),PC (PPC) and PI (PPI). We note that while we include steps in these algorithms for broadcasting data to and checking for new data from other processes, in our current implementation these steps do not necessarily occur at every iteration Algorithm 10 Parallelized Branch-and-price (PBP) set R BP =,RD BP = while MP has not been solved do select an unevaluated node while RMLP has not been solved do solve RMLP only considering columns of R BP \ RD BP valid for this node solve the pricing problem to find an improving vector r to add to R BP Broadcast R BP \ RD BP,(x RMLP,y RMLP,z RMLP ) to PPC and PPI processes Broadcast π to PPI processes Check for set RD PC from PPC process and set RBP D = RBP D RPC D if found Check for new solution from PPC,PPI processes end while branching if necessary end while Algorithm 11 Parallelized Primal Solution Construction (PPC) R PC =,RD PC = while Not told to stop do check for new R BP,(x RMLP,y RMLP,z RMLP ) from PBP process and set R PC = R PC R BP \ RD PC if new R BP found check for new best solution value v BEST from all processes if exists call PC with R PC,RD PC,(x RMLP,y RMLP,z RMLP ),vbest broadcast RD PC to PBP process and new solution if found to all processes end while A natural parallelization scheme is to dedicate a single process to the execution of PBP, m 1 processes to the execution of PPC and m 2 processes to the execution of PPI. The key information that needs to be passed between the processes is the set of restrictions R, those which have been solved, RD, and the current best solution (x BEST P,yP BEST ), which can be done by passing r BEST = NyP BEST. In the application discussed below, passing the set of 47

57 Algorithm 12 Parallelized Primal Solution Improvement (PPI) R PI = while Not told to stop do check for new R BP,(x RMLP,y RMLP,z RMLP ),π from PBP process, set R PI = R BP if found check for new best solution (x BEST,y BEST ) from all processes call PI with R PI,(x RMLP,y RMLP,z RMLP broadcast new solution if found end while P P ),π and (xbest P,y BEST P ) restrictions R is easily manageable and does not lead to communication delays because the set does not get too large during the execution of the algorithm and a restriction, which is defined by a 0-1 vector, can be encoded compactly. In addition, while conceptually the sets R D are passed between processes, it is unnecessary to pass the actual r s. Specifically, PBP assigns a unique identifier to each r R BP which is also shared with other processes. This allows PPC to pass just the identifiers of the elements in RD PC. Since PPI generates it s own r which are unknown to PBP we chose to not pass them to avoid potential communication delays. We summarize the passing of data between processes in Figure 1. To save space Figure 1: Data sharing between processes we omit superscripting the sets R,R D with their process owner. Although not depicted in the figure, we also pass solutions between PPI processes to facilitate path-relinking in the best-relink-other-solution neighborhood. Finally, we note that if a PPC process runs out of 48

58 restrictions to solve, then instead of idling until it receives a new R BP it can temporarily change its mode to that of a PPI process. 3.3 Multi-Commodity Fixed-Charge Network Flow We next discuss how BPGS can be used to solve instances of the single-path variant of MCFCNF, the formulation of which is presented in Section 2.2. In keeping with the approach presented in Sections 3.1 and 3.2, we use the equivalent objective function max c ij (d k x k ij) + f ij y ij k K (i,j) A (i,j) A The BPGS approach presented in Section 3.2 has many generic tasks. In particular, once we have chosen a matrix N to dictate the structure of the restriction P N (r), the algorithms PC and PI require no further specialization. To apply the approach to MCFCNF, we chose the matrix N = I to indicate how the approach can generalize variable fixing heuristics. Hence, P N (r) consists of the extra constraints y ij r ij (i,j) A. Thus, we are in essence searching the space of feasible network designs for the optimal solution to P. Given the dual variables π ij, the pricing problem is max π ij r ij α (i,j) A subject to x k ij x k ji = δi k i N, k K, (16) j:(i,j) A j:(j,i) A d k x k ij u ij r ij (i,j) A, (17) k K r ij {0,1} (i,j) A, (18) x k ij {0,1} k K, (i,j) A. (19) Note that while the feasible region of the pricing problem is the same as that of the original problem, the objective function is significantly different and makes the problem easier to solve than the original. Since it is a maximization problem and the dual variables 49

59 π are nonnegative, the objective function encourages the variables r ij to be set to 1 even in solutions to the linear programming relaxation (LPR), thus hopefully avoiding highly fractional solutions. However, there is also an optimal solution with r ij = 1 (i,j) A. This is not desirable since the restricted problem associated with this r will be too loosely constrained to be solvable. To eliminate this problem we use the structure of an optimal solution to MCFCNF. In particular, given our assumption that f ij 0 (i,j) A, we can assume that in an optimal solution an arc is installed only if it is used by a commodity. Thus we add the inequalities r ij x k ij (i,j) A (20) k K to the pricing problem. We also observe that c ij 0 (i,j) A implies that the route a commodity takes from its source node to its sink node will not contain any cycles. Thus, to reduce circuity in the routes chosen for commodities by the pricing problem, we add the constraints x k ij 1 j N, k K, (21) j:(i,j) A i:(i,src(k)) A and j:(sink(k),j) A x k ij x k ij 0 k K, (22) 0 k K. (23) The first set ensures that a commodity does not leave a node multiple times, whereas the next two sets ensure that a commodity never enters (leaves) its source (sink). Finally, we have chosen to try to strengthen the bound V MLP through the use of wellknown valid inequalities for MCFCNF. Specifically, we augmented BP to a branch-priceand-cut algorithm, adding steps that dynamically add the disaggregated constraints (6) and the cover inequalities k C xk ij C 1 to RMLP where a cover with respect to an arc (i,j) is a set C K such that k C dk > u ij. 3.4 Computational Results There are many questions to answer regarding the performance of BPGS. Since it is an exact algorithm, we chose to benchmark it against a commercial MIP solver: CPLEX

60 This allows us to assess the capability of BPGS to produce high quality primal solutions, tight dual bounds, and proofs of optimality. BPGS has been implemented in C++ and uses CPLEX as its MIP/LP solver. All experiments were performed on a machine with 8 Intel Xeon CPUs running at 2.66 GHz and with 32 GB RAM. The basic parallelization consisted of 1 PBP process, 1 PPC process and 2 PPI processes. Message passing between processes was implemented via a combination of MPI ([17]) and text files. The use of 4 processors seems reasonable as many high-end desktops come with 4 processors. Finally, for all experiments we allowed our code to run for 15 minutes and all restrictions P N (r) were solved with an optimality tolerance set to 1% and a time limit of 5 minutes. In our first set of experiments, we ran CPLEX for 6 hours. Since we want to assess both primal- and dual-side performance of BPGS, CPLEX was run with MIPEmphasis set to balanced, which means that CPLEX uses tactics intended to find a proven optimal solution quickly ([20]). The disaggregate constraints x k ij y ij are implemented as CPLEX user cuts, for the CPLEX experiments and when branch-and-price guided search solves restrictions P N (r). Unless otherwise noted, computation times are reported in seconds. Of the three sets of MCFCNF instances presented in [16] and identified as C, C+ and R, we chose to use the instances in C due to their diversity in size and difficulty. The instances have been used to benchmark the performance of meta-heuristics designed for the MCFCNF variant in which a commodity may be split across multiple paths (0 x k ij 1). However, the instances remain feasible when splitting is not allowed. The instances are classified as follows: F denotes instances where the ratio of fixed to variable cost for an arc is high and V otherwise, T denotes instances that are tightly capacitated relative to total demand and L otherwise. Finally, the naming scheme is #nodes #arcs #commodities (F V ) (T L). Unfortunately, we do not have access to the parallel version of CPLEX However, the results clearly show that BPGS produces near-optimal solutions quickly. Furthermore, the frequency with which BPGS produces a better solution in 15 minutes than CPLEX in 3 hours suggests that it is likely to be competitive with a parallel version of CPLEX. 51

61 3.4.1 Primal Side Table 6 shows the results for the 31 instances considered. (Note that we have used the minimization form of MCFCNF.) More specifically, we give the value of the best primal solution found by CPLEX in 6 hours (CPLEX Primal) and the time taken by CPLEX to find that solution (CPLEX TTB), the value of the best primal solution found by BPGS in 15 minutes (BPGS Primal) and the time taken by BPGS to find that solution (BPGS TTB), a comparison of the quality of primal solutions produced by BPGS and CPLEX in the form of 100 (BPGS Primal CPLEX Primal)/(BPGS Primal), in case CPLEX finds a better solution than BPGS, the time required to find the first better solution (Time to Beat), the value of the dual bound found by CPLEX (CPLEX Dual) and the resulting optimality gap 100 (CPLEX Primal CPLEX Dual)/(CPLEX Primal) (CPLEX Gap), the value of the dual bound found by BPGS (BPGS Dual) and the resulting optimality gap 100 (BPGS Primal BPGS Dual)/(BPGS Primal) (BPGS Gap), the quality of the solution found by BPGS measured against the best dual bound 100 (BPGS Primal max(cplex Dual, BPGS Dual))/(BPGS Primal) (BPGS Best Gap), a comparison of the strength of the dual bounds produced by the BPGS and CPLEX in the form of 100 (CPLEX Dual BPGS Dual)/(CPLEX Dual) (Dual Gap), the time CPLEX needed to prove the optimality of a solution when it was able to do so (Time to Opt CPLEX), and the time BPGS needed to prove the optimality of a solution when it was able to do so (Time to Opt BPGS). 52

62 We see that BPGS produces high-quality solutions quickly since it produces primal solutions in 15 minutes that are on average within 2.16% of optimality (only 0.02% worse than CPLEX). We observe that CPLEX also performs quite well, producing primal solutions in 6 hours that are on average within 2.14% of optimality. Furthermore, for 22 of the 31 instances BPGS produces a better primal solution. However, we note that we are counting instances where CPLEX quickly produces a solution whose value is within 1% of the optimal value, yet is worse than the value of the solution produced by BPGS (e.g. instance F-T). For these instances, CPLEX may have found an even better solution if given a smaller optimality tolerance. Therefore, we next focus on the 21 instances (presented in boldface) for which CPLEX produced a solution within 1% of the optimal value that is better than the solution produced by BPGS or did not find a provably optimal solution at all. For 10 of these 21 instances BPGS produces a better solution in 15 minutes than CPLEX produces in 6 hours. For 10 of the remaining 11 instances, CPLEX requires nearly 3 hours (on average) to produce a solution that is better than what BPGS produces in 15 minutes (and those solutions are only 0.53% better on average). Furthermore, we observe that for three of the instances, namely F-L, V-T, and V-L, the best solution found by BPGS was produced by the PPC process whereas the best solutions for all other instances were produced by a PPI process. Thus local improvement seems to be quite effective. As each PPI process uses three different schemes for creating restrictions that represent a neighborhood of the best-known solution, we next analyze the performance of each. The augment-best neighborhood was responsible for the majority of the solution improvement over all instances at 86% followed by the best-relink-other-r neighborhood at 11% and the best-relink-other-solution neighborhood at 3%. One reason that the best-relink-other-solution neighborhood is not that effective may be the lack of intelligence in the selection of the solution to link it with. Currently, it selects the other solution randomly without considering how similar of dissimilar it is. 53

63 54 Table 6: Primal and Dual Comparison with default CPLEX CPLEX BPGS Gap with Time CPLEX BPGS BPGS Gap Dual Time to Opt Instance Primal TTB Primal TTB CPLEX To Beat Dual Gap Dual Gap Best Dual Gap CPLEX BPGS V-L 423, , , , V-T 399, , , , F-T 669, , , , V-L 95,273 20,082 94, , , F-L 138,287 15, , , , , V-T 99,074 19,599 98, , , F-T 139,898 20, , , , V-L 433, , , , F-L 599, , , , V-T 502, , , , F-T 648, , , , V-L 76,392 12,915 76, , , F-L 119,304 21, , , , V-T 76,486 18,401 76, ,401 75, , F-T 110,417 20, , , , V-L 54, , , , F-L 96,348 7,766 96, , , V-T 53, , , , F-T 99,396 4,952 99, , , V-L 114,902 20, , , , , F-L 152,026 21, , , , , V-T 116,485 21, , , , , F-T 156,660 21, , , , , V-L 48, , , , F-L 60,384 12,488 60, ,849 59, , , V-T 47, , , , F-T 56,804 1,330 56, , , , V-L 99,087 20,643 98, , , F-L 139,839 21, , , , V-T 97,092 18,122 98, ,639 94, , F-T 132,820 20, , , , , Average , ,

64 A closer examination of the instances for which CPLEX finds a better solution than BPGS, presented in Table 7, reveals that this happens most often for instances with 400 commodities. This may indicate that our choice of N = I to define restrictions is not appropriate for instances with a large number of commodities. Even if P N (r) only considers 200 arcs, with 400 commodities the IP still has 80,000 binary variables and may be too large to solve quickly. Table 7: Results by Number Commodities in an Instance # Commodities # Instance # Instances CPLEX Avg. Time CPLEX Needed Found Better to Find Better , , ,869 The time to best information in Table 6 only provides partial insight on performance. To supplement this information, we present in Figure 2 more detailed information concerning the progression of the quality of the solution produced by BPGS over the course of its execution for the four instances with 20 nodes, 230 arcs and 200 commodities. At each point in time, we show the optimality gap for the best solution produced up to that point (as measured against the dual bound produced by CPLEX after 6 hours). We see that for each of the four instances BPGS has produced a solution that is within 5% of optimality in less than eight minutes. Thus, the time to best somewhat inflates the time required for BPGS to produce a solution of high quality. For example, for instance F-T, BPGS finds its best solution (of value 139,718) after 818 seconds. However, the previous best solution (found after 416 seconds) was of value 139,719 and both are better than what CPLEX produces in 6 hours. Given the similarities in both nature and goal of BPGS and local branching, we compare BPGS with CPLEX implementation of local branching. In Table 8, we give the value of the best solution produced by BPGS in 15 minutes and the the value of the best solution produced by CPLEX local branching in 1 hour. We focus our analysis on the difficult instances presented in boldface in Table 6. For 14 of the 21 instances BPGS finds a better solution in 15 minutes than CPLEX with local branching finds in 1 hour. Enabling local branching allows CPLEX to find a better solution than BPGS for only one new instance: 55

65 Optimality Gap 15.00% 13.50% 12.00% 10.50% 9.00% 7.50% 6.00% 4.50% 3.00% 1.50% 0% V-L F-L V-T F-T Minutes Figure 2: Primal-side Performance Over Time F-T. We conclude from these experiments that BPGS is quite competitive with the implementation of local branching in CPLEX. Next we study whether increasing the number of processes used by BPGS enables it to find good solutions even more quickly. Thus we compare our original parallelization that consisted of 4 processes broken down into 1 PBP process, 1 PPC process and 2 PPI processes with 6 processes broken down into 1 PBP process, 1 PPC process and 4 PPI processes. In Table 9 we present the results of this set of experiments. For each instance we present in boldface the best solution when there is not a tie or the lower time to best solution when there is. We note that for 21 of the instances using 6 processors either enabled the approach to find a better solution or an equivalent solution in less time than when we used 4 processors. We also note that for some instances (such as F-T or V-L) the time taken to find the best solution is decreased dramatically when we allot 6 processors to the approach Dual Side To assess the dual-side performance of BPGS, we look again at the results reported in Table 6. We observe that while the dual bound produced by BPGS in 15 minutes is worse than the bound produced by CPLEX after 6 hours it is only 7.5% worse on average. Furthermore, at the end of 15 minutes BPGS produced a primal solution, a dual bound, and a provable optimality gap that was only 9.5% on average. 56

66 Table 8: Primal Comparison with CPLEX and Local Branching CPLEX + LB BPGS Gap with Time Instance Primal TTB Primal TTB CPLEX + LB To Beat V-L 95,046 3,118 94, F-L 141,844 3, , V-T 99,508 3,600 98, F-T 139,625 2, , , V-L 76,572 3,423 76, F-L 119,611 3, , V-T 76,816 3,600 76, F-T 111,522 2, , V-L 54, , F-L 96,697 1,363 96, F-T 99,386 1,835 99, , V-L 115,109 3, , F-L 153,541 3, , V-T 116,664 2, , , F-T 161,437 3, , F-L 60,640 3,266 60, , F-T 56, , V-L 100,660 2,576 98, F-L 143,044 3, , V-T 97,713 2,788 98, , F-T 137,959 3, , Average ,463 Examining the 13 instances that CPLEX solves to optimality, we see that 8 of those instances are solved at the root node through a combination of primal heuristics and valid inequalities added to strengthen the dual bound. Our implementation of BPGS does not have such enhancements, but is still able to solve 3 of them and produce primal solutions within 1% of the dual bound produced by CPLEX for the others, which implies that BPGS is unable to prove optimality only because its own dual bound is not strong enough Comparison with IP Search Lastly we compare in Table 10 the performance of BPGS with the IP Search procedure for the MCFCNF presented in the previous chapter. We note that the comparison is not completely fair as the results for BPGS were achieved with the use of CPLEX 11.2 while the IP Search results were achieved with CPLEX 9. For each instance we present in boldface the best solution when there is not a tie or the lower time to best solution when there is. We see that for 23 instances BPGS either finds the best solution or a solution equivalent to the one found by IP Search in less time. Although the IP Search procedure was designed specifically for the MCFCNF, the solution it produced was 1% worse on average than the one produced by the more general BPGS. What is most striking is how much stronger the 57

67 Table 9: Primal Side Comparison of 4 and 6 Processors 4 Processors 6 Processors Instance Primal TTB Primal TTB V-L 423, , V-T 398, , F-T 668, , V-L 94, , F-L 138, , V-T 98, , F-T 139, , V-L 430, , F-L 597, , V-T 501, , F-T 643, , V-L 76, , F-L 118, , V-T 76, , F-T 110, , V-L 54, , F-L 96, , V-T 53, , F-T 99, , V-L 114, , F-L 152, , V-T 116, , F-T 157, , V-L 47, , F-L 60, , V-T 47, , F-T 56, , V-L 98, , F-L 139, , V-T 98, , F-T 134, , dual bound produced by BPGS is than the dual bound produced by IP Search. Given that for these instances the extended formulation used by BPGS did not provide a stronger linear relaxation than the compact one, this is most likely due to the fact that BPGS performs its primal and dual side tasks in parallel whereas for IP Search these tasks compete for CPU time. Having studied the potential of IP-based search on an academic network design problem, the MCFCNF, we next turn our attention to a large-scale network design problem of realworld interest. Specifically, we study the applicability of IP-based search concepts to the service network design problem for Less-Than-Truckload freight transportation carriers. 58

68 Table 10: Primal and Dual Comparison with IP Search BPGS IP Search Gap with BPGS IP Search Instance Primal TTB Primal TTB BPGS Dual Gap Dual Gap V-L 423, , , , V-T 398, , , , F-T 668, , , , V-L 94, , , , F-L 138, , , , V-T 98, , , , F-T 139, , , , V-L 430, , , , F-L 597, , , , V-T 501, , , , F-T 643, , , , V-L 76, , , , F-L 118, , , , V-T 76, , , , F-T 110, , , , V-L 54, , , , F-L 96, , , , V-T 53, , , , F-T 99, , , , V-L 114, , , , F-L 152, , , , V-T 116, , , , F-T 157, , , , V-L 47, , , , F-L 60, , , , V-T 47, , , , F-T 56, , , , V-L 98, , , , F-L 139, , , , V-T 98, , , , F-T 134, , , , Average

69 CHAPTER IV IP-BASED SEARCH IN PRACTICE National less-than-truckload (LTL) carriers run high-volume freight transportation operations, often spending millions of dollars in transportation and handling costs each week. A LTL motor carrier transports shipments that typically occupy only 5-10% of trailer capacity. Hence, transporting each customer shipment directly from origin to destination is not economically viable. As a result, LTL carriers collect and consolidate freight from multiple shippers to increase trailer utilization, referred to as the load factor, and route freight through a network of consolidation terminals between origin and destination. The savings generated by increasing trailer load factors through consolidation is partially offset by other costs; transferring freight between trailers generates a handling cost, and terminalto-terminal routing increases the total time and distance each shipment requires to reach its destination. This chapter focuses on methods for planning how freight should be routed from origin to destination through the terminal network, specifying where along the way it is transferred from one trailer to another. In the LTL industry, this service network design problem is known as constructing a load plan. Customer service and pricing pressures have made effective load plans more critical than ever to a carrier s success. National LTL carriers now compete with both so-called super-regional carriers (resulting from mergers of regional LTL carriers), and also LTL service offerings from traditional package express companies like UPS and FedEx; this has led to pressure on pricing, as well as increased competition on customer service including transit time and on-time reliability. LTL shipments are quoted a service standard from origin to destination in business days. Historically these standards were long enough (often 5 or more business days) that service only loosely constrained freight routing decisions. Today, service standards of 1, 2, and 3 days are much more common (Figure 3 presents a freight profile by service standard for 60

70 a national carrier), and these tighter standards must be enforced when planning shipment paths. Shorter service standards reduce opportunities for consolidation (since consolidation introduces handling time and circuity time penalties), especially if rigid operating rules are also enforced (such as a shipment transferred at most one breakbulk per day). Modern freight routes are more complex, and a shipment may visit multiple breakbulks in a single operating day. As a result, carriers need methods for designing load plans that accurately model how short service standards constrain shipment paths and the consolidation opportunities that still exist. 1 Day 2 Day 3 Day 4 Day 5 Day Figure 3: # Shipments by Service Standard LTL load plans are typically constructed using a number of self-imposed simplifying rules that are later relaxed during actual operations. For example, a traditional load plan assumes that the same freight routing decisions are executed every day. On a given operating day, terminal managers then often override the plan in an attempt to reduce linehaul costs given daily freight volume fluctuations; a simple example is to build a trailer that skips a planned next handling terminal. By relaxing some of these simplifying rules when planning, it may be possible to build improved load plans that generate cost savings and require less manual replanning. For example, one such relaxation would be to plan for predictable daily volume fluctuations by allowing for different freight routing decisions on different weekdays. 61