Understanding Some Unexpected Failures of the Theory of Constraints Product Mixes utoria: lexandre Linhares bstract: The theory of constraints proposes that, when production is bounded by a single bottleneck, the best product mix heuristic is to select products based on their ratio of throughput per constraint use. This is not true for cases when production is limited to integer quantities of final products. We demonstrate four facts, all of which go directly against current thought in the TOC literature. For example, there are cases on which the optimum product mix includes products with lowest product margin and lowest ratio of throughput per constraint time, simultaneously violating the margin heuristic and the TOC-derived heuristic. We start with a simple product mix example previously analyzed in the literature and introduce a new product which dramatically alters the results obtainable from a theory of constraints perspective. It is thus crucial to understand the nature of these unexpected failures to obtain optimum product mixes. Such failures are due to the combinatorial complexity (NP- Hardness) of the product mix decision problem, a fact that is demonstrated here. 1. Introduction The theory of constraints is a remarkably successful operations management philosophy, centered on the idea of focusing managerial attention to the local constraints which inhibit the global performance of an entire system (Goldratt and Cox 1984; Goldratt and Fox 1986; Goldratt 1990a; Goldratt 1990b). Over the last two decades it has gathered much momentum, with the creation of organizations such as the Goldratt Institute and the TOC Center in Dayton, Ohio. It has also spawned a host of products, such as the OPT software and an innovative bestselling management novel. Blackstone (2001) reviews some of its core ideas and fields of application. The focus of our study will be on the problem of selecting the optimum product mix under the theory of constraints, which is deemed as an improvement over traditional practices (Gupta et al 2002; Kee and Schmidt 2000; Wahlers and Cox 1994). Consider a facility with a set of products to build, but without the capacity (i.e., a fixed time horizon) required to meet the demand for all of them. Let us suppose that this facility must deal with integer quantities of final products. In this case a product mix decision must be made, with the obvious tradeoff of prioritizing some product lines at the expense of others. traditional method for selection of the product mix is given by selecting the products with highest individual product margin with higher priority, regardless of the time spent on the bottleneck(s). Let us name this method as the margin heuristic. The theory of constraints, however, proposes that product lines be selected according to their ratio of throughput per time spent on the system constraint(s). Let us refer to this approach as the TOC-derived heuristic. The heuristic has been formally stated in numerous TOC publications (e.g., Goldratt 1990a); for the convenience of the reader let us follow from Fredendall and Lea (1997): TOC-derived Product Mix Heuristic Step 1: Identify the system s constraint(s): (a) Calculate the required load on each resource to produce all the products. The constraint or bottleneck (BN) is the resource whose market demand exceeds its capacity. Step 2: Decide how to exploit the system s constraint(s):
(a) Calculate the contribution margin (CM) of each product as the sales price minus the raw material (RM) costs; (b) Calculate the ratio of the CM to the products processing time on the bottleneck resource (CM/BN); (c) In descending order of the products CM/BN, reserve the BN capacity to build the product until the BN resource s capacity is exhausted; (d) Plan to produce all the products that do not require processing time on the bottleneck (i.e., the free product) in descending order of their CM. (Fredendall and Lea 1997, p. 1535-1536) It was believed, when originally suggested, that this TOC-derived heuristic would obtain the best combination of products for all cases. In fact, it does obtain optimum solutions when production is allowed to be fragmented. However, studies by Plenert (1993) and by Lee and Plenert (1993) clearly demonstrated by comparing the product mixes obtained with the TOC-derived heuristic with those obtained with integer linear programming that when production must be done over integer quantities the TOC-derived heuristic could fail to find the optimum product mix in the case of multiple constraining resources. These rather interesting results launched an increasing series of studies for improved policies (Fredendall and Lea 1997; Hsu and Chung 1998) and for advanced heuristics such as genetic algorithms (Onwubolu and Mutingi 2001a; Onwubolu and Mutingi 2001b) and tabu search (Onwubolu 2001), providing new product mix methods for the case of multiple constrained resources. fter all, this multiple constraining resources scenario framed the original context on which Plenert (1993) demonstrated the shortcomings of the TOC-derived heuristic. The situation, however, is in fact more complex than it seems. In this paper, we demonstrate the following facts, all of which go directly against current thought in the theory of constraints literature: FCT 1. There are cases on which the TOC-derived heuristic fails even with a single bottleneck. FCT 2. There are cases on which the TOC-derived heuristic fails to obtain a higher profit than the traditional product margin heuristic. FCT 3. There are cases on which the optimum product mix includes products with lowest product margin and lowest ratio of throughput per constraint time, violating both the traditional heuristic and the TOC-derived heuristic. FCT 4. There are strong reasons to believe that an efficient and optimum heuristic is simply impossible. 2. Review of the Blackstone (2001) example Let us start the discussion with an example from Blackstone (2001). Figure 1 presents an hypothetical facility capable of producing three products: product X, product Y, and product Z. Product X sells for $90 and has a weekly demand of 50 units; product Y sells for and has a weekly demand of 75 units, and product Z sells for $70 and has a weekly demand of 100. The facility has five work stations (, B, C, D, ) and the products are devised from four types of raw materials (, RM2, RM3, and RM4). 2
Production of X is started with two units of RM2 processed at station for utes each. One of them is taken to station C for 1utes of processing while the other is taken to station D also for 1utes. These materials are then joined, along with a new unit of, at station, in a ute process. Hence X has $40 of material cost. X $90 50/Week Y 75/Week Z $70 100/Week $10 D 1 D C 1 C B RM2 RM2 RM3 RM4 $10 Figure 1. fter Blackstone (2001): a hypothetical facility. Production of Y is started with a unit of RM2 processed at station for utes, and a unit of RM3 processed at station B also for utes. fter station is finished, the resulting material flows to station D for 1utes and then to station for an additional 10 minutes. fter station B processes RM3, the resulting material is taken to stations C (5 minutes), D (utes), and finally it is joined with the material resulting originally from RM2, in station, in a ute process. Material costs for Y equal $30. Capacity requirements per unit Station Load (minutes) Product X Product Y Product Z Product X Product Y Product Z Total requirement 20 10 5 1000 750 500 2250 Station B 0 10 10 0 750 1000 1750 Station C 15 5 5 750 375 500 1625 Station D 15 25 10 750 1875 1000 3625 Station 5 10 5 250 750 500 1500 Demand 50 75 100 Selling price 90 100 70 Raw materials 40 30 25 Labor 10 10 5 Overhead 30 30 15 Product cost 80 70 45 Product margin 10 30 25 Table 1. Summary data of the Blackstone (2001) example 3
Product Z is made out of one unit of RM3 and one unit of RM4. RM3 is initially processed at B for utes, then followed by utes at C, utes at D, and 5 minutes at, where it will be joined with the unit of RM4 processed for utes at. The material costs for Z are $25. word of caution: the reader might have noticed that there are some incorrect data in Blackstone s paper. For example, in that paper, figure 6 reveals that product Z requires 15 minutes of processing at station C, while the next figure will place that number at only 5 minutes. It is impossible to tell from that text which numbers were the correct ones, but this does not affect the arguments in either way. In here we are thus referring to the data from the matrix in Blackstone (2001) s figure 7, and not from the facility scheme of Blackstone (2001) s figure 6. We thus have on table 1 a summary of the load on the stations where it may be concluded that there is not enough capacity in station D, which clearly makes it the only system constraint. Now a decision must be made: which products should be produced? This is the product mix problem under a bottleneck. Operating costs of the system are given by $10 of labor per hour and $30 of overhead per hour, which brings a total of $8000 per week, given 5 stations operating for 40 hours. However, if we break these costs per product, different product margins emerge. Table 1 gives Blackstone s (2001) labor costs and overhead costs per product, which, when added to raw material costs, let us obtain the product margin of products X, Y, and Z. Product mix by product margin Product X Product Y Product Z Selling price $90 $70 Raw material $40 $30 $25 Throughput/unit $50 $70 $45 Totals Production 0 75 52 127 Throughput/product $0 $5.250 $2.340 $7.590 Operating expense $8.000 Plant profit -$410 minutes of D per unit 15 25 10 minutes of D per product 0 1875 520 2395 Table 2. Product mix decision following the product margin heuristic. Under the traditional viewpoint, Blackstone argues, companies tend to view products as either dogs which have low profit margins, or as stars with great margins. This view is compatible with that of determining a product mix by selecting the products with highest margins first. Under this product margin heuristic, a company should prefer to produce all demand of 75 units of product Y (margin: $30), followed by 52 units of the product with the next best margin, which is Z (margin: $25). Its production of only 52 units of Z, instead of its whole demand of 100 units, is due to the constraints of the bottleneck. The bottleneck, station D, should still have utes left of available capacity, but we may assume that no products can be produced in this timeframe, and so this product mix based on margins leads to a profit of -$410 (Table 2). 4
Now, this decision (and its associated profit) does not consider the actual time that products spend on the constraint. In this case the product with highest margin is also the product with highest utilization of the bottleneck. This shortcoming leads us to the proposal of the theory of constraints: to include products based on their comparative ratio of throughput per constraint minute. This is done in table 3, where product Z, with a ratio of $4,50/minute, is produced to meet its total demand of 100 units. This decision consumes 1000 minutes of the bottleneck. The next product, product X with a ratio of $3,33, is also produced to meet its total demand of 50 units, consuming 750 minutes of the bottleneck. With the remaining minutes, the heuristic then allocates the bottleneck to work on product Y, and there is capacity to produce 26 units. This new product mix generates $8.820, raising profit to $820 per week. Product mix from a theory of constraints standpoint Product X Product Y Product Z Selling price $90 $70 Raw material $40 $30 $25 Throughput/unit $50 $70 $45 Constraint minutes 15 25 10 Throughput/constrnt.min. $3,33 $2,80 $4,50 Totals Production 50 26 100 176 Throughput/product $2.500 $1.820 $4.500 $8.820 Operating expense $8.000 Plant profit $820 minutes of D per unit 15 25 10 minutes of D per product 750 650 1000 2395 Table 3. Product mix decision following the throughput per constraint time (TOC) heuristic generates greater profit than selection by product margin. On the words of Blackstone (2001): this example proves emphatically that products do not have profits, companies do. Making decisions based on product profit while ignoring the impact of the product on the constraint is clearly suboptimal. The correct decision variable for determining product mix is Throughput per Constraint Minute. (Blackstone 2001, p.1062) In the next section, a new product is introduced into Blackstone s example. It will let us look closer at this heuristic of throughput per constraint time. 3. Introducing pathological product lpha Let us now introduce a new product into Blackstone s (2001) example, product lpha. On a first reading, product lpha will seem very different from the products already offered. Its parameters are very distinct from those of products X, Y, and Z. But this is precisely intended to be the case, in order to show the limitations of the heuristic in this new special case. Product lpha is expensive and sells for $6630. It is created from 2 units of raw material 3, i.e., it has $30 of material cost, and it demands 1650 minutes of processing time in station D. This means that it adds further burdens on the system constraint, and only on the system constraint. The weekly demand for lpha is a single unit. Thus as we can see in table 4, the load on bottleneck station D skyrockets to 527utes. 5
Now, how does the heuristic of selecting products based on their ratio of throughput per constraint time fare in this case? Product Z still leads this parameter, with a ratio of $4,50 per constraint minute. It is followed by the new offering, product lpha, with a ratio of $4,00; and then by products X (ratio of $3,33) and finally Y (ratio of $2,80). So the heuristic tells us to start production by fulfilling all demand for product Z. This will in turn require the bottleneck to work for 1000 minutes. fter demand for Z is fulfilled, the bottleneck will have 1400 minutes remaining. But this is not enough time to produce the product with the next best ratio of throughput per constraint time, as product lpha requires 1650 minutes of bottleneck time. So the TOC-derived heuristic would either lead to an unfeasible solution which exceeds the bottleneck availability, or it would have to skip production of lpha and produce the same mix as that one before the introduction of lpha, which, as we have just seen, leads to a profit of $820. X $90 50/Week Y 75/Week Z $70 100/Week lpha $6630 1/Week D 1650 min $10 D 1 D C 1 C B RM2 RM2 RM3 RM4 $10 RM3 RM3 Figure 2. Introduction of product lpha. If, however, one does not produce the item with best ratio of throughput per constraint minute at all, and instead resorts to producing only product lpha and the demand for product X, the total throughput would grow, turning the higher profit of $1100. (Notice also that this higher profit assumes that operating expenses are held fixed at $8000, but according to Blackstone (2001), operating expenses are based on $1600 per station per week, and this work schedule does not require station B to be active at all. So if operating costs under this schedule drop to $6400, total profit would grow from the $820 seen above to $2700, only by selecting a different product mix than that pointed out by the TOC-derived heuristic.) This goes against current thought in the TOC literature and demonstrates FCT 1 given in the introduction: There are cases on which the TOC-derived heuristic fails even with a single constrained resource. So there seems clearly to be a problem with the TOC-derived heuristic. The skeptical reader may not be convinced and argue that the introduction of product lpha is too unconventional, that its parameters are much too distinct from those of the previous 6
products, and that it is the inadequacy and unrealistic nature of these numbers that somehow make up for this failure of the TOC-derived heuristic. This is not the case (though the numbers of product lpha really do seem quite artificial). The TOC-derived heuristic does not lead to optimum product mixes in a much larger number of cases. Capacity requirements per unit Station Load (minutes) Product X Product Y Product Z Product lpha Product X Product Y Product Z Product lpha Total Requirement 20 10 5 0 1000 750 500 0 2250 Station B 0 10 10 0 0 750 1000 0 1750 Station C 15 5 5 0 750 375 500 0 1625 Station D 15 25 10 1650 750 1875 1000 1650 5275 Station 5 10 5 0 250 750 500 0 1500 Demand 50 75 100 1 Selling price 90 100 70 6630 Raw materials 40 30 25 30 Labor 10 10 5 275 Overhead 30 30 15 825 Product cost 80 70 45 1130 Product margin 10 30 25 5500 Table 4. Introduction of product lpha places additional pressure on the system constraint. Product lpha: a pathological case for the theory of constraints Product X Product Y Product Z Product lfa Selling price $90 $70 $6.630 Raw material $40 $30 $25 $30 Throughput/unit $50 $70 $45 $6.600 Constraint minutes 15 25 10 1.650 Throughput/constrnt.min. $3,33 $2,80 $4,50 $4,00 Totals Production 50 0 0 1 Throughput/product $2.500 $0 $0 $6.600 $9.100 Operating expense $8.000 Plant profit $1.100 minutes of D per unit 15 25 10 1650 minutes of D per product 750 0 0 1650 2400 Table 5. Profit rises to $1100 (or to $2700 if station B is allowed to shut down), violating the TOC-derived heuristic in a single constrained resource case. In order to clarify this issue, let us consider in the following section the simplest possible case of catastrophic failure of the TOC-derived heuristic. This will help us in isolating the problem and in making the underlying reason clear. 4. Isolating the phenomenon Figure 3(a) presents an extremely simple case of a set of products {#1,#2,#3,#4}, a single work station and a single raw material (worth ). Our planning period, and the capacity of station, is a single work day, or 8 hours. The selling price of each final product is a direct function of the time spent on station, and the demand for the day is one product #1, one product #2, one product #3, and one product #4. Since the production of these items would require 19 hours, station is clearly a system constraint and is obviously the only one. Now what is the best product mix? Let us see how the product margin heuristic and the TOCderived heuristic fare in this peculiar case. Let us first consider the classic method of product mix, by selecting the products with highest margins first (Fig 3(b)). In this case, since all products are derived from raw material 1, we need only to select the product with highest selling price, which is product #1. This product would thus consume 6 hours of station, and the remaining hours would not be 7
sufficient time to produce any of the remaining demanded items. Thus, total throughput would equal $500 under this heuristic. Product 1 $600 Product 2 $550 Product 3 Product 4 Product 1 $600 Product 2 $550 Product 3 Product 4 6 hours 5 hours 6 hours 5 hours (a) a simple case with one bottleneck (b) result of the product margin heuristic: $500 Product 1 $600 Product 2 $550 Product 3 Product 4 Product 1 $600 Product 2 $550 Product 3 Product 4 6 hours 5 hours 6 hours 5 hours (c) result of the TOC-derived heuristic:$450 Figure 3 (a-d). The simplest case of such failure. (d) optimum solution violates both policies:$600 Now, by using the TOC-derived heuristic in figure 3(c), we would start by selecting the products with highest relation of throughput/constraint time, which turns out in this case to be product 2, with a $110/hour ratio (a ratio larger than, the ratio of all the remaining items). This product would consume 5 hours of station time, and there would not be sufficient time for the production of any remaining items, leading to a total throughput of $450. This demonstrates FCT 2: There are cases on which the TOC-derived heuristic fails to obtain a higher profit than the traditional product margin heuristic. Now, the optimum heuristic for this case is exactly to violate both the traditional product margin heuristic and the theory of constraints heuristic and to produce the items with lowest product margin and lowest ratio of throughput per constraint time! The simultaneous violation of both policies leads to a total throughput of $600. This demonstrates FCT 3: There are cases on which the optimum product mix includes products with lowest product margin and lowest ratio of throughput per constraint time, violating both the traditional heuristic and the TOC-derived heuristic. The reason for such unanticipated results is given in the following section. 8
5. The reason for these failures The simple case presented in the previous section may have led the reader to the following obvious conclusion: Proposition 1. Selection of a product mix in a constrained facility is NP-Hard. Proof. Reduction from the knapsack problem. To demonstrate that selection of a product mix in a constrained facility is NP-hard, we must demonstrate how a particular polynomial-time method for its optimum solution would also be capable of solving a known NP-hard problem (e.g., Garey and Johnson 1979). Consider the following optimization problem: we are given a sack and a set of N items of value V i and weight W i. The knapsack problem asks us to select a subset of items which maximizes the sum value of items placed in the sack, while the weighted sum of these items lies at most at a particular threshold T (Garey and Johnson 1979). This famous problem is NP-hard to solve, which means that there is no known method that will produce an optimum solution under an efficient (polynomial) time frame, as a function of N. It is easy to see that any exact method the product mix problem under TOC can be used to solve the knapsack problem. Let us consider a production setting with a single bottleneck (i.e., station ) and N possible products. The products correspond to the items to be (or not to be) placed on the sack; so, for each item, let us create a corresponding product. s in the previous example, each product consists of raw material processed in station, the bottleneck. Let each product have a raw material price of $0. The selling price of each product is given by the value of each item V i. The load required (time in minutes) for each product in the bottleneck is given by weight W i of each corresponding item. Let the capacity of station (i.e. available time for processing) equal the sack s maximum weight allowed, T. If we make these straightforward mappings then an optimum solution of the product mix problem is also an optimum solution to the original knapsack problem. Thus any exact algorithm for optimum product mix under TOC can also be used for the solution of the NPhard knapsack problem. This demonstrates that selection of a product mix in a constrained facility is also in itself an NP-hard problem. In the above example and proof, we have considered the 0-1 knapsack problem, where a binary decision either withdrawn or placed a specific item on the sack. The situation with the product mix decision under a constraint is closer, however, to the traditional knapsack problem that asks for the number of each specific type of item to be placed on the sack. This change in perspective does not change the major proposition that the problem is NP-Hard and it may be unpractical to attempt to obtain optimal solutions to all cases. The reader should have noticed that this result obviously implies that the corresponding problem under multiple constraints is also NP-Hard. The reasoning is the following. Suppose we have an optimum product mix algorithm for the case of multiple constraints. This algorithm can be used to solve the case of a single constraint, as for example by adding a new constrained resource and a new product. Let the value of the product be zero and the new constrained resource only be used to produce the new product (and be the only one so). solution to the transformed multiple constraint problem is a direct solution to the single constraint one. So an algorithm for the multiple constrained problem would solve 9
optimally an NP-Hard problem, which immediately demonstrates that the multiple constraints case is also obviously NP-Hard. Since an exact method for selection of a product mix in a constrained facility would imply that P=NP launching a revolution in science and mathematics, this result demonstrates FCT 4: There are strong reasons to believe that an efficient and optimum heuristic for the product mix decision under the theory of constraints is simply impossible. It also explains Plenert s (1993) results and illustrates why researchers have not been able to find a simple optimum heuristic for product mix under multiple constrained resources as we have seen here, the TOC-derived heuristic fails even on facilities with a single constrained resource. It seems thus more reasonable to expect that the best possible that may be obtained for large instances are high quality approximations given by the advanced heuristics (such as genetic algorithms) that have been under study recently (see, for instance, Fredendall and Lea 1997; Hsu and Chung 1998; Onwubolu 2001; Onwubolu and Mutingi 2001a; Onwubolu and Mutingi 2001b). There are two obvious limitations on this result: First, we are dealing with cases where we have perfect information. This supposition does not reflect numerous industrial environments that exhibit rapidly shifting bottlenecks, major deviations in processing time for each station, etc. While these suppositions are shared with a large part of the literature (e.g., Goldratt 1990a, Fox 1987), there is still a clear need for further research of cases with imperfect and non-deterministic information. nother limitation is that even if items are allowed to be completed in part over days and weeks (instead of in whole numbers during the planning period), then the TOC-derived heuristic is in fact optimal. This last point is worth developing. It is crucial to distinguish the TOC philosophy from the TOC-derived product mix heuristic. This paper is in no way a criticism of the theory of constraints. TOC is a management philosophy that does not require, never mentions, and is completely independent of the integer assumption. The results presented here only demonstrate that the straightforward extension of the heuristic for the integer case may be problematic. This does not imply anything for the theory of constraints as a management philosophy (Mabin and Balderstone 2003; Mabin and Gibson 1998; Mabin and Davies 2003). It does, however, gives us a new important research problem: since TOC is based on breaking constraints, how to properly use the TOC philosophy to deal with the combinatorial complexity demanded in the integer production case. fter all, we do not want to turn the product mix decision itself into a bottleneck. 6. References BLCKSTON, J.H., Theory of constraints a status report. International Journal of Production Research, v.39, p.1053-1080, 2001. FOX, R., Theory of constraints. N Conference proceedings, p.41 52, 1987. FRDNDLL, L.D. and L, B.R., Improving the product mix heuristic in the theory of constraints. International Journal of Production Research, v. 35, p.1535-1544, 1997. GRY, M. and JOHNSON, D., Computers and Intractability: a guide to the theory of NPcompleteness, W.H. Freeman & Co., San Francisco, 1979. 10
GOLDRTT,.M., and COX, J., The goal: a process of ongoing improvement, Croton-on- Hudson, NY: North river press, 1984. GOLDRTT,.M. and Fox, R.., The race, Croton-on-Hudson, NY: North river press, 1986. GOLDRTT,.M., Sifting information out of the data ocean: the haystack syndrome, Croton-on-Hudson, NY: North river press, 1990a. GOLDRTT,.M., What is this thing called the theory of constraints and how should it be implemented? Croton-on-Hudson, NY: North river press, 1990b. GUPT, M.C., BXNDL, S.J. and P.S. RJU, Integrating BM/TOC approaches for performance improvement: a framework and application. International Journal of Production Research, v. 40, p.3225-3251, 2002. HSU, T.C., and S.H. CHUNG, The TOC-based algorithm for solving product mix problems. Production Planning & Control, v. 9, p.36-46, 1998. K, R., and C. Schmidt, comparative analysis of utilizing activity-based costing and the theory of constraints for making product-mix decisions. International Journal of Production conomics, v. 63, p.1-17, 2000. L, T.N., and G. PLNRT, Optimizing theory of constraints when new product alternatives exist. Production and Inventory Management Journal, v. 34, p.51-57, 1993. MBIN, V., and BLDRSTON, S., The performance of the theory of constraints methodology: analysis and discussion of successful TOC applications. International Journal of Operations & Production Management, v. 23, p.568-595, 2003. MBIN, V., and DVIS, J., Framework for understanding the complementary nature of TOC frames: insights from the product mix dilemma. International Journal of Production Research, v. 41, p.661-680, 2003. MBIN, V., and GIBSON, J., Synergies from spreadsheet LP used with the theory of constraints a case study. Journal of the Operational Research Society, v. 49, p.918-927, 1998. ONWUBOLU, G. C., Tabu search-based algorithm for the TOC product mix decision. International Journal of Production Research, v. 39, p.2065-2076, 2001. ONWUBOLU, G.C., and M. MUTINGI, genetic algorithm approach to the theory of constraints product mix problems. Production Planning & Control, v. 12, p.21-27, 2001a. ONWUBOLU, G. C. and M. MUTINGI, Optimizing the multiple constrained resources product mix problem using genetic algorithms, International Journal of Production Research, v. 39, p.1897-1910, 2001b. PLNRT, G., Optimizing theory of constraints when multiple constrained resources exist. uropean Journal of Operational Research, v. 70, p.126-133, 1993. WHLRS, J.L., and J.F. Cox, III, Competitive factors and performance measurement: applying the theory of constraints to meet customer needs. International Journal of Production conomics, v. 37, p.229-240, 1994. 11