Accepted in International Journal of Production Economics, Vol. 135, No. 1, 2012

Transcription

1 Accepted in International Journal of Production Economics, Vol. 35, No., 0 Production planning of a ybrid manufacturing/remanufacturing system under uncertainty witin a closed-loop supply cain Jean-Pierre Kenné, Pierre Dejax and Ali Garbi 3 Mecanical Engineering Department, Laboratory of Integrated Production Tecnologies, University of Quebec, École de tecnologie supérieure, 00, Notre Dame Street West, Montreal (Quebec), Canada, H3C K3 Department of Automatic Control and Industrial Engineering, Logistics and Production Systems Group, Ecole des Mines de Nantes, IRCCyN, BP Nantes Cedex 3 - France 3 Automated Production Engineering Department, Production Systems Design and Control Laboratory, University of Quebec, École de tecnologie supérieure, 00, Notre Dame Street West, Montreal (Quebec), Canada, H3C K3 Abstract Tis paper deals wit te production planning and control of a single product involving combined manufacturing and remanufacturing operations witin a closed-loop reverse logistics network wit macines subject to random failures and repairs. Wile consumers traditionally dispose of products at te end of teir life cycle, recovery of te used products may be economically more attractive tan disposal and remanufacturing of te products also pursue sustainable development goals. Tree types of inventories are investigated in tis paper. Te manufactured and remanufactured items are stored in te first and second inventories. Te returned products are collected in te tird inventory and ten remanufactured or disposed of. Te objective of tis study is to propose a manufacturing/remanufacturing policy tat would minimise te sum of te olding and backlog costs for manufacturing and remanufacturing products. Te decision variables are te production rates of te manufacturing and te remanufacturing macines. Te Optimality conditions are developed using te optimal control teory based on stocastic dynamic programming. A computational algoritm, based on numerical metods, is used for solving te optimal control problem. Finally, a numerical example and a sensitivity analysis are presented to illustrate te usefulness of te proposed approac. Te structure of te optimal control policy is discussed depending on te value of costs and parameters and extensions to more complex reverse logistics networks are discussed. Key words: Reverse logistics, closed-loop supply cains, manufacturing/remanufacturing, optimal control, production Planning, stocastic dynamic programming, Numerical metods.. Introduction Te management of return flows induced by te various forms of reuse, or recovery of products and materials in industrial production processes as received growing attention trougout te last two decades among practitioners as well as scientists. Tis is due to te constant searc for increased productivity, a better service to clients, as well as environmental 0. Tis manuscript version is made available under te CC-BY-NC-ND 4.0 license ttp://creativecommons.org/licenses/by-nc-nd/4.0/ DOI : 0.06/j.ijpe

2 concerns and te goals of sustainable development in general. Work in tis area may be found as early as te 900 s. As an example, Amezquita and Bras (996) analyzes te remanufacturing of an automobile clutc and Lund (996) edited a compreensive report on te remanufacturing industry. According to te American Reverse Logistics Executive Council, Reverse logistics can be viewed as te process of planning, implementing and controlling te efficient, cost effective flow of raw materials, in-process inventory, finised goods and related information, from te point of consumption back to te point of origin, for te purpose of recapturing teir value or proper disposal (Rogers and Tibben-Lembke, 998). As mentioned in Fleiscmann et al. (997), reverse logistics encompasses te logistic activities all te way from used products no longer required by te user to products again usable in a market. Tere is a major distinction between material recovery (recycling) and added value recovery (repair, remanufacturing). Tey describe te general framework of Reverse distribution, composed of te Forward cannel (going from te suppliers and troug te producers and distributors to te consumers, and of te Reverse cannel, going back from te consumers troug collectors and recyclers, to te suppliers or producers. In all cases, te reuse opportunities give rise to a new material flow from te user back to te spere of producers. Te management of tis material backward flow opposite to te conventional forward supply cain flow is te concern of te recently emerged field of «reverse logistics». Reverse logistics systems ave been classified into various categories depending on te caracteristics tat are empasized. Tus several classifications may be found in te literature: Seaver (994) focuses on design considerations for te different types of recovery operations based upon te Xerox Corporation experience; Fleiscmann et al. (997) propose a compreensive review of quantitative approaces and distinguis te types of returned items; Rogers and Tibben-Lembke (998) analyze trends and practices; Tierry et al. (995) analyze te main options for recovery. Teses autors distinguis two categories or reverse logistic systems and four types or return items or services. Te two categories of reverse logistic systems are open and closed-loop systems, depending on weter te return operations are integrated or not wit te initial operation. Te four categories of return items or services are te following: reusable items (suc as returned products or pallets), repair services (were te products are sent

3 back to te consumer after repair), remanufacturing, an industrial process in wic used, end of life products are restored to like-new conditions and put back into te distribution system like te new products and te recycling of raw materials and wastes. Many case studies and papers proposing models ave been publised in te area of reverse logistics as well as several reviews. See De Brito et al. (003) for a review of case studies; Dekker et al. (004) for a review of quantitative models for closed-loop supply cains; Geyer and Jackson (004) propose a framework to elp identify and assess supply loop constraints and strategies and tey apply it to te recycling and reuse of structural steel Sections in te construction sector; and Bostel et al. (005) for a review of optimization models in terms of strategic, tactical and operational planning. Special issues of journals ave been devoted to te area of reverse, or closed-loop logistics: Verter and Boyaci (006) edited a special issue journal on optimization models for reverse logistics. Guide and Van Wassenove (006, Part and Part ) edited a two parts feature issue journal on Closed-Loop Supply Cains. As can be seen from te numerous publised literatures on reverse logistics, many industrial cases ave been reported and many scientific planning or optimization models ave been proposed. Tese cases and models differ by numerous caracteristics. One of te te most complex situation is found in remanufacturing activities. Traditional areas for remanufacturing are te automotive or aeronautic sectors (macinery and mecanical assemblies suc as aircraft engines and macine tools) as well as te remanufacturing of toner cartridges (see te report by Lund (996) about te remanufacturing industry. An example is Hewlett-Packard, wic collects empty laser-printer cartridges from customers for using again (Jorjani et al. (004)). Oter examples are described in te next Section. Individual repair requirements for every product returned, and coordination of several interdependent activities makes production planning a igly sopisticated task in tis environment. In contrast wit traditional manufacturing systems (forward direction), no welldetermined sequence of production policies exist in remanufacturing (backward or reverse direction). Tis is illustrated by Sundin and Bras (005) wo describes a model of 3

4 remanufacturing steps not following a specific order. In many cases, te original manufacturer is also in carge of collecting, refurbising, and remanufacturing te used products. Te production planning in a ybrid context, involving manufacturing and remanufacturing facility, referred to as forward-reverse logistics in a closed-loop network, is very complex especially in te presence of uncertainties and under te consideration of te overall system s dynamics. Muc of te work in te area of remanufacturing is devoted to te strategic design of te closed-loop system, or to te tactical or operational planning of production and management of inventories, considering discrete periods of time. See te next Section for a more compreensive review of literature related to te remanufacturing activities. In tis paper we address te area of an integrated combined manufacturing / remanufacturing witin a closed-loop supply cain subject to uncertainties. We study te production planning of a ybrid manufacturing/remanufacturing system in a dynamic continuous time stocastic context. Uncertainties are due to macine failures and continuous time modeling seems to us a better framework to consider stocastic aspects. We found no suc approac in te publised literature on remanufacturing. We develop a generic stocastic optimization model of te considered problem wit two decision variables (production rates of manufacturing and remanufacturing macines) and two state variables (stock levels of manufactured and remanufactured products). Our goal is to caracterize optimal policies for te production rates of te new products as well as te remanufactured products, depending on te values of te model parameters and costs. Our modeling approac could possibly be applied to many industrial cases suc as tose mentioned in tis Section or te next were macines can be subject to failures and teir production rates can be continuously controlled. We feel te assumptions made can be acceptable, at least approximately, in many cases, but tis sould be verified as te first step of an industrial application. It is assumed in particular tat te remanufactured products can be considered meeting te same quality level as te new products so tat bot type of products can be distributed like new. We also assume tat te demand for new products is deterministic and known in advance and te proportion of returned products to be possibly remanufactured is a known proportion of demand. A typical example migt be te remanufacturing of copier cartridges, or aeronautical parts. 4

5 Te rest of te paper is organised as follows. In Section, te literature review is presented. Te model notations, te assumptions and te control problem are presented in Section 3. A numerical example and a sensitivity analysis (carried out to analyse te effect of te used products return ratio and te backlog costs), are given in Sections 4 and 5, respectively. Discussions and te direction for future works are presented in Section 6. Te paper concludes wit a sort summary in Section 7.. Literature review In tis Section, we focus on cases and models related to remanufacturing activities. A number of practical cases ave been reported in various contexts of application. Many works are devoted to te design of te network system or to production or inventory planning troug discrete optimization models. De Brito et al (003) discuss network structures and report cases pertaining to te design of remanufacturing networks by te original equipment manufacturers or independent manufacturers, te location of remanufacturing facilities for copiers, especially Canon copiers and oter equipments and te location of IBM facilities for remanufacturing in Europe. Tey also present case studies on inventory management for remanufacturing networks of engine and automotive parts for Volkswagen and on Air Force depot buffers for disassembly, remanufacturing and reassembly. Finally, tey present case studies on te planning and control of reverse logistics activities, and in particular inventory management cases for remanufacturing at a Pratt & Witney aircraft facility, yielding decisions of lot sizing and sceduling. As mentioned above teir survey on design, planning and optimization of reverse logistic networks, Bostel et al (005) propose a classification of problems and models according to te ierarcical planning orizon. Strategic planning models are devoted to network design problems, wile tactical and operational models to a number of sort terms issues. In tis context tey discuss a number of inventory management models wit reverse flows, including periodic and continuous review deterministic and stocastic inventory models. Tey also present production planning models, including for disassembly levelling and planning and production 5

6 planning and sceduling models involving return products. Tese models may include remanufacturing activities. Te special issue publised by Verter and Boyaci (006) contains tree papers optimization models for facility location and capacity planning for remanufacturing and a paper on assessing te benefits of remanufacturing options. In te introduction to teir Feature issues, Guide and Van Vassenove (006) discuss assumptions of models for reverse supply cain activities, and in particular for remanufacturing operational issues and remanufactured product market development. Several papers deal wit optimal policies for remanufacturing activities, pertaining to acquisition, pricing, order quantities, lot sizing for products over a finite life cycle. In recent years tere as been considerable interest in inventory control for joint manufacturing and remanufacturing systems in forward-reverse logistics networks. A forwardreverse logistic network establises a relationsip between te market tat releases used products and te market for new products as mentioned by El-Sayed et al. (008). Wen te two markets coincide, ten it is called a closed-network, oterwise it is called an open loop network (Salema et al. (007)). In Ostlin et al. (008), seven different types of closed-loop relationsips for gatering cores for remanufacturing ave been identified (ownersip-based, service-contract, direct- order, deposit-based, credit-based, buy-back and voluntary-based relationsips). Several disadvantages and advantages are described in Ostlin et al. (008) according to suc types of relationsips. By exploring tese relationsips, a better understanding can be gained about te management of te closed-loop supply-cain and remanufacturing. In a close loop network, te kind of planning problems arising and te adequacy of traditional production planning metods depend, to a large extent, on te specific form of reuse considered. Material recycling surely does involve new production processes. Returned parts and products may ave to be transformed back into raw material by means of melting, grinding, etc. However, te difficulty lies in te tecnical conversion to usable raw materials rater tan in managerial planning and control of tese activities. From a production management point of view, tese activities are no different from oter production processes. Consequently, conventional production planning metods sould 6

7 suffice to plan and control recycling operations for products tat can be sent back to te market, in order to recapture te maximum value. Te order of remanufacturing steps are very muc dependent on te type of product being remanufactured, were it is collected from, its relationsip to te user and manufacturer, volumes etc. Te situation may become more complex if disassembly is required prior to te actual recycling process. Dobos (003) found optimal inventory policies in a reverse logistics system wit special structure wile assuming tat te demand is a known function in a given planning orizon and te return rate of used items is a given function. Te demand is satisfied from te first store, were te manufactured and remanufactured items are stored. Te returned products are collected in te second store and ten remanufactured or disposed of. Dobos (003) minimized te sum of te olding costs in te stores and costs of te manufacturing, remanufacturing and disposal. Te necessary and sufficient conditions for optimality were derived from te application of te maximum principle of Pontryagin (Seierstad and Sydsaeter (987)). Teir results are limited to deterministic demand and return process wit no consideration on te dynamics of production facilities. Taking a closer look at te dynamic caracteristic of te production planning problems, one can notice tat te stocastic optimal control teory, suc as in Akella and Kumar (986), Deayem et al. (009), Hajji et al, (009) and references terein, is not yet used in reverse logistics. Kibum et al. (006) discussed te remanufacturing process of reusable parts in reverse logistics, were te manufacturing as two alternatives for supplying parts: eiter ordering te required parts to external suppliers, or overauling returned products and bringing tem back to «as new» condition. Tey proposed a general framework for te considered remanufacturing environment and a matematical model to maximize te total cost savings by optimally deciding (i) te quantity of parts to be processed at eac remanufacturing facility, (ii) te number of purcased parts from subcontractor. Te decision is optimally made using a mixed-integer programming model. Te study presented in Cung et al. (008) analysed a close-loop supply cain inventory system by examining used products returned to a reconditioning facility were tey are stored, remanufactured, and ten sipped back to retailers for retail sale. Te findings of 7

8 te study presented in Cung et al. (008) demonstrate tat te proposed integrated centralized decision-making approac can substantially improve efficiency. Te majority of te previous models are based on matematical / linear programming. An example of suc models can be found in El-Sayed et al. (008) were a multi period multi ecelon forward reverse logistics network model is developed. Te control of te manufacturing and remanufacturing production facilities, based on teir dynamics is very limited in te literature. Kiesmuller and Scerer (003) considered a system were remanufacturing processes are integrated into a single production environment. In tis type of ybrid environment, new and reused products are processed separately. Tis control approac of ybrid production/remanufacturing systems leads to te definition of two different inventory positions or levels to control and syncronize eac of te subsystems. Te repair/remanufacturing system studied by Pellerin et al. (009) differs from tis environment as it responds to bot remanufacturing orders and unplanned repair demands witin te same execution system. In addition, tere is a need to consider multiple remanufacturing rates for te current case. Tis additional complexity calls for a different control approac. Garbi et al. (008) treated suc a problem by proposing a control policy for repair and remanufacturing system wit limited capacity and solved te problem using a simulation approac. Te stocastic model presented in Pellerin et al. (009) and Garbi et al. (008) is based on a remanufacturing problem witout any consideration on manufacturing planning problems. Attention in te production planning problem in manufacturing systems subject to failures and repairs started growing wit te pioneering work of Risel (975); it as since been followed by extensions, as in Akella and Kumar (986), Kenné et al. (003), and Dong-Ping (009). Tose extensions are based on continuous time stocastic models and te dynamic programming approac. Te control policies are obtained eiter analytically (one macine, one product) or numerically for multiple macines or multiple products. We didn t found available results in ybrid manufacturing/remanufacturing wit continuous control models in a stocastic context. As mentioned in te introduction, a stocastic dynamic model is developed in tis paper to optimize te performance of given forward reverse logistics network. Te considered reverse 8

9 logistics network establises a relationsip between te market tat releases used products and te market for new products. Very few optimization models for te control of supply cains wit reverse flows are available in literature under uncertainty (witout any simplified assumptions on te existence of stocastic processes). Terefore, in view of previous researces on te various specific areas of remanufacturing, our researc focuses on developing a new generic model based on stocastic optimal control teory under uncertainties due to te dynamics of te production facilities. Te proposed approac for te control of a stocastic ybrid manufacturing/remanufacturing system differs from existing approaces because it integrates te manufacturing/remanufacturing system s dynamics. Te practical implication of te proposed model is examined troug a numerical example and experimental analysis. 3. Proposed control model Tis Section presents te notations and assumptions used trougout tis article, and te optimal control problem statement. x () t x () t x () t 3 () t () t () t d c d 3. Notations stock level of manufactured products stock level of remanufactured products stock level of te returned products stocastic process of te ybrid system stocastic process of te manufacturing macine stocastic process of te remanufacturing macine demand rate of customers remanufacturing demand rate d r m nsp d p manufacturing demand rate d remanufacturing supplier rate disposal rate d cr product return rate from te market c r c r c c c repair cost for te manufacturing macine repair cost for te remanufacturing macine inventory olding cost for manufactured products inventory olding cost for remanufactured products backlog cost for manufactured products 9

10 c C disposal cost disposal backlog cost for remanufactured products q transition rate from mode to of te process () t wit, {,,3,4} k q transition rate from mode i to j of te process () t wit i, j{,0} and k,0 ij Q g() J () v() transition rate matrix vector of limiting probabilities instantaneous cost function total cost value function discount rate production rate of te manufacturing macine u () t u () t production rate of te remanufacturing macine u maximal production rate of te manufacturing system max u maximal production rate of te remanufacturing system max 3. Context and Assumptions Te following summarizes te general context and main assumptions considered in tis paper.. Te model is a time continuous (not multi-period as usually assumed in te literature).. Customers locations are known and fixed 3. Te returned quantities are known functions of te initial customer demand. 4. Te customer demand is known and subject to a constant rate over time. 5. Te quality of remanufactured products is not different from manufactured products. 6. Te potential locations of suppliers, facilities, distributors, etc. are known. 7. Material, manufacturing, non-utilized capacity, sortage, transportation, olding, recycling, remanufacturing, and disposal costs are known for eac location. 8. Te maximal production rates of eac macine are known. 9. Te sortage cost depends on te sortage quantity and time (average value ($/product/unit of time)). 0. Te olding cost depends on te mean inventory level (average value ($/product/unit of time)).. Te manufacturing and remanufacturing systems are unreliable product units subject to breakdown and working in a parallel structure. k 0

11 Assumption is an original caracteristic of our approac, due to te consideration of macine breakdowns wit te stocastic control tecnique. Oter works consider discrete periods of time. Assumptions and 6 are classical assumptions in supply cain networks models. Assumption 3 is often made in tese types of models: te rate of returned is a fixed, known proportion of te initial demand. See te list of assumptions of models made for reverse supply cain activity in te Introdution to te feature issue by Guide and Van Wassenove (006 part,table ): eiter constant return rates or normally distributed return rates are proposed. Tis latter would assumption would be te subject of furter researc. Assumption 4 is also common to deterministic demand distribution models. Assumption 5 is commonly made in remanufacturing models, under possible quality control or inspection conditions before te decision of remanufacturing is made. Assumption 7 about known unit cost is classical. Assumption 8 is common in production planning. Assumptions 9 and 0 are common in inventory management. Assumption is te major motivation of our work. It is a classical assumption in production planning, but innovative in te remanufacturing area. 3.3 Problem statement Te considered ybrid manufacturing/remanufacturing system consists of two macines wic are subject to random breakdowns and repairs denoted M and M for manufacturing and remanufacturing, respectively. We consider a one product recovery system were demands can be fulfilled from an inventory for serviceable items. Tis inventory can be replenised by production or by remanufacturing used and returned items. Anoter inventory is available for te stock keeping of te returned items aead of te remanufacturing process. Tus, returned items can be remanufactured, disposed of, or be old on stock for later remanufacturing. Te situation is illustrated in Figure. Te system beaviour is described by a ybrid state comprising bot a continuous component (stocks of products) and a discrete component (modes of te macines). Te continuous component consists of continuous variables x ( t), x ( t ) and x () 3 t corresponding to te stock level of manufactured products, remanufactured products and returned items. Tese variables are described by tree differential equations based on te dynamics of te involved tree stock levels illustrated in Figure.

12 dx () t u( t) dm x(0) x dt dx () t u( t) dr x(0) x dt dx3 () t dcr dnsp d p x3(0) x dt 3 () Te serviceable inventories x () t and x () t are replenised by manufactured and remanufacturing items, respectively, wit dc dm dr. We assumed tat te product return rate from te market and te disposal rate are suc tat te remanufacturing macine M is never starved. Tus, te decision variables are u () t andu () t ; and te state variable are x () t and x () t for given dm, dr, dc, dcr, d p and d nsp. To acieve its main production goal, wile meeting part demands from manufacturing plants, te company sould determine ow many returned products sould be trown into te remanufacturing process to be remanufactured as new ones. Te objective of te ybrid manufacturing/remanufacturing system is tus to provide a given portion of te overall production from new spare parts (returned products). For example, could be 40% of d c wit dr dcr (e.g., te value of d r depends on te objective of te remanufacturing activities). Te average supply rate of te remanufacturing macine is equivalent to its production rate given tat tere is no store available on it. For example, if te remanufacturing is producing at its maximal rate u max, ten te disposal rate is set to drc max d cr u if te stock level of returned products is equal to a target value (fixed in tis paper by a disposal policy not studied erein), to avoid any starvation of te remanufacturing macine. Te state of a macine can be classified as operational, denoted by, and under repair, denoted 0. Let s say i( t), i,, denotes te state of te macine i M wit value in,0 B. i Let us also define q and q as failure rates of M and M, 0 0 q and q as repair rates of M and 0 0 M. Te dynamics of te ybrid system is described by a continuous time Markov process wit values in B B B ( t) ( t) ( t) transition rates from state to state are denoted (,),(,0),(0,),(0,0),,3,4. Te q (wit,,,3,4 ) and

13 determined from te failure and repair rates of M and M as in Table. Te transition diagram, describing te dynamics of te considered manufacturing/remanufacturing system is presented in Figure. For te considered ybrid system, te corresponding 4 4 transition matrix Q is one of an ergodic process. Hence, () t is described by te matrix Q q were q verify te following conditions: q q 0 ( ), () Te transition probabilities are given by: wit t0 q (3). q t t if t t t q. t t if t lim 0 for all, B. t Te set of te feasible control policies ( ), includingu ( ) and u ( ), depends on te stocastic process t and is given by: u u u umax t u umax t ( ),, 0, Ind ( ), 0, Ind ( ) (5) were wit Let t ; umax and umax are te maximal production rates for M and M, respectively, Ind ( ) g be te cost rate defined as follow: if ( ) is true 0 oterwise (4) g, x, x c x c x c x c x c ( d ( t) C ), B (6) p disposal were x max 0, x, x max 0, x i i and i i c is a constant defined as follows: c c Ind ( t) c Ind ( t) 3 ( c c ) Ind ( t) 4 r r r r 3

14 Our objective is to control te production rates u () and discounted cost given by:,, t,, 0, 0,, 0 u so as to minimize te expected J x x e g x 0 x dt x x x x (7) were is te discount rate. Te value function of suc a problem is defined as follows:,, inf,, v x x J x x B (8) ( u( ), u( )) ( ) In appendix A, we present te properties of te value function v. given by equation (8) by sowing tat suc a function satisfies te Hamilton-Jacobi-Bellman (HJB) equations. Te properties of te value function and te way to obtain HJB equations can be found in Kenné et al. (003). Suc equations describe te optimality conditions for manufacturing and remanufacturing planning problems. Numerical metods used to solve te proposed optimality conditions are also presented in Appendix A. In te next Section, we provide a numerical example to illustrate te structure of te control policies. 4. Analysis of results In tis Section, we present a numerical example for te ybrid manufacturing/remanufacturing system presented in Section 3. Te system capacity is described by a four states Markov process wit te modes in B,,3,4. We used te computational domains D G G suc tat: : 0 40, : 0 40 G x x G x x (9) wit and. Te manufacturing/remanufacturing system will be able to meet te customer demands (represented by te demand rate d c ), over an infinite orizon and reac a steady state if: ( umax umax ) umax 3umax dc (0) 4

15 were, k,,3, is te limiting probability of te system at mode k. Note tat te limiting k probabilities of modes,, 3 and 4 (i.e.,,, 3 and 4 ), are computed as follows: 4 Q( ) 0 and i () were (,, 3, 4) ; and Q() is te corresponding 44transition rate matrix. Table summarizes te parameters used in tis paper. Te feasibility condition given by equation () is satisfied for te cosen parameters if te raw material for te remanufacturing macine is always available (as discussed previously). We obtain a 6.6% availability of te system (i.e., AVs 6.6 ), given by te following expression: i 00 AVs ( umax umax ) umax 3umax dc () d c Te policy improvement tecnique is used to solve te system of equations (A.4)-(A.7), see appendix, for te data presented in table wit d r d c R r were R r is a return rate. Te first scenario investigated in tis paper, for wic te return rate (backward direction) is defined as te quotient of te average returns serviceable products D R and te average demands of (i.e., ` R / d / d ) provides te manufacturing and r R D r c remanufacturing policies for R 0.5 (i.e., d d 50% ). As a result, te value function r D representing te cost and te production rates of manufacturing and remanufacturing macines are obtained and depicted in Figures 3, 4a and 4b (for manufacturing), 5a and 5b (for remanufacturing), respectively. r Te production rates of te manufacturing and remanufacturing macines in teir operational mode (i.e., mode ) are presented in Figures 4 and 5. Te obtained manufacturing and remanufacturing policies in modes and 3 recommend to set te manufacturing rate (mode ) or te remanufacturing rate (mode 3) to teir maximal values given tat te ybrid system is feasible only at mode. Tere is no production in mode 4 given tat bot macines are down. For te manufacturing policy, Figure 4 sows tat tere is no need to produce parts, using M for comfortable stock levels. Ten te production rate is set to zero wen te stock level of manufactured products is greater tan 5 products as illustrated in figure 6. Te structure of te remanufacturing control policy is similar to te one of te manufacturing process wit a different c 5

16 tresold level. As illustrated in Figure 5, te production rate is set to zero wen te stock level of remanufacturing products is greater tan 3 products. Te tresold values of 5 and 3 products for manufacturing and remanufacturing are obtained from te numerical resolution of equations (A.4) to (A.7) and well illustrated in Figures 4(b) and 5(b). Te quantified feedback policy obtained for tis basic case and its implementation are illustrated in figure 6. Recall tat, in te considered forward-reverse logistics network, te remanufactured process brings returned products back to as new condition. Te bigger value of te remanufacturing tresold level, compared to te manufacturing tresold level, is mainly due to te fact tat te considered mean time to failure of M is greater tan te mean time to failure of M (i.e., MTTF MTTF or q M M q ). 0 0 From te obtained results, and based on te illustration of Figure 6, te computational domain is divided into tree regions were te optimal production control policy consists of one of te following rules:. Set te production rate of te manufacturing or remanufacturing system to its maximal value wen te current stock level is under te corresponding tresold value ( Z for manufacturing macine M and Z for remanufacturing macine M ).. Set te production rate of M to te demand rate d m wen te current stock level of manufactured products is equal to Z. Set also te production rate of M to te demand rate d r wen te current stock level of remanufactured products is equal to Z. 3. Set te production rates of manufacturing or remanufacturing macines to zero wen teir current stock levels are larger tan Z or Z. Te control policy obtained is an extension to te so-called edging point policy given tat te previous tree rules respect te structure presented in Akella and Kumar (986). Tus, te production policy for te manufacturing and remanufacturing at mode are given respectively by: u if x ( ) Z u ( x, x,) d d if x ( ) Z 0 oterwise max c r (3) 6

17 and u if x ( ) Z u ( x, x,) d if x ( ) Z 0 oterwise max r Using te control policies given by equations (3) and (4), te company will minimize te total cost due to remanufacturing so tat eventually it can maximise its total profit. Te previous results ave been obtained for a return rate equal 50%. Let us first confirm te structure of te control policies troug a sensitivity analysis; and ten observe te system for different return rates. Te aim of te sensitivity analysis is also to validate and illustrate te usefulness of te model developed in tis paper. (4) 5. Sensitivity analysis Sensitivity analyses are conducted on backlog cost parameters to gain insigt into te proposed stocastic model. Te proposed model is also validated in tis Section troug a set of experimental data reflecting practical industrial situation by considering a return rate different from te one used in te previous Section. Extensions, based on some assumptions of te developed matematical model, are discussed in te next Section. We performed a couple of experiments using te numerical example presented previously. Te results sown in Tables 3, 4, 5 and 6 represent two different situations in forward-reverse logistics system analysis presented below for returned rates set to 50% and 5% respectively. Figure 7, Tables 3 and 4 are based on te variation of manufacturing/remanufacturing backlog costs for a 50% return rate. Te corresponding trend of variation for te tresold levels (manufacturing and remanufacturing) and te incurred costs (represented by te average value function v( Z, Z ) ) is illustrated in Figure 7(a) and 7(b), respectively. One can observe tat wen te backlog costs c and c increase, te tresold values Z and Z also increase. Te overall cost also increases as well. In Tables 3 and 4, te experiment corresponding ( c, c ) (0,0), ( Z, Z ) (3,8) and v(3,8) 594 is igligted and is considered as a to basic case of te first block of backlog costs variations (i.e., for a return rate of 50%). Te parameters of te control policy move as predicted from a practical view point. 7

18 Te sensitivity analysis performed in tis paper, validates te proposed approac and sows te usefulness of te proposed model given tat te control policy is well defined by te equations (3)-(4) and parameters obtained from results analysis. We also conducted a similar sensitivity analysis for a return rate of 5% given tat we assumed a known return rate instead of trying to find it trow an optimization problem. Figure 7, Tables 5 and 6 are based on te variation of manufacturing/remanufacturing backlog costs for a 5% return rate. As in te case of 50% return rate, in Tables 5 and 6, te experiment corresponding to ( c, c ) (0,0), ( Z, Z ) (45,7) and v(3,8) 75 is igligted and is considered as a basic case of te second block of backlog costs variations (i.e., for a return rate of 5%). Te corresponding trend of variation for te tresold levels (manufacturing and remanufacturing) and te incurred costs (represented by te average value function v( Z, Z ) ) is illustrated in Figure 8(a) and 8(b) respectively. Wen te backlog costs c and c increase, te tresold values z and z also increase as in te case of 50% return rate. Due to te fact tat te return rate is 5%, te manufacturing and remanufacturing stock levels converge to 6 and 9 serviceable parts as teir corresponding backlog cost is large. Te restriction tat te remanufacturing process contributes just for 5% of te overall production give a large tresold level for te M to ensure sufficient serviceable parts to edge against future failures of te macines. 6. Discussions and extensions Te proposed ybrid manufacturing/remanufacturing model can be applicable to te various industries after specific customization. However, as te proposed model is introduced wit restrictive assumptions, many future works are accordingly needed. Above all, te proposed model wit backward information (remanufacturing) can be extended by adopting more industry practices and so te relevant complex matematical model need to be developed. Since te proposed model is formulated as a markovian stocastic optimal model, te computational burden for optimal solution increases exponentially as te size of problem increases (number of macines, number of products, uncertainty on product demands and returns). Tus, in te future, 8

19 an efficient euristic algoritm needs to be developed in order to solve te large-scale problems in real industrial environment. Recall tat, for te system considered previously and related to a one manufacturing macine, one remanufacturing macine and one-product, te control policy described by equations (3) and (4) is completely known for given Z and Z complex manufacturing system consisting of for remanufacturing, producing parameters Z ki, and kj wit,,3 For a more m macines for manufacturing, n macines p different part types, te control policy depends on te Z, for k,, p, i,, m, j,, n HJB equations (as in equation (A.)) is almost impossible for large m, n and. Te solution of te p since te dimension of te numerical sceme to be implemented increase exponentially wit te complexity of te system. Suc a dimension is given by te following equation: p p m n mn p 3 ( j ) ( j ) j j Dim N x N x (5) were N ( x ) card[ G ( x )] wit k for manufacturing inventories and k for kj kj remanufacturing inventories. Te numerical grids for te manufacturing and remanufacturing stock levels are G( x j) and G( x j), respectively. Eac macine as two modes (i.e., mn modes for m n macines) and its production rate can take tree values, namely maximal production rate, demand rate and zero for manufactured and remanufactured products (i.e., 3 m n p possibilities for m -manufacturing macines, n -remanufacturing macines products an p -products system). For example, in te case of macines, 5 types of products ( m, n 5), wit N( xj ) 00, N( xj ) 00, j,,5, from equation (5), we ave: Dim Te related numerical algoritm cannot be implemented on today computers. Suc problems are classified in te control literature as complex problems (Gerswin (990)). For te aforementioned complex problem (i.e., wit m, n and p ), a ierarcical control approac as in Kenné and Boukas (003) or a combination of te control teory and te simulation based experimental design as in Garbi and Kenné (003) can be used to obtain a near optimal control policy. Te use of simulation is more appropriate in tis context because it provides a tool to understand ow te system beaves by carrying out «wat-if» assessments and to identify wic 9

20 factors are more important for furter more detailed analysis (response surface analysis and optimization). Tis could give te possibility to develop a more general model in reverse logistic network including capacity limits, multi-product management and uncertainty on product demands and returns. 7. Conclusion Te number of applications and scientific publications in te field of reverse logistics as been steadily growing, reflecting te increasing importance of tis subject, mainly as a result of society s accrued environment awareness and searc for economic productivity. However, taking a closer look to te dynamic caracteristic of te production planning problems, one can notice tat reported work is mostly based on non macine-dynamic models and tus, general models are missing, in particular models tat relate reverse and forward cains. In tis paper, te production planning model of a ybrid manufacturing/remanufacturing system in reverse logistics as been proposed in a continuous time stocastic context. We developed te stocastic optimization model of te considered problem wit two decision variables (production rates of manufacturing and remanufacturing macines) and two state variables (stock levels of manufactured and remanufactured products). By controlling bot te manufacturing and remanufacturing rates, we obtained a near optimal control policy of te system troug numerical tecniques yielding straigtforward decision rules. We illustrated and validated te proposed approac using a numerical example and a sensitivity analysis. We finally discuss te extension of te proposed model to te case of a forward-reverse logistics network involving multiple products, multiple macines and uncertainty on product demands and returns. Acknowledgements Te autors want to tank two anonymous referees wose critics and suggestions contributed to significantly increase te quality of tis paper. 0

21 Appendix A Optimality conditions and numerical approac Regarding te optimality principle, we can write te HJB equations as follows: v( x, x, ) min g( x, x, ) ( u dm) vx ( ) ( u dr ) vx( ) ( ) v(, ( x, )) v(, a, x) ( u,, u ) ( ) B (A.) Te optimal control policy ( u ( ), u ( )) denotes a minimiser over ( ) of te rigt and * * side of equation (9). Tis policy corresponds to te value function described by equation (7). Ten, wen te value function is available, an optimal control policy can be obtained as in equation (9). However, an analytical solution of equations (A.) is almost impossible to obtain. Te numerical solution of te HJB equations (9) is a callenge wic was considered insurmountable in te past. Boukas and Haurie (990) sowed tat implementing Kusner s metod can solve suc a problem in te context of production planning. Let us now develop te numerical metods for solving te optimality conditions given by equation (9). Suc metods are based on te Kusner approac as in Kenné et al. (003), Hajji et al. (009) and references terein. Te main idea beind tis approac consists of using an approximation sceme for te gradient of te value function v( x, x, ). Let and denote te lengt of te finite difference interval of te variables x and x respectively. Te value function v( x, x, ) is approximated by v ( x, x, ) and v ( x, x, ) as in equation (0). v (, xi i, ) v ( x, x, ) ( ui di) if ui di 0 i vx ( x, x, ) ( u ) i i di (A.) v ( x, x, ) v (, xi i, ) ( ui d) oterwise x wit i,, d dm and d dr. Wit approximations given by equation (0) and after a couple of manipulations, te HJB equations can be rewritten as follows: v ( x, x, ) min x x x x g( x, x, ) ( u, u ) ( ) ( / ) ( / ) p ( ) v ( x, x, ) r x i p ( ) v ( x,, ) p ( ) v (, x, )

22 were ( ) is te discrete feasible control space or te so-called control grid and te oter terms used in equation (A.3) are defined as follows: p xi u d u d c r ui di if ui di 0 ( ) x 0 oterwise p xi di ui if ui di 0 ( ) i 0 oterwise (A.3) p ( ) Te system of equations (A.3) can be interpreted as te infinite orizon dynamic programming equation of a discrete-time, discrete-state decision process. Noting tat p ( x ) p ( ) ( ) ( ) ( ) x p x p x p, te terms px i ( ), px i ( ) and p ( ), for all, can be considered as transition probabilities for a controlled Markov cain on a discrete state space representing te control grid to be considered for te numerical solution of HJB equations. Te term/( / ), for all B, corresponds to a positive discount factor wic is bounded away from. Te obtained discrete event dynamic programming can be solved using eiter policy improvement or successive approximation metods. In tis paper, we use te policy improvement tecnique to obtain a solution of te approximating optimization problem. Te algoritm of tis tecnique can be found in Kusner and Dupuis (99). Te discrete dynamic programming equation (A.3) yields te following four equations: () (,,) () (,,) g(, x, x, u, u ) px v x x px v x x v ( x, x,) min ( u 3, u) () ( / ) ( / ) p () v ( x, x,) p () v ( x, x,3) (A.4)

23 () (,,) () (,,) g (, x, x, u,0) px v x x px v x x v ( x, x,) min u 4 () ( / ) ( / ) p () v ( x, x,) p () v ( x, x,4) (A.5) (3) (,,3) (3) (,3) g(3, x, x,0, u ) px v x x px v x v ( x, x,3) min u 4 (3) ( / ) ( / ) p (3) v ( x, x,) p (3) v ( x, x,4) (A.6) (4) (,,4) (4) (,,4) g(3, x, x,0,0) px v x x px v x x v ( x, x,4) ( / ) ( / ) p (4) v ( x, x,) p (4) v ( x, x,3) (A.7) 3

24 References Akella, R., Kumar, P. R., Optimal Control of Production Rate in a Failure Prone Manufacturing System, IEEE Trans. on Automatic Control, AC 3, 6-6, 986 Amezquita, T. and Bras, B., Lean remanufacture of an Automobile Clutc, Proceedings of First International Working Seminar on Reuse, Eindoven, Te Neterlands, p 6, 996 Bostel, N., Dejax, P., Lu, Z., Te Design, Planning and Optimization of Reverse Logistic Networks, in Logistics Systems: Design and Optimization, A. Langevin, D. Riopel, editors, Springer, 7-, 005 Boukas, E.K., Haurie, A., Manufacturing Flow Control and Preventive Maintenance: A Stocastic Control Approac, IEEE Trans. on Automatic Control, 33, 04-03, 990 Cung, S. L., Wee, H.M., Po-Cung, Y., Optimal Policy for a Closed-Loop Supply Cain Inventory System Wit Remanufacturing, Matematical and Computer Modelling, 6, , 008 Dobos, I., Optimal Production Inventory Strategies for a HMMS-Type Reverse Logistics System, International Journal of Production Economics, 8-8(), , 003 De Brito, M. P., Dekker, R., Flapper S. D. P., Reverse Logistics - a Review of Case Studies, ERIM Report Series Reference No. ERS LIS, 003 Deayem, N.F.I., Kenné, J.P., Garbi, A., Hierarcical Decision Making in Production and Repair/Replacement Planning wit Imperfect Repairs Under Uncertainties, European Journal of Operational Researc, 98( ), 73-89, 009 Dekker, R., Fleismann, M., Interfut, K., Van Wassenove L., edts., Reverse Logistics. Quantitative models for closed-loops supply cains, 004 Dong-Ping Song., Production and Preventive Maintenance Control in a Stocastic Manufacturing System, International Journal of Production Economics, 9(): 0-, 009 El-Sayed, M., Afia, N., El-Karbotly, A., A Stocastic Model for Forward Reverse Logistics Network Design Under Risk, Computers & Industrial Engineering, In Press, 008 Fleiscmann, M., Bloemof-Ruwaard, J.M., Dekker R., Van der, L.E., Van Nunen, J. A. E. E., Van Wassenove, L.N., Quantitative Models for Reverse Logistics: A review, European Journal of Operational Researc, 03(), -7,997 Garbi, A., Kenné, J.P., Optimal Production Control Problem in Stocastic Multiple- Product Multiple-Macine Manufacturing Systems, IIE transactions, 35, 94-95, 003 Garbi, A., Pellerin, R., Sadr, J., Production Rate Control for Stocastic Remanufacturing Systems, International Journal of Production Economics, (), 37-47, 008 Gerswin, S. B., Manufacturing Systems Engineering, Prentice Hall, 994 Geyer, R., Jackson, T., Supply Loops and Teir Constraints: Te Industrial Ecology of Recycling and Reuse. California Management Review 46 (), 55-73, Winter

25 Guide V.D.R. and Van Vassenove, L.N., Closed-Loop Supply Cains, Feature Issue (Part ), Production and Operations Management, Vol. 5, No 3, 006 Guide V.D.R. and Van Vassenove, L.N., Closed-Loop Supply Cains, Feature Issue (Part ), Production and Operations Management, Vol. 5, No 4, 006 Hajji, A., Garbi, A., Kenné, J. P., Joint Replenisment and Manufacturing Activities Control in Two Stages Unreliable Supply Cain, International Journal of Production Researc, 47(), 33-35, 009 Jorjani, S., Leu, J., Scott, C., Model for Te Allocation of Electronics Components to Reuse Options, International Journal of Production Researc, 4(6), 3-45, 004 Kenné, J.P., Boukas, E. K., Hierarcical Control of Production and Maintenance Rates in Manufacturing Systems Journal of Quality in Maintenance Engineering, 9, 66-8, 003 Kenné, J.P., Boukas, E.K., Garbi, A., Control of Production and Corrective Maintenance Rates in a Multiple-Macine, Multiple-Product Manufacturing System, Matematical and Computer Modelling, 38, , 003 Kibum, K., Iksoo, S. Juyong, K., Bongju, J., Supply Planning Model for Remanufacturing System in Reverse Logistics Environment, Computers & Industrial Engineering, 5( ), 79-87, 006 Kiesmuller, G. P., Scerer, C. W., Computational Issues in a Stocastic Finite Horizon One Product Recovery Inventory Model, European Journal of Operational Researc, 46( 3), , 003 Kusner, H.J., Dupuis, J., Numerical Metods for Stoastic Control Problems in Continuous Time, Springer-Verlag, 99 Lund, R., Te Remanufacturing Industry: Hidden Giant, Boston, Massacusetts: Boston University, 996 Ostlin, J., Sundin, E., Bjorkman, M., Importance of Closed-Loop Supply Cain Relationsips for Product Remanufacturing, International Journal of Production Economics, Volume 5, , 008 Pellerin, R., Sadr, J., Garbi, A., Malamé,R., A Production Rate Control Policy for Stocastic Repair and Remanufacturing Systems, International Journal of Production Economics, In Press, 009 Risel, R., Dynamic Programming and Minimum Principles for Systems Wit Jump Markov Disturbances, SIAM journal on Control, 3, , 975 Rogers, D. S., Tibben-Lembke, R.S., Going backwards: reverse logistics trends and practices, Center for Logistics Management, University of Nevada, Reno, Reverse Logistics Executive Council, 998 Salema, M. I. G., Barbosa-Povoa, Ana Paula, Novais, A. Q., An Optimization Model for te Design of a Capacitated Multi-Product Reverse Logistics Network Wit Uncertainty, European Journal of Operational Researc, 79(3), , 007 5