Optimal control of a remanufacturing system

Transcription

1 int. j. prod. res., 1 september 2004, vol. 42, no. 17, Optimal control of a remanufacturing system K. NAKASHIMAy*, H. ARIMITSUy, T. NOSEy and S. KURIYAMAz An optimal control problem of a remanufacturing system under stochastic demand is studied. The system is formulated by a Markov decision process, which is a class of stochastic sequential processes in which the reward and transition probability depend only on the current state of the system and the current action. The models have gained recognition in such diverse fields as engineering, economics, communications, etc. Each model consists of states, actions, rewards and transition probabilities. The paper considers two types of inventories: the actual product inventory in a factory and the virtual inventory used by a customer. The state of the remanufacturing system is defined by both inventory levels. One can obtain the optimal production policy that minimizes the expected average cost per period. The paper also considers some scenarios under various conditions and shows the example of controlling the remanufacturing system. 1. Introduction The escalating growth in consumer waste in recent years has started to threaten the environment. Currently, product recovery is practised in part because of the escalating deterioration of the environment and in part because of profit motives. Product recovery aims to minimize the amount of waste sent to landfill sites by recovering materials and parts from old or outdated products by means of recycling and remanufacturing. Product recovery includes collection, disassembly, cleaning, sorting, repairing, reconditioning, reassembling and testing (Gupta and Taleb 1991, Brennan et al. 1994). The present paper deals with an optimal control problem of a remanufacturing system under stochastic variability such as demand. The system is formulated into a Markov decision process (MDP) (Howard 1960, Puterman 1994). The MDP is a class of stochastic sequential processes in which the reward and transition probability depends only on the current state of the system and the current action. MDP models have gained recognition in such diverse fields as economics, communications, transportation, etc. Each model consists of states, actions, rewards and transition probability. In the engineering field, for example, the approach is used for controlling the production system (Ohno and Ichiki 1987, Ohno and Nakashima 1995). Choosing an action as a production quantity in a state generates rewards and/or costs and determines the state at the next decision epoch through a transition Revision received May ydepartment of Industrial Management, Osaka Institute of Technology, Omiya, Asahi-ku, Osaka, Japan. zsetsunan University, Faculty of Business Administration and Information, 17-8, Ikedanakamachi, Neyagawa, Osaka, Japan. *To whom correspondence should be addressed. nakasima@dim.oit.ac.jp International Journal of Production Research ISSN print/issn X online # 2004 Taylor & Francis Ltd DOI: /

2 3620 K. Nakashima et al. probability function. One can then obtain the optimal production policy that minimizes the expected average cost per period in the optimal production control problem. The paper considers two types of inventories: the actual product inventory in a factory and the virtual inventory used by customers. It defines the state of the remanufacturing system by considering both inventory levels. It then obtains the optimal production policy that minimizes the expected average cost per period. The paper also considers some scenarios under various conditions and optimizes the remanufacturing system. Section 2 briefly summarizes remanufacturing environments and the relevant literature. Section 3 considers a single-item remanufacturing system under stochastic demand. The system is formulated into an undiscounted MDP to determine the optimal control policy that minimizes the expected average cost per period. Finally, the numerical results of controlling the remanufacturing system under various conditions are shown. 2. Remanufacturing environments and a literature review The paper focuses on the operational aspect of product recovery in the remanufacturing environment with stochastic variability. Remanufacturing systems have to make operational, manufacturing, inventory, distribution and marketing-related decisions (Stock 1992, Kopicky et al. 1993). In general, the existing methods for traditional production systems cannot be used for remanufacturing systems directly. Remanufacturing environments are characterized by their highly flexible structures. Flexibility is required to handle the uncertainties, e.g. collection and discard, that are likely to arise. In addition, note that the products used by consumers will be collected and recovered and used as parts for remanufacuring. Gungor and Gupta (1999) and Moyer and Gupta (1997) review the literature in the area of environmentally conscious manufacturing and product recovery. They summarize much of the area including industrial examples, modelling and solutions. Minner (2001) notes there are the two well-known streams in product recovery research area: stochastic inventory control (SIC) and material requirement planning (MRP). The present paper restricts itself to SIC. As for the periodic review models, Cohen et al. (1980) develop the product recovery model in which the collected products are used directly. Inderfurth (1997) discusses the effect of non-zero lead times for orders and recovery in the different model. As for continuous review models, Muckstadt and Isaac (1981) deal with a model for a remanufacturing system with non-zero lead times and a control policy with the traditional (Q, r) rule. Van der Laan and Salomon (1997) suggest push and pull strategies for the remanufacturing system. Guide and Gupta (1999) present a queuing model to study a remanufacturing system. Kiesmuller (2003) discusses optimal policy for a one-product recovery system with lead times. The policy was composed of the optimal manufacturing rate, remanufacturing rate and disposal rate over a finite-planning horizon. All the above studies, however, considered demand and procurement as independent being in the systems. Nakashima et al. (2002) deal with a product recovery system with a single class of product life cycle. They propose a new analytical approach to evaluate the system using a Markov chain and give numerical examples for various conditions.

3 Optimal control of a remanufacturing system Optimization of the system We formulate a remanufacturing system with stochastic variability such as demand using a discrete time Markov decision model. Consider a single process that produces a single-item product. The finished products are stocked in the factory and faced according to customer demand. Traditional inventory management focuses on only the inventory in the factory. The remanufacturing system, however, should focus on the outdated products collected from customers. That is, the remanufacturing producers have to consider the products in use as part of the future inventory. The products used by consumers are taken here to be the virtual inventory. It is important to manage the virtual inventory until products are collected and used in remanufacturing as well as controlling the inventory on hand The model Figure 1 shows a remanufacturing production system. Remanufacturing preserves the product s or the part s identity, and performs the required disassembly and refurbishing operations to bring the product to a desired level of quality with remanufacturing cost. On the other hand, we define normal manufacturing as producing the products using new resources. The number of products by normal manufacturing at period t, P(t), is chosen as an action, k, i.e. k ¼ P(t). Products are produced by normal manufacturing and/or remanufacturing with the parts taken back from customers. All production begins at the start of the period and all products are completed by the end of the period. All the products bought by consumers are new. It is assumed that the number of finished products and that of the products bought by consumers are I(t) and J(t), respectively. If the backlog occurs, I(t) is negative. Demand in successive periods, D(t), is composed of independent random variables with a known identical distribution. When sold, products are remanufactured at the remanufacturing rate, l, with the remanufacturing cost including the collecting cost, and products in use are discarded at the discarded rate,, with an out-of-date cost. It is supposed that l þ Formulation The state of the system is denoted by: sðtþ ¼ðIðtÞ, JðtÞÞ: ð1þ P(t) Inventory on hand I(t) Factory λ o oo D(t) Customer Virtual Inventory J(t) µ Figure 1. Remanufacturing system.

4 3622 K. Nakashima et al. The transition of the each inventory is given by: Iðt þ 1Þ ¼IðtÞþkþlJðtÞ DðtÞ Jðt þ 1Þ ¼JðtÞ ljðtþ JðtÞþDðtÞ ð2þ ð3þ The action space in s(t), K(s(t)) is defined by: KðsðtÞÞ ¼ f0,..., maxf0, Imax IðtÞ ljðtþgg: The transition probability is defined as: 8 >< PrfDðtÞ ¼dg if sðt þ 1Þ ¼ðIðtÞþkþlJðtÞ d, P sðtþsðtþ1þ ¼ JðtÞ ljðtþ JðtÞþdÞ, >: 0 otherwise: The expected cost per period in state s(t) under k, r(s(t),k) is given by: where C H C N C R C B C O rðsðtþ, kþ ¼C N k þ C R ljðtþþc H ½IðtÞŠ þ þ C B ½ IðtÞŠ þ þ C O JðtÞ, holding cost per unit, normal manufacturing cost of a new product, remanufacturing cost of a product, backlog cost per unit, out-of-date cost per unit. S and S are a set of all possible states and the total number of states, respectively. Let us number the state, s n as s (¼ 1,..., S ). An undiscounted MDP that minimizes the expected average cost per period, g, is formulated as the following optimality equation: ( ) g þ v s ¼ min k2kðsþ rðs, kþþ X s 0 2S P ss 0ðkÞv s 0 ð4þ ð5þ ð6þ ðs 2 SÞ; ð7þ where v s is the relative value when the production system starts from state s (Howard 1960). An optimal production policy is determined as a set of k that minimizes the right-hand side of equation (7) for each state s using a policy iteration method (Howard 1960, Puterman 1994). 4. Numerical results This section shows the numerical example of controlling the remanufacturing system described in section 3. It is assumed that the maximum number of products inventory, Imax, is 5 and that of backlog demand, Imin, is set as 5. The maximum number of the virtual inventory is 10. The cost parameters are as follows: C H ¼ 1, C N ¼ 2, C R ¼ 3, C B ¼ 10 and C O ¼ 10: The distribution of the demand is given by: Pr D n ¼ D 1 2 Q þ j ¼ Q 1 Q, ð0 j QÞ, j 2 where D ¼ 2 and Q is an even number and variance ( 2 )isq/4. ð8þ

5 Optimal control of a remanufacturing system Optimal control policy One can obtain an optimal control policy that minimizes the expected average cost per period using a policy iteration method. It is assumed that the remanufacturing rate, l, is 0.2 and the discarded rate,, is 0.5. Table 1 shows the optimal control policy in case variance of demand is 0.5. Note that the optimal number of normal production is restricted to the number of remanufacturing production. The minimum expected cost per period, g, is 11.5 under this optimal control policy Effect of the remanufacturing rate and of the variance of the demand Figure 2 shows the behaviour of the minimum costs under two types of variance, i.e. 2 ¼ 0.5 and 1.0. As the remanufacturing rate increases, the minimum cost State (I, J ) Action, k (I, J ) k (I, J ) (I, J ) k (I, J ) k (I, J ) k (I, J ) k ( 5, 0) 7 ( 4, 7) 2 ( 2, 3) 3 ( 1, 10) 0 (1, 6) 0 (3, 2) 0 (4, 9) 0 ( 5, 1) 6 ( 4, 8) 2 ( 2, 4) 3 (0, 0) 3 (1, 7) 0 (3, 3) 0 (4, 10) 0 ( 5, 2) 6 ( 4, 9) 1 ( 2, 5) 2 (0, 1) 2 (1, 8) 0 (3, 4) 0 (5, 0) 0 ( 5, 3) 5 ( 4, 10) 1 ( 2, 6) 1 (0, 2) 2 (1, 9) 0 (3, 5) 0 (5, 1) 0 ( 5, 4) 5 ( 3, 0) 6 ( 2, 7) 0 (0, 3) 1 (1, 10) 0 (3, 6) 0 (5, 2) 0 ( 5, 5) 4 ( 3, 1) 5 ( 2, 8) 0 (0, 4) 1 (2, 0) 1 (3, 7) 0 (5, 3) 0 ( 5, 6) 4 ( 3, 2) 4 ( 2, 9) 0 (0, 5) 0 (2, 1) 0 (3, 8) 0 (5, 4) 0 ( 5, 7) 3 ( 3, 3) 4 ( 2, 10) 0 (0, 6) 0 (2, 2) 0 (3, 9) 0 (5, 5) 0 ( 5, 8) 3 ( 3, 4) 3 ( 1, 0) 4 (0, 7) 0 (2, 3) 0 (3, 10) 0 (5, 6) 0 ( 5, 9) 2 ( 3, 5) 2 ( 1, 1) 3 (0, 8) 0 (2, 4) 0 (4, 0) 0 (5, 7) 0 ( 5, 10) 2 ( 3, 6) 2 ( 1, 2) 3 (0, 9) 0 (2, 5) 0 (4, 1) 0 (5, 8) 0 ( 4, 0) 6 ( 3, 7) 1 ( 1, 3) 2 (0, 10) 0 (2, 6) 0 (4, 2) 0 (5, 9) 0 ( 4, 1) 5 ( 3, 8) 1 ( 1, 4) 2 (1, 0) 2 (2, 7) 0 (4, 3) 0 (5, 10) 0 ( 4, 2) 5 ( 3, 9) 0 ( 1, 5) 1 (1, 1) 1 (2, 8) 0 (4, 4) 0 ( 4, 3) 4 ( 3, 10) 0 ( 1, 6) 1 (1, 2) 1 (2, 9) 0 (4, 5) 0 ( 4, 4) 4 ( 2, 0) 5 ( 1, 7) 0 (1, 3) 0 (2, 10) 0 (4, 6) 0 ( 4, 5) 3 ( 2, 1) 4 ( 1, 8) 0 (1, 4) 0 (3, 0) 0 (4, 7) 0 ( 4, 6) 3 ( 2, 2) 4 ( 1, 9) 0 (1, 5) 0 (3, 1) 0 (4, 8) 0 Table 1. Optimal control policy Variance=0.5 Variance=1.0 Minimum cost Remanufacturing rate 9 Figure 2. Behaviour of minimum cost.

6 3624 K. Nakashima et al. decreases in each variance case. All the minimum costs in the case 2 ¼ 0.5 are less than those in the case 2 ¼ 1.0. This also illustrates the importance of the smoothing production. 5. Conclusions This paper has dealt with the optimal control problem of the remanufacturing system under stochastic variability. It considered two types of inventories: the actual product inventory in the factory and the virtual inventory used by consumers. The system was formulated into the undiscounted MDP. It obtained the optimal production policy that minimized the expected average cost per period using the policy iteration method. Finally, it showed the examples of controlling the remanufacturing system optimally under various conditions. Numerical results illustrated the property of the optimal control of the remanufacturing system. This means that the proposed approach is applicable to various systems by choosing the parameters according to their conditions. For future research, we suggest that the product lifetime should be considered to model the remanufacturing systems more precisely. Acknowledgement The authors thank anonymous referees for valuable comments and helpful suggestions. References BRENNAN, L., GUPTA, S. M. and TALEB, K. N., 1994, Operations planning issues in an assembly/disassembly environment. International Journal of Operations and Production Management, 14, COHEN, M. A., NAHMIAS, S. and PIERSKALLA, W. P., 1980, A dynamic inventory system with recycling. Naval Research Logistics Quarterly, 27, GUIDE, JR, V. D. R. and GUPTA, S. M., 1999, A queueing network model for remanufacturing production systems. In Proceedings of the Second International Seminar on Reuse, Eindhoven, the Netherlands, 1 3 March, pp GUNGOR, A. and GUPTA, S. M., 1999, Issues in environmentally conscious manufacturing and product recovery: a survey. Computers and Industrial Engineering, 36, GUPTA, S. M. and TALEB, K. N., 1994, Scheduling disassembly. International Journal of Production Research, 32, HOWARD, R. A., 1960, Dynamic Programming and Markov Processes (Cambridge, MA: MIT Press). INDERFURTH, K., 1997, Simple optimal replenishment and disposal policies for a product recovery system with lead-times. OR Spektrum, 19, KIESMULLER, G. P., 2003, Optimal control of a product recovery system with lead time. International Journal of Production Economics, 81 82, KOPICKY, R. J., BERG, M. J., LEGG, L., DASAPPA, V. and MAGGIONI, C., 1993, Reuse and Recycling: Reverse Logistics Opportunities (Oak Brook: Council of Logistics Management). MINNER, S., 2001, Strategic safety stocks in reverse logistics supply chains. International Journal of Production economics, 71, MOYER, L. and GUPTA, S. M., 1997, Environmental concerns and recycling/disassembly efforts in the electronics industry. Journal of Electronics Manufacturing, 7, MUCKSTADT, J. A. and ISAAC, M. H., 1981, An analysis of single item inventory systems with returns. Naval Research Logistics Quarterly, 28, NAKASHIMA, K., ARIMITSU, H., NOSE, T. and KURIYAMA, S., 2002, Analysis of a product recovery system. International Journal of Production Research, 40, OHNO, K. and ICHIKI, K., 1987, Computing optimal policies for controlled tandem queueing systems. Operations Research, 35,

7 Optimal control of a remanufacturing system 3625 OHNO, K. and NAKASHIMA, K., 1995, Optimality of a Just-in-Time production system. In Proceedings of APORS 94 (Singapore: World Scientific), pp PUTERMAN, M. L., 1994, Markov Decision Processes (New York: Wiley). STOCK, J. R., 1992, Reverse Logistics (Oak Brook: Council of Logistics Management). VAN DER LAAN, E. A. and SALOMON, M., 1997, Production planning and inventory control with remanufacturing and disposal. European Journal of Operational Research, 102,