A DEEP Q-LEARNING NETWORK FOR SHIP STOWAGE PLANNING PROBLEM

Transcription

1 POLISH MARITIME RESEARCH Specal Issue 2017 S3 (95) 2017 Vol. 24; pp /pomr A DEEP Q-LEARNING NETWORK FOR SHIP STOWAGE PLANNING PROBLEM Yfan Shen 1 Nng Zhao 2* Mengjue Xa 1 Xueqang Du 2 1 Scentfc Research Academy, Shangha Martme Unversty, Chna 2 Logstcs Engneerng College, Shangha Martme Unversty, Chna * correspondng author ABSTRACT Shp stowage plan s the management connecton of quae crane schedulng and yard crane schedulng. The qualty of shp stowage plan affects the productvty greatly. Prevous studes manly focuses on solvng stowage plannng problem wth onlne searchng algorthm, effcency of whch s sgnfcantly affected by case sze. In ths study, a Deep Q-Learnng Network (DQN) s proposed to solve shp stowage plannng problem. Wth DQN, massve calculaton and tranng s done n pre-tranng stage, whle n applcaton stage stowage plan can be made n seconds. To formulate network nput, decson factors are analyzed to compose feature vector of stowage plan. States subject to constrants, avalable acton and reward functon of Q-value are desgned. Wth these nformaton and desgn, an 8-layer DQN s formulated wth an evaluaton functon of mean square error s composed to learn stowage plannng. At the end of ths study, several producton cases are solved wth proposed DQN to valdate the effectveness and generalzaton ablty. Result shows a good avalablty of DQN to solve shp stowage plannng problem. Keywords: Deep Q-Leanng Network (DQN); Contaner termnal; Shp stowage plan; Markov decson process; Value functon approxmaton; Generalzaton INTRODUCTION In Recent years more and more researchers devoted themselves to the study of marne scence, port logstcs and so on[1-3], cause ocean s one of the mportant resources for human bengs. Especally n ports, researchers have made great contrbutons to contaner termnal equpment [4-7] and plannng [8-10] ntellgence. In contaner termnals, stowage plan s one of the most mportant and tme consumng plannng phase. In present tme, stowage plannng s manly made by hand wth computer assstance. Such manual plannng management mode reles heavly on experence of planners, whch costs labor and tme. Wth automaton currency n contaner termnal, the manual plannng hnders management automaton process. At the same tme, contaner shps has been larger and larger n recent tme, whch ncreases plannng labor and tme consumpton. Under such crcumstances, stowage plannng automaton or ntellgent stowage plannng has been a crtcal technology to be broke through n contaner termnal management to optmze both cost and effcency. Prevous studes of contaner shp stowage plannng focus on stowage plannng model and algorthms to solve ths problem. In terms of stowage plannng model, Master Bay Plan and In-Bay Plan have been the manstream. Among Master Bay Plan, Todd D S and Sen P [11] propose a Master Bay Plan model mnmzng reshufflng, wth trmmng moment, heelng moment, shp stablty and poston as constrants. A GA s desgned to solve ths problem. Zhao N and M W J [12] made a mult-objectve mxed nteger programmng model wth shp stablty factors and operaton factors as constrants to optmze reshuffles and yard crane effcency. 102 POLISH MARITIME RESEARCH, No S3/2017 Unauthentcated

2 The proposed MIP model can only solve small scale problems wth tradtonal plannng solver. Moura A and Olvera J et al [13] proposed a MIP model optmzng total transportaton cost wth shppng lne n consderaton. Amone In-Bay Plan, Avrel M and Penn M [14] proposed a MIP model mnmzng reshuffles. Proposed algorthm can solve small scale problems. J.J.Shelds [15] made a comparson between model solvng outputs and actual loadng outputs to valdate proposed model. Ima et al [16-18] proposed mult-objectve MIP model mnmzng reshuffles. Numercal experments reveals more bnary varables and bnary constrants would sgnfcantly ncrease complexty and sgnfcantly lower solvng effcency. Haghan and Kasar et al [19] proposed a MIP model wth turnaround tme and shp parameters as constrants. In terms of stowage plannng algorthm, most researches prefer ntellgent optmzaton algorthms. Álvarez et al [20] proposed a tabu-search algorthm wth multple ntal solutons to solve the problem optmzng movng dstance of stackers, shuffles and contaner weght dstrbuton n shp. Numercal experments show that proposed tabu-search algorthm can solve cases wth more than 100 contaners n short tme whle MIP solvers cannot solve cases wth more than 40 contaners. Km et al [21-24] proposed beam-search algorthm to solve stowage plannng problem. Y.Lee et al [25] decomposes stowage plannng problem nto smaller scale sub-problems usng herarchy theory to solve stowage plannng problem. An Ant Colony Optmzaton-Tabu Search hybrd algorthm s proposed, and numercal experment shows superorty of proposed hybrd algorthm over orgnal ndependent algorthms. These analyses shows that stowage plannng studes at the moment concentrate on composng a MIP model and desgn a heurstc algorthm to solve the model. Such method performs well n small scale cases whle has lmtatons such as poor performance n large scale cases and weak ablty of generalzaton. Thus n ths paper a deep renforcement learnng algorthm s proposed to solve stowage plannng problem. Intellgent agent of stowage plannng s traned to solve stowage plannng problem effcently and mantan better generalzaton. before land sde slots. Ths sequence relatonshp between slot locatons vares between Deck Stowage Plan and Hold Stowage Plan. Deck has more constrants to ensure shp stablty and operaton safety. 2. Shp slot weght lmt factor Before In-Bay Plannng, Master Bay Plan has preplanned allocaton to suggest a weght lmt for each shp slot to guarantee shp stablty and securng capacty. Thus each slot has a weght range constrant. 3. Heavy-over-lght lmt factor Theoretcally, heavy contaners should be loaded under lght contaners to ensure shp stablty. Whle n actual plannng, heavy contaners are allowed to load over lght contaners f these contaners weghs close. Thus, a heavyover-lght lmt factor s appled to formulate ths constrant. OPTIMIZATION OBJECTIVES 1. Starcase shape sequencng n deck stowage plannng In terms of loadng sequence, contaners should better be loaded n star shape, whch means avod nsert a contaner between contaners to mprove loadng operaton effcency and safety. 2. Mnmzng reshuffles When a contaner needs to be loaded before contaners over tself n contaner yard, reshuffle s needed. Reshuffles caused by stowage plan should be mnmzed durng plannng to mprove loadng effcency. 3. Mnmzng yard crane shfts When contaners wth adjacent planned loadng sequence locates n dfferent yard bays or even n dfferent yard blocks, yard crane needs to shft from one bay to another to load these two contaners. Unreasonable plan causes yard crane to shft back and forth to pck contaners, whch affects loadng effcency. Thus, yard crane shfts should be mnmzed to mprove loadng effcency. CONTAINER SHIP STOWAGE PLANNING PROBLEM DECISION FACTORS In stowage plannng process, several factors needs to be consdered to ensure seaworthness of contaner shp and mprove operaton effcency. 1. Shp slot locaton and sequence relatonshp factor To ensure effcency durng shp loadng process, shp slots has relatve loadng sequence relatonshp such as slot 8401 can only be loaded when the slot rght under slot 8401 whch s 8201, and relatve sea sde slots should better be loaded DEEP REINFORCEMENT LEARNING ALGORITHM FOR STOWAGE PLANNING PROBLEM MARKOV DECISION PROCESS L. S. Shapley frst proposes Markov Decson Processes (MDP) n stochastc games research. R.Bellman then proposes dynamc programmng method to solve general sequencal problem. R.A.Howard and D.Blackwell proposed general theoretcal framework and effectve method for MDP. A MDP s a 5-tuple ( st, at, t rtπ,, ), where s t s a fnte set of states, a t s a fnte set of actons avalable from state s t, POLISH MARITIME RESEARCH, No S3/ Unauthentcated

3 r t s mmedate reward (or expected mmedate reward), T s the transt functon from state s t to s t + 1, π s the strategy or polcy The problem of MDP s to fnd a polcy π that specfes actons that the decson maker wll choose when n state s t. MDP FOR STOWAGE PROBLEM MDP for stowage plannng problem s formulated accordng to bass of MDP. Fg. 2 shows a example of stowage plannng MDP. -500, f stowage plan s avalable r = 500, f stowage plan s not avalable (1) 1 0, else r2 = 10 * φ(5) (2) r3 = -30* φ(9) (3) Formula (1) s the reward of avalablty, (2) s the reward of reshufflng, (3) s the reward of yard crane shftng. 4. Stowage Plannng Evaluaton Functon and Acton Evaluaton Functon Fg. 1. Slot Scheme. π t 1 λ t 1 t (4) s ' π v ( s) = T( s' sa, )[ r ( s' sa, ) + v ( s ) s = s] t 1 λ t 1 (5) s' π Q ( sa, ) = Ts ( ' sa, )[ r ( s' sa, ) + vs ( )] ( ) ( ', )[ ( ', ) ( ')] (6) * * V s = T s sa Rs sa + λv s a s ' Fg. 2. MDP for Stowage Plannng Problem 1. Stowage State In stowage plannng, t s stowage sequence, t=0 s the ntal state when no contaner s loaded, t=1 s the next state when the frst contaner s loaded, t=2 s the state when the second contaner s loaded, and so on. In Fg. 2, S 0 s the ntal state when the whole bay s empty, S 1 s the next state when the frst contaner C1 s stowed, and then S 2. As s shown, when n S 0, there are several avalable actons, whch s to stow whch contaner n whch avalable slot. S1 ( C2, M 1) means stow C2 nto M1 slot n state S 1, S2( C6, M2 S1( C2, M 1)) means two contaners are stowed, frst stow C2 nto M1 slot and then stow C6 nto M2 n state S Stowage Acton In stowage plannng, an acton s to mate a contaner wth a slot, whch means stow ths contaner n ths slot. In dfferent state, avalable actons are dfferent. In Fg. 2, when n S 0, f stowage constrants are gnored, there are 36 avalable actons or mate of contaners to slots. 3. Stowage Reward In renforcement learnng, reward represents the envronment. In stowage plannng learnng, reward manly expresses objectves and constrants. Snce the result of a stowage plan s evaluated by avalablty, reshufflng, yard crane shftng, and these evaluatons have dfferent scales of mportance, the stowage reward s as follow. (, ) ( ', )[ ( ', ) ( ')] (7) * * Q sa = T s sa Rs sa + λv s s ' π s the stowage strategy or polcy, λ s the dscount factor, whch represents the nfluence of next stowage move to ths move, T( s' sa, ) s the probablty of takng acton a to get state s ' n state s, Rs ( ' sa, ) s the reward of takng acton a to get state s ' n state s, vs () s expected reward of takng varous actons n state s, or expected reward of stowng other contaners after state s, Qsa (, ) s the total reward of takng acton a n state s, V * () s s the mum reward n state s, Q * (, sa ) s the mum reward of takng acton a n state s. STOWAGE PLANNING FEATURES The dmensons of dfferent shp bays are usually dfferent n stowage plannng, and renforcement learnng needs a tranng set wth same dmensons. Thus, stowage features are ntroduced to approxmate dfferent stowage states. In ths research, 9 features are selected as the feature vector of a stowage state, Φ= { φ(1), φ(2), φ(8), φ(9)} φ (1) = W / W (8) φ (2) = T / T (9) 104 POLISH MARITIME RESEARCH, No S3/2017 Unauthentcated

4 () Wj- W / W, f T > 1 φ (3) = () W - W / W, f T = 1 (10) rewards to mze the fnal reward and make the polcy better. φ (4) = P S (11) φ (5) = F / F (12) φ (6) = X X / X (13) j Fg. 3. Framework of Renforcement Learnng φ (7) = X X / X (14) k φ (8) = ( G G ) + ( G G ) / 2G (15) j k The dfference between Deep Q-Learnng and Q-Learnng s that the look up table s replaced by deep neuron network to update Qsa (, ), whch enables effectveness n super large state space scale. And the deep neuron network can be traned wth mnmzng lost functon L( w ) whch updates n each teraton. 1, f ths contaner s located n the same yard bay wth the φ (9) = prevous one 0, else (16) (8) Represents the normalzed weght of selected contaner. (9) Represents the normalzed ter number of selected slot. (10) Represents the normalzed weght gap between selected contaner and the contaner located rght under t on the shp. (11) Represents the potental of selected match of contaner and slot (or acton), whch means number of remanng lghter contaner mnus avalable shp slots above selected slot, or expresson of nfluence of selected acton to later stowage plannng. (12) Represents normalzed reshuffles caused by ths acton. (13) Represents normalzed sequental gap between selected contaner wth contaners located left of selected. (14) Represents normalzed sequental gap between selected contaners wth contaners located rght under selected contaner. (15) Represents normalzed sum of sequental gap between selected contaner wth contaners located left of selected and sequental gap between selected contaners wth contaners located rght under selected contaner. (16) Represents whether ths contaner locates n the same yard bay wth the prevous one. DEEP REINFOREMENT LEARNING ALORITHM FOR STOWAGE PLANNING PROBLEM Fgure 3 shows the framework of renforcement learnng or Q-Learnng for stowage plan. In the ntal state of learnng, the ntellgent agent s lke a naïve planner, every acton the planner take wll have a reward to update Qsa, (, ) and the agent wll decde next acton for next state dependng on updated Qsa, (, ) ths s the teraton of renforcement learnng. Actually, the agent learns from teratons of attempts and L w r Qs a w Qsaw 2 ( ) =Ε [( + γ ( ', '; ) (, ; )) ] a ' Target (17) s ' s the next state, and a ' s the next acton. The partal n the w drecton s n (18). L( w) =Ε [( r+γ Qs ( ', a'; w) w sars,,, ' a ' Qsaw (, ; )) Qsaw (, ; )] w (18) Stochastc Gradent Descent s used to optmze the lost functon, and the weght updates after every teraton, whch s qute smlar to tradtonal Q-Learnng algorthm. In order to approxmate reward for new states that never appeared before, a evaluaton functon approxmaton functon s ntroduced to mprove generalzaton ablty. Unlke supervsed learnng, renforcement learnng doesn t have known tags for tranng, tags are obtaned through teratons. Whle a state and an acton s updated, the change of weght for ths match can affect other matches, whch causes neffectveness of prevous state and acton matches, and then t causes longer tranng tme or even falure of tranng. Thus, an experence replay method s ntroduced to prevent neffectveness. Experence replay stores the experence of tme t as ( φ, a, r, φ + ) n experence hstory queue D, and then D s t t t t 1 stochastcally sampled as ( φj, aj, rj, φ j + 1) to do mn-batch to update the weght. Ths ensures every hstory ponts are consdered when updatng a new data pont. Experence replay stores all prevous states and acton n a sequence to mnmze objectve functon when Q-Functon updates. L( w) =Ε [( r+γ Qs ( ', bw ; ) ( sars,,, ')~ U( D) b Qsaw 2 (, ; )) ] (19) POLISH MARITIME RESEARCH, No S3/ Unauthentcated

5 D represents a sequence of prevous states and actons, U( D ) s a unform dstrbuton among experence sequence D. Experence replay based upon unform dstrbuton lowered data dependency to mprove learnng robustness. The deep network for the stowage plannng problem s desgned as follows. 1. Input layer and output layer For stowage plannng problem, the nput layer s a matrx of feature vector of stowage samples, the output layer s the approxmate Q-value. Thus, the number of nodes n nput layer s 9, the number of nodes n output layer s Number of hdden layers Generally, more hdden layer makes hgher precson of approxmaton, whle more hdden layer costs more tranng and greater probablty of over-fttng. In ths case, 9 hdden layer s accepted. Tab. 1. DQN Algorthm for Stowage Plannng DQN Tranng Algorthm for Stowage Plannng Intalze experence hstory queue D wth length N Intalze Qsaw (, ; ) wth random weght w0 For each stowage epsode loop: Intalze observaton sequence s1 = { x1} and feature sequence φ1 = φ( s1) For each step n an epsode loop: Select an acton to perform n state s wth ε ( soft) greedy Update reward and extract feature φ1 = φ( s1) Save experence tuple ( φt, at, rt, φ t + 1) nto experence hstory queue D Collect samples ( φj, aj, rj, φ j + 1) wth sze of random samplng mn-batch Transform sample ( φj, aj, rj, φ j + 1) nto tranng tuple ( xk, yk) xk = φj, yk = r + λ a' Q ( φ j + 1, a', w 1) Update network weghts of tranng set {( xk, y k)} m accordng to wl( ) w wth stochastc gradent descent Loop untl end of states s Loop untl end of epsodes 3. Number of nodes n hdden layers To avod over-fttng and mantan better generalzaton ablty, the number of nodes n hdden layer should be mnmzed when the precson s assured. Number of nodes n hdden layer s related to number of nodes n nput layer, number of nodes n output layer, complexty of learnng problem, transton functon and sample data. Too few nodes causes poor tranng performance, and too many nodes causes less system error but may cause over-fttng. 4. Actvaton functon There are three wdely used actvaton functons, TanH, Sgmod and Relu ( Rectfed Lnear Unts). Relu has better tranng performance especally n attenuaton of gradent and network sparsely. Thus, Relu s used as the actvaton functon of ths research. Relu : f(x)= (0, x ) (20) The desgned deep neuron network for stowage plannng problem s shown n Fgure 4. Accordng to deep network desgn, DQN tranng algorthm s desgned, pseudo code for DQN Algorthm for Stowage Plannng s shown n Table 1, and flowchart n Fgure 5. Fgure 4. Deep Neuron Network for Stowage Plannng Problem Fg. 5. Flowchart for DQN Algorthm for Stowage Plannng 106 POLISH MARITIME RESEARCH, No S3/2017 Unauthentcated

6 STOWAGE CASE STUDY OF DQN STOWAGE PLANNING CASE DESCRIPTION In ths case, producton data of Nngbo Port s used to verfy proposed method. Selected shp bay has 19 slots, 19 correspondng contaners locate n 4 yard bays n 2 blocks. Shp bay s shown n Fgure 6, ths bay has 4 ters and 5 rows, each weght box s a slot to be stowed. Contaner dstrbuton n yard s shown n Fgure 7. Number nsde each box n Fgure 7 s the weght of each contaner. Parameter setup for stowage plannng s shown n Table 2 and parameter setup for DQN learnng algorthm s n Table 3. Random exploraton rate ε ndcates that n the ntal state of teratons, the random exploraton rate equals to 1 to mprove exploraton. After each 1% of total teratons, the exploraton rate decrease by a step of 0.09 to reach 0.1 when teratons fnsh. Wth ths descendng, the agent can focus on optmzed soluton gradually to converge whle keepng a moderate ablty of exploraton. STOWAGE RESULT ANALYSIS The proposed DQN s traned wth producton data for teratons, whch costs 2 hours and 46 mns. The traned DQN can fnsh the test case n seconds, and the stowage result of the test case s as n Fgure 8. The upper left fgure shows the weght dstrbuton of stowage, the upper rght fgure shows the sequence of stowage. The boxes are flled wth dfferent colors to dstngush ts orgnal yard bay. In ths stowage plan, 1 reshuffle and 3 shfts are needed to fnsh loadng of ths shp bay, of whch 3 shfts are necessary (because there are 4 yard bays n total). The reshuffle of contaner wth sequence 18 s unnecessary, but t s stll a good soluton. Wth all that, the effectveness of DQN traned wth producton data s valdated. Fg. 8. Stowage result of test case GENERALIZATION ABILITY ANALYSIS Fg. 6. Shp Bay Layout Fg. 7. Contaner Dstrbuton Tab. 2. Parameter setup for stowage plannng Heavy-over-lght lmt factor δ Reshuffle weght w1 Yard crane shft weght w2 0.5 t 3 1 Tab. 3. Parameter Setup for DQN Learnng Algorthm Learnng rato α Dscount factor λ 1* Random exploraton rate update nternal Experence replay depth Random exploraton rate ε 1 to 0.1 wth step of Number of nodes n hdden layers 1% Generalzaton of Data wth Same Sze To verfy the generalzaton of same sze data, another stowage case wth 19 contaners s ntroduced. Ths case (case. 2) comes from a dfferent shp of same port. Stowage result has 1 reshuffle and 4 shfts, 4 shfts of wtch are all necessary. Wth comparson wth port planners stowage results, the stowage plan of DQN shows a good performance. Manual plan costs 121 seconds on average, whle DQN can complete the calculaton n 0.073s. Ths case study shows a good result n terms of generalzaton of same sze data. 2 Generalzaton of Data wth Dfferent Sze To verfy the generalzaton of dfferent sze data, a stowage case wth 40 contaners s ntroduced. Ths case (case. 3) has a bg dfference wth the prevous one both n case sze and contaner dstrbuton. Result of DQN of ths case shows some heavy-over-lght contaners, whle the weght gaps are all n the heavy-over-lght lmt. The result has 12 reshuffles and 11 shfts, 4 shfts are unnecessary. For the complexty of ths case, port planners show varetes n ther plans, wth an average of 10.2 reshuffles and 9.6 shfts. Port planners takes 237s to make the plan whle DQN costs 0.131s. Thus, DQN shows comparable ablty n ths case wth human compettors wth much better tme consumpton. Ths case study shows a good result n terms of generalzaton of dfferent sze data. POLISH MARITIME RESEARCH, No S3/ Unauthentcated

7 ROBUSTNESS ANALYSIS In stowage plannng DQN learnng, robustness of algorthm refers to whether the tranng algorthm can get good DQN wth varous stowage plannng cases. In generalzaton analyss part, DQN traned wth case. 1 s used to plan case. 2 and case. 3. To verfy the robustness of proposed algorthm, case. 2 and case. 3 are used as tranng set to get new DQNs. Plannng results of dfferent DQNs are shown below. Tab. 4. Tranng parameters and tme consumpton Tranng Set No. of Contaners Iteratons Tranng tme Case h 46 mn Case h 53 mn Case h 21 mn Table 4 shows that these three tranng has same teraton setup, and wth same case sze, the tranng tme s qute smlar. Tab. 5. Comparson of DQNs plannng results Tranng Set Case. 1 Case. 2 Case. 3 Test Case Case. 1 Case. 2 Case. 3 Case. 1 Case. 2 Case. 3 Case. 1 Case. 2 Case. 3 Reshuffles Shfts Tranng Tme As n Table 5, dfferent test cases shows good result wth dfferent traned DQNs, and the effcency of dfferent DQNs are qute smlar, whch means nfluence of dfferent tranng cases and test cases are neglgble. Thus, the proposed algorthm has good stablty and robustness. CONCLUSIONS In ths study, a DQN and a learnng method for ths DQN s proposed to solve shp stowage plannng problem, novatons are as follows. 1. Introduces deep learnng algorthm to solve plannng problem. Wth DQN, massve calculaton and tranng s done n pre-tranng stage, whle n applcaton the plannng problem can be solved n seconds 2. Objectves and constrants of shp stowage plannng problem are transformed to feature vectors to extract stowage polces wth deep learnng algorthm automatcally. Polces from data tends to have less bas than desgned heurstcs n prevous studes. 3. Experence replay s ntroduced n DQN to enforce generalzaton and robustness of proposed algorthm. 4. Provded reference to solvng plannng problem n contaner termnals such as yard storage plannng and equpment schedulng. ACKNOWLEDGEMENTS Ths research was supported by the Natonal Natural Scence Foundaton of Chna (No ), the Scence and Technology Commsson of Shangha Muncpalty (No.15YF , No ), Mnstry of Educaton of the PR Chna (No ), Shangha Muncpal Educaton Commsson (No. 14ZZ140), Shangha Martme Unversty (No.2014ycx040). REFERENCES 1. M. Omar, S. S. Supad Integrated models for shppng a vendor s fnal producton batch to a sngle buyer under lnearly decreasng demand for consgnment polcy. Sans Malaysana 41.3: C. M, Z. W. Zhang, Y. F. Huang, Y. Shen, A fast automated vson system for contaner corner castng recognton. Journal of Marne Scence and Technology- Tawan, 24(1): DOI: /JMST X. P. Ru, X. T. Yu, J. Lu, et al An algorthm for generaton of DEMs from contour lnes consderng geomorphc features. Earth Scences Research Journal, 20(2): G1-G9, 20(2):G1-G9. 4. Y. Shen, An Ant-Collson Method of Slp Barrel for Automatc Shp Loadng n Bulk Termnal. Polsh Martme Research, 23(s1). 5. C. M, Y. Shen, W. J. M, Y. F. Huang, Shp Identfcaton Algorthm Based on 3D Pont Cloud for Automated Shp Loaders. Journal of Coastal Research, 2015(SI.73): DOI: /SI C. M, Z. W. Zhang, X. He, Y. F. Huang, W. J. M, Two-stage classfcaton approach for human detecton n camera vdeo n bulk ports, Polsh Martme Research, 22(SI.1): DOI: /pomr C. M, H. W. Lu, Y. F. Huang, W. J. M, Y. Shen, Fatgue alarm systems for port machne operators. Asa Lfe Scences, 25(1): Yfan S, Nng Z, Wejan M Group-Bay Stowage Plannng Problem for Contaner Shp. Polsh Martme Research, 23(s1). 9. Mengjue X., Nng Z, Wejan M Storage Allocaton n Automated Contaner Termnals: the Upper Level. Polsh Martme Research, 23(s1). 10. C. M, X. He, H. W. Lu, Y. F. Huang, W. J. M, Research on a Fast Human-Detecton Algorthm for Unmanned Survellance Area n Bulk Ports. Mathematcal Problems n Engneerng. DOI: /2014/ POLISH MARITIME RESEARCH, No S3/2017 Unauthentcated

8 11. D. S. Todd, P. Sen, A Multple Crtera Genetc Algorthm for Contanershp Loadng Internatonal Conference on Genetc Algorthms, East Lansng, M, Usa, July. DBLP, N. Zhao, W. J. M, Robust approach n stowage plannng at contaner termnals. IEEE proceedng of the 4th Internatonal Conference on Intellgent Logstc System, A. Moura, J. Olvera, C. Pmentel, A Mathematcal Model for the Contaner Stowage and Shp Routng Problem. Journal of Mathematcal Modellng and Algorthms n Operatons Research, 12(3): M. Avrel, M. Penn, Exact and approxmate solutons of the contaner shp stowage problem. Computers & Industral Engneerng, 25(1-4): J. J. Shelds, Contanershp Stowage: A Computer- Aded Preplannng System. Marne Technology, 21(4): A. Ima, T. Mk, A heurstc algorthm wth expected utlty for an optmal sequence of loadng contaners nto a contanerzed shp. Journal of Japan Insttute of Navgaton, 80: (n Japanese). 23. K. H. Km, Y. M. Park, K. R. Ryu, Dervng decson rules to locate export contaners n contaner yards. European Journal of Operatonal Research, 124: K. H. Km, J. S. Kang, K. R. Ryu, 200. A beam search algorthm for the load sequencng of outbound contaners n port contaner termnals. OR Spectrum, 26: Y. Lee, J. Kang, K. R. Ryu, K. H. Km, Optmzaton of Contaner Load Sequencng by a Hybrd of Ant Colony Optmzaton and Tabu Search, Natural Computaton Lecture Notes n Computer Scence, 3611, CONTACT WITH THE AUTHORS Nng Zhao Logstcs Engneerng College Shangha Martme Unversty Shangha Chna 17. A. Ima, E. Nshmura, K. Sasak, S. Papadmtrou, Soluton comparsons of algorthms for the contanershp loadng problem. Proceedngs of the Internatonal Conference on Shppng: Technology and Envronment, avalable on CD-ROM. 18. A. Ima, E. Nshmura, K. Sasak, S. Papadmtrou, Soluton comparsons of algorthms for the contanershp loadng problem. Proceedngs of the Internatonal Conference on Shppng: Technology and Envronment, avalable on CD-ROM. 19. A. Haghan, E. I. Kasar, A model for desgnng contaner loadng plans for contanershps. In: 80th Transportaton Research Board Annual Meetng, Washngton, DC, USA. 20. J. F. Álvarez, A heurstc for Vessel plannng n a reach stacker termnal. Journal of Martme Research Jmr, 3(1): págs K. H. Km, Analyss of rehandles of transfer crane n a contaner yard. APORS-Conference, 3: K. H. Km Evaluaton of the number of rehandles n contaner yards. Computers & Industral Engneerng, 32: POLISH MARITIME RESEARCH, No S3/ Unauthentcated