Optimization under Uncertainty. with Applications

Transcription

1 with Applications Professor Alexei A. Gaivoronski Department of Industrial Economics and Technology Management Norwegian University of Science and Technology 1

2 Lecture 3 Multistage stochastic programming problems Example: Asset liability management Stochastic Gradient methods Example 1: Inventory management Example 2: Service pricing on social networks Example 3: Empty container problem Example 4: Water resources management Different actors: connection with game theory Moving window: service resource allocation 2

3 Stochastic program with recourse We obtain the so called recourse formulation

4 Deterministic equivalent If we consider the discrete case we can link to a scenario every possible realization of the random variable In this case we can write the large scale deterministic equivalent

5 Deterministic equivalent Large scale deterministic equivalent

6 Nonanticipativity If it could be possible to forecast the future we could have a different first stage decision for each scenario

7 Nonanticipativity Since we can not forecast the future we need to enforce nonanticipativity, setting With nonanticipativity constraints we require that the decisions taken in different scenarios that at a given stage look the same have to coincide This is what is automatically done in the recourse formulation, setting a unique first stage decision

8 Multistage problems Multistage recourse problems Recourse on the recourse on the recourse on the recourse It is an extension of the two stage model and it is formulated with a nested structure

9 Event Tree

10 Multistage recourse formulation Multistage recourse formulation

11 Dynamic asset and liability management 11

12 Asset and Liability Management (ALM) Scenario tree At each date manager assesses new information about prices, volatilities, interest rates and take portfolio rebalancing decision Borrowing, liabilities 12

13 Asset and Liability Management (ALM) Sequence of investment decisions First stage (t=0) Prices and portfolio composition are known Inventory balance constraint: Asset endowment after rebalancing Initial asset endowment Asset bought Asset sold Cashflow balance constraint Value sold Initial asset value Liabilities value Borrowed value Value bought Value invested short term 13

14 Asset and Liability Management (ALM) Trading date t Cashflow balance 14

15 Asset and Liability Management (ALM) Trading date t Inventory balance 15

16 Asset and Liability Management (ALM) Trading date t Inventory balance constraint Cash flow constraint End of horizon constraint Different other constraints: regulatory, etc 16

17 Asset and Liability Management (ALM) Objective: maximize utility of terminal wealth Other objectives depend on particular setting (deviation fom target index, etc.) Resulting model 17

18 Stochastic gradient methods: application fields nonlinear problems which still possess some global structure, e.g. convex or with a few convex global patterns continuous distributions with relatively large number of random parameters parametrized dynamic policies simulation and optimization synthetic description Properties 1 Properties 2

19 Stochastic gradient methods Problem Method Projection Stochastic (quasi)gradient example:

20 Stochastic gradient methods Step size Matrix stepsize Local approximation of the objective Tradeoff between here and now and wait and see Smoothing for multiextremal problems

21 Simulation and optimization Simulation decision variables Optimization observations, estimates

22 Inventory management, demand Inventory order issued when order arrives after delay τ (s,s) inventory policy Finding optimal (s,s) policy combining simulation and optimization Excel spreadsheet without delay with delay

23 Parametrized dynamic policies Original problem General solution Parametrized solution Approximating problem 23

24 SQG: software for solving stochastic programming problems with stochastic gradient methods System structure 24

25 SQG: SQG, optimization continued engine March 2009, University of Cagliari 25

26 Example: supply chain management 26

27 Histogram near optimal point 27

28 Section of the objective function 28

29 Section of the objective function, detail 29

30 Excel spreadsheet SQG solution 30

31 Solution of complex SP problems with SQG Two examples: Differentiated service pricing on social network Transportation between supply/demand nodes with inventories Common feature: complex networks where small cases are solvable with normative approach, but optimization of simulation model allow to arrive much, much further Numerical method: stochastic inertial finite differences Avoids different pitfalls that often happen along this road Some insights from numerical experiments 31

32 Example 1: Differentiated service pricing on social network Service provider who maximizes his profit Population of customers connected in social network Service provider Social network: nodes with links that randomly change in time 32

33 Example 2: Transportation between supply/ demand nodes with inventories Set of nodes connected with links (roads, waterways ) with supply/ demand (or both) for a collection of items Transportation (trucks, ships) operates on links that takes items in excess and bring them where they are lacking Transportation time, finite inventories, costs of transportation, inventory, backlog Decisions taken before one step before inventory is known: where to take and where to bring Transportation of empty containers between ports 33

34 Differentiated service pricing on social network How to price a service offered to participants in a social network in order to maximize profit? Uniform price? Or maybe differentiated price? Offer discounts to well connected participants? If yes, how much? We develop model and integrated simulation and optimization tool that answers these questions 34

35 Model of social network Set of nodes i=1:n Discrete time t=1,,t, A t (w) incidence process: a stationary Markov process with values in the space of NxN matrices; the element a t ij with value between 0 and 1 describes the strength of connection between nodes i and j. the set of neighbors of node i at time t Service provider (SP) provides service described by set of parameters x (subscription and unit usage prices for different user categories, QoS, SLA, ) decided by SP. SP can monitor usage and structure of the network (possibly with errors). 35

36 Model of social network, continued Customers decide about subscription y i t (0 or 1) and volume of usage z it. Subscription y i t of user i at time t+1 depend on Current subscription decision y i t Service parameters x Knowledge about the service usage among network neighbors from B i t (ω): the network effect function v it (ω) Random event ω Thus, subscription process is defined as follows: and more specifically by transition probabilities 36

37 Model of social network, continued The volume of traffic z i t of customer i at time t also depends on the service parameters and network effects: Revenue of SP from user i is defined by subscription and traffic: Example: x=(x i1,x i2 ) where x i1 is the subscription flat rate and x i2 is unit traffic rate for user i, then Total revenue during the time horizon 1,,T: 37

38 Revenue maximization on social network Decision problem od service provider: Find service parameters x that maximize F(x) under constraints where X is admissible set. This is very nontrivial optimization problem. Difficulties: randomness and transient behavior, precise values of objective function can be obtained only after long simulation process 38

39 Possible optimization strategies: many pitfalls here Sample path optimization often advocated but will fail here 1. Select very long time horizon T 2. Generate all the random numbers ω* necessary for simulation during this time horizon 3. Use off-shelf nonlinear optimization software to solve the problem 4. Advantage: no effort is needed for implementation and tuning of optimization algorithm 5. Drawback: will not work due to highly irregular behavior of the sample path objective 39

40 Example A Fixed constant incidence matix Network effect function is between 0 and 1; the more users from the neighborhood of user i are subscribed the closest it is to 1 The single service parameter is its price x per unit of traffic, the same for all users 40

41 Example A, continued Subscription/unsubscription probabilities are deterministic functions of price and network effect where is the maximal price the user i is willing to accept and b i, q i are some parameters between 0 and 1 The traffic function Revenue 41

42 Example A, continued Sample path objective functions for T=20 and T=10000 Irregular behavior persists irrespective of the length of the time horizon, local maxima are just about everywhere. Off-shelf optimization codes will fail 42

43 Answer: stochastic (quasi)gradient methods (SQG) Method for solution of stochastic optimization problems of the type Iterative process Projection operator ξ s is an estimate of the gradient of the objective function: ρ s is a stepsize: Stochastic approximation: Kiefer & Wolfowitz, Ermoliev, Kushner, Pflug, Shapiro 43

44 SQG, continued Important difference: transient behavior, we can observe only estimate with the property Convergence theorem: Functions F s (x) are convex and bounded on open set Set X is convex and compact ρ s is nonnegative and Nonstationarity condition: Then with probability 1 44

45 Lot of work is needed to adapt this method to optimization of simulation models Transient behavior Integration issues between simulation and optimization Obvious strategies do not work, for example brute force finite differences At each step: 1. Simulate the system from some initial state for current values of service parameters x s during time horizon T, obtain the observation of objective u 0s =R T (x s,w s ). 2. Do the same for the values of x i s that differ from x s in that the i-th variable is incremented by the value δ s of finite differences, obtain the estimate u is. Do it for all the service parameters. 3. Compute the i-th component of the estimate of the gradient ξ i s =(u is -u 0s )/ δ s 4. Perform one step of the SQG method, obtain x s+1 5. Go to step 1. This takes forever because T should be sufficiently large to get rid of the transient effects. 45

46 Integrated simulation and optimization Intertwine tightly simulation and optimization: change service parameters according to SQG method every simulation step Perform n+1 parallel simulations for the current point and n shifts with the finite difference step for each of the n parameters using common random numbers Utilize previous information in the estimation of function values necessary for the finite differences This has an effect of filtering out both noise and transient effects Since optimization steps are so lightweight they can be performed by millions in a few minutes on laptop 46

47 Integrated simulation and optimization, continued Optimal values of service parameters are obtained after the end of single simulation run consisting of n+1 simulation threads 47

48 Full description of the algorithm 48

49 Full description of the algorithm, continued 49

50 Tuning of the algorithm on the problem with known solution If the network has fixed structure the process is described by Markov chain with 2 N states. For small N one can find stationary probabilities of the chain and find the optimal prices using offshelf NLP software. This is used as a benchmark 50

51 Study of networks with different structure Network of type A Global leader 1 connected with everybody Local leaders 2 connected with half of the rest and the leader Ordinary nodes 3 connected only with leader1 and one of the local leaders 2 51

52 Study of networks with different structure, continued Network of type B Global leader 1 connected with everybody L local leaders 2 connected with M followers and the global leader Followers 3 connected with global leader 1 and one of the local leaders 2 52

53 Optimal prices, networks with fixed structure Differentiation does matter 53

54 Optimal revenues, networks with fixed structure Differentiation does matter, but random errors in connections can destroy the picture 54

55 Optimal revenues, networks with fixed structure Differentiation does matter, errors in transition probabilities do not matter much 55

56 Optimal prices, networks with randomly changing structure If connections flip randomly the prices converge 56

57 Optimal revenues, networks with randomly changing structure If connections flip randomly the revenues grow due to increased network effects 57

58 Example 2: Transportation between supply/ demand nodes with inventories Set of nodes connected with links (roads, waterways ) with supply/ demand (or both) for a collection of items Transportation (trucks, ships) operates on links that takes items in excess and bring them where they are lacking Transportation time, finite inventories, costs of transportation, inventory, backlog Decisions taken before one step before inventory is known: where to take and where to bring Transportation of empty containers between ports 58

59 Comparison of scenario tree deterministic equivalent with optimization/simulation 59

60 A small problem due to huge dimension of deterministic equivalent 3 ports, 1 type of conatiners,1 ship that travels in one day between ports, 7 days, forecasts of demand/supply are known, normal distribution around forecasts, independent forecast errors Binary tree 7 periods on each of 3 demands, 7 days: 2.1*10 6 scenarios, 1.7*10 7 variables, 1.2*10 7 constraints, CPLEX 12.1, 8 hours of OPL 6.3 time of single core 2.93 GHz, 8 GB of memory, 23 GB virtual memory allocated Trinary tree first 4 periods: 5.3*10 5 scenarios, 1.8*10 7 variables, 1.1*10 7 constraints, 34 hours of OPL 6.3 time What if we use the most powerful cloud available? Trinary tree first 5 periods 1.4*10 7 scenarios, 3.6*10 8 variables, 2.2*10 8 constraints 60

61 Which tree to choose Conclusion: binary tree is superior to other trees for the same number of scenarios 61

62 Comparison for different levels of uncertainty 62

63 Management of water system 63

64 Management of water system Large uncertainty in inflows, secondary source demand Often demand it is not possible to satisfy fully: programmed and not programmed deficit Some very costly ways to transport water by pumping Costs: two levels of penalty for not satisfaction of demand + electricity costs tor pumping + penalty for water loss due to overflow 64

65 Combination of optimization and Cost observations simulation Optimization of pumping rules by SQG Update of parameters Available water Natural inflows Optimal flows by solving LP Demand Pumping rules 65

66 More than one decision maker: stochastic optimization + game theory 66

67 Case study subscription Provider 2 Customers price price price service request subscription Provider 1 67

68 Point of view of Provider 1 Objective: set prices within given constraints in order to maximize profit increase market share. Decision framework Model of customer response Model of competitors behavior Features Uncertainty Absense of equilibria Dynamics 68

69 Decision process of Provider 1 Each decision period for a given decision predict customer response using customer model predict competition response using competition model select optimal policy from profit/market share model which depends on response of customers and competition 69

70 Decision process of Provider 1 profit model customer model model of competition - decisions of Provider 1 - decisions of Provider 2 - decisions of customers (subscriptions) 70

71 Roadmap Develop a single period model of this decision process without uncertainty Use it as a building block for construction of models which represent uncertainty and dynamics Approach: selected models of game theory (Stakelberg games) + stochastic programming 71

72 I. Deterministic single period model I.1 Model of customer behavior I.2 Model of competition I.3 Profit model 72

73 I.1 Model of customer behavior Price dynamics t=0 t=1 t=2 time price charged by provider i at time period t number of customers who choose provider i at time t amount of first time customers subscribed to provider i at time t customers which switch from provider j to provider i at time t 73

74 demand Demand vs price f(h) d 0 h price h 0 74

75 I.1 Model of customer behavior - basic price - price increment One period model: 75

76 Provider 1 Provider 2 Customers Decisions: - price to charge to customers for the service - price to charge to Provider 2 for leased capacity - price to charge to customers for the service - amount of capacity to lease from Provider 1 - total number of subscriptions to Provider 1 service - total number of subscriptions to Provider 2 service 76

77 I.2 Model of competition Profit model for Provider 2: - revenue from service provision - cost for leasing capacity - cost of service provision fixed provision cost variable provision cost 77

78 I.2 Model of competition, continued Prediction of competition response For fixed values of Provider 1 policies y find response z(y) of Provider 2assuming that he maximizes his profit subject to Quality of Service constraints average required capacity per user 78

79 I.2 Model of competition, continued Under additional assumptions this has the following solution i.e. predictions of response of Provider 2 depend linearly on policies of Provider 1 79

80 Profit model of Provider 1 - revenue from service provision - revenue from leasing of capacity to Provider 2 - cost of service provision explicit expressions for z are available from prediction model 80

81 Profit model of Provider 1, continued subject to constraints This problem may or may not be concave 81

82 Uncertainty and dynamics Prediction Decision Period 1 Period 2 - decisions which Actor 1 takes during Period 1 - decisions which Actor 2 takes during Period 1 - decisions which Actor 1 takes during Period 2 - decisions which Actor 2 takes during Period 2 - uncertain parameters which become known by Period 2 - uncertain parameters which remain unknown by Period 2 82

83 Period 2: prediction For given obtain prediction for decision of Provider 2 during Period 2 by solving problem Period 2: decision Find decision by solving problem 83

84 Period 1: prediction For given obtain prediction during Period 1 by solving problem for decision of Provider 2 Period 1: decision Find decision by solving problem 84

85 Looks very complicated! But, can be simplified considerably using approaches of stochastic programming - finite number of scenarios to represent distributions - deterministic equivalents - use commercial software as building blocks 85

86 Pricing example:combined decision service prices for periods 1 and 2 capacity prices for periods 1 and 2 bound on the amount of leased capacity capacity expansion at the beginning of Period 2 86

87 Pricing example, continued uncertain parameters: 87

88 Pricing example, continued 88

89 Pricing example, prediction period 1 where 89

90 Example: Prediction for period 2 90

91 Numerical Experiments Pricing problem including uncertainties Both one stage model and two stage intermediary model Implementation in Matlab, using quadratic programming solvers Varying parameters to simulate different situations 91

92 Single stage; profit function Provider 1 92

93 Two stages; profit function Provider 1 93

94 Summary III It is feasible to process uncertainty resulting from decisions of other players within framework of stochastic optimization combination of game theory approaches and stochastic programming Numerically tractable: Matlab and some other NLP solvers 94

95 Summary Stochastic optimization at the current stage has tools for finding robust solutions of different design and planning problems under uncertainty More effort toward nonlinear problems and integer problems is needed 95