Grid Economics. Integrating Economic Principles in Grid Resource Management

Transcription

1 Grid Economics Integrating Economic Principles in Grid Resource Management Kurt Vanmechelen, Jan Broeckhove, Gunther Stuer Research Group Computational Programming

2 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 1

3 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 2

4 Preliminaries Grid Computing - Form a distributed shared infrastructure by linking resources across organizational and administrative boundaries - Increase infrastructural efficiency - Increase flexibility Resource Management - Orchestrate and optimize resource usage 3

5 Motivation (I) Progress Grid Technology - Virtualization through middleware developments - Successful deployments on different scales Reinstate computing model with shared resources Grid usage policies which user gets - What? - At what cost? - When? - For how long? Current state - Static policies - System centric resource management 4

6 Motivation (II) Incentives? - Introduce costs, budgets, currency, market - Self organizing user-centric resource management Step towards Grid Bussiness Models - Sell cycles (pay-per-use) SunGrid (1$ / CPU hour) HP IPS, IPS+,... - Obtain cycles GIMP EFF ($ for 1,000,000 digits) Importance increases as we move up the Grid adoption curve 5

7 The Grid Value Chain R1 R2 R3 S1 S2 S3 A1 A2 RP1 RP2 SP1 SP2 AP1 AP2 C1 Infrastructure Market Service Market Application Market C2 Infrastructure Services Applications Participates Provides 6

8 Grid Market Models Resource Model - Types - Quality attributes Consumer Model - Job model - Budget model - Value model (demand) Provider Model - Allocation model - Value model (supply) Pricing Model 7

9 Pricing Model How are prices set? Several mechanism possibilities - Bargaining - Bartering - Auctions Single Unit / Multiple Unit / Combinatorial Sealed-bid / Dutch / English / Commodity Market with equilibrium pricing -... Need for evaluation 8

10 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 9

11 Commodity Market p = (p 1,...,p n ) ξ(p) = D(p) - S(p) Price Adjustment - Gradient based methods - Least squares methods - Simplex methods - Lagrangian Pattern Search - Smale s Method

12 Resource Model Commodity resource categories for CPU - CPU slots are traded - Slot performance category Pricing per category - Cost for 1 time unit of usage - Consumer pays fixed rate for duration of job Eases budgetary planning Alternatives: Variable rate, Variable with CAP 11

13 Consumer Model Host job queue - CPU bound (running time t r ) - Atomic Deadline for finishing jobs (optional) Budget - Periodically replenished - Leftovers discarded - Expenditures spread out Formulate demand on categories - Minimize turnaround time single job - Minimize cost - User-level preferences

14 Provider Model Offer resources for each category Strategies for determining supply, given p - Full Provision - Linear Provision - Linear Provision with minimum prices Elasticity attribute E - Amount of steps for avg(s i ) 13

15 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 14

16 Grid Economics Simulator Discrete event simulator - Controlled deterministic environment - Graphical UI Configuration & Analysis - Scale - Efficiency Compare different market models - Performance / Scalability - Price correctness - Utilization - Fairness - User-level objective functions 15

17 Simulator Architecture User Interface Layer Configuration Layer Output Management Layer FIFO EDF... Commodity Market English Auction... Non-Economic Resource Management Economic Resource Management Domain Layer Consumer Model Provider Model Resource Model 16

18 Commodity Pricing GES JVM Scheduler P1 P2 P3 New step Schedule Controller C1 C2 Excess Demand Provider Register RMI Registry C3 Matlab Environment Matlab JVM Equilibrium Price Vector Find EQ Price Vector Pricing Minimize ED Adjust Price Matlab Optimization Server Optimization Routines 17

19 18

20 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 19

21 Example Grid Market Scenario 678 CPU slots in Fast category in Slow category 150 market participants consumers - 50 providers Jobs added - Every 100 steps - 10% probability 20

22 Strongly varying job load Oversupply scenario Prices follow demand closely Pricing neutralizes demand shocks Prices reflect valuations 21

23 Constant job load Overdemand scenario Keep job queues constant Prices ~ constant Prices reflect valuations Full utilization 22

24 Fairness 4 budget groups All groups get fair share Overdemand & oversupply case 23

25 Result Table 24

26 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 25

27 Integration with EGEE/BEgrid Resource Broker Ghent CE Brussels CE UA CE KUL CE Economic Broker Grid Bank Market Agent Market Agent Market Agent Market Agent Market Agent Grid Market Market Agent Market Agent 26

28 Agenda Introduction Commodity Market Model Grid Economics Simulator Example Scenarios BEgrid Integration Future Work & Conclusions 27

29 Future Work Limitations - Coarse grained pricing - Locality not integrated in pricing model - Limitations for 1-on-1 negotations - Currently only spot market Combine commodity approach with other market mechanisms Scalability study Real-world deployment 28

30 Conclusions Economic models for Grid Resource Management - Dynamically adapt to valuations of Grid users - Increase perceived value of the Grid infrastructure - Incentives for well-considered resource usage / sharing Large market design space - Need for evaluation - Developed simulator and framework for Grid market designs Commodity Market model - Properties of price correctness, fairness, efficiency and price stability - A lot of research challenges ahead... 29

31 Questions? 30