inteliscaler Workload and Resource Aware, Proactive Auto Scaler for PaaS Cloud

Transcription

1 inteliscaler Workload and Resource Aware, Proactive Auto Scaler for PaaS Cloud Paper #10368 RS Shariffdeen, UKJU Bandara, DTSP Munasinghe, HS Bhathiya, and HMN Dilum Bandara Dept. of Computer Science & Engineering, University of Moratuwa, Sri Lanka

2 Introduction 2

3 Autoscaling at PaaS Level Reactive Nature Inability to adapt to complex workload patterns Not considering the execution time (Spin Up/Down time) Rule-Based System Event triggered system to manage on-demand resources User Involvement User is required to have a deeper understanding of the domain to utilize the auto scaling mechanism, define rules, and threshold 3

4 Research Goal Autoscale computing resources in a PaaS cloud environment based on current workload and resource usage, while predicting the workload to reduce cost and meet desired QoS/SLA goals. Research Contributions Performance study of an existing auto scaler A workload forecasting technique A penalty-based proactive scaling method Evaluation of feasibility of smart killing on various IaaS providers Analysis of different combinations of scaling methods Implementation of proposed combination on Apache Stratos and AWS 4

5 Our Solution Key features Proactive Better predictive ability Better resource utilization Considering resource acquisition costs and service degradation penalties Evaluation Against Apache Stratos 5

6 Analysis of Stratos Auto Scaling Performance 6

7 Analysis of Stratos Auto Scaling Performance Workload Workload Analysis RUBiS Workload RUBiS Workload Moraspirit Workload 7

8 Analysis of Stratos Auto Scaling Performance Workload Analysis Prediction Evaluation Actual workload Apache Stratos prediction using S = ut + 0.5at2 8

9 Analysis of Stratos Auto Scaling Performance Workload Analysis Resource Comparison High Threshold Total Instances Spawned = 12 Actual Workload Low Threshold Total Instances Spawned = 14 Resource Allocation 9

10 Workload Prediction 10

11 Workload Prediction Base Models Ensemble Technique Evaluation Base Model Selection ARIMA - Linear model with ability to capture seasonal behavior Neural Network - Nonlinear model which is data driven and adaptable Exponential Model - Non linear exponential models have no counterparts in ARIMA Naive Prediction - As an error correction step 11

12 Workload Prediction Base Models Ensemble Technique Evaluation Ensemble Technique Ensemble forecast for (t+h) is weighted average over the individual models Weights are calculated from the inverse error of the forecasts. E.g.: 12

13 Workload Prediction Base Models Ensemble Technique Evaluation Determination of Weights Effectiveness of Ci depends on how you use the past forecast error from the i-th model Averaged Error over past data Most recent error t t We fitted the past forecasting error in i-th model using exponential smoothing model and calculated the contribution coefficients Ci based on the result t 13

14 Workload Prediction Base Models Ensemble Technique Evaluation Comparison of Results 14

15 Resource Allocation 15

16 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Scaling Algorithm n = max v3 = 0 v2 n = min v1 T 16

17 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Scaling Algorithm: Example Penalty Function 17

18 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Awareness of Resource Pricing Model Considerations Each IaaS provides many different instance types Non-uniform pricing policies AWS hourly billing vs. GCE per-minute billing Solution Cost optimize considering IaaS pricing model separately Smart Killing for AWS 18

19 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Resource Scaling Flow 19

20 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Resource Scaling - Approaches 20

21 Resource Allocation Algorithm Penalty Based Optimization Resource Cost Model Simulation - Cost 21

22 Overall Solution 22

23 Overall Solution Architectural Design Tests on Mock IaaS Tests on AWS EC2 Architectural Design 23

24 Overall Solution Architectural Design Tests on Mock IaaS Tests on AWS EC2 Tests on AWS EC2 Stratos inteliscaler 24

25 Summary Performance Evaluation Current auto scaler implementation in Apache Stratos Workload Prediction Evaluating current workload prediction techniques Proposing a new ensemble workload prediction technique for PaaS Resource Allocation Comparison of different combinations of resource allocation methods Proposing a penalty-based resource allocation technique Overall Solution Implementation and testing on Apache Stratos 25

26 Further Improvement Defining different penalty functions based on required QoS Bidding for Spot Instances on AWS Inclusion of a Performance Model Support for Heterogeneity 26

27 Q&A 27

28 Thank You 28

29 Overall Solution Architectural Design Tests on Mock IaaS Tests on AWS EC2 Tests on Mock IaaS Stratos - Low Threshold Stratos - High Threshold inteliscaler 29

30 Introduction Cloud Terminology Auto Scaling Apache Stratos Upcoming Sections Performance Evaluation Current auto scaler implementation in Apache Stratos Workload Prediction Evaluating current workload prediction techniques Proposing a new ensemble workload prediction technique for PaaS Resource Allocation Comparison of different combinations of resource allocation methods Proposing a penalty-based resource allocation technique Overall Solution Implementation and testing on Apache Stratos Project Summary 30

31 Performance Evaluation Workload Simulation Platforms Apache Stratos Evaluation Workload Types Nature Synthetic Empirical Pattern On-and-Off Growing Stable Sudden Peak Cyclic Bursting 31

32 Performance Evaluation Workload Simulation Platforms Apache Stratos Evaluation Why Simulation? Clouds exhibit varying demands, supply patterns, system sizes and computing resources. Cloud users have heterogeneous, dynamic and competing QoS requirements. Cloud applications have varying performance, workload, and dynamic application scaling requirements. 32

33 Performance Evaluation Experimental Platforms Workload Simulation Platforms Apache Stratos Evaluation Synthetic Workload Generators CloudSim Httperf MDCSim Faban GreenCloud Rain Experimental Workloads Application Benchmarks 98 World Cup RUBiS Google Cluster Data RUBBoS ClarkNet TPC-W 33

34 Weight Calculation 34

35 Research Overview Contributions Achievements Performance study of an existing auto scaler Research Papers A workload forecasting technique A penalty based proactive scaling method Evaluation of feasibility of smart killing on various IaaS providers Evaluation of various combinations of scaling methods Implementation of proposed combination on Apache Stratos Acceptance of Adaptive Workload Prediction for Proactive Auto Scaling in PaaS Systems for International Conference Internet of Things and Cloud Computing Technologies, Singapore Working on another paper for International Cloud Computing Conference - IEEE Cloud 2016, USA Research Grant AWS Education Research Grant for 1 year 35

36 Introduction Cloud Terminology Auto Scaling Apache Stratos Apache Stratos A PaaS framework Can be set up on existing IaaS AWS, GCE, Azure Private cloud support Why Stratos? Open source Apache Community support Mock IaaS - eliminates need and cost of using actual IaaS resources Architecture Services via cartridge instances (PHP, MySQL, Tomcat, etc.) Managed by modules (Cloud Controller, Auto Scaler, etc.) Pub-sub messaging 36

37 Resource Allocation Challenges Penalty Based Optimization Resource Cost Model Resource Allocation Current PaaS Auto Scalers User need to define threshold parameters Autoscaler unaware of the pricing models of IaaS Inteliscacler Cost and QoS aware scaling algorithm to calculate resource count required Pricing model aware resource scaling 37

38 Introduction Cloud Terminology Auto Scaling Apache Stratos Stratos Auto Scaling 38

39 Workload Prediction Statistical Machine Learning Proposed Model Literature - Prediction Techniques : Statistical Naïve Prediction Moving Average Exponential Smoothing 39

40 Workload Prediction Statistical Machine Learning Proposed Model Literature - Prediction Techniques : Statistical Autoregressive Integrated Moving Average Model(p,d,q) A combination of, Differencing(d) Autoregression(p) Moving Average(q) 40

41 Workload Prediction Statistical Machine Learning Proposed Model Literature - Prediction Techniques : Machine Learning Autoregressive Neural Networks Lagged values of the time series can be used as inputs to a neural network. 41

42 Workload Prediction Statistical Machine Learning Proposed Model Challenges In PaaS Workload Prediction A PaaS cloud may be used to build different applications with vastly different workload patterns. As the workload dataset grows with time, predictive model should evolve and continuously learn the latest workload characteristics. The workload predictor should produce results within a bounded time. The predictor should produce sufficiently accurate results over a sufficiently large time horizon. 42

43 Workload Prediction Statistical Machine Learning Proposed Model Evaluation of Existing Models - Datasets 43

44 Workload Prediction Statistical Machine Learning Proposed Model Evaluation of Existing Models - Results Each model performs well on some datasets while performing worse on other datasets A single model will not perform well in all online training scenarios BUT, Different applications on PaaS have different workload characteristics Dynamic adjustment of models is required for robust results 44

45 Analysis of Stratos Auto Scaling Performance Our Simulator Workload Analysis Our Simulator Workloads Synthetic o Rain-Toolkit o Httperf Empirical o Sports website (Moraspirit Log) o 98 World Cup Trace o Rackspace Cloud Data (Hosting a HR System) o Google Cluster Data Platform 4 Server Nodes Apache Stratos 4.1 setup on top of AWS Google Cluster Data Benchmark RUBiS 45

46 Analysis of Stratos Auto Scaling Performance Experimental Setup Our Simulator Workload Analysis Server Node Specification Intel Core i3 1.6 GHz 4 GB DRR3 RAM 500 GB Hard Disk MySQL 5.5. Apache 2.4 PHP 5.5 Client Specification Intel Core i5 2.6 GHz 4 GB DRR3 RAM Rain Tool Kit Java

47 Analysis of Stratos Auto Scaling Performance Our Simulator Workload Analysis AWS Setup 47

48 Analysis of Stratos Auto Scaling Performance Our Simulator Workload Analysis Stratos Prediction 48

49 Analysis of Stratos Auto Scaling Performance Our Simulator Workload Analysis Stratos Scaling Process Policies Application Deployment Policy Auto-Scaling Policy o Load Average Threshold o Memory Consumption Threshold o Requests in Flight Threshold 49

50 Project Summary 50

51 Introduction Cloud Terminology Auto Scaling Apache Stratos Summary Performance Evaluation Current auto scaler implementation in Apache Stratos Workload Prediction Evaluating current workload prediction techniques Proposing a new ensemble workload prediction technique for PaaS Resource Allocation Comparison of different combinations of resource allocation methods Proposing a penalty-based resource allocation technique Overall Solution Implementation and testing on Apache Stratos 51

52 Project Summary Challenges Faced Further Improvement Key References Challenges Faced Resource and Infrastructure Cost Insufficient sources for Workload Data Setting up Apache Stratos Paas Cloud on AWS EC2 52

53 Project Summary Challenges Faced Further Improvement Key References Key References N. Roy, A. Dubey, and A. Gokhale, Efficient Autoscaling in the Cloud using Predictive Models for Workload Forecasting, in proc. Intl. Conf. on Cloud Computing (CLOUD), Washington, DC, July H. Alipour, Y. Liu, and A. Hamou-Lhadj, Analyzing Auto-scaling Issues in Cloud Environments, in proc. 24 th Annual Intl. Conf. on Computer Science and Software Engineering, Markham, Ontario, Canada, Nov C. Bunch, V. Arora, N. Chohan, C. Krintz, S. Hegde, and A. Srivastava, A Pluggable Autoscaling Service for Open Cloud PaaS Systems, in proc. 5th IEEE Intl. Conf. on Utility and Cloud Computing (UCC), Chicago, IL, Nov J. Yang, C. Liu, Y. Shang, Z. Mao, and J. Chen, Workload Predicting-Based Automatic Scaling in Service Clouds, in proc. 6th IEEE Intl. Conf. on Cloud Computing, Santa Clara, CA, Jul G.F.S. Garg, R. Buyya, and W. Li, Revenue Maximization Using Adaptive Resource Provisioning in Cloud Computing Environments, in proc. ACM/IEEE 13th Intl. Conf. on Grid Computing (GRID), Beijing, China, M. Mao and M. Humphrey, Auto-scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows, in proc. Intl. Conf. for High Performance Computing, Networking, Storage and Analysis, New York, NY,