Cost Optimization of Elasticity Cloud Resource Subscription Policy

Transcription

1 IEEE TRANSACTIONS ON SERVICES COMPU TING, MANUSCRIPT ID 1 Cost Optimization of Elastiity Cloud Resoure Subsription Poliy Ren-Hung Hwang 1, Chung-Nan Lee 2, Yi-Ru Chen 1, Da-Jing Zhang-Jian 2 1 Dept. of Computer Siene and Information Engieering, National Chung Cheng University, Taiwan 2 Dept. of Computer Siene and Information Engieering, National Sun Yat-sen University, Taiwan Abstrat In loud omputing, resoure subsription is an important proedure w hih enables ustomers to elastially subsribe to IT resoures based on their servie requirements. Resoure subsription an be divided into tw o ategories, namely long term reservation and on-demand subsription.although ustomers need to pay the upfront fee for a long term reservation ontrat, the usage harge of reserved resoures is generally muh heaper than that of the on-demand subsription. To provide a better Internet servie by using loud resoure, servie operators will expet to make a trade-off betw een the amount of long term reserved resoures and that of on-demand subsribed resoures. Therefore, how to properly make resoure provision plans is a hallenging issue. In this paper, w e present a tw o-phase algorithm for servie operators to minimize their servie provision ost. In the first phase, w e propose a mathematial formulae to ompute the optimal amount of long-term reserved resoures. In the seond phase, w e use the Kalman filter to predit resoure demand and adaptively hange the subsribed on-demand resoures suh that provision ost ould be minimized. We evaluated our solution by using real-w orld data. Our numerial results indiated that the proposed mehanisms are able to signifiantly redue the provision ost. Index Terms Priing and resoure alloation, Strategi information systems planning, Modeling and Predition 1 INTRODUCTION 1.1 Bakground Cloud omputing [1] is now one of the prevailing Information Tehnology (IT) trends that delivers IT resoures as a servie through the Internet, suh as omputing and storage. However, instead of being a new tehnology itself, the onept of loud omputing has evolved from the base of grid omputing [2 ] [3], utility omputing, luster omputin g [4], and distri b- uted systems in general. By integrating these tehnologies, loud omputing has proven distint advantages of good salability, reliability and availability. In addition, loud omputing allows ustomers to rent resoures on a pay-per-use-basis, and provides more ost-effetive plans. Servies of loud omputing appliations an be roughly divided into three ategories: Infrastruture as a Servie (IaaS) [5] [6], Platform as a Servie (PaaS) [7], and Software as a Servie (SaaS) [8]. In this paper, we mainly foused on the servie provision issue on IaaS, whih abstrats hardware resoures (e.g., server, storage, and network bandwidth) into a pool of omputing resoures and virtualization infrastruture. IaaS providers (e.g., Amazon EC2 [9] and GoGrid [10]) build flexible loud solutions aording to the hardware requirements of ustomers; furthermore, it lets ustomers run operating systems and software appliations on virtual mahines (VMs). Customers merely pay for the resoures that are atually used. Therefore, IaaS has beome more popular and useful for a wider range of onsumers in reent years. For example, to host web appliation servies, servie operators would apply IaaS resoure subsription plans to dynamially adjust servie apaity to satisfy a time-varying demand. While subsribing IaaS resoures, the web servie operators aimed to provide a ertain Servie Level Agreement (SLA) with their lients, e.g., a guarantee on request resonse time. The resoure provisioning of IaaS allows onsumers to elastially inrease or derease the system apaity by hanging onfigurations of omputing resoures. Moreover, loud providers have multiple usage-based priing models based on different VM onfigurations, suh as different CPU ores, memory size, and rental osts. Cloud providers generally offer at least two subsription plans (i.e., reservation and on-demand plans) to their ustomers. The on-demand plan is typially more expensive than the reservation plan beause the former allows VMs to be dynamially aquired at anytime without a ommitment and harged on a pay-per-use-basis. On the other hand, with the reservation plan, users need to pay an upfront fee for the ontrat. Then, the r e- served VMs an be utilized at a heaper usage ost during the time of ontrat. In this way, ustomers ahieve signifiant ost savings. However, there are some unpreditable situations, suh as unertain demand, inurring over- and under- provisioning problems. For exa mple, th e time-varying workload flutuation inreases the diffiulty of demand estimation. O w- ing to the error-prone demand estimation and omplex ombination of loud resoures, ustomers usually make inappropriate subsription plans. Clearly, the resoure subsription problem an be divided into two sub-problems: how many long-term Digital Objet Indentifier /TSC xxxx-xxxx/0x/$xx x IEEE /13/$ IEEE

2 2 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID resoures to be reserved and how many on-demand resoures to be aquired. If the long term reserved resoures are more than the atual demand, it auses waste of the upfront fee. On the other hand, if the r e- served resoures are less than the atual demand, additional resoures need to be subsribed on-demand, whih are more expensive than usage ost of long term reserved resoures. In addition, dynamially requested resoures may not be granted or ready to use instantaneously. In fat, initializing a new VM requires approximately 5-15 minutes [11] delay. This delay may also auses violation of the SLA between the web servie operator and its lient, whih often invokes signifiant penalties. To alleviate this problem, a promising mehanism is to prepare extra resoures ahead of time by prediting traffi demand. Furthermore, different ways of inreasing resoures also need to be taken into aount, suh as VM migration and repliation. 1.2 Motivation and Objetive More and more web-servie providers run their web appliations on the loud beause it provides lower ost, fast deployment and high salability. In partiular, IaaS is apable of dynamially providing virtual infrastruture aording to the demand of users and offering flexible provisioning plans. In this paper, we bring th e viewpoint that webservie providers, who not only desire to improve reliability and salability via various traits of loud, but effetively redue operational ost. The operational ost we onsidered inludes an upfront fee of a long term resoure reservation ontrat, usage ost of initiating the reserved resoures, usage ost of resoures aquired on-demand, and a penalty ost of SLA violation due to insuffiient resoures alloated. We onsidered two phases of a resoure provisioning plan: a long term reservation plan and an on-demand alloation plan. Our objetive was to minimize the operational ost by virtue of optimal resoure reservation and preditive adjustment of resoure usage. The following tehniques were used to ahieve the objetive: For long term resoure reservation, we aimed to find the amount of resoures to be leased suh that the operational ost ould be minimized, assuming that insuffiient resoures at any time instane ould be dynamially and instantaneously alloated ondemand. The resoure reservation plan inluded the lease period, types of VM (servers) and their quantity to be reserved. We derived equations for alulating the expeted operational ost assuming that we an estimate the workload demand distribution based on historial data. Upper bound and lower bound of the minimum resoure alloation were then derived. For on-demand resoure alloation, we first adopted the Kalman Filter to predit workload demand. Gi v- en the workload demand, the VM onfiguration problem was formulated as a Linear Programming problem whih was then transformed into a Knapsak Problem. The VM onfiguration problem took into aount of VM launh or shutdown to reflet the hange of workload demand. It also onsidered the time onstraints of leasing a ertain type of VM and time required for VM launhing. Finally, penalty of insuffiient resoure provision was also onsidered. Both heuristi algorithm and dynami programming tehniques were used to find the solution of the Knapsak Problem whih gave the optimal VM onfiguration for the next operation period. 1.3 Contributions In this paper, our major ontributions are summarized as follows: We introdued a loud ustomer entri view and two-phase resoure provision plans. We proposed mathematial formulae to find the upper bound and lower bound of the optimal amount of resoure needed to be alloated for long term resoure reservation problem. Our simulation results verified our proposed formulation. We exploited a preditive-based resoure management to adaptively onfigure VMs. The optimal deision was formulated as a linear programming problem and was solved effiiently by the related Knapsak Problem. Real workload data and priing model of Amazon EC2 [9] were used to demonstrate the effetiveness of our algori thm. The results showed the proposed algorithm signifiantly redued the operational ost. The rest of this paper is organized as follows: Related works are reviewed in Setion 2. Our system model and assumptions are illustrated in Setion 3. The problem formulation and the proposed solutions are presented in Setion 4. Experimental results are presented in Setion 5. Finally, onlusions are stated in Setion 6. 2 RELATED WORK 2.1 Priing Model of Cloud Resoures The onept of the business model on omputing servies or resoure usage might draw upon the utility omputing [12] like our eletri servies at home. Utility omputing took advantage of a ritial idea: payper-use-basis. Moreover, loud omputing developed a broader appliation. For example, web-appliation servies had been operated and marketed by use of a pay-per-use-basis model. In IaaS loud environment, a variety of omputing resoures an be ombined to form different types of VM; eah with a different ombination of apaities of different resoures. Take Amazon as an example, there are multiple VM types, and eah of them has its own onfiguration and prie as shown in Table 1. In addition, there are three different purhasing models (i.e., reserved, on-demand, and

3 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY 3 spot) that give users the flexibility to manage their finane effetively. Table 1. Cloud server onfigurations and their pries defined by Instane Types Small Medium Large Extra Large Amazon EC2 (obtained in De. 2011). Reserved On- Instane (1yr Term) Demand Spot front Configuration Up- Cost/hr Cost/hr Cost/hr 1 ECU, 1.7GB RAM, 160GB disk $160 $0.024 $0.080 $ ECU, 3.75GB RAM, 410GB disk $320 $0.048 $0.160 $ ECU, 7.5GB RAM, 850GB disk $640 $0.096 $0.320 $ ECU, 15GB RAM, 1690GB disk $1280 $0.192 $0.640 $ Resoure Provisioning for Cloud Computing Customers, suh as web appliation providers, an gain the resoure alloation via a resoure subsription. Meanwhile, when loud providers aept a request from a ustomer, they have to instantiate the speified resoure. As a result, how to explore the issue of resoure provisioning an be divided into two viewpoints, namely loud provider- and ustomer- entri view. In [13], Tsakalozos et al. proposed a miroeonomiinspired approah to determine the number of VMs to be alloated to eah user by their finanial apaity, then maximizing per-user s profit and effetiveness of resoure sharing. [14] aimed to optimize the resoure onsumptions in the luster-based loud systems, and both the geneti algorithm (GA) and the reonfiguration algorithm were developed to optimize the system state at low transition overhead. In [15], Ying et al. proposed a two adaptive ontrol approah (i.e., the resoure onsumption optimization loop and the resoure alloation loop) to manage infrastruture r e- soures and improve resoure utilization. Different from the above studies whih attempted to maximize revenue and resoure utilization for loud provid ers, [16][17][18] mainly foused on the ustomer-entri view. There are many aspets that needed to be investigated from this view, suh as the onduted way of subsription, resoure demand deision, and prie-qos tradeoffs. In this work, we foused on the resoure provisioning problem from the ustomer-entri view. In earlier work suh as [19] [20], they were apable of deiding the optimum number of the VM with single type through performane analysis; thereby orr e- sponding to the minimum ost. Hu et al. [19] applied a queueing model to analyze the response time distribution for two lasses of job; moreover, a heuristi algorithm was developed to obtain the smallest number of servers without violating the SLAs. Similarly, [20] used a queueing model to build an adaptive provisioning mehanism. However, in a real world situation, most of IaaS providers offer several priing options: reservation, on-demand, and spot. Reservation option is suitable for long term provisioning. A ustomer signs a long term lease ontrat with the IaaS provider to reserve a fixed amount of resoures. Usually, it inludes an upfront fee for signing the ontrat and a usage fee per instane and unit of time for atual use of the resoures. On-demand option is suitable for dynami provisioning. A ustomer an ask for resoures on a pay-peruse-basis at any time. The Amazon EC2 also provides the spot option, whih allows a ustomer to submit a bid prie to ompete with remained resoures. Zhao et al. [21] analyzed the preditability of the time-varying pries of spot instanes in Amazon EC2 to minimize rental ost for running elasti appliations in a loud environment. Sine resoures purhased through this option are not guaranteed to be available, we shall not onsider this option in this paper. Mao et al. [22] proposed an auto-saling mehanism and sheduled VM instane startup and shut-down ativities by onsidering the deadline for running a job, users budget, and multiple instane types. In [16], a system alled Kingfisher was developed to minimize the loud tenant s deployment ost and meet servie demand at the same time. Kingfisher took into aount the ost of eah VM type as well as the transition time of adjusting apaity. Both [16] and [22] did not take a global view of long-term resoure provisioning, but only onsidered how to alloate optimal server apaity at eah sheduling instane. In ontrast, we investigated both reservation plan and on-demand plan. Chaisiri et al. [17] also studied the optimal long term reservation plan and on-demand plan. They formulated the optimal long term reservation plan problem as a stohasti programming model by assuming that the distribution of workload demand and prie model was known in prior. However, the omputation omplexity is too high and the assumption is not pratial. In our previous work [18], we onsidered the same problem, but only single type of VM was onsidered. In [23], Mark et al. adopted a demand foreaster to predit the future workload suh that ourrenes of overprovisioning ould be redued. Nevertheless, their predition mehanism was only used in the reservation phase and the delay of dynami resoure pro vision was negleted in the on-demand phase. In [24], two provisioning algorithms for long- and short-term planning were proposed. The long-term plan, like above work, was determined by subsribing reserved instanes for the long-term usage. Inside the long-term plan, multiple short-term plans were triggered to provide enough numbers of spot instanes for apaity supporting. Even though they developed the short-term plan, th ey overlooked the time delay to launh a new VM.

4 4 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID In most of the aforementioned studies, the lateny required to hange VM onfiguration was not onsidered. However, loud providers usually need some preparation time to hange th e VM onfiguration, either migrating to a different type of VM or repliating a VM instane. As a onsequene, a ustomer may need to wait for a ertain amount of time to be able to atually aquire the requested resoure. This kind of VM onfiguration hange delay may ause SLA violation if the on-demand provisioning plan does not take the onfiguration lateny into aount. Therefore, in this work, VM onfiguration delay was onsidered in the proposed proative, preditive-based dynami resoure management. 2.3 Resoure Demand Predition During th e on-demand phase, knowing the wor k- load demand is essential for resoure provisioning. However, it is not realisti to assume the workload demand an be known in prior. Therefore, several studies had foused on workload predition and alloated loud resoures based on the predition. For example, Islam et al. [25] developed preditionbased resoure measurement and provisioning strategies using Neural Network and Linear Regression to satisfy upoming resoure demand. Adaptive sliding window size was proposed to improve the auray of foreasting. However, their experiments foused on the auray of the foreasting models, and they did not take into aount the impat of VM onfiguration delay, n either showed how to hange VM onfiguration based on the predited workload. Caron et al. [26] proposed an auto-saling module based on identifying past resoure usage patterns whih were similar to that of the present usage. They used the Knuth -Morris-Pratt (KMP) as the string mathing algorithm to predit the future workload. Nonetheless, it was quite time onsuming to searh for similar patterns over the entire set of historial data. In this work, we proposed to predit workload demand based on the Kalman filter [27] whih requires far less omputation and memory. 3 PROBLEM SATEMENT AND ASSUMPTIONS We onsider the long- and short-term VM provisioning problem from the view point of a web-servie operator. The time interval for short-term planning, denoted by t, is relatively muh smaller than the longterm planning, denoted by T [24]. (For example, 5 minutes vs. 1 year.) For long-term provisioning, we aim to determine the optimal number of VMs needs to be reserved. For short-term provisioning, we aim to determine how to onfigure VMs to provide suffiient servie apaity for time varying workload. 3.1 Cloud Computing Environment We assume that an IaaS provider offers multiple VM types. Eah type features differen t hardware speifiations. Besides, there are three different rental osts, inluding an upfront fee for long term reservation, a usage harge of reserved resoures, and an on-demand ost on dynamially alloated resoures. These osts are normalized to per short-term time interval hereafter for ease of ost alulation. Let V = {V1, V2,, VM} denote the set of VM types and M be the total number of VM types supported by the IaaS provider. Eah VM type has its own hardware speifiation and servie apaity. Let Ci denote the apaity of Vi whih orresponds to the maximum number of onurrent users or the servie request rate that an be supported by an instane of Vi without violating the quality of servie guarantee. 3.2 Priing Model As shown in Table 1, we ondier two priing models: reserved and on-demand. The osts of a launhed on-demand instane and a launhed reserved instane of type Vi (per time t) are and respetively. Let denote the upfront fee (per time t) for long term reserving an instane of Vi. As a ommon pratie for priing models, we assume >. Furthermore, we define three priing funtions, namely (X), Φ(X), and Λ(X), to represent the upfront fee for a long term reservation, usage harge of launhing reserved VMs, and the ost of launhing ondemand VMs respetively where X= {n1,, nm} is a provision onfiguration alloated by a subsription plan and ni denotes the number of instanes of Vi. 3.3 Resoure Demand and Penalty Funtion It is important for a web-servie operator to maintain its servie quality to its users all the time. Profiling tehnique is ommonly used by appliation providers to measure their servie quality based on workload profile and resoure provision onfiguration profile. As a onsequene, appliation providers usually d e- ploy a monitoring module whih onstantly traes the workload (e.g., the number of simultaneous online users or servie request rate) and resoure utilization. With the profiling tehnique, we assume that the measured workload ould be transferred to resoure demand and the apaity of a VM an thus be derived. For example, we an define the apaity of a type i VM to be the maximum number of simultaneous online users or servie request rate that it an support without violating the quality of servie guarantee made by the appliation provider. Due to the stohasti nature of the servie request pattern, the web-servie operator is not able to know the workload of the next time instane. As a result, resoure under provisioning ould happen. Namely, it is possible that even with a workload predition teh-

5 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY 5 nique, the apaity of the provided VMs is not suffiient to meet the suddenly raised workload (burst r e- quests). This situation would beome worse if the resoure subsription plan is onservative. When resoure is under provisioned, quality of servie will be degraded and ause potential revenue loss to the appliation provider. We propose the onept of penalty to the QoS or SLA violation and define a penalty funtion whih is quantified by Eq.(1), (1) where p is the penalty fator. As expressed in (1), the penalty is proportional to the resoure demand d. The rationale is that all the on-line users will be affeted when the resoure provision beomes insuffiient. We will onsider the penalty funtion in the short-term provisioning phase. 3.4 System Arhiteture and Problem Statement The overview of our system arhiteture is shown in Figure 1. The proposed framework onsists of two roles: Servie Provider and IaaS Provider. In the arhiteture, the Resoure Provisioner omponent of IaaS Provider takes the responsibility for adjusting the deployment of VM repository based on the Servie Provider s request. for dynami resoure alloation, and determines an optimal VM onfiguration for resoure provision. Resoure Broker, whih performs adaptive planning and delivers resoure subsription to the IaaS Provider. In this work, we propose a two-phase optimization proess to obtain the optimal solutions for subsribing the resoure from the IaaS. Figure 2 shows the flow hart of the proposed two-phase optimization proess: Figure 1. Operational overview of our system arhiteture. The arhiteture of the Servie Provider onsists of the following key omponents: Monitoring Engine, whih traes the workload (the number of simultaneous online user) and resoure utilization. Workload Analyzer, whih generates the analysis of workload via some analytial models. Predition Model, whih makes use of Kalman filter aording to the analysis of workload to pr o- vide demand predition. Elastiity Planner, whih exeutes our approah Figure 2. Tw o-phase optimization proess. First phase: long-term planning for optimal resoure reservation The long-term planning algorithm aims at determining the number and types of VMs to be provisioned so that the operational ost is minimized for the next lease ontrat period, given the historial data or distribution of workload demand. In this phase, we show how to formulate a mathematial model that alulates the operational ost given a set of workload demands and resoure onfiguration, and then we derive an upper and lower bound for VM onfiguration whih meets the workload demand while minimizing the operational ost. Seond phase: short-term planning for dynami resoure alloation After starting the reservation ontrat, the proess enters the seond phase. During this ontrat period, the reserved VMs an be launhed with a usage harge to meet time-varying demand. If workload demand exeeds the apaity of reserved VMs, more VMs an be flexibly provisioned with an on-demand subsription plan at anytime. In this phase, we apply the Kal-

6 6 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID man filter and Linear Progra mmin g to dynamially onfigure VMs. As a result, our short-term planning algorithm enables us to aquire minimum usage harges and effetive resoure utilization. 4 METHODOLOGY As stated in Setion 3, the optimization proess has two phases, and their main funtionalities are longterm resoure reservation optimization and effetive short-term resoure alloation. In this setion, we desribe the proposed two-phase planning algorithms. 4.1 Long Term Resoure Reservation Solution Notations used in this setion are summarized in Table 2. Reall that T is the time duration of the lease ontrat. We assume that historial data of workload demand, D, of the past lease period is known, where. Given D, let R={n1,, nm} be the optimal provision onfiguration for the lease ontrat and r be the apaity provided by R, r=. Table 2. List of notations used in Long-term planning phase. Vari abl e Meaning T Time period for the long-term planning phase D The set of demands,. Probability that for any given time t. Cumulative distribution funtion for Long-term provision ost when reservation is R R Optimal long-term reservation; R={n 1,, n M} r Total apaity provided by R, r = Optimal provision onfiguration when the demand d an be U d served by R (i.e., d r) Optimal provision onfiguration when the demand d exeeds r V d (i.e., d>r) Under reservation indiation funtion: B(d) (X) Upfront fee (per unit of short-term time, t) Upfront fee for an instane of type i VM. Φ(X) Usage harge of launhing reserved VMs, Usage harge of launhing an instane of reserverd type i VM. Λ(X) Cost of launhing on-demand VMs Cost of launhing an instane of on-demand type i VM Let denote the provisioning ost for time T when the long-term resoure reservation is R. In this phase, we derive by assuming that on-demand resoure is dynamially alloated based on the priori known workload demand suh that QoS is always satisfied. is given by the following equation: where the first item in the equation is the upfront fee, the seond item is the usage harge for launhing reserved VMs when demand is less than long term reservation apaity, the third item is the usage harge of launhing all the reserverd apapity when the demand exeeds the reserved apaity, and the fourth item is the ost for on-demand alloated VMs to serve the exeeded demand. The objetive is to minimize. In [31], we had derived an upper bound for alulating the amount of long-term resoures need to reserved for a general ase where demand is a ontinuous random variable. However, the IaaS provider has multiple priing models and multiple VM types. As a result, it is diffiult to ompute the differential of (R), Φ(R), and Λ(R) with respet to r. To make the problem tratable, and make the presentation easier to understand, we will derive the optimal amount of long-term reserved resoures with a simplified model where the demand is a disrete random variable and only one single type of VM is onsidered. Let be the probability that for any given time t. Let the average provisioning ost per unit of time, denoted by, be given by Sine T is relatively muh larger than t, an be written as follows: Now we simplify equation (3) by onsidering only one type of VM and the demand is given by number of required VMs (i.e., ). Furthermore, beomes the number of single type VM reserved for long-term planning. Let R={n} and r=n in the following equations. From Table 1, we also observed that for a single type of VM, (n), Φ(n), and Λ(n) are linear funtions of n. Speifially, ( n)=, Φ(n)=, and Λ(n)=. Therefore, we rewrite equation (3) as follows: It is obvious that minimizing is equivalent to minimize. Notie that (n), Φ(n), and Λ(n) are monotoni inreasing funtions and. Therefore, will derease as r inreases from 0, reah a minimum value, and then inrease again when r ontinues to inrease [17]. Now let us assume that r* is the optimal number of VMs to be reserved for long-term planning. That is,, for any possible r. In partiular, and. Thus we have:, (5), (6), (7), (8) and, (9), (10)

7 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY 7, (11). (12) Equations (8) and (12) give an upper bound and lower bound of the CDF of the optimal number of reserved VMs. Thus, based on these two equations we will be able to ompute the optimal solution. Now we show how to use the result of single type VM soltion for the original problem with multiple VM types. We first simplify the problem to a single type VM problem by only seleting the VM that has the best apaity/prie (CP) ratio. Under this onstraint, the workload demand is transformed to demand of the number of the best CP ratio VMs. And equations (4)~(12) will be derived based on the priing model of the VM with the best CP ratio. Based on equations (8) and (12), we will be able to obtain the optimal reservation r*. However, sine we have transformed the problem with multiple VMs to a single type of VMs, equations (8) and (12) suggest that we ould do a searh for the best ombination of multiple VM types with apaity falls between the apaity of and, where is the apaity of the VM with the best CP ratio. For example, if r* is 8, we an set the searh range of reserved workload demand to be between 7 CbestCP and 8CbestCP. Eah value in the range is regarded as the reserved demand of the following Integer Linear Programming formulation. Minimize (13) Subjet to (14) 0 M (15) The objetive funtion (13) is to minimize th e upfront ost of reserved VMs. Here, ni is a deision variable whih denotes the number of type i VM that is subsribed in the long term lease ontrat. For eah reserved demand, we alulate the set of ni through the Unbounded Knapsak Problem [28]; then using the historial data to determine whih one has the minimum total ost. 4.2 On-Demand Resoure Alloation Figure 3 shows a real trae of wordload demand of a learning management system. As shown in Figure 3, the demand flutuates over the monitoring time. Flutuations of demand would ause an under provisioning problem if we only rely on long term reserved resoures. For over provision ase, that is, when workload demand is less than the reserved resoure, attention is also required for the usage harge of launhing a reserved VM. Therefore, proper onfiguring VMs aording to the real-time workload demand is ritial to redue the operational ost. A straight forward way to onfigure VMs for next short-term planning interval is based on the measured demand of urrent interval. However, as we will show in our numerial results, onfiguring VMs based on some predition mehanism will signifiantly redue the operational ost. In addition, we also address the problem of provision lateny required by the IaaS provider to hange VM onfiguration. Figure 3. The aggregate historial w orkload demand traked by monitoring engine. Predition mehanism Several workload or network traffi predition mehanisms had been studied in the literature [25, 26]. Quite a few of them required historial data to train their system parameters or build their predition mo d- el. In this work, our predition mehanism is based on the Kalman filter beause that it has low omputation omplexity, does not require a training phase, and has been shown to be a good predition model for network traffi [29, 30]. We assume that the VM onfiguration an be re-onfigured at the beginning of eah shortterm planning interval; default length is 5 minutes. Our predition model will predit future resoure d e- mand based on the demand measured by the monito r- ing engine during the urrent time interval. Speifially, the measured workload demand is defined as the maximum number of simultaneously users aessing the servie or the maximum request rate; and the measured resoure demand is the minimum apaity to support the measured workload demand without violating the QoS or SLA guarantee. In this work, the system state of the Kalman filter is modeled as an M/M/ //M queue as shown in Figure 4 [29, 30], where N is the maximum number of users of the web appliation. Figure 4. Queueing model for the traffi environment. Let x(t){0, 1,...,N} and y(t){0, 1,...,N} denote the number of online users and requests at time t respe-

8 8 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID tively. Let Pn(t) denote the probability that n users enter the online servie system at time t; thereby, the Chapman-Kolmogorov equation is given as follows [29]: {( N ( n 1)) Pn 1( t) ( n 1) Pn 1( t)} {[( N n) n] Pn ( t (16) From (16), we an obtain the mathema tial formulation for the Kalman filter, whih inludes two steps: Predition step ~ x ( xˆ( k 1) Bu ( k 1) (17) ~ 2 P( pˆ( k 1) B Q( n 1) (18) Update step xˆ( ~ x ( K( [ y( Cx ~ ( ] (19) ~ ~ 2 K( P( C /[ P( C R( ] (20) Pˆ( [1 K( C] ~ p( (21) ( t )L C [1 e ] (22) t ( t )L D [( t ) L (1 e )] (23) t B D / C (24) N u (25) The meaning of the notations in the above formulae are illustrated as follows: Q( and R( are model and observation noise ovariane respetively, P( is error ovariane matrix, is the mean value of the model noise, and K( is the filter gain. Short-term planning algorithm (SPA) Table 3 summaries the notations used in our shortterm planning algorithm. Before we desribe details of the proposed algorithm, we desribe the operation flow first whih is shown in Figure 2. Table 3. List of notations used in SPA. Vari abl e Meaning r m The VM apaity requirement for urrent demand r The urrent-launhed VM apaity r r The overall VM apaity of reserved resoures r The preditive VM apaity assoiated Δ-time-ahead p demand Δ Time intervals in look-ahead demand predition proess I The urrent-launhed VM onfiguration I r T he VM onfiguration in reservation ontrat I o T he VM onfiguration subsribed via on-demand plan As shown in Figure 2, the operation flow starts from on-line measurement of workload and resoure demand performed by the Monitoring Engine. The observed demand is passed to the Workload Analyzer whih analyzes the maximum VM apaity requirement, r m, for the last planning interval. Subsequently, the predition model generates the predited apaity requirement, rp, for the next planning interval. At the same time, VM Repository provides the Elastiity Planner with the data of urrent-launhed VM onfiguration, I, and reserved VM onfiguration, Ir. Their VM apaities are r and rr respetively. Then, the Elastiity Planner onduts the SPA algorithm based on the above information to determine an adaptive planning. Finally, the Resoure Broker applies the adaptive planning to deliver resoure subsription. In the following, we desribe the proposed shortterm planning algorithm (SPA). Depending on the values of rp, r, and rr, the SPA lassifies the resoure planning senarios into three ases whih are illustrated as follows: Short-term planning algorithm (SPA) Input: r m, r p, r, r r, I, I r Output: the updated I, whih is used for adaptive planning Initialization: I := { 0 } // VM onfiguration subsribed via on-demand plan is empty o Proedure: 1 if r < r then // on-demand subsription is required r p 2 I o 3 I ILP1_OnDemand ( r p = I + I r o 4 else if r 5 I 6 else if r 7 I < r p -r, Δ) r then // launhing more reserved VMs is required ILP2_AdjustVMConfiguration( I r > r then p ShutDownSpareVMs( I, Δ, I, r ) p, r ) //need to shutdown some reserved VMs p 8 end if 9 UpdatePreditionModel( r m ) //use Kalman filter to predit next demand 10 return I End Proedure Figure 5. Short-term palnning (SPA) algorithm. Senario 1 (lines 1-3): Resoure on demand In this senario, the predited demand (rp) exeeds the apaity of all reserved VMs (rr), thus the Resoure Broker must operate the on-demand option to subsribe more VMs. Optimal solution of the on-demand VM onfiguration is formulated as an Integer Programming (IP) problem: Minimize (26) Subjet to (27) (28) 0 M (29) The objetive funtion (26) is to minimize the rental ost of on-demand VMs. Deision variable ni denotes th e number of VMs of eah type that is su b- sribed in the adaptive planning. Thus, ni may be any non-negative integer value as expressed in (29). The onstraint (27) ensures that the amount of subsribed resoure apaity exeeds the inadequay of the reserved resoures. VM re-onfiguration delay is also onsidered in the formulation. Let Ti(ni) be the startup delay funtion for provisioning of ni VMs of type i. The onstraint (28) restrits the VM deployment time within Δ. Senario 2 (lines 4-5): Adjust VM onfiguration In this senario, although the predited demand an be served by reserved VMs, but it exeeds the apaity of urrent VM onfiguration (r). Therefore, reonfiguring launhed VMs from the reserved VM pool (Ir) is nees-

9 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY 9 sary. Likewise, we formulate the VM re-onfiguration problem as an IP as follows: Minimize (30) Subjet to (31) (32) {0,1} (33) Here, we use a slightly different notation. Eah VM in the reserved VM pool (Ir) is assigned a unique ID (index) and its apaity is denoted by Cj. Let xj be a binary value that indiates whether the jth VM in Ir will be launhed in the updated onfiguration or not, that is: The objetive funtion (30) is to minimize the usage harge of reserved VMs. The onstraint (31) guarantees suffiieny of the re-onfigured VM apaity. In onstraint (32), the lateny of reonfiguration is alulated aording to a state-transition funtion : (35) where the first row indiates that the jth VM has a state transition from OFF ( ) to ON ( ). Moreover, we also regard the above IP formulation as a 0/1 Two-dimensional Knapsak Problem and aquire a feasible solution of minimum ost via dynami programming [28] or a heuristi algorithm [16]. Senario 3 (lines 6-7): Shutdown spare VMs In this senario, the predited demand is less than the urrently onfigured VM apaity. However, based on the priing model of Amazon EC2, we add a onstraint that a launhed VM will be harged based on a ertain period of time, e.g., 1 hour. Therefore, the orresponding ation is to shut down some launhed VMs, whih had nearly a full hour of operation first, until the provisioning apaity is just above the predited demand. 5 EVALUATION In this setion, the experimental evaluation of the pr o- posed algorithm is presented. We design a series of simulation-driven experiments based on the realworld data obtained from a web-based learning servie platform. The number of online users in eah 5-min interval was reorded as time series data from to Experiment Setup We first present the parameter settings of a loud omputing environment used in this performane evaluation. We adopt the priing models set by Amazon EC2, as shown in Table 1. Sine we re-onfigure VMs every short-term planning interval, we present the normalized upfront and usage osts per short-term planning interval in Table 4. To onvert the workload to VM apaity, we follow the analysis results of [16] where empirial profiling was resorted to determine the maximum request rate whih VM type an sustain for a TPC-W appliation. Sine the number of simultaneously on-line users is a ommon profiling metri used in e-learning systems, server apaity is set based on this metri. Speifially, in our simulations, we set the servie apaity of the four VM types to be 10, 30, 65, and 100 on-line users, respetively. Based on Table 4, the CP ratios of the four VM types are 10.00, 10.00, 10.83, and 8.33 respetively. The Large type VM has the best CP ratio. Table 4. Cost of eah VM type per planning interval (5-min.). Instane Type Small $ $ $ Medium $ $ $ Large $ $ $ Extra Large $ $ $ Validation of Optimal Resoure Reservation Estimation We first validate the proposed long term reservation solution. Two experiments were performed. We first ompared optimal long-term reservation of our solution to that of [17]. We then addressed the issue of multiple types of VMs. In the first experiment, we ompare the result of equations (8) and (12) to the result presented in Figure 7 of [17]. In order to do the omparison, parameters are set aording to that of Table II in [17]. Speifially, only one type VM is onsidered and the upfront fee (alled first stage ost in [17]) is set to $227.5/year (equivalent to $0.026/hr), usage fee for reserved VM (alled expending fee in [17]) is 0.03/hr, and ondemand ost is 0.085/hr. The demand distribution is assumed to be normal distribution with mean 25.5 and variane 6. Based on equations (8) and (12), we have and. Sine and, the optimal number of VMs to be reserved ( is 26. The optimal reservation suggested in [17] is 30. We notie that reserving 30 VMs is unreasonably high as the probability that the demand will higher than 30 is less than In other words, nearly 96.7% of time the atual demand is less than 30 VMs. We suspet the differene may due to the fat that the VMs were reserved in units of 5, therefore, 26 was not onsidered in [17]. It may also due to some other settings that are not onsidered in this work, suh as ost of storage and network. We verify our optimal reservation using the exhaustive searh method whih alulates and ompares the osts of all possible number of reserved VMs

10 10 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID (i.e., from 1 to 35). Results are depited in Figure 6, whih is in the same format of Figure 7 in [17]. As we an observe that total osts of reserving 26 VMs and 30 VMs are and , respetively. The results indiated that reserving 26 VMs yields the minimum total ost. As ompared to Figure 7 of [17], we also observe that the expending ost and on-demand ost shown in [17] are far muh too high whih may due to the onsideration of storage and network ost. For example, the expending ost for launhing 30 VMs all year long is only $7,884/year ($0.03*24*265*30), but it was shown to be more than $100,000 in Figure 7 of [17] whih is more than 12 times more. In the seond experiment, we use the real-world data olleted by the e-learning system. Sine the Large type VM has the best CP ratio, the parameters used in equations (8) and (12) are set as follows:,,. Thus, based on equations (8) and (12), we have and. The PDF and CDF of the olleted resoure demand (in unit of Large type VM) are shown in Figure 7. Inverting the CDF at , as shown in Figure 7 (b), we get the number of VMs to be reserved is between 7 and 8. Sine the realworld data was olleted every 5 minutes for two months, we are able to use the olleted data to o m- pute the operational ost for these two months given we have reserved a VM onfiguration. Furthermore, the olleted data is workload demand, i.e., the nu m- ber of on-line users. Thus, we an verify the optimal VM onfiguration obtained by the solution of Integer Linear Programming. Based on the apaity of a Large type VM, the workload demand an be obtained whih is between 455 and 520. For eah workload demand, we an also apply the ILP formulation to find the best VM onfiguration and its orresponding operational ost. Figure 8 shows the operational osts of the best VM onfigurations for various workload demands of applying the ILP solution. As we an observe that the optimal operational ost based on the olleted realworld workload demand is $437 and the VM onfiguration is 7 Large and 1 Medium VMs that have apaity of supporting 485 on-line users. Furthermore, we study the impat of on-demand ost on the number of reserved VMs. Figure 9 shows that the optimal number of VMs needs to be reserved as inreases from $ to $ As we an observe from Figure 9, the optimal number of reserved VMs inreases as inreases. The result is intuitive, as we inrease the on-demand ost, we need to reserve more VMs in the long term lease ontrat to avoid the hane of on-demand VM provisioning. PDF Number of required VMs (a) (b) Figure 7. The probability (a) and umulative (b) distribution funtions of the real data. CDF Number of required VMs Figure 8. Comparing ost of different number of reserved VMs based on the real data. Figure 6. Comparing the number of VMs reserved in first phase by proposed method and that of [17]. Figure 9. Comparing number of reserved VMs for different ondemand ost unit.

11 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY Impat of Penalty Cost and Lateny of VM reonfiguration When resoure provisioning is insuffiient to meet the workload demand, QoS will be violated, whih results in a penalty for the appliation provider. We use the penalty fator to present suh a situation. The value of the penalty fator ertainly affets the operational ost and determines the importane of QoS guarantee. As the penalty fator inreases, the more important the QoS is and the events of QoS violation should be dereased. Figure 10 shows the operational ost under a different penalty setting. Apparently, as the penalty fator inreases, the operational ost also inreases. (Here, usage ost inludes upfront fee.) workload demand ahead of time suh that suffiient resoures ould be provisioned in time. 5.4 Benefits of Predition Mehanism In this subsetion, we examine the benefit of using workload predition mehanism. Firstly, we explain the setup of the relevant parameters for the Kalman filter presented in Setion 4.2. Then, we define the evaluation riteria. Finally, we analyze and interpret the results of our experiments. Figure 10. Impat of penalty fator on the total ost. Subsequently, we disuss the impat of lateny of VM re-onfiguration on the rate of resoure insuffiieny and penalty ost. In this subsetion, we merely onsider a passive provisioning. That is, on-demand resoure alloation does not use the predition mehanism. We firstly define delay of VM-reonfiguration as shown in Figure 11. For example, at the beginning of t1, we used the measured workload of t0, i.e., d0, as the ondemand resoure requirement r1 for t1. However, due to the delay of VM re-onfiguration (assume the delay is 1), the resoure requested at the beginning of t1 will be provisioned at the beginning of t2. Therefore, the alloated apaity r1 based on d0 will be used to serve the demand of d2. In Figure 12, as the delay inreases, the number of intervals in whih the provisioned apaity is insuffiient suh that QoS is violated during these intervals and the orresponding penalty also inrease. Figure 11. The definition of delay unit. To summarize, knowing how to redue the ourrene rate of insuffiieny of provisioned apaity is the key to minimize the overall operation ost. The most fundamental approah is to be able to predit Figure 12. Impat of alloation lateny on resoure insuffiieny and penalty. The parameters of the M/M/ //M queue are set as follows: N = 1000, L = 5 minutes, = 1, = (40 minutes), and arrival rate λ(t) is derived as follows: (36) where l(t) denotes the number of users who login to the system at time t. We evaluate the auray of the predition based on two metris: Mean Absolute Perentage Error (MAPE), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). Besides, we define the formula whih measures the gain of invoking the predition mehanism as follows: (37) Here, C and C are the operation ost without and with the predition mehanism respetively. A higher value of Gain implies larger ost savings through the predition mehanism. Figure 13 shows the measured real-world workload, the predited workload for the next planning interval based on the measured workload of urrent interval, and the apaity offered via the optimal VM onfiguration generated by our proposed on-demand planning algorithm. (Only the measured data of last two weeks are shown in Figure 13.) The alulated MAPE, RMSE, and MAE are equal to 0.049, and 18.47, respetively.the parameter N is set to slightly larger than the maximum number of users observed in the two-month historial data. As we an observe that the predition urve losely resembles the demand urve, but is slightly larger for most of the time. The

12 12 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID urve of the apaity of servers alloated is jagged due to the apaity of eah type of VM differs signifiantly large and there is a minimum launhing time onstraint of a VM whih is set to one hour (i.e., when a VM is launhed, it must stay ative for at least one hour). predited workload for the next seond interval, we regard the predited workload for the next interval as the orreted value, then used it as the observed sa m- ple in the Update step of the Kalman filter; the predi t- ed workload for the next third interval is similarly performed. As a result, the inrease in the urve of multistep predition beomes faster as shown in Figure 16. For the error of the predited workload for three di f- ferent planning intervals, MAPE values are 0.049, 0.076, and respetively. Figure 13. Compariosn of Preditions, resoure alloations, and atual demands (only the data of the last tw o weeks are shown). Figure 14 shows the number of intervals in whih the provisioned apaity is insuffiient suh that QoS is violated during these intervals. As we an observe that with our predition mehanism, the number of inte r- vals in whih QoS is violated has been signifiantly redued. Figure 15. Comparing the improvement of 1-step-ahead predition. Next, we use the orresponding preditions to solve the problem of VM-reonfiguration delay (e.g., 2- step-ahead predition orresponds to delay of 2). Fi g- ure 17 shows the improvement of 2-step and 3 -step preditions. Figure 14. The effetiveness of the predition mehanism to the resoure insuffiieny. Figure 15 also shows the effet of VM reonfiguration lateny. Without th e predition mehanism, the VM is re-onfigured based on the workload of the urrent planning interval. With predition, the VM is re-onfigured based on the predited workload of the next planning interval. If VM reonfiguration delay is larger, it affets the operational ost of the no predition ase more than that of the proposed alg o- rithm with Kalman filter predition. However, here we used the 1-step-ahead predition and its improvement dereases as onfiguration delay inrases. As shown in Figure 16, the gap between delay of 2 and 3 is smaller than that of delay of 1 and 2. It is beause the predition is made for next planning interval only. Therefore, we extend our Kalman filter predition mehanism to be able to predit the workload for up to the next three planning intervals. When alulating the Figure 16. Comparing the preditions among different steps of ahead of time w here sr is the measured w orkload (demand). Figure 17. The Gain of eah pair whih has orresponding predition to the delay. 5.5 Comparison of Predition Mehanisms As aforementioned in Setion 2.3, several predition mehanisms for future resoure demands. In this setion, we ompare the performane of two ommon-

13 HWANG ET AL.: COST OPTIMIZATION OF ELASTICITY CLOUD RESOURCE SUBSCRIPTION POLICY 13 ly adopted mehanisms with Kalman filter, namely Hidden Markov Model (HMM) [18] and Neural Network (NN) [25]. HMM is a statistial model whih desribes a Markov proess with unknown variables. It an be used to analyze the unknown variables of a Markov proess from some observable parameters. It reuiqres a set of historial demand data as training set to build HMM. With the onstruted HMM, it is albe to predit future demands based on urrent observations (see [18] for details). On the other hand, a neural network onsists of the input(i), hidden(h) and ou t- put(o) layers [25]. The neurons have onnetions (synapses) to the next layer at eah layer. The neural network is fed with input vetors and eah synapse is assoiated with a weight. Error Corretion Neural Network (ECNN), a supervised learning model, is adopted for training the neural network. The ECNN adjusts the weights by using the bakpropagation (BP) algorithm. In our experiments, the learning rate of BP is set to 0.7. Furthermore, there is only one hidden layer in the neural network and the number of neurons in the input, hidden and output layer is set to 6, 4, 1, respetively. In our experiments, the surprised learning model ontinues to update the weights in the NN during the on-line predition phase. Table 5 ompares serveral performane metris of Kalman filter (KF), HMM, and NN. (Notation is given in Setion 5.4.) As we an observe that FK and HMM yield very ompetitive predition auray and are slightly better than NN. However, we notied that the total ost using HMM as the predition mehanism is muh higher than the other two mehanisms. To fu r- ther investigate this phenomenon, we examin e the usage ost (inludes upfront fee) and penalty ost of these three mehanisms in Figure 18. Apparently, the penalty ost of HMM is muh higher than that of the other two mehanisms. By examining the predited demands of HMM, we found that HMM tends to underestimate the demand whih results in under provisioning. Therefore, although the predited demand of HMM is very lose to the atual demand, but due to underestimation, it yields muh higher penalty ost. This problem was also observed in [18] and a remedy predition mehanism in whih the predited demand is set to the output with the onditional probability equaling to 85 perentile is proposed. Table 5. Comparison of performane of KF, HMM, and NN. Performane Metri KF HMM NN MAPE RMSE MAE Total Cost (penalty=0.5) Figure 18. Comparison of the total ost of FK, HMM, and NN under different penalty fator. 6 CONCLUSION Reently, IaaS infrastruture beomes a popular platform for appliation providers to deploy their appliations. However, IaaS providers offer many types of VM onfiguration and prie them differently. Fu r- thermore, they also offer several priing models. It raises an interesting issue to appliation providers on how to effetively provision or subsribe VM resoures from an IaaS provider. In this paper, we formulated the resoure provisioning problem as a two phase resoure planning problem. In the first phase, we foused on determining the optimal long term resoure provisioning. We proposed some mathematial formulae to ompute the optimal long term resoure onfiguration to minimize the expeted operational ost. In the seond phase, we proposed a Kalman filter predition model for prediting resoure demand. We then formulated the optimal resoure onfiguration for the predited demand as an Integer Progra mming pro b- lem and transformed it to an Unbounded Twodimensional Knapsak Problem whih an be solved via dynami programming or heuristi algorithms. Several issues had also been onsidered in our work, inluding impat of lateny of VM re-onfiguration, and minimum rental time onstraint for launhing a VM. We evaluated our proposed solutions based on workload data from a real system and Amazon EC2 s priing model. Our numerial results showed that the proposed long term resoure planning algorithm was able to yield near optimal operational ost. The results also showed that the proposed on-demand planning algorithm signifiantly redued the operational ost and was able to ope with the lateny of VM reonfiguration. In future, we plan to evaluate our solutions with larger resoure demand from some real web appliation systems. ACKNOWLEDGMENT This researh has been funded in part by the National Siene Counil under the Grants NSC E-194-

14 14 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID 012-MY3 and NSC E MY3. REFERENCES [1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Ratterson, A. Rabkin, I. Stoia, and M. Zaharia, A View of Cloud Computing, in Communiations of the ACM, vol. 53, April 2010, pp [2] I. Foster, Y. Zhao, I. Raiu, and S. Lu, Cloud Computing and Grid Computing 360-degree Compared, in Pro. of Grid Computing Environments Workshop (GCE), Nov. 2008, pp [3] S. Zhang, X. Chen, S. Zhang, and X. Huo, The Comparison Between Cloud Computing and Grid Computing, International Conferene on Computer Appliation and System Modeling (ICCASM), Ot. 2010, pp. V11-72-V [4] H. Brunst, W.E. Nagel, and A. D. Malony, A Distributed Performane Analysis Arhiteture for Clusters, in Pro. IEEE International Conferene on Cluster Computing (CLUSTER 03), De. 2003, pp [5] Gong, J. Liu, Q. Zhang, H. Chen, and Z. Gong, The Charateristis of Cloud Computing, in Pro. on the IEEE International Conferene Parallel Proessing Workshops 2010 (ICPPW 10), Sept. 2010, pp [6] W. Dawoud, I. Takouna, and C. Meinel, Infrastruture as a Servie Seurity: Challenges and Solutions, in Pro. the 7th International Conferene on Informatis and Systems 2010(INFOS 10), Cairo, Marh 2010, pp [7] M. Bonifae, B. Nasser, J. Papay, S. C. Phillips, A. Servin, X. Yang, Z. Zlatev, S. V. Gogouvitis, G.Katsaros, K. Konstanteli, G. Kousiouris, A. Menyhtas, D. Kyriazis, Platform-as-a-Servie Arhiteture for Real- Time Quality of Servie Management in Clouds, in Pro. Fifth International Conferene on Internet and Web Appliations and Servies 2010(ICIW 10), Barelona, May 2010, pp [8] W. Sun, X. Zhang, C. J. Guo, P. Sun, H. Su, Software as a Servie: Configuration and Customization Perspetives, IEEE Congress on Servies Part II 2008, Beijing, Sept. 2008, pp [9] Amazon EC2, [10] GoGrid, [11] A. Li, X. Yang, S. Kandula, and M. Zhang, CloudCmp: Comparing Publi Cloud Providers, in Pro. of Internet Measurement Conferene, 2010, pp [12] M. A. Rappa, The Utility Business Model and the Future of Computing Servies, IBM Systems Journal, vol. 43, no. 1, pp , [13] K. Tsakalozos, H. Kllapi, E. Sitaridi, M. Roussopoulos, D. Paparas, and A. Delis, Flexible Use of Cloud Resoures through Profit Maximization and Prie Disrimination, in Pro. of the 27th IEEE International Conferene on Data Engineering (ICDE 2011), April 2011, pp [14] L. He, D. Zou, Z. Zhang, K. Yang, H. Jin and S. Jarvis, Optimizing Resoure Consumptions in Clouds, in Pro. of the 12th IEEE/ACM International Conferene on Grid Computing (Grid 2011), 2011, pp [15] Z. Ying, H. Gang, L. Xuanzhe, and M. Hong, Integrating Resoure Consumption and Alloation for Infrastruture Resoures on-demand, in Pro. IEEE 3rd International Conferene on Cloud Computing, Jul. 2010, pp [16] U. Sharma, P.J. Shenoy, S. Sahu, and A. Shaikh, A Cost-Aware Elastiity Provisioning System for the Cloud, in Pro. International Conferene on Distributed Computing Systems, July 2011, pp [17] S. Chaisiri, B. S. Lee, and D. Niyato, Optimization of Resoure Provisioning Cost in Cloud Computing, IEEE Transations on Servies Computing, Feb. 2011, pp [18] W.-R. Lee, H.-Y. Teng and R.-H. Hwang, Optimization of Cloud Resoure Subsription Poliy, IEEE CloudCom 2012, Taipei, Taiwan, De. 3-6, [19] Y. Hu, J. Wong, G. Iszlai, and M. Litoiu, Resoure Provisioning for Cloud Computing, in Pro. of the 2009 Conferene of the Center for Advaned Studies on Collaborative Researh, 2009, pp [20] R.N. Calheiros, R. Ranjan, and R. Buyya, Virtual Mahine Provisioning Based on Analytial Performane and QoS in Cloud Computing Environments, in Pro. International Conferene on Parallel Proessing, Sept. 2011, pp [21] H. Zhao, M. Pan, X. Liu, X. Li, and Y. Fang, Optimal Resoure Rental Planning for Elasti Appliations in Cloud Market, the 26th IEEE International Parallel and Distributed Proessing Symposium, Shanghai, China, [22] M. Mao, J. Li and M. Humphrey, Cloud Auto-Saling with Deadline and Budget Constraints, in Pro. of 11th ACM/IEEE International Conferene on Grid Computing (Grid 2010), 2010, pp [23] C.C.T. Mark, D. Niyato, and C. Tham, Evolutionary Optimal Virtual Mahine Plaement and Demand Foreaster for Cloud Computing, in Pro. IEEE International Conferene on Advaned Information Networking and Appliations (AINA), 2011, pp [24] S. Chaisiri, R. Kaewpuang, B. S. Lee, and D. Niyato, Cost Minimization for Provisioning Virtual Servers in Amazon Elasti Compute Cloud, in Pro. International Symposium on Modeling, Analysis and Simulation, of Computer and Teleommuniation Systems (MASCOT), July 2011, pp [25] S. Islam, J. Keung, K. Lee, and A. Liu, An Empirial Study into Adaptive Resoure Provisioning in the Cloud, Future Generation Computer Systems, vol. 28, pp , Jan [26] E. Caron, F. Desprez, and A. Muresan, Foreasting for Grid and Cloud Computing On-Demand Resoures Based on Pattern Mathing, in Pro. Cloud Computing Tehnology and Siene (CloudCom), 2010, pp [27] G. Welh and G. Bishop, An Introdution to the Kalman Filter, University of North Carolina at Chapel Hill, Chapel Hill, NC, [28] S. Martello and P. Toth. Knapsak Problems: algorithms and omputer implementations, John Wiley & Sons, Chihester New York, [29] A. Kolarov, A. Atai, and J. Hui, Appliation of Kalman Filter in High- Speed Networks, in Pro. IEEE GLOBECOM. Communiations Conf., vol. 1, San Franiso, 1994, pp [30] T. Anjali, C. Bruni, D. Iaoviello, G. Koh, and C. Soglio, Filtering and Foreasting Problems for Aggregate Traffi in Internet Links, Performane Evaluation, vol. 58, no. 1, pp , Ot [31] Yi-Ru Chen, Cost Optimization of Elastiity Cloud Resoure Subsription Poliy, Master Thesis, Department of Computer Siene and Info. Eng., National Chung Cheng University, Taiwan, July Ren-Hung Hwang reeived his Ph.D. degree in omputer siene from University of Massahusetts, Amherst, Massahusetts, USA, in He joined the Department of Computer Siene and Information Engineering, National Chung Cheng University, Chia-Yi, Taiw an, in 1993, where he is now a distinguished professor of the department of Computer Siene and Information Engineering. His researh interests inlude wireless networks, Internet of Things, and loud omputing. He is a senior member of the IEEE. Chung-Nan Lee reeived his Ph.D. degree in eletrial engineering from the University of Washington, Seattle, in Sine 1992, he has been w ith National Sun Yat-Sen University, Kaohsiung, Taiwan, where he is now a Professor in the Department of Computer Siene and Engineering. His urrent researh interests inlude multimedia over w ireless networks, loud omputing, and evolutionary omputing. Yi-Ru Chen reeived her master degree from the department of Computer Siene and Information Engineering, National Chung Cheng University. She is now a software engineer in Cyberlink. Her researh interest is in loud omputing. Da-Jing Zhang-Jian reeived MS degrees in Computer Siene and Engineering from National Sun Yat-sen University, Kaohsiung, Taiwan, in He is urrently working toward the PhD degree in the Department of Computer Siene and Engineering at the National Sun Yat-sen University. His researh interests inlude loud omputing, pow er management and omputer graphis.