Online Resource Scheduling under Concave Pricing for Cloud Computing

Transcription

1 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems Online Resource Scheduling under Concave Pricing for Cloud Computing Rui Zhang, Kui Wu, Minming Li, Jianping Wang City University of Hong Kong, Hong Kong {minming.li, University of Victoria, Canada Abstract With the booming cloud computing industry, computational resources are readily and elastically available to the customers. In order to attract customers with various demands, most Infrastructure-as-a-service (IaaS) cloud service providers offer several pricing strategies such as pay as you go, pay less per unit when you use more (so called volume discount), and pay even less when you reserve. The diverse pricing schemes among different IaaS service providers or even in the same provider form a complex economic landscape that nurtures the market of cloud brokers. By strategically scheduling multiple customers resource requests, a cloud broker can fully take advantage of the discounts offered by cloud service providers. In this paper, we focus on how a broker can help a group of customers to fully utilize the volume discount pricing strategy offered by cloud service providers through cost-efficient online resource scheduling. We present a randomized online stack-centric scheduling algorithm (ROSA) and theoretically prove the lower bound of its competitive ratio. Three special cases of the offline concave cost scheduling problem and the corresponding optimal algorithms are introduced. Our simulation shows that ROSA achieves a competitive ratio close to the theoretical lower bound under the special cases. Trace-driven simulation using Google cluster data demonstrates that ROSA is superior to the conventional online scheduling algorithms in terms of cost saving. Index Terms Cloud computing; Bulk purchasing; Concave pricing Introduction In the past few years, we have witnessed the tremendous development of cloud computing, with more and more cloud service providers jumping on the cloud bandwagon. Along with the stable growth of large scale public cloud providers like Amazon EC[], Windows Azure[9] and Rackspace[], small scale cloud providers such as ReadySpace[] and GoGrid[] have vigorously emerged. Despite the hype about cloud computing, however, the actual adoption rate of cloud computing is still behind expectation [9], especially outside the United States. Clearly, to the entire cloud industry, it is crucial to stimulate end users participation in cloud computing. From an individual cloud service provider s perspective, it is important to keep its competitiveness among peer cloud service providers. As analyzed in [6], the only way to cloud computing success is to develop adequate pricing techniques. In an Infrastructure-as-a-Service (IaaS) cloud, the cloud provider dynamically segments the physical machines, using virtualization technologies, to accommodate various virtual machine (VM) requests from its customers. In principle, the customers only need to pay for the resource they actually consumed. Nevertheless, the pay-as-you-use pricing model is presently only ideological due to the high complexity in monitoring and auditing resource usage, such as network bandwidth, virtual CPU time, memory space, and so on. Consequently, real-world charging schemes in IaaS cloud have become absurdly complicated []. For instance, cloud providers usually adopt an hourly billing scheme, even if the customers do not actually utilize the allocated resources in the whole billing horizon [5]. In the current cloud market, many cloud providers offer big discount for reserved and longterm requests [][][6]. In addition, cloud providers usually give volume discount to customers with requests of large quantity, e.g., Amazon EC cloud [] gives % discount for customers spending $5, or above on reserved instances and % discount for customers spending $, or above. The diverse pricing schemes and various discount offers among different IaaS service providers or even within the same provider form a complex economic landscape way beyond the control of individual end users. This leaves opportunities for the cloud brokers to emerge as mediators between the customers and the providers. Following the above trend, dedicated cloud brokers are emerging to help customers make better purchase decisions. Recent work shows that cloud brokers who mediate the trading process between the customers and the cloud providers can significantly reduce the cost for the customers while helping the cloud providers to reshape or smooth out the burst in the incoming VM requests [5]. Recent market study expects that the global cloud services brokerage market will be worth $.5 billion US dollars by 8 []. A cloud broker can help reduce the cost of customers through temporal multiplexing and spatial multiplexing of resources. By temporal multiplexing, the broker takes advantage of providers hourly billing cycles to use a customer s unused resource for executing other customers tasks [][5][5]. The goal is to maximize resource utilization so that more customers can be accommodated and in return each can pay less. By spatial multiplexing, the broker takes advantage of volume discount by packing multiple customers resource requests to meet the providers high threshold for bulk resource purchase, thus, the total cost can be reduced and each can pay less consequently. While the advantages of temporal multiplexing have been thoroughly investigated before [5], the benefit of spatial multiplexing remains less explored (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

2 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems Job Job Job Time (a) Resource Job Volume discount threshold Job Job Time (b) Resource Job Job Job Job Time (c) Fig. : Example of conventional scheduler not producing the schedule with optimal cost. When being offered with volume discount from a cloud service provider, end customers may be willing to adjust the execution speed of their jobs, especially those time-flexible and interruptiontolerant tasks, so that a higher volume discount can be enjoyed due to the higher amount of total requested resource of the jobs from a group of customers. We use an example to illustrate that conventional scheduling may not lead to the optimal cost under volume discount. As shown in Fig. (a), we have three incoming jobs. Job arrives at time with a deadline of 5, a workload (which is measured by the amount of requested resource) of 6 and a maximum processing speed of. Job arrives at time with a deadline of 7, a workload of and a maximum processing speed of. Job arrives at time 6 with a deadline of 9, a workload of 6 and a maximum processing speed of. Suppose that the threshold for volume discount is, a conventional scheduler may schedule a job with its maximum processing speed starting from the instant when the job is submitted, as shown in Fig. (b). Under this schedule, two units of workload from job can enjoy the volume discount. We can observe that postponing the starting time for processing job to time and dividing the execution of job into two segments give better opportunity in enjoying volume discount, as shown in Fig. (c). Though different cloud service providers may offer different pricing strategies with volume discount, a pricing strategy with volume discount can be modeled as a concave function in general, i.e., the total cost of two separate purchases for resource amounts r and r, respectively, should be no less than the cost of a single purchase of the same total resource amount r + r. To discover cost efficient online scheduling algorithm under a concave cost function, this paper makes the following contributions: Under a generic concave cost function, we investigate the basic features that a cost optimal scheduling should possess. Three special cases of the concave cost scheduling problem are introduced, namely, scheduling under a linear function with a fixed activation cost, laminar-structured job requests, and unit job requests with agreeable deadlines. We show that each special case can be solved offline using a polynomial algorithm. We propose an online request reshaping algorithm, called randomized online stack-centric scheduling algorithm (ROSA), under a generic concave cost function. We theoretically prove the lower bound of its competitive ratio [7] and evaluate its performance with trace-driven simulation. In this example, the processing speed of a job is assumed to be equal to its instantaneous resource consumption which is charged accordingly. using Google cluster data. Experimental results show that ROSA achieves a competitive ratio close to the theoretical lower bound under the special case cost function and is superior to the conventional online scheduling algorithm in terms of cost saving. The rest of the paper is organized as follows. In Section, we formulate the concave cost job scheduling problem. In Section, we analyze the properties that an optimal schedule should possess. In Section, Section 5, and Section 6, we study three special cases of the concave cost scheduling problem, scheduling under a linear function with a fixed activation cost, laminar-structured job requests, and unit job requests with agreeable deadlines, respectively. In Section 7, we propose and study a randomized online algorithm, ROSA, which achieves low competitive ratio with a linear complexity. Section 8 presents our experimental results using Google cluster data. Section 9 concludes the paper. Problem Formulation This paper considers the resource scheduling problem for IaaS clouds, where multiple customers may submit job requests at random instants with random workload that should be fulfilled before specified deadline to a broker. We assume that the inter-arrival times for job requests are arbitrary. We assume that the processing time for each job is deterministic and known to the broker given the resource allocated to the job. The broker is responsible for purchasing computational resource from IaaS clouds, allocating resource to and executing jobs, as well as meeting job deadlines. The deadlines specified by the customers are flexible. Different from PaaS cloud, where the customers directly submit job requests to cloud service providers, brokers mediate the process by organizing the job requests in a manner which benefits the most from the volume discounts provided by the cloud provider. Both the cloud provider and the customers benefit from this mediation. Individual customers can enjoy volume discounts which often require a large volume of job requests. The cloud provider benefits from the revenue boosted by the brokerage. To ease analysis, we assume that time is slotted, and jobs arrive at the beginning of a time slot. In any unit time slot, a job either is allocated with no resource or uses allocated resource in the whole time slot, unless otherwise stated. For convenience, a set of symbols listed in Table are used. Assume that n job requests, J, J,..., J n, are submitted during the time interval [, t], where t is an arbitrary time instance. Respectively, let ti a, td i, w i denote the arrival time, deadline, and the workload of job request J i. We set an upper limit on the resource that could be allocated to task J i at any time instant, denoted by where w i. We introduce to reflect the case that the execution of a task cannot be further accelerated given additional resource. A task J i can be denoted by a tuple < ti a, td i, w i, > (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

3 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems We assume that the cloud provider has abundant computing capacity at any time instant t. The broker is not restricted to give equal amount of resource to each running job. The broker can adjust the computing resource allocated to each task J i at time t, denoted by r i (t). Naturally, r i (t) = indicates that no resource is assigned to J i at time t. If J i has already been partially processed, it is paused at t. This assumption is theoretically reasonable and practically feasible. Theoretically, as long as each task meets its deadline, the scheduler should have the freedom to assign the resources in order to reduce the cost. Practically, there are many approaches of dynamically adjusting the resources allocated to a running job. For example, the resource allocation for tasks implemented in Apache Hadoop [5] can be controlled by dynamically adjusting the number of mappers. Formally, we require that r i (t) be piecewise constant with finitely many discontinuities. We define R(t) as the total allocated resource at time t, i.e., R(t) = i r i (t). The framework is generic since, r i (t), and R(t) can represent any type of resource such as CPU, memory, network bandwidth, and disk. The broker purchases computational resource from IaaS clouds and has to pay for the resource cost. The broker intends to meet all job deadlines while reducing the total resource cost. We model the resource cost as follows. Associated with the allocated resource at time t, R(t), is a resource cost, which can be approximated by a non-decreasing function f ( ), i.e., Instantaneous resource cost at time t = f (R(t)). () The customers evaluate the broker based on two factors: Whether the job deadlines are met and the price they need to pay for their jobs. If the broker can get discount for the total resource cost of all jobs. It can redistribute the discount to every single job so that all customers can benefit from it. A trivial example would be using a proportional cost sharing scheme, i.e., the cost to pay for a job is proportional to the amount of resource the job uses. Therefore, the cost for job J i at time t is calculated as: ri(t) R(t) f (R(t)). A feasible job schedule consists of resource functions r i (t), i =,..., n, defined over the entire time axis that satisfy: t d i t a i r i (t)dt w i, i =,..., n, () r i (t), t [ti a, td i ], i =,..., n, () r i (t) =, t [ti a, td i ], i =,..., n. () The optimal resource scheduling problem is to find a feasible schedule that minimizes the total cost: min r i(t) C = f (R(t))dt (5) Significantly different from previous work on speed scaling [], [6], [8], the cost function is not assumed to be convex in our case. Instead, it is approximated as a concave function. The optimal task scheduling problem turns out to be minimizing a concave function, which is hard to solve. The lack of convexity in the cost function invalidates all existing solutions such as those in [], [6], [8]. Note that linear programming (LP) with rounding approximation is commonly used for constrained optimal job scheduling problems [8] []. In Section, we demonstrate that by proving properties of optimal solutions, elegant scheduling algorithm can be found when finding an appropriate LP solution is hard. TABLE : List of symbols used in Section Symbols Descriptions t, t i, ti a {} R + Time instants [t, t ] A continuous time interval between t and t, inclusive. J i, i N A job request with index i ti a The arrival time of job request i ti d The hard deadline (specified by the job submitter) of job request i w i R + The amount of workload to be completed for job i w i, R + The maximum resource requirement for job i, assumed equivalent to the maximum processing speed of job i r i (t) {} R + Instantaneous resource allocation and processing speed of job i at time instant t R(t) = i r i (t) Instantaneous resource allocation of all submitted jobs at time instant t f ( ) {} R + Cost function Offline Resource Scheduling Minimization with a concave cost function usually falls into the class of NP-hard problems, for example, the concave network flow problem []. This partially suggests the hardness of our scheduling problem. Though we have not formally proved its NPharness, we have discovered the properties of optimal scheduling with a general concave cost function. These properties provide us with valuable insights on making cost-efficient decisions in offline and online resource scheduling. Furthermore, these properties have inspired us to find an optimal offline scheduling algorithm for a special concave cost function. In this section, we present the properties that an optimal schedule should have and point out why it is hard to come up with an optimal scheduling algorithm with polynomial complexity. Additional symbols used are listed in Table. Lemma. Assume that f ( ) is a positive, non-decreasing, and piecewise concave function. Then for any time interval [t, t ], an arbitrary positive function x(t), and any positive value max t [t,t ] x(t), the following inequality holds: t t t t f (x(t))dt t t f ()dt + f ( )dt + ( f (v) f ())δ( w i w i ), (6) where w = t x(t)dt, v = w t i wi u, δ(y) = if y > and otherwise, and t = t + wi. TABLE : List of symbols used in Section Symbols Descriptions x(t), t {} R + An arbitrary positive function Rounded down A, A, A i, A Schedules of all job requests R i (t) R(t) r i (t) f (x(t)) f (x(t) + R i (t)) δ(y), y {} R + δ(y) = when y > otherwise δ(y) = 5-99 (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

4 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems Proof. We have t f (x(t))dt t t ( ) f (x(t)) f () = x(t) + f () dt t x(t) t ( ) f (ui ) f () x(t) + f () dt = = t t f ( ) x(t)dt + t t t f ( )dt + t t t t f ()dt +( f (v) f ())δ( w i w i ). f ()( x(t) )dt The practical meaning of Lemma is as follows. We assume that the cost function is positive, non-decreasing, and piecewise concave. The resource cost of a job schedule is computed using (5). The cost optimal way to schedule job J i in an interval where no other jobs are scheduled is to allocate the maximum possible resource,, to J i for reducing its processing time. With Lemma, it is easy to prove the following lemma. Lemma. Assume that the cost function is positive, nondecreasing, and piecewise concave. Assume that there is a feasible schedule, denoted by A, for a given set of n tasks J,..., J n. Let [t k, t k+ ] be an arbitrary time interval, such that, for all tasks J i, i =,..., n, r i (t ) = r i (t ) where t, t [t k, t k+ ]. If A allocates to task J i the resource r i (t) <, t [t k, t k+ ], then there exists another feasible schedule, A, that behaves the same as A for t [t k, t k+ ] while r i (t) =, t [t k, t k+ ]. In addition, the total resource cost of allocation A is no larger than that of A. Proof. Define a time instant t m such that t k t m t k+ and t m = t k + tk+ t k r i (t)dt. We can compress the execution time of task J i into the time period [t k, t m ] with less total cost, by assigning J i the maximum resource in [t k, t m ]. Let us call this new schedule A. Denote R i (t) = j i r j (t). Define f i (x(t)) = f (x(t) + R i (t)). Since the cost function f ( ) is positive, non-decreasing, and piecewise concave, so is f i ( ). Based on Lemma, tk+ t k f i (r i (t))dt tm t k f i ( )dt + tk+ t m f i ()dt +( f i (v) f i ())δ( w i w i ), (7) where w = t k+ r t i (t)dt, v = w w k, and δ is the indicator function as in Lemma. Equivalently, tk+ t k f (R(t))dt tm t k f (R i (t) + )dt + tk+ t m f (R i (t))dt +( f (v + R i ) f (R i ))δ( w i w i ), (8) where R i can be any feasible value of R i (t), t [t m, t k+ ], meaning that the remaining amount of job i (i.e., v) could be allocated to any time interval in [t m, t k+ ]. Inequality (8) means the resource cost of A is not less than that of A. In addition, since the schedule A is feasible, schedule A is also feasible as it does not violate any deadlines. The practical meaning of Lemma is that whenever resource is allocated to a job J i, the optimal solution should allocate its maximum resource. Lemma. Assume that the cost function, f ( ), is positive, non-decreasing, and piecewise concave. Given a job J i =< ti a, td i, w i, > which can be fully allocated in either [t, t ] or [t, t ]. Both R i = R i (t), t [t, t ] and R i = R i (t), t [t, t ] are constants. Without loss of generality, R i R i. Then, the resource cost of allocating J i to [t, t ] with processing speed is no greater than the resource cost of allocating J i to [t, t ] with processing speed. Proof. It is sufficient to prove the following inequality: f ( + R i ) y + f (R i )(t t y) + f (R i )(t t ) f ( + R i ) y + f (R i )(t t y) + f (R i )(t t ), (9) where < y wi. By concavity of f ( ) and the condition that R i R i, the following inequality holds: ( f ( + R i ) f (R i )) ( f ( + R i ) f (R i )), This implies that, y ( f ( + R i ) f ( + R i )) y ( f (R i ) f (R i )) Therefore, (9) holds. Note that Lemma and Lemma are correct for any concave cost function f ( ). Connecting Lemma to the resource scheduling problem, we immediately get the following corollary. Corollary. Assume that the cost function is positive, nondecreasing, and concave. If a schedule A is optimal, then, there does not exist a scheduled job J i such that a segment of the allocated processing time for J i can be reassigned to another time interval which has a higher scheduled resource excluding the resource allocated to J i. Unfortunately, the converse of Corollary is not true. We have the following counter example. Let J =<,,, >, J =<,, 5, 5 >, J =<,, 5, 5 > and J =<,,, > be four jobs to be scheduled. We assume that the cost function is f (x) = x. Fig. (a) shows a globally optimal schedule with the optimal cost 6.9 which satisfies the property of Corollary. Fig. (b) shows a non-optimal schedule which also satisfies the condition of Corollary with a cost of Scheduling under a Linear Function with a fixed activation cost In this section, we study the concave cost job scheduling problem under a special piecewise concave cost function which is a linear function with a fixed activation cost. The instantaneous resource cost at time t becomes, f (R(t)) = H R(t) + K δ(r(t)) () where H, K R +. H stands for the unit resource price, K stands for the fixed activation cost and δ(r(t)) = if R(t) > and δ(r(t)) = otherwise. A linear function with a fixed activation cost is commonly used in practice, which has a straightforward business justification. In many cases, whenever some service is provided, it will incur a fixed activation cost that is independent of the service quantity, for example, the cost incurred for system 5-99 (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

5 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems 5 Resource J J J J Time (a) Resource J J J J Time Fig. : (a) A globally optimal schedule (b) A non-optimal schedule which satisfies the condition of Corollary (b) monitoring and administration, or a minimum revenue asked by the service provider for any single transaction. For an arbitrary schedule A, the cost can be written as, f (R(t))dt = K D(A) + H w i () where D(A) is defined in Table. As H i w i is constant when all jobs are scheduled, the problem becomes minimizing D(A). In the literature of scheduling, this problem is referred to as activation cost minimization. Chang et al [8] studied a similar problem with the following constraints: () At any time instance, the number of scheduled jobs is limited by a constant. () All jobs can be processed in a unit time interval. () All jobs can be preempted only at integer time points. They showed that the problem can be solved offline using linear programming. In this section, constraint () is relaxed. We assume: (a) Jobs can be preempted at any time points. (b) The number of scheduled jobs at any time instant is unbounded. This problem is solvable using Linear Programming []. In the rest of this section, an optimal online algorithm which schedules jobs greedily and sequentially is introduced []. This greedy approach is more intuitive. The purpose of introducing this special case cost function and the greedy algorithm is to assess the proposed online algorithm introduced in Section 7. TABLE : List of symbols used in Section, Section 5, and Section 6 Symbols H R + K R + D(A) S J L i τ I Descriptions Unit resource price Activation cost The length of the union of all time intervals with at least one job allocated in schedule A A set of job requests Latest possible start time of J i Smallest latest possible start time amount all jobs An ordered list of time instances Definition. We first define the term latest possible start time for job J i =< ti a, td i, w i, > as L i = ti d wi where w i stands for the remaining workload of J i during the scheduling process. Let τ = min i {L i } indicate the smallest latest possible start time. Lemma. Let S J = {J i i =,..., n} be a set of n jobs to be scheduled and τ = min i {L i } be the smallest latest possible start time among all jobs. Assume that the cost function is a linear function with a fixed activation cost, as defined in (). There exists an optimal schedule in which no jobs are scheduled before τ. Proof. Suppose there is a set of n jobs, S J = {J i i =,..., n} and J i =< t a i, td i, w i, >, to be allocated in time interval [, T], where i T is the maximum time instance. Let g be an arbitrary schedule of jobs in S J. Let us show that we can reallocate the workload which is scheduled in time interval [, τ] by g to time interval [τ, T] without incurring the total cost. Let I (g) be the union of time intervals with at least one job scheduled in g. Let I denote the entire time space, i.e. I = [, T]. Step : We reallocate the workload of J i to the time intervals (I (g) {t τ < t < ti d }) as much as possible. Let us call the resulting schedule g. According to (), C(g ) C(g). Let x i be the amount of workload from J i that is allocated in [, τ] by g. Step : Let S J = {J i x i >, J i S J } and J m =< tm, a tm, d w m, u m > be such that tm d = min Ji S (td J i ). We reschedule the workload of J m which is scheduled in [, τ] (with an amount of x m ) by g into the intervals ((I I (g)) {t τ < t < tm}). d This is feasible as the entire workload of J m can be scheduled in [τ, T]. After the reallocation, all workload of J m is scheduled in [τ, T]. Step : For convenience, let I(J m, g) and I(J m, g ) be the sets of time intervals, where J m is allocated in g and in g, respectively. We reallocate the workload of J i S J from (I(J m, g) [, τ]) to (I(J m, g ) [τ, T]). This is also feasible as ti d tm. d Let us call the resulting schedule g. We repeatedly perform Steps - on g and any subsequent schedules until all jobs are reallocated to [τ, T]. Let us call the resulting schedule g k. Let I k (g) denote the union of intervals with at least one job scheduled in g k. By Step and, it is obvious that I k (g) I (g). Hence, C(g k ) C(g). The same procedure can also be applied to an optimal schedule. Hence, there exists an optimal schedule such that no jobs are scheduled before τ. Under the linear cost function (), it is trivial that, if a job has to be allocated to a particular time interval, allocating other jobs as much as possible to this time interval will not incur additional cost. Combining with Lemma, we find a deterministic procedure which leads to an optimal schedule under the cost function (). This procedure is presented in Algorithm. As there are a pair of nested loops which iterate over n jobs, Algorithm has an complexity of O(n q), where q is the maximum number of reallocation to be performed on one job in each step. Theorem. If the cost function is in the form of (), scheduling by Algorithm is optimal. Proof. The proof directly follows by Lemma. In this section, a special case cost function and an algorithm to perform cost optimal job allocation under such cost function were introduced. In Section 8, we use this offline algorithm to evaluate the randomized online algorithm introduced in Section (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

6 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems 6 Input: S J, the set of all jobs; Output: A, cost optimal job schedule while S J do Calculate L i for all J i S J using Definition ; τ = min i L i ; 5 tmax d = 6 for J i S J do 7 if L i = τ then 8 Schedule J i in [τ, ti d ] in A; 9 S J = S J {J i }; tmax d = max(tmax, d ti d); end end for J i S J do if τ < ti d then 5 Schedule the remaining workload of J i as much as possible in interval [τ, tmax] d with resource in A; 6 Update w i as the amount of the unscheduled workload of J i ; 7 if w i = then 8 S J = S J J i ; 9 end end end end Algorithm : Optimal scheduling algorithm with a linear cost function plus positive fixed activation cost 5 Laminar-structured Job Requests In this section, we study a special pattern of job requests under general concave cost functions. The condition for this special pattern can be described as follows. For any pair of jobs, J =< t a, td, w, u > and J =< t a, td, w, u >, such that t a ta, exactly one of the following conditions hold: (). t d ta or (). t d td. In other word, interval [ta, td ] either contains interval [t a, td ], or the two intervals are mutually exclusive to each other. A simple example is demonstrated in Fig.. In Fig., the time intervals, in which the jobs can be allocated, are represented using circles. We mark the circles using their corresponding jobs, J i, i N. We draw an arrow from circle J i to circle J j if and only if ti a t a j and ti d t d j. Let us call this a laminar structure since the circles not pointed by any arrows form the top lamina, the circles only pointed by one arrow form the second lamina, and so on. In J J J J J 5 t a t a t a t d t a t d Time t d t 5 a t d t 5 d Fig. : Laminar-structured job requests this section, we show that an job schedule which minimizes (5) can be found by allocating jobs lamina by lamina from bottom to top. If we omit the arrow from J i to J j when there exist arrows from J i to J k and from J k to J j, a laminar structure can be represented using a number of disjoint tree structures. Laminar structure is interesting as it is often used to model job requests created from recursive procedures []. It is often studied in contrast to agreeable deadlines which is introduced in the next section. In distributed systems, it is common that during the execution of the main task several subtasks are invoked which have to be finished before the deadline of the main task. This can be modeled using a laminar structure. In this section, we show that the offline cost optimal scheduling problem for laminar-structured job requests can be solved using a polynomial time algorithm. We use tree structure and laminar structure interchangeably. top lamina nd lamina rd lamina J J J J Fig. : Laminar model for the job requests shown in Fig.. Here, each job is represented as a node. An arrow from J i to J j indicates that the time interval [t a i, td i ] contains the time interval [ta j, td j ]. If the laminar structure of a set of job requests, S J, is composed of several disjoint trees, we can combine them as a single tree by adding a dummy job request which has the earliest arrival time and the latest deadline. A subtree of S J is a subset of S J which is also a subtree of the tree structure representation of S J. For example, J, J, and J form a subtree of the laminarstructured job requests shown in Fig.. Before introducing the optimal scheduling algorithm, we first study the properties an optimal schedule should possess for laminar-structured job requests. Lemma 5. If a schedule for a set of laminar-structured job requests, S J, is cost optimal under a concave cost function, the schedule is optimal for each subtree of S J. Proof. We prove Lemma 5 by contradiction. Let S J be a set of laminar-structured job requests. Let A be an optimal schedule for S J. We assume that there exists a subset of S J, S J S J, which is also a subtree in the tree representation of S J, such that S J is not allocated with its minimum cost. We break down this problem into two cases. In the first case, no jobs in S J S J is allocated to a time interval that overlaps with the scheduled processing time of the jobs in S J. This implies that the cost of other jobs are independent with the jobs in S J. As A is not optimal for S J, A cannot be optimal for S J. In the second case, there exist a set of jobs, S J S J S J which are allocated to time intervals that overlap with the scheduled processing time of the jobs in S J. By definition, the jobs in S J can only be on higher levels than the jobs in S J with respect to the tree representation of S J. Therefore, the union of the time intervals where the jobs in S J can be allocated contains that of the jobs in S J. By Lemma, A is not optimal. Hence, contradiction is reached in both cases. Definition. Let S J be a set of jobs and A be an arbitrary schedule of S J. The resource consumption profile of A, RCP(S J, A), J (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

7 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems 7 is a set of tuples (resource, length) with distinct resource values. Each tuple in RCP(S J, A) stands for the total length of time intervals with the corresponding allocated resource value. For example, the resource consumption profile of the schedule shown in Fig. (a) is {(, )}. The resource consumption profile of the schedule shown in Fig. (b) is {(, ), (, )}. Lemma 6. The optimal schedules of a set of laminar-structured job requests have the same resource consumption profile. Proof. Let S J be a set of laminar-structured job requests. We show that all optimal schedules of S J have the same resource consumption profile by induction. For the base case, we need to show that the optimal solutions of the leaf nodes in the tree representation of S J have the same resource consumption profile. This is trivial by Lemma, as the optimal schedule under a concave cost function allocates the maximum resource to each job. Now, we assume that the root node of S J, J root, has m direct children, J, J,..., J m, and the corresponding subtrees rooted from these children are denoted as S, S,..., S m. We assume that the optimal schedules for each of S i, i =...m have the same resource consumption profile, RCP(S i, A i ), i =...m, where A i denotes an optimal schedule of S i. We need to show that it is also true for S J. Let A and A be two optimal schedules of S J. By Lemma 5, we know that the optimal schedule of S J consists of optimal schedules for S i, i =...m. This implies that RCP(S i, A) = RCP(S i, A ), i =...m. According to Lemma and Lemma, the optimal schedule of S J consists of a greedy allocation of J root. As the time interval which J root can be allocated contains that of all other jobs. The greedy allocation leads to a unique resource consumption profile, i.e., RCP(S J, A) = RCP(S J, A ). According to Lemma 5 and Lemma 6, we propose an optimal offline scheduling algorithm for laminar-structured job requests, shown in Algorithm. Algorithm has a computational complexity of O(nm) where n is the number of jobs and m is the average number of disjoint time intervals allocated to a job. In the worst case, a job needs to be allocated into n disjoint time intervals. Therefore, a loose upper bound of the computational complexity can be O(n ). 6 Unit job requests with agreeable deadlines In this section, we introduce another special case of the concave cost job scheduling problem. This special case consists of two key conditions which can be described as follows. Firstly, all of the jobs can be processed in a unit time slot, i.e., w i = () Secondly, the interval of a job cannot be fully contained by the interval of any other job, i.e., for two arbitrary jobs, J =< t a, td, w, u > and J =< t a, td, w, u >, [t a, td ] [ta, td ] [ta i, td i ], i =, () Inequality (), also known as the condition of agreeable deadlines, is often used to model job requests which are supposed to complete in the order they are submitted. We show that this special case concave cost job scheduling problem can be solved using dynamic programming in polynomial time. Input: S J, a set of laminar-structured job requests (with the corresponding laminar structure); A, an empty schedule of jobs in S J ; Output: A cost optimal schedule for jobs in S J ; function Schedule Laminar(S J, A) : if S J = then 5 return A; 6 end 7 Select a job, J i, with no children in the laminar structure of S J ; 8 while J i has unallocated workload do 9 In A, allocate J i with resource to [t i, t j ], which is available to J i, such that R i (t), t [t i, t j ] is constant and is the highest among other available time intervals; end remove J i from S J and update its laminar structure representation; return Schedule Laminar(S J J i, A); end function Schedule Laminar Algorithm : Optimal offline scheduling algorithm for laminar-structured job requests Lemma 7. If a concave cost job scheduling problem satisfies (), there exists a cost optimal schedule such that, for any pair of jobs, they are either allocated to exactly the same time intervals or allocated to completely different time intervals. Proof. Lemma 7 is true if the following inequality holds for an arbitrary concave cost function, f ( ), and arbitrary variables {} R+, i =,,,. min{ f (u + u + u ) + f (u ), f (u + u + u ) + f (u )} f (u + u ) + f (u + u ) () The inequality holds due to the concave property of f ( ). According to Lemma and Lemma 7, the special case problem can be transformed into an integer programming problem with integer variables indicating the unit time slot each job is allocated to. Now, we show an interesting property of the optimal schedules, which is derived from (), for this special case problem. Lemma 8. Let S J be a set of job requests which complies with () and (). Let J =< t a, td, w, u > and J =< t a, td, w, u > be an arbitrary pair of jobs in S J (without loss of generality, t a < ta ). Let A be an optimal schedule of S J. Let [t, t + ] and [t, t + ] be the unit intervals where J and J are allocated by A, respectively. We have t t. Proof. We prove Lemma 8 by contradiction. Let us assume that an optimal schedule A allocates J and J to [t, t +] and [t, t +], respectively. Without loss of generality, t > t. Note that both J and J can be allocated in either [t, t + ] or [t, t + ]. By Lemma, A is not optimal. Lemma 8 indicates that, for this special case concave cost scheduling problem, cost optimal schedulers allocate jobs in the order they arrive. According to Lemma 7 and Lemma 8, we introduce an optimal and polynomial scheduling algorithm using dynamic programming, shown in Algorithm. Here, jobs are associated with time 5-99 (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

8 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems 8 Input: S J ; Output: A, cost optimal job schedule sort S J in ascending order of job arrival time; let I be an empty list of time instances sorted in ascending order; let M A be an empty list of schedules; 5 get the first job, J, from S J ; 6 insert t a and td into I; 7 let A be an empty schedule; 8 associate J with [t a, td ] in A; 9 insert schedule A into M A ; while S J do get the next job, J i, from S J ; insert ti a and ti d into I; for A M A do let [t α, t β ] be the last interval with jobs associated in A; 5 if ti a t α then 6 let A be a copy of A; 7 associate J i with [t α, t β ] in A; 8 associate J i with [t β, ti d] in A ; 9 insert A into M A before A; else if ti a t β then associate J i with [ti a, td i ] in A; else if α ti a < t β then let A be a copy of A; associate jobs which were associated with [t α, t β ] to [t α, ti a ] in A; 5 associate J i with [t β, ti d ] in A; 6 associate jobs which were associated with [t α, t β ] to [ti a, t β] in A ; 7 associate J i with [ti a, t β] in A ; 8 insert A into M A before A; 9 end end for A, A M A do let [t α, t β ] be the last interval with associated jobs in A; let [t α, t β ] be the last interval with associated jobs in A ; if [t α, t β ] = [t α, t β ] and the total resource of the jobs associated with [t α, t β ] are the same for A and A then 5 if A is more cost efficient than A then 6 Remove A from M A ; 7 else 8 Remove A from M A ; 9 end end end Return A with the lowest cost in M A ; end Algorithm : Optimal scheduling algorithm for unit job requests with agreeable deadlines intervals. If a job is associated with a time interval, this job can be allocated to any unit time slot in the interval. If multiple jobs are associated with a time interval, they must be allocated to the same unit time slot in the interval. The main idea is to consider the jobs one by one and cumulatively in the order they arrive. When a job is taken into consideration, we enumerate all possible schedules for the job and the jobs which have already been considered. To prevent the number of possible schedules from growing exponentially, after considering each job, we index the solutions with respect to their last interval with associated jobs and the total resource requirement of the jobs associated to it. For each index value, we only keep the solution with the lowest cost (refer to Algorithm : line to line ). By Lemma 8, the eliminated schedules cannot be optimal. Algorithm is polynomial since the number of stored schedules after considering each job is linear to the number of jobs. Assume that n jobs J i =< ti a, td i, w i, >, i =,..., n are sorted in ascending order with respect to ti a. In the worst case scenario, we have tn a < t d. After considering the last job, there will be exactly n stored schedules. The computational complexity of Algorithm is O(qn ) where n is the number of jobs and q is the average number of unit time slots in the interval of a job. In the worst case, q = n. For clarity, we demonstrate Algorithm using the following example. Let us assume that there are four job requests: J =<,, 5, 5 >, J =<, 6,, >, J =<, 8, 9, 9 >, and J =< 5, 9,, >. Algorithm finds the optimal schedule as follows. In the first step, we consider J. The stored schedule is {([, ], J )} which indicates that J is scheduled in a unit time slot in [, ]. Now, J can be scheduled in either [, ] or [, 6]. So, {([, ], J )} extends to two new schedules with different cost, {([, ], J ), ([, 6], J )} and {([, ], J, J )}. Here, {([, ], J ), ([, 6], J )} denotes a schedule which allocates J to a unit time interval in [, ] and allocates J to a unit time interval in [, 6]. {([, ], J, J )} denotes a schedule which allocates both J and J to the same unit time interval in [, ]. Now, let us consider J. Extending from the first stored schedule, we get two new schedules, {([, ], J ), ([, 6], J ), ([6, 8], J )} and {([, ], J ), ([, 6], J, J )}. Extending from the second schedule, we get {([, ], J, J ), ([6, 8], J )}. There are only two distinct values of last interval with associated jobs, namely ([6, 8], J ) and ([, 6], J, J ). As {([, ], J, J ), ([6, 8], J )} has a lower cost than {([, ], J ), ([, 6], J ), ([6, 8], J )}, the former is kept. J arrives at 5 which is in the middle of [, 6]. G extends to {([, ], J, J ), ([6, 8], J ), ([8, 9], J )} and {([, ], J, J ), ([6, 8], J, J )}. G extends to {([, ], J ), ([, 5], J, J ), ([8, 9], J )} and {([, ], J ), ([5, 6], J, J, J )}. We return the schedule with the lowest cost according to the cost function f ( ). 7 A Randomized Online Algorithm In this section, we introduce an efficient online scheduling algorithm with a positive, non-decreasing and concave cost function f ( ). The basic idea of our online algorithm is to stack the processing times of multiple jobs whenever possible and run the jobs with the maximum possible resource in order to reduce the total cost. We prove the lower bound for the competitive ratio of the proposed online algorithm against the optimal schedule. 7. Online Scheduling and Expected The online resource scheduling problem assumes that, at any time instant t, the scheduler only knows the tasks which arrive upon or 5-99 (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

9 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems 9 before t. The scheduler does not rely on any knowledge of future information. Online task scheduling is required in many cases, because the cloud service provider or service broker may not have information of all tasks in advance and has to make decision with information available so far. We denote by C ON the total cost induced by an online scheduling algorithm and by C the total cost induced by the offline optimal scheduling algorithm. We use competitive ratio which is defined below to evaluate how close the online scheduling is to the offline optimal algorithm. Definition. An online scheduling algorithm is β-competitive if there exists a constant θ such that C ON βc + θ (5) holds for any input. We call β the competitive ratio. If the online algorithm is not deterministic, C ON is a random variable and in this case (according to [7]), the competitive ratio is defined as: Definition. A randomized online scheduling algorithm is β- competitive if there exists a constant θ such that E[C ON ] βc + θ (6) holds for any input, where E is the expectation taken over the random choices made by the online algorithm. It is clear that the competitive ratio is higher than, and the smaller the competitive ratio, the closer the online algorithm to the offline optimal solution. 7. Randomized Online Stack-Centric Scheduling Algorithm (ROSA) To present ROSA formally, we first define the concept of task density of a time interval. Definition 5. Assume that there are a set of n jobs J i =< ti a, td i, w i, >, i =,..., n. Let A be a schedule of a subset of the n jobs. The task density of a time interval [t, t ] is defined as n DEN([t, t ]) = δ [t, t ] [t j, t k ], (7) i= where [t j, t k ], j, k N stand for the intervals where J i is scheduled in A. Here, δ([t j, t k ]) = if [t j, t k ] and otherwise. The basic idea of our online algorithm, shown in Algorithm, is to sequentially allocate jobs in the order they are submitted. Algorithm makes local optimal schedule on allocating each job, using information available so far. The local optimality of Algorithm can be verified using Lemma and Lemma. When the scheduler allocates the processing time for a job, J i =< ti a, td i, w i, >, it always allocates the job with its maximum possible resource, u. Also, when scheduling the workload of J i, we consider the time intervals within the range of [ti a, td i ] in the order that the interval with the highest scheduled workload comes first. After allocating as much workload as possible to the current time interval, we go on to the next interval until all of the workload of J i is accommodated. In this way, the resulting schedule after allocating each job always complies with the conditions of both Lemma and Lemma. If multiple time intervals have the same density, Algorithm selects a random interval from them to proceed. Such randomization offers an opportunity for the current j,k task in consideration to be processed along with future unknown incoming tasks. We refer to this algorithm as a randomized online stack-centric algorithm (ROSA). Note that ROSA is appropriate for online job scheduling when any concave or piece-wise concave cost function is used. Initialization: an ordered list of time instants I = ; while an task J i arrives do Insert time instants ti a and ti d into I; Find all subintervals [ti a, td i ], each representing a time period in between two adjacent time instants in I, and mark them as unprocessed; 5 while w i > do 6 Select the unprocessed subinterval [ti a, td i ], denoted by [t, t ], that has the highest task density (randomly select one if there is a tie); 7 if DEN([t, t ]) = then 8 if t > t wi then 9 Return task J i is infeasible; end Randomly select an instant t [t, t wi ]; Allocate to J i in [t, t + wi ]; Insert t and t + wi into I; else 5 Allocate to J i in [t, min{t, t + wi }]; 6 Insert t + wi into I if it is smaller than t ; 7 end 8 Update w i = max{, w i t t }; 9 Mark this subinterval as processed; end end Algorithm : ROSA- Randomized online stack-centric algorithm Theorem. All online scheduling algorithms have a competitive ratio no less than f (n )+ f (n+) + f (n), where n is the total number of jobs and f is a positive, non-decreasing, and concave cost function. Proof. We prove the theorem based on Yao s minimax principle [7], i.e., to establish a lower bound on the performance of a randomized algorithm, it suffices to find an appropriate distribution of inputs, and to prove that no deterministic algorithm can have the cost smaller than the lower bound against that distribution. As such, we specify a random instance of the problem and analyze what any algorithm could attain in expectation on this random instance. At time, one task, denoted as task, with u = n+, w = (n + ), and deadline of instant n arrives. The first group of n tasks, tasks,..., n with =, w i = (i =,..., n), arrive randomly during the time interval (, n ], all having the same deadline of instant n (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

10 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems The second group of n tasks, tasks n+,..., n with =, w i = (i = n+,..., n) arrive randomly during the time interval [n +, n ], all having the same deadline of instant n. First, we derive the optimal total cost on the random instance. Since the first group of tasks and the second group of tasks have no overlap in time, tasks in the first group have no way to be scheduled with any task in the second group. Obviously, the optimal schedule on this random instance is to equally split the workload of task into two parts, and then schedule the first half and the second half of task with tasks of the first group and tasks of the second group, respectively. The optimal total cost is constant and equals f (n). We only need to consider reasonable deterministic online algorithms. We call an online algorithm reasonable if it has the following properties: ) The algorithm makes schedules upon arrival of job requests and only with information available so far, and when the schedule of a job is determined, the algorithm should not change the schedule at a later time. The job requests are processed in consecutive time intervals unless it is more cost efficient to split the processing intervals. ) Whenever resource is allocated to a job J i, the algorithm should allocate its maximum resource. ) When there is not enough information to make a better schedule for a task, the algorithm should not split the workload of the task. The first property is because the algorithm needs to be online; the second property is because of Lemma ; the third property is because ROSA works in the same way (refer to Algorithm : line 7 to line ). Any reasonable deterministic online algorithm will have to start scheduling task at some point in time before time n (otherwise the deadline cannot be met). Consider an algorithm that makes a schedule to execute task, which will be scheduled in two consecutive unit time slots, at time t [, n ]. There are three possible scenarios: ) Case t [, n ]: In this case, task has to be scheduled with jobs in the first group. Clearly, the cost of any online algorithm on scheduling task and the jobs in the first group is no less than the cost of the best solution, which is f (n+)+ f (n). That is, the minimum cost for executing these jobs is to stack all jobs together, resulting in the cost of f (n + + n ) = f (n) for the overlapping period and the cost of f (n+) for finishing the rest workload of job. Similarly, the cost of any online algorithm on scheduling jobs in the second group is no less than the minimum cost, which is f (n ). Therefore, the total cost of any online algorithm is no less than f (n )+ f (n+)+ f (n). ) Case t [n, n + ]: Any online algorithm has a cost no less than f (n ) + f (n + ) + f (n). (8) For t [n, n + ], Equation (8) represents the best possible solution that stacks all jobs in the first group into the time period [n, n] and stacks all jobs in the second group into the time period [n +, n + ]. ) Case t [n +, n ]: The analysis of this case is similar to that in the first case. The lower bound of the online algorithm is f (n ) + f (n + ) + f (n). To summarize, the total cost of any reasonable deterministic online algorithm on this random instance is no less than f (n ) + f (n + ) + f (n). Since the total cost of the optimal offline solution is f (n), Theorem follows. From Theorem, we easily have the following corollaries: Corollary. Assume that the cost function has the form f (x) = n α, where < α <. The competitive ratio of all online scheduling algorithms is no less than + α when n. Corollary. Assume that the cost function is in the form of (). All online scheduling algorithms have a competitive ratio no less than when n. Based on Corollary, the lower bound of the competitive ratio of any online scheduling algorithms is.7 when α =.5. While the lower bound of Corollary is meaningless in the sense that the competitive ratio has to be larger than, our experimental evaluation in Section 8 shows that ROSA approaches this (meaningless) lower bound closely, meaning that empirically ROSA is nearly optimal. 8 Experimental Evaluation In this section, we carry out four sets of experiments. The first, second, and third sets of experiments aim at evaluating the tightness of the competitive ratio analysis introduced in Theorem for the three special cases, linear function with a fixed activation cost, laminar-structured job requests, and unit job requests with agreeable deadlines. The fourth set of experiments aims at evaluating the performance of ROSA under a generic concave cost function in comparison to other conventional online scheduling algorithms using the Google cluster trace data []. 8. Competitive ratio analysis for a linear function with a fixed activation cost We test the competitive ratio of ROSA with the special cost function defined by (). As we have theoretically proved the optimality of Algorithm in Section, we use Algorithm to compute the optimal cost for each scenario. Apart from the parameters, ti a, td i, w i and, of job J i introduced in Section, we have some additional control parameters for the experiment, as listed in Table. In addition, the deadline of J i, ti d, is generated as follows. t d = w i ( + ρ) + t a, (9) Where ρ is a parameter which controls the flexibility of the incoming jobs for resource scheduling. We choose the value of ρ from {, 5,, } in this experiment. The result in Fig. 5 shows that the competitive ratio of ROSA tends towards as the number of simulated jobs increases. Fig. 5 also shows that the competitive ratio decreases when ρ increases. This indicates that, given more flexibility of job execution time, ROSA can achieve a better competitive ratio. This also means that customers with stringent deadlines gain less from volume discounts compared to customers with loose deadlines. From Fig. 5, we can also see that the competitive ratio largely depends on the ratio between the fixed cost K and the unit price H (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

11 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems.... ρ = ρ = 5 ρ = ρ =.... ρ = ρ = 5 ρ = ρ =.... ρ = ρ = 5 ρ = ρ = Total Number of VM Requests Total Number of VM Requests Total Number of VM Requests (a) (b) (c) Fig. 5: Competitive ratio of ROSA with a linear pricing function plus positive fixed cost. (a) H =, K =. (b) H =, K =.5 (c) H =, K =.5.5 ROSA FIRSTFIT RANDOM.5.5 ROSA FIRSTFIT RANDOM ROSA FIRSTFIT RANDOM 6 8 Number of job requests (a) α = Number of job requests (b) α = Number of job requests (c) α =.75 Fig. 6: Performance comparison for the online scheduling algorithms ROSA, Firstfit, and Random for laminar-structured job requests under concave cost function f (x) = x α 5 5 ROSA FIRSTFIT RANDOM ROSA FIRSTFIT RANDOM 5 ROSA FIRSTFIT RANDOM 6 8 Number of job requests (a) α = Number of job requests (b) α = Number of job requests (c) α =.75 Fig. 7: Performance comparison for the online scheduling algorithms ROSA, Firstfit, and Random for job requests with agreeable deadlines under concave cost function f (x) = x α TABLE : Parameters used for evaluation Parameter Description H, K Parameters for the cost function described in (), vary from three different settings : H =, K =. (small fixed cost), H =, K =.5 (medium fixed cost) and H =, K = (large fixed cost). T Number of hours of the simulation period, fixed to d max The maximum time duration a job has to be processed ( wi d ui max, i), fix to 5 u max The maximum resource allocated to a job ( u max, i), fixed to.5 N Number of jobs, vary from 5 (sparse) to 5 (dense) α It controls the cost function f (x) = x α ρ As introduced in (9) 8. Competitive ratio analysis for laminar-structured Job Requests We conduct the following experiment to compare the performance of ROSA with two other conventional online scheduling algo (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See rithms, a naïve random scheduler, referred to as Random, and the first-fit scheduler, referred to as Firstfit for laminar-structured job requests. The Random algorithm randomly allocates job requests within their corresponding intervals. The Firstfit algorithm processes the incoming jobs at their arrival time immediately with a finishing time as early as possible. In this experiment, we randomly generate a number of job requests which comply with the laminar-structure and schedule the job requests using three scheduling algorithms introduced above. The experiment is repeated for three cost functions, f (x) = x α, α =.5,.5,.75. We compute the lower bound of the cost using Algorithm and show the competitive ratio of each online algorithm against the number of job requests in Figure 6. We observe that ROSA outperforms the other two online algorithms significantly with steady competitive ratios.,.5, and. when α =.5,.5, and,

12 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems Cost Ratio ROSA : FIRSTFIT RANDOM : FIRSTFIT BF : FIRSTFIT ROSA* : FIRSTFIT ROSA$ : FIRSTFIT Cost Ratio ROSA : FIRSTFIT RANDOM : FIRSTFIT BF : FIRSTFIT ROSA* : FIRSTFIT ROSA$ : FIRSTFIT Cost Ratio ROSA : FIRSTFIT RANDOM : FIRSTFIT BF : FIRSTFIT ROSA* : FIRSTFIT ROSA$ : FIRSTFIT ρ (a) α = ρ (b) α = ρ (c) α =.75 Fig. 8: Performance comparison for the online scheduling algorithms ROSA, Firstfit, and Random with continuous concave cost function f (x) = x α.75, respectively. Note that these competitive ratios are smaller than the theoretical lower bound discovered in Theorem. This, however, is reasonable since Theorem is to calculate expected competitive ratio while laminar-structured job requests have a nice structure that lead to a better than average competitive ratio. We can also observe that the competitive ratio of Firstfit is slightly greater than that of Random. This result is also not surprising since it is very common to find a set of laminar-structured job requests in which Firstfit produces a schedule with an above average cost. 8. Competitive ratio analysis for unit job requests with agreeable deadlines With mostly the same settings as introduced in the previous subsection, we carry out the following experiment to compare the performance of ROSA against two conventional online scheduling algorithms, Firstfit and Random for unit job requests with agreeable deadlines. In this experiment, we randomly generate job requests with unit processing time and agreeable deadlines. We plot the competitive ratio of the three online scheduling algorithms against the lower bound computed using Algorithm in Figure 7. Figure 7 demonstrates that ROSA has much lower competitive ratio than the other two online algorithms. The average competitive ratio of ROSA is.6,., and.5 when α =.5,.5, and.75, respectively, which is very close to the theoretical lower bound,.,., and.9, we discovered in Theorem. 8. Trace driven simulation for generic concave cost job scheduling In the second set of experiments, we conduct simulations based on Google cluster data [] which has been widely used to perform cloud computing related simulations. The trace data contains a large number of job records coming from 9 users. It is recorded for a 9-day duration in May on a cluster of K physical machines. The size of the trace data is over 8 GB. Here, we consider the jobs with explicit computational tasks which can be processed in a deterministic duration given the precise processing power of the machine used. To align with our problem, we preprocessed the data to eliminate the jobs that are not naturally finished (e.g. web services). This leaves us with 7 million job records coming from users. Google cluster data is suitable for evaluating the proposed scheduling algorithm as it provides job requests with a large variety of resource requirement. The job requests submitted by different users exhibit different patterns in term of inter-arrival time and job length. We first analyze the feature of the trace data. Fig. 9 shows the distribution of the processing length of the jobs. It shows Fig. 9: data Percentage Job Length (Hour) Distribution of job execution length for Google cluster Percentage Job request inter arrival time (mins) Fig. : Distribution of job inter-arrival times for Google cluster data an approximate exponential distribution. To generate sample job requests for the broker, we arbitrarily form groups of ten users and perform evaluation on each group. Fig. shows the distribution of job inter-arrival times for the trace data. It can be seen that it is hard to fit the inter-arrival times to a known distribution. A similar observation and a more detailed analysis on the job arrivals could be found in [7] as well. This further justifies the usefulness of our online algorithm, which works for arbitrary job arrivals. The job records of the trace data contain three attributes relevant to our simulation, job arrival time, job workload, and job resource requirement. For an arbitrary job J i, the job arrival time, ti a, indicates when the job is available to the scheduler. The CPU rate requirement,, indicates the maximum processing speed and the maximum instantaneous resource consumption of. Burst tasks from the same user within second are considered as one request.. We use the CPU rate requirement as the resource requirement of the jobs for simplicity (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

13 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems J i. We obtain the job workload, w i, by multiplying the average CPU rate with the processing time of J i. Similar to the previous experiment, we set the total elapsed time, T, as and we generate the deadline of J i using the method introduced in (9) with ρ as the control parameter. The cost of a scheduling decision is computed using (5) with two types of cost functions: (a) f (x) = x α, < α <, and (b) a piecewise linear function defined by (), which complies with the concavity property. Cost Ratio ROSA : FIRSTFIT RANDOM : FIRSTFIT, x = x +, x < f (x) = x+9 5, x < 6 x+8, x > 6 () We use four online algorithms, ROSA, Firstfit, backfilling (BF) and Random, to schedule each of the groups of job traces. A backfilling algorithm under abundant resource is implemented. When allocating resource for a job request, BF prioritizes time slots with less allocated resource and earlier time slots to flatten the resource utilization throughout the entire period. We run the simulations with six different values of ρ,, 5,, 5, and 5, respectively. The cost function f (x) = x α are varied by setting α to three different values,.5,.5 and.75. As the Firstfit algorithm produces the same scheduling decision for different values of ρ, we use it as a baseline algorithm. We report the cost ratio of ROSA over Firstfit, denoted by ROS A : First f it, and the cost ratio of Random over Firstfit, denoted by Random : First f it. The result is shown in Fig. 8. As shown in Fig. 8, the mean cost ratios between ROSA and Firstfit over grouped user traces are smaller than under all parameter settings. Especially for the cases when α =.5, the mean ratios are.8967,.7595,.665,.69,.5597 and.985 for ρ =, 5,, 5,, and 5, respectively. Practically, the customers are able to get a % cost saving if the interval between the arrival time and deadline of each job is two times of the duration for processing the job at its maximal speed. As expected, the backfilling algorithm achieves the opposite to ROSA since flattening resource utilization under a concave cost function reduces the opportunity for enjoying volume discount. Throughout the paper, we assumed that the broker can accurately estimate job execution time. Now, let us analyze the performance of ROSA when the job execution time estimation is inaccurate. We assume that the estimated job execution time (executed with maximum speed) consists of a normally distributed noise with zero mean and a variance of σ. Therefore, scheduling using unmodified ROSA will violate 5% of the deadlines while over-provisioning the other 5% of the job requests. Here, we provide a simple fix. We multiply the workload of all jobs by +σ to reduce the probability of deadline violation to.5%. We report the resource cost ratio between ROSA under inaccurate job execution time estimation and Firstfit under accurate job execution time estimation in Figure 8. We can observe that when σ =. and σ =.5, denoted by ROS A : First f it and ROS A$ : First f it, respectively, ROSA still significantly outperforms all other online algorithms in terms of resource cost. Fig. shows the cost ratio assessment for the three online algorithms with a piecewise linear cost function defined by (). We can observe that the cost ratio between ROSA and Firstfit is.88,.768,.6,.58 and.5 when ρ =, 5,, 5 and, respectively. Under the function defined by (), the customers are able to obtain similar benefit as that under the cost function of f (x) = x ρ Fig. : Performance assessment for the online algorithms ROSA, Firstfit and Random with a piecewise linear cost function To show that ROSA can achieve real-time job scheduling, we report the average execution time of ROSA compared to Firstfit which are both implemented using C++ in Table 5. We can observe that, although the execution time of ROSA is significantly larger than that of Firstfit, it is able to allocate 5 job requests within one minute. This indicates that ROSA meets the requirement for real-time job scheduling. TABLE 5: Comparison of average execution time over trials between ROSA and Firstfit (in milli-seconds) 9 Conclusions Number of job requests Firstfit ROSA Cloud is an emerging computing market where cloud providers, brokers, and users share, mediate, and consume computing resource. With the evolution of cloud computing, Pay-as-you-go pricing model has been diversified with volume discounts to stimulate the users adoption of cloud computing. This paper studies how a broker can schedule the jobs of users to leverage the pricing model with volume discounts so that the maximum cost saving can be achieved for its customers. We have analyzed the properties that an optimal solution should have and studied three special cases of the concave cost scheduling problem. We developed an online scheduling algorithm and derived its competitive ratio. Simulation results on a Google data trace have shown that the proposed online scheduling algorithm outperforms other conventional scheduling algorithms. Although continuous concave cost functions and piece-wise linear cost functions are used to conduct the evaluation, the properties proved and the online algorithm proposed apply to all piecewise concave cost functions. The work is the initial step towards studying the behaviors and strategies of cloud service providers, brokers, and end customers when offering or facing a pricing model with volume discounts. It opens a door for many interesting problems along the line. For example, how a cloud service provider could determine its pricing schemes (with volume discounts) given the rational customer behavior of cost saving along with other competitors to increase its revenue. To enjoy volume discounts, the customers are encouraged to provide loose deadlines, since tight deadlines leave a small window for cost saving. Loose deadlines, however, may 5-99 (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See

14 information: DOI.9/TPDS.5.799, IEEE Transactions on Parallel and Distributed Systems degrade user experience. As such, further research is required to obtain better trade-off decisions. In addition, the online scheduling problem that allows job migration from one physical machine to another is interesting and deserves further investigation. Finally, assigning job requests from different users to the same physical machine may lead to potential security risks such as covert channel attacks and denial of service attacks. Finding a trade-off between the gain from volume discounts and the induced security risks is also an interesting research problem. Acknowledgement This work is partially supported by Hong Kong Research Grants Council under GRF, CityU 79, and by the Natural Sciences and Engineering Research Council of Canada (No ). References [] Alibaba. Alibaba cloud computing. [] Amazon. Amazon elastic compute cloud (amazon ec). amazon.com/cn/ec/. [] L. Andrew, A. Wierman, and A. Tang. Optimal speed scaling under arbitrary power functions. ACM SIGMETRICS Performance Evaluation Review, 7():9, 9. [] A. Antoniadis and C.-C. Huang. Non-preemptive speed scaling. Journal of Scheduling, 6():85 9,. [5] Apache. Apache hadoop. [6] N. Bansal, H. Chan, and K. Pruhs. Speed scaling with an arbitrary power function. In Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 69 7, 9. [7] A. Borodin and R. El-Yaniv. Online Computation and Competitive Analysis. Cambridge University Press, New York, NY, USA, 998. [8] J. Chang, H. Gabow, and S. Khuller. A model for minimizing active processor time. In Algorithms ESA, volume 75 of Lecture Notes in Computer Science, pages 89. Springer Berlin Heidelberg,. [9] P. Charalampous. Increasing the adoption rates of cloud computing. the adoption rates of cloud computing. [] C. Fu, Y. Zhao, M. Li, and C. J. Xue. Maximizing common idle time on multi-core processors with shared memory. Submitted to International Conference on Embedded Software (EMSOFT ),. [] Gogrid. [] N. Gohring. Confirmed: Cloud infrastructure pricing is absurd. confirmed-cloud-iaas-pricing-absurd. [] Google. Google cluster data. googleclusterdata/. [] G. Guisewite and P. Pardalos. Algorithms for the single-source uncapacitated minimum concave-cost network flow problem. Journal of Global Optimization, ():5 65, 99. [5] T. Henzinger, A. Singh, V. Singh, T. Wies, and D. Zufferey. Flexprice: Flexible provisioning of resources in a cloud environment. In Cloud Computing (CLOUD), IEEE rd International Conference on, pages 8 9,. [6] Huawei. Huawei cloud service. [7] D.-C. Juan, L. Li, H.-K. Peng, D. Marculescu, and C. Faloutsos. Beyond poisson: Modeling inter-arrival time of requests in a datacenter. In Advances in Knowledge Discovery and Data Mining, pages Springer,. [8] S. Khuller, J. Li, and B. Saha. Energy efficient scheduling via partial shutdown. In Proceedings of the Twenty-first Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, pages 6 7,. [9] Microsoft. Microsoft windows azure. [] Preweb. Cloud services brokerage (csb) market. releases/cloud-services/brokerage-market/prweb7.htm. [] Rackspace. [] Readyspace. [] S. Ren, Y. He, and F. Xu. Provably-efficient job scheduling for energy and fairness in geographically distributed data centers. In Distributed Computing Systems (ICDCS), IEEE nd International Conference on, pages, June. [] R. Van den Bossche, K. Vanmechelen, and J. Broeckhove. Cost-efficient scheduling heuristics for deadline constrained workloads on hybrid clouds. In Cloud Computing Technology and Science (CloudCom), IEEE Third International Conference on, pages 7,. [5] W. Wang, D. Niu, B. Li, and B. Liang. Dynamic cloud resource reservation via cloud brokerage. In ICDCS,. [6] C. Weinhardt, A. Anandasivam, B. Blau, N. Borissov, T. Meinl, W. Michalk, and J. Stosser. Cloud computing a classification, business models, and research directions. Business & Information Systems Engineering, (5):9 99, 9. [7] A. Yao. Probabilistic computations: Toward a unified measure of complexity. In Proceedings of IEEE Symp. Foundations of Computer Science (FOCS), pages 7, 977. [8] F. Yao, A. Demers, and S. Shenker. A scheduling model for reduced cpu energy. In Proceedings of IEEE Symp. Foundations of Computer Science (FOCS), pages 7 8, 995. Rui Zhang received his B.S. and M.S. degrees from the Department of Mathematics, Imperial College London, London, in and, respectively. He is currently a Ph.D. student at the Department of Computer Science, City University of Hong Kong. Kui Wu received the B.Sc. and the M.Sc. degrees in Computer Science from Wuhan University, China in 99 and 99, respectively, and the Ph.D. degree in Computing Science from the University of Alberta, Canada, in. He joined the Department of Computer Science at the University of Victoria, Canada in and is currently a Professor there. His research interests include mobile and wireless networks, network performance evaluation, and cloud computing. Minming Li is currently an associate professor in the Department of Computer Science, City University of Hong Kong. He received his Ph. D. and B.E. degree in the Department of Computer Science and Technology at Tsinghua University in 6 and respectively. His research interests include algorithm design and analysis in wireless networks and embedded systems, combinatorial optimization and algorithmic game theory. Jianping Wang is an Associate Professor in the Department of Computer Science at City University of Hong Kong. She received the B.S. and the M.S. degrees in computer science from Nankai University, Tianjin, China in 996 and 999, respectively, and the Ph.D. degree in computer science from the University of Texas at Dallas in. Her research interests include dependable networking, optical networks, cloud computing, service oriented networking and data center networks (c) 5 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See