CHARACTERISTICS OF FAIRNESS METRICS AND THEIR EFFECT ON PERCEIVED SCHEDULER EFFECTIVENESS

Transcription

1 CHARACTERISTICS OF FAIRESS METRICS AD THEIR EFFECT O PERCEIVED SCHEDULER EFFECTIVEESS J. gubr Department of Computer Scence Makerere Unversty P.O.Box 7062 Kampala, Uganda ngubr@ct.mak.ac.ug M. van Vlet Informatcs and Informaton Scence Radboud Unversty jmegen Toernooveld 1, 6525 ED, jmegen, The etherlands maro@cs.ru.nl ABSTRACT Parallel job schedulers are mostly evaluated usng performance metrcs. Deductons however can be msleadng due to selectve job starvaton (unfarness). To choose a better scheduler, therefore, there s a need to compare schedulers for farness as well. Performance and farness, however, have mostly been studed ndependently. We examne characterstcs of three approaches to farness evaluaton n parallel job schedulng. We examne how they represent job starvaton and other aspects of dscrmnaton. We show that the mpled unfarness s not always starvaton/dscrmnaton n practce. We use smultaneous consderaton of performance and farness and compare deductons wth scheduler effectveness derved from group-wse performance evaluaton. We observe that due to possble msrepresentaton of starvaton by farness metrcs, schedulers shown as superor may not be so n practce. KEY WORDS Performance, Farness, Effectveness, Co-allocaton. 1 Introducton Parallel job schedulng has been an actve feld of research for over a decade [1]. The satsfacton users get from the scheduler can vary from scheduler to scheduler. Broadly, we consder a scheduler that most satsfes the users to be the most effectve. The choce of an effectve scheduler s hard to make. One of the reasons could be that the (performance) metrc used s napproprate [2]. The approprateness of the metrc depends on the schedulng set up. If the metrc s not approprate, users satsfacton/dssatsfacton cannot be deduced from the metrc values. Even n cases where performance metrc used s approprate, the results may be msleadng due to selectve starvaton of jobs. Some jobs may be starved at the expense of others and the starvaton s not adequately mpled by the metrc. An mpled good/poor performance may therefore be unrealstc n practce. An effectve scheduler needs be both far and wth good performance. Whle a lot of research has been done on performance of parallel job schedulers [3], less (lke [4][5][6][7][8]) has been done on farness. Performance and farness have also been studed ndependently yet the trend of one cannot be deduced from another. Makng a choce of a good scheduler needs smultaneous consderaton of performance and farness. Ths s because n some cases, good performance s got by starvng some jobs at the expense of others (unfarness). Measures of dsperson [6][4], Far start-tme analyss [5] and the Resource Allocaton Queung Measure (RAQFM) [17][18] are common farness metrcs. These measures can, sometmes, gve msleadng deductons for parallel job schedulers. They can ndcate unfarness n cases where there s no starvaton (and vce versa) whch may lead to wrong deductons. We use typcal parallel job schedulng cases and examne metrc characterstcs to dentfy nstances where a scheduler starvng no jobs can be deemed unfar (and vce versa). We then compare selected nstances of multcluster schedulers by smultaneous consderaton of performance and farness. We valdate the results by usng group-wse performance evaluaton. Prevous studes [10] [11][12][13] show that group-wse evaluaton reveals job favortsm/starvaton and hence scheduler effectveness. We observe that there are crcumstances where measures of farness do not represent all (un)far scenaros accurately. They can, therefore, lead to counter-ntutve deductons. They are not relable n evaluatng scheduler effectveness n parallel job schedulng. We dscuss the concepts and evaluaton of performance and farness n Secton 2. We then dscuss scheduler effectveness n Secton 3. We study farness evaluaton by dsperson, far start-tme analyss and RAQFM n Sectons 4, 5 and 6 respectvely. We descrbe the expermental studes n Secton 7 and present the results n Secton 8. We dscuss our fndngs n Secton 9 and make conclusons together wth suggestons for future research n Secton Performance and Farness Evaluaton We now dscuss the concepts and evaluaton of scheduler performance and farness.

2 2.1 Performance Performance can be envsaged as a measure of user satsfacton. For example, a user may need to have the results to hs request at the earlest possble tme. Average Response Tme (ART) can be used as a user based performance metrc. Other performance metrcs nclude Average Watng Tme (AWT), Job Slow-down (JSD) and throughput. An approprate metrc depends on the schedulng envronment [2]. In a real tme scenaro for example, AWT s not an approprate metrc. The proporton of jobs that are processed before the deadlne can be a better metrc. The system set up also determnes the nterpretaton that can be derved from the metrc. Whle the deductons from AWT and ART are the same for space slcng systems, they are dfferent for tme slcng schedulers. The average value of the performance metrc does not gve an n-depth vew of performance. Ths can be due to dfferences n job schedulablty [14]. Prevous studes show that group-wse performance analyss s benefcal. Ths s because starved and favored jobs can easly be dentfed. gubr and van Vlet [11][14] show that schedulablty s hghly dependant on job characterstcs. Dfferent job groups (grouped by sze, number of components and wdth of wdest component) have dfferent performances. The rates at whch these performances change wth scheduler parameters are also dfferent. Srnvasan et al. [15] use job groups to compare robustness of schedulers for moldable jobs. They show that dfferent (sze based) job groups have dfferent (n some cases opposte) performance changes as scheduler parameters change. Some jobs therefore have a trend opposte that of the entre job stream average. More work by Srnvasan et al. [12][13] show that the performance dfference between conservatve and aggressve backfllng s actually n (sze based) job groups. The domnatng group dctates the overall job stream relatve performance. Evaluaton usng job groups therefore help n dentfyng starvaton and favortsm among the jobs. Groups however need to be chosen carefully. The job characterstcs that determne the favortsm/dscrmnaton should be used to generate the groups. 2.2 Farness The nterpretatonal beneft of group-wse evaluaton s due to the fact that t reveals relatve performance. A good scheduler needs to be far to all the jobs. It should not starve some of the jobs at the expense of others. Though less studed n parallel job schedulng, farness s a very mportant aspect n all queung systems. Possbly, t s because of farness that queues came nto exstence. Rafael et al. [16] observed that people n queues are more annoyed by perceved unfarness than by the tme spent n queues. There two man perceptons of farness. The frst (called FCFS style) uses the users expected order of servce whle the second uses equal resource dstrbuton among the users. The second approach s only applcable n tme-slcng cases. Snce we consder space-slcng, we focus of FCFS style only. The FCFS farness has been evaluated mostly usng the far start-tme approach, measures of dsperson and RAQFM. In far start-tme approach [5], a job s consdered unfarly treated f ts processng s delayed by job(s) that arrved after t. The dea behnd s that n a far settng, a job should not be delayed by another that arrved after t. Let us consder a job J n a job stream. The actual start tme t a s the tme J starts processng when the entre job stream s processed. The far start tme t f s the tme J starts processng f no job arrved after t (e. t s the last job n the queue). If t a >t f, then J s unfarly treated. The mean of t a tf for all unfarly treated jobs s used to determne scheduler farness. In dsperson approach [4][9], measures of dsperson lke standard devaton and coeffcent of varaton are used to deduce farness. The dea behnd s that f a scheduler favors some jobs at the expense of others, then there s an ncrease n dsperson among the jobs performance. The hgher the dsperson, the more unfar the scheduler s. Jan et al. [4] proposed that a good farness metrc should be populaton sze and scale ndependent, bounded and contnuous. They use the dsperson approach and proposed the farness coeffcent κ = 1 whch satsfes the characterstcs. 1+CV 2 In RAQFM [17][18], the underlyng prncple s that all jobs are enttled to an equal share of the system resources. If at a tme t there are (t) jobs, then each job s 1 enttled to (t) of the system resources. If job J s gven a proporton S (t), then the temporal dscrmnaton D (t) s gven by D (t) =S (t) 1 (t).ifj enters the queue at tme t e and completes processng at tme tc, then the total dscrmnaton s gven by: D = t c t e S (t) 1 dt (1) (t) The overall dscrmnaton of the system s got by Var[D ]. Ths s because for non dlng systems, E[D ]=0. Dfferent approaches to measurng farness can lead to counter-ntutve deductons. The metrcs can mply unfarness where there s no dscrmnaton. Lkewse, there can be cases of dscrmnaton and the metrcs do not mply them. We study such cases deeper n Sectons 4, 5 and 6. Impressve farness mpled by these measures does not always translate nto scheduler effectveness n practce. 3 The Concept of Scheduler Effectveness Broadly, effectveness of a scheduler depends on how well t treats all the users. Users (jobs) should get the best possble performance and some of the jobs should not be dscrmnated at the expense of others. The general behavor of queung systems s determned by senorty and servce requrements [18] of jobs. Job senorty can be looked at

3 as the tme the jobs have spent n the queue and servce requrements can be looked at the processor hours requred to process a job. We note that servce requrements n parallel job schedulng s a bt more complcated than that used n [17]. In [17], they only consder the job runtme and n parallel job schedulng, there s an element of job sze as well. Snce we consder rgd jobs, the job sze aspect cannot be transformed nto the approach n [17]. Ideally, jobs would be processed n the order of senorty. However, parallel jobs have a bg varance n szes. A strct Frst Come Frst Served (FCFS) scheduler leads to system fragmentaton and hence capacty loss. Much as FCFS gves a straght forward far treatment of jobs, jobs do not perform equally. Ths s due to 1. Job servce requrements: Dfferent jobs have dfferent szes and runtmes. A job followng short jobs s bound to wat less compared to that followng a long job. Lkewse, the tme a job wats, whle at the head of the queue ncreases wth ts sze. Bg jobs have to wat as the system accumulates enough processors. 2. System and queue states: Typcal supercomputng envronment have peak and off peak hours. Jobs that arrve n off peak hours enjoy better performance snce they fnd shorter queues and a less busy system. The reverse s true for peak hours. Farness, therefore, does not necessarly mply performance equty. However, each job needs to get the best possble performance. An effectve scheduler can mprove the performance of some jobs so long as t does not do t at the expense of others. In fact, snce capacty loss may be reduced, dong t may be at the beneft of others. Ths s applcable n schedulers lke Conservatve backfllng [19], Aggressve backfllng [20] and backfllng wth look ahead [23]. To evaluate scheduler effectveness, we need to look for evdence that the best performng jobs are not dong so at the expense of others. Therefore, we can use the worst performng jobs. Jobs can be grouped by the characterstc that constran ther schedulablty. Prevous studes [10][14][11][15] show that group wse studes gve detaled performance dfferences wthn the job stream. 4 Farness by Measures of Dsperson In ths approach, we compute the extent of dsperson n performance n the job stream. Ths can be done by parameters lke standard devaton (σ), varance (σ 2 ), coeffcent of varaton (C V ) and farness ndex (κ). Low dsperson mples a far scheduler. The measures do not gve consstent deductons. Due to the nconsstence, Vassupngayya et al. [8] observed that they are not suffcent to measure farness. A hgh value of mean performance for example, lowers the value of C V. In metrcs lke AWT and ART, we get a case where deteroraton n performance mples an ncrease n farness whch s counter ntutve. More to that, there are cases where an ncrease n dsperson does not come from selectve job starvaton. To llustrate ths, we consder a job stream scheduled by FCFS and Conservatve backfllng (BF) as a case study. Let us consder onlne jobs J 1,J 2, J scheduled by FCFS and BF. When scheduled by FCFS, AW T = µ, and varance = σ 2 and J has watng tme t w. When scheduled by BF, let AW T = µ, varance = σ 2 and the watng tme for J be t w. We compare the ndvdual job performance, the average performance and the extent of dsperson among the jobs performance (farness). Theorem 4.1 t w t w J =1, 2 Proof If no job backflls, CB FCFS. Therefore: t w = t w J =1, 2 In BF, a backflled job delays no other job. It starts processng earler than when scheduled by FCFS (drect beneft). Removng a job from the queue for backfllng shortens the queue. Ths may lower the reservaton tmes of jobs behnd t and those yet to arrve (ndrect beneft). If set S conssts of jobs drectly or ndrectly affected by backfllng, then t w > t w J S Snce a job ether benefts from BF or retans ts FCFS reservaton tme: t w t w Theorem 4.2 µ µ Proof By defnton: µ = 1 t w and µ = 1 From the valdty of Theorem 4.1, Theorem 4.2 s also vald. t w Let us defne two non-negatve parameters δµ and δt as: δµ = µ µ and δt = t w t w (2) Lemma 4.1 If we break the job stream nto two dsjont sets S 1 and S 2, where J m S 1 ff t w m µ and J n S 2 ff t w n >µ, then: (µ t w m )δt m (µ t w n )δt n J m S 1 J n S 2 Proof Snce µ s a measure of central tendency for t w 1,tw 2, tw, then: (µ t w m )= (µ t w n ) (3) J m S 1 J n S 2 In FCFS, a job keeps n queue because; () t has not reached the head of the queue or () t s at the head of the queue but cannot ft n the avalable free processors.

4 Factor () affects all jobs equally whle factor () affects large jobs more. Small jobs therefore perform better than large jobs. Ths mples that small jobs domnate S 1 and large jobs domnate S 2. BF utlzes processors that would be dle and small jobs mostly beneft from t [12]. Small jobs therefore get bgger performance mprovement compared to large jobs. Ths mples that () for jobs n S 1, µ t w m > 0 and δt m 0 and () for jobs n S 2, µ t w n > 0 and δt n 0. Multplyng µ t w m wth a relatvely large δt m makes LHS grow (absolutely) faster than RHS where µ t w n s multpled wth a relatvely small δt n. Theorem 4.3 σ 2 σ 2 Proof From the defnton of varance: σ 2 = 1 = (t w µ) 2 and σ 2 = 1 = (t w µ ) 2 (4) From (4), σ 2 σ 2 can be wrtten as: 1 = 1 [(t w µ ) 2 (t w µ) 2] elmnatng t w (t w µ + t w µ)(t w µ t w + µ) and µ usng (2), we get: Theorem 4.4 If κ and κ are the farness coeffcents for FCFS and BF respectvely, then κ κ Proof By defnton, κ =(1+CV 2 ) 1, from Theorem 4.3 and 4.2: σ 2 σ 2 σ σ and µ µ µσ σµ σ µ σ σ C V C V Snce C V and C V are postve, then CV 2 C2 V κ κ Theorems 4.1, 4.3 and 4.4 show that: () o job n BF perform worse than n FCFS, therefore BF does not starve any job as compared to FCFS () Dsperson n BF s hgher than that n FCFS. Dsperson, therefore, measures relatve rather than absolute starvaton. Snce however jobs have dfferent servce requrements and fnd dfferent queue and system states, t should be expected n practce. If we are to use dsperson as a measure of farness, we would conclude that FCFS s farer than BF yet the hgher dsperson n BF s not a result of starvaton. Actually, the ncrease n dsperson s due to dfferent rates at whch dfferent jobs get mprovements n performance as they are scheduled by BF nstead of FCFS. There s, therefore, a practcal loophole n usng dsperson to deduce unfarness/starvaton n parallel job schedulng. 5 Farness by Far Start Tme Analyss 1 (2(t w µ)+δµ δt )(δµ δt ) = 1 (δµ δt ) (t w µ)(δµ δt ) = 1 (δµ δt ) (t w δµ tw δt µδµ + µδt ) [ ] = 1 (δµ δt ) µδt t w δt [ 1 (δµ δt ) ] µδt t w δt Ths smplfes to: σ 2 σ 2 = 1 (δµ δt ) (µ t w )δt (5) In far-tme analyss, we consder a job to be unfarly treated f t s delayed by another that arrved after t. Ths approach has some short comngs Frst, ths approach cannot compare schedulers where delays are not explctly caused by later arrvng jobs. Sometmes, a job can be delayed due to poor packng of jobs that arrved before t. Such a case s not catered for. Secondly, equvalence n farness does not always mply equvalence n satsfacton. Let us consder a case where we compare FCFS and BF. Snce FCFS processes n the strct order of arrval, there can be no nstance where a job s delayed by another that arrved after t. Ths mples that there s no unfarly treated job n the job stream. Let us also consder a case where the job stream s scheduled by BF. In BF, the jobs are allowed to volate the FCFS order but only f the jobs jumped are not delayed. Ths mples that n BF as well, there s no job whch s delayed by another that arrved after t. Ths gves us a concluson that FCFS and BF are equally as far. The benefts of backfllng [19], n such a case, show no effect on farness. Ths, defntely, s counter-ntutve. Usng the two sets defned n Lemma 4.1, RHS of (5) can be rewrtten as: Thrdly, ths approach uses a dynamc pont of reference. The far start tme of a job s dependent on the [ ] 1 (δµ δt ) (µ t w m)δt m + (µ t w n )δt n schedulng algorthm. Ths mples that a sngle job has J m S 1 J n S 2 dfferent far start tmes for dfferent schedulers. The devatons, computed from these far start tmes have dfferent (6) (δµ δt ) 2 > 0 and usng Lemma 4.1, RHS of Equaton mplcatons n real lfe. Let us consder two schedulers A (6) s non negatve. and B where a certan job J has far start tmes 10 and 50.

5 If the actual start tmes are say 20 and 30 respectvely, far start tme approach wll ndcate scheduler A to be more unfar compared to scheduler B. Ths s despte the fact that the job owner actually gets more satsfacton from scheduler A than from B. Lastly, the approach consders the dsadvantage caused to a job by a later arrvng job. Ths, n some cases, may be small and local that t does not gve the global extent and shft n user satsfacton. To llustrate ths, let us consder a job J k n a queue. We look at some of the crcumstances that can alter ts reservaton tme. () Job J k may jump (n cases lke Ft Processors Frst Served [25] or backfllng) and get processed earler than t would. Let the jump beneft caused be b j k. () Some of the jobs ntally ahead of J k jump (lke n cases of FPFS and Backfllng) utlze the exstng dle processors. Ths effectvely ncreases the rate at whch the load s processed and shortens the queue whch may mprove the reservaton of J k. If the beneft J k gets from the jumpng of J s b k, and a set SB k of jobs creates such benefts to J k, then the net extra reservaton beneft R 1 k of J k s gven by: R 1 k = b k (7) J S k B () Some of the jobs ntally ahead of J k jump but nterferes wth the packng scheme, ncreases fragmentaton and leads to a worse reservaton of J k. Ths s more lkely n approaches lke aggressve backfllng and FPFS. Let the dsadvantage suffered by J k due to J be d k, and the set of such jobs be SD k. The total dsadvantage n reservaton R 2 k s gven by: R 2 k = d k (8) J S k D (v) Some of the jobs ntally behnd J k jump n such a way that J k gets a dsadvantage n reservaton. Ths can be due to ncreased fragmentaton or long executon tmes. If such jobs have a set SD k and J causes a dsadvantage d k, then the total dsadvantage n reservaton R 3 k suffered by J k s gven by: R 3 k = d k (9) J S k D The net change n reservaton for J k s got by summng up the benefts and the losses for J k. If the net change s gven by R, then: R = b j k + b k d k d k (10) J S B J S D J S D The approach n [5] used only R 3 k to evaluate farness. Lke shown n (10), other factors are also nvolved. A dsadvantage R 3 k s not necessarly a net dsadvantage. In some schedulers lke aggressve backfllng, t rarely go beyond a few job from the head of the queue [21]. Chang et al. [21] propose that f we are to prevent a bg effect of backflled jobs delayng jobs other than that at the head, makng about the frst four reservatons s good enough. Sometmes, however, the a decson made on one job can have an effect that goes far deep nto the queue [22]. Studes n [11] actually show that even wth no consderaton of job run-tmes (lke n FPFS), all jobs acheve a net beneft. The contrbuton of R 3 k can be substantal, neglgble or non-exstent dependng on the schedulng envronment. In cases where ts substantal, far-tme approach gves a bg estmaton of starvaton otherwse t does not. 6 RAQFM RAQFM was proposed n [17] and analyzed n [18]. It uses the dfference between the resources allotted to a job and that t s farly enttled to. The RAQFM n [17] was meant for a sngle server faclty. It s able to ntutvely explan the far most decsons to be made n naturally challengng scenaros. For example, t adequately handles a case where n a supermarket a customer wth very few groceres s behnd one wth a lot of groceres. If used n parallel job schedulng, t brngs n a problem of what s perceved to be far and what s actually far. It can clam dscrmnaton where users get deal servce. For example, f two jobs needng 2 and 6 processors wth runtme 5 each arrve smultaneously nto a 10 node cluster, they mmedately get the servce. However, RAQFM wll consder the frst job to have temporal dscrmnaton ( ) 5= 3 2 and the second job to have ( ) 5= 1. The frst job s 2 dscrmnated whle the second s favored yet both get the deal servce. As part of future work n [17], the modfcaton of RAQFM to ft mult-server cases s stll n progress. 7 Expermental Evaluaton We now expermentally compare deductons from smultaneous comparson of group wse performance wth a combnaton of job stream performance and farness. 7.1 The model We consder a system of homogeneous clusters C 1,C 2, C served by a sngle queue and scheduler. The clusters are connected by a fast WA wth neglgble communcaton overheads. Jobs are onlne, rgd, and can be broken nto multple components and co-allocated. Job duratons are unknown to the scheduler but fnte. Job szes are generated from a dstrbuton D(q) defned on an nterval [n 1,n 2 ] (0 < n 1 <n 2 ).InD(q), the probablty p, that a job has sze,

6 s gven by: p = { 3q f s a power of 2 Q q Q f s not a power of 2 (11) Jobs wth sze greater than a threshold thres are broken nto components. 7.2 The Schedulng algorthms We consder three schedulng algorthms. The frst s FCFS. Jobs are queued and allocated to clusters n ther (strct) arrval order. The second s Ft Processors Frst Served (FPFS) [25]. Ths s lke FCFS but f the job at the head of the queue cannot ft n the system, the frst job deeper n the queue, that fts s scheduled. The job at the head of the queue s protected form starvaton by fxng the number of tmes t s jumped to maxjumps. If t s jumped maxjumps tmes, no other job s allowed to jump t untl enough space has been freed to start ts processng. We use FPFS(x) to represent FPFS when maxjumps = x. The thrd s the Greedy mult-cluster scheduler [24]. Jobs are queued n the arrval order. Each job has a prorty ndcator whch s the product of the tme t has spent n the queue and ts hardness value. The hardness value s an approxmaton of job schedulablty (sze n ths case). The scheduler pcks the hghest prorty job from the frst depth jobs that fts n the avalable processor hole. Lke n FPFS, the job at the head of the queue s protected from starvaton by lmtng the number of tmes t can be jumped to maxjumps. We use Greedy(j, d) to represent the greedy scheduler when maxjumps = j and depth = d. 7.3 Performance and Farness Evaluaton We use ART performance metrc. To measure farness, we use σ and C V. We use sze to estmate job schedulablty and generate four sze based groups Q 1,Q 2,Q 3, and Q 4. Groups are bounded by the (sze) lower quartle, medan and upper quartle. 7.4 Expermental set up We smulate a 5 cluster system of 20 processors per cluster. Jobs are generated from D(0.85) over [1, 38]. Job nterarrval and executon tmes are exponentally dstrbuted wth mean 0.64 and 10.0 respectvely (load = 0.786). We take thres = 11 and jobs wth sze > thres are broken nto up to 4 components. The probablty that a job (sze > thres) s broken nto 2, 3, or 4 components s 1 3. To break a job of sze x nto n components, the frst n 1 components are made to have a wdth x n and the n th component to have x (n 1) x. Results n are recorded wth a maxmum absolute error of 0.05 at 95% confdence nterval. We consder FCFS, FPFS(10), Fgure 1. Performance and farness: C V (top), σ(bottom) FPFS(50), Greedy(10,10), Greedy(10,50), Greedy(50,10) and Greedy(50,50). 8 Results 8.1 Performance and Farness We summarze the performance and farness (by σ and C V ) n a 2 2 matrx (Fgure 1). We observe that σ and C V do not produce consstent deductons on farness. The order of farness among schedulers s not the same for C V and σ. Ths s n lne wth [8] that measures of dsperson are nsuffcent n evaluatng farness. 8.2 Group wse evaluaton We now present job stream and group wse performance n Fgure 2. We observe that FCFS and Greedy(50,50) gve the worst and best performance respectvely. The same

7 same performance for a scheduler to be far. Snce farness s meant to measure the user s feel of dscrmnaton, there s a need to model what the user expects n real lfe. The devaton can be used to measure how close to the deal the scheduler reaches. An approach lke dsperson assumes equal performance whch actually s not the case n practce. Ths makes the deductons beng made on a non-realstc foundaton. Whle n ths paper we do not propose an approach to measure farness, we examne the exstng approaches for ther representaton n the real lfe schedulng scenaro of parallel jobs. We also nvestgate how one of the exstng approaches relates wth deductons from group wse evaluaton. Deductons from farness and performance evaluaton do not rhyme wth those of group-wse evaluaton. Snce the approaches may msrepresent users needs, ther deductons do not necessarly gve an accurate vew of user satsfacton. 10 Concluson and Future Work Fgure 2. Performance of dfferent schedulers for the job stream (top) and groups (bottom) trend s observed n group wse evaluaton. However, the entre job stream and group wse performance trends are not always the same. Greedy(10,50) and FPFS(10) are equal for the entre job stream but Greedy(10,50) s better for group wse evaluaton. More to that, FPFS(10) s better than Greedy(10,10) for the entre job stream though the reverse s true for group wse evaluaton. The dfferences n performance and farness mply that usng performance alone does not always gve a good dea of scheduler effectveness. The extent of job starvaton (whch s nadequately measured by measures of dsperson) needs to be put nto consderaton. 9 Dscusson Choosng an approprate performance metrc s a hard decson to make [2]. Ths s because dfferent schedulng scenaros can have dfferent ways qualty of servce can be envsaged. After choosng a performance metrc, ts outcome can be msleadng because of relatve job starvaton. Ths s a general concept of scheduler farness. Farness has been measured n other queung systems and several metrcs have been proposed. Those metrcs, however, are not necessarly approprate n the parallel job schedulng case. They therefore lead to counter ntutve conclusons. The man problem s how to accurately represent starvaton n an envronment where jobs are havng dfferent requrements, varyng arrval rates and need not to have the We have shown that the commonly used farness metrcs do not always accurately represent the users needs as far as dscrmnaton and favortsm s concerned. There are crcumstances where a far scenaro s ndcated as un far and vce versa. Ths leads to a problem of the applcablty of the metrcs as well as the relablty of the deductons. More research on ths work can be on comng up wth robust metrcs for farness n parallel job schedulng. There s however work n ths drecton (Raz et al [17] ndcate t as ongong work). There s also a need to study n more detal the possble ptfalls that may render a farness metrc unrepresentatve of the real lfe envronment. References [1] D.G. Fetelson, L. Rudolph, E. Frachtenberg and U. Schwegelshohn (eds) The Job Schedulng Strateges for Parallel Processng Workshop Seres. fet/parsched/ [2] E. Frachtenberg and D.G. Fetelson, Ptfalls n Parallel Job Schedulng Evaluaton, Proc. 11 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 3834, Cambrdge, MA, 2005, [3] D.G. Fetelson, L. Rudolph and U. Schwegelshohn, Parallel Job Schedulng - A Status Report, Proc. 10 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 3277, ew York, Y, 2004, [4] R. Jan, D. Chu and W.R. Hawe, A Quanttatve Measure of Farness and Dscrmnaton for Resource Allocaton n Shared Computer Systems, Techncal Report TR-301, Dgtal Equpment Corporaton, 1984.

8 [5] G. Sabn, G. Kochar and P. Sadayappan, Job Farness n on-preemptve Job Schedulng, Proc Internatonal Conference on Parallel Processng, Montreal, Canada, 2004, [6] G. Sabn and P. Sudayappan, Unfarness Metrcs for Space-sharng Parallel Job Schedulers, Proc. 11 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 3834, Cambrdge, MA, 2005, [7] U. Schwegelshohn and R. Yahyapour, Farness n Parallel Job Schedulng, Journal of Schedulng, 3(5), 2000, [8] S. Vasupongayya and S -H. Chang, On Job Farness n on-preemptve Parallel job Schedulng. Proc. 17 th IASTED Conf. on Parallel and Dstrbuted Computng and Systems, Phoenx, AZ, [9] B. Av-Itzhak and H. Levy, On Measurng Farness n Queues, Advances of Appled Probablty, 36(3), 2004, [10] D. G. Fetelson, Metrc and Workload Effects on Computer Systems Evaluaton. Computers 36(9), 2003, [11] J. gubr and M. van Vlet, Group-wse Performance Evaluaton of Processor Co-allocaton n Mult-cluster Systems, Proc. 13 th Workshop on Job Schedulng Strateges for Parallel Processng, Seattle, WA, 2007, [12] S. Srnvasan, R. Kettmuthu, V. Subraman and P. Sadayappan, Selectve Reservaton Strateges for Backfll Job Schedulng, Proc. 8 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 2537, Ednburgh, Scotland, UK, 2002, [13] S. Srnvasan, R. Kettmuthu, V. Subraman and P. Sadayappan, Characterzaton of Backfllng Strateges for Parallel Job Schedulng, Proc Internatonal Conference on Parallel Processng Workshops, Vancouver, BC, Canada, 2002, [14] J. gubr and M. van Vlet, The Impact of Jobs Physcal Characterstcs on ther Schedulablty n a Mult-cluster System. Proc. 2 nd Internatonal Conference on Computng and ICT Research, Makerere Unversty - Kampala, Uganda, 2006, [17] D. Raz, H. Levy and B. Av-tzak, A Resource- Allocaton Queung Farness Measure, SIGMETRICS Performance Evaluaton Revew, 32(1), 2004, [18] B. Av-Itzhak, H. Levy, and D. Raz, Davd. A resource allocaton queueng farness measure: propertes and bounds. Queueng Systems 56(2), 2007, [19] A. W. Mu alem and D. G. Fetelson, Utlzaton, predctablty, workloads, and user runtme estmates n schedulng the IBM SP2 wth backfllng. IEEE Trans. Parallel & Dstrbuted Syst. 12(6), 2001, [20] D. Lfka, The AL/IBM SP schedulng system,proc. 1 st Workshop on Job schedulng Strateges for Parallel Processng, LCS 949 Santa Barbara, CA, 1995, [21] S-H. Chang, A. Arpac-Dusseau, and M. K. Vernon, The mpact of more accurate requested runtmes on producton job schedulng performance, Proc. 8 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 2537, Ednburgh, UK, , [22] D. Tsafrr and D.G. Fetelson, Instablty n parallel job schedulng smulaton: The role of workload flurres, Proc 20 th Internatonal Parallel and Dstrbuted Computng Symposum, Rhodes, Greece, [23] E. Shmuel and D. G. Fetelson, Backfllng wth lookahead to optmze the performance of parallel job schedulng. Proc. 9 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 2862, Seattle, WA, 2003, [24] J. gubr and M. van Vlet, Usng the Greedy Approach to Schedule Jobs n a Mult-cluster Systems, Proc. Internatonal Conference on Parallel and Dstrbuted Processng Technques and Applcatons, Las Vegas, V, 2006, [25] K. Ada, H. Kasahara and S. arta, Job schedulng scheme for pure space sharng among rgd jobs, Proc. 4 th Workshop on Job Schedulng Strateges for Parallel Processng, LCS 1459, Orlando, FL, 1998, [15] S. Srnvasan, S. Krshnamoorthy and P. Sadayappan, A Robust Schedulng Technology for Moldable Schedulng of Parallel Jobs, Proc. IEEE Conf. on Cluster Computng, Kowloon, Hong Kong, 2003, [16] A. Rafael, E. Kedm, D. Vslul and G. Barron, Queues and Farness: A Multple Study Expermental Investgaton,