DoubleAuction-Inspired Meta-SchedulingofParallel ... · Grids are composed of distributed...

Double Auction-Inspired

Meta-Scheduling of Parallel

Applications on Global Grids

Saurabh Kumar Garg1, Srikumar V enugopal2, James Broberg1 and

Rajkumar Buyya1

1Cloud Computing and Distributed Systems (CLOUDS) Laboratory,Department of Computer Science and Software Engineering,

The University of Melbourne, Australia,2School of Computer Science and Engineering

The University of New South Wales, Sydney, Australia

Email: {sgarg, brobergj, raj}@csse.unimelb.edu.au

Meta-schedulers map jobs to computational resources that are part of a grid,such as clusters, that in turn have their own local job schedulers. ExistingGrid meta-schedulers either target system-centric metrics, such as utilization andthroughput, or prioritize applications based on utility metrics provided by theusers. The system-centric approach gives less importance to users’ individualutility, while the user-centric approach may have adverse effects such as poorsystem performance and unfair treatment of users. Therefore, this paper proposesa novel meta-scheduler, based on the well-known double auction mechanism thataims to satisfy users’ service requirements as well as ensure balanced utilisationof resources across a grid. We have designed valuation metrics that commodifyboth the complex resource requirements of user applications and the capabilitiesof available computational resources. Through simulation using real traces, wecompare our scheduling mechanism with other common mechanisms widely usedby both existing market-based and traditional meta-schedulers. The results showthat our meta-scheduling mechanism not only satisfies up to 15% more userrequirements than others, but also improves system utilization through load

balancing.

Keywords: Grid Computing, Resource Allocation, Meta-Scheduling, Auction

1. INTRODUCTION

Grids are composed of distributed high-performancecommodity clusters and supercomputers managedby batch job schedulers such as Portable BatchScheduler (PBS) [1]. These distributed resourcesin the production grids are mostly managed bymeta-schedulers that interact with the local jobschedulers at each resource site in a grid to determinethe most appropriate resource for executing anapplication submitted by a user. Meta-scheduling isdifferent from cluster-level scheduling as it involvesmatching of the multiple concurrent applications todifferent distributed resources rather than dispatchingapplications to individual cluster nodes within a singledomain. Examples of such meta-schedulers includethe Maui/Moab scheduling suite [2], gLite WorkloadManagement System [3], and GridWay [4].Whilst Grids have matured with respect to the

integration of different components, users have also

developed sophisticated Quality of Service (QoS)requirements for application execution, and are ready tocompensate resource providers for delivering an agreedlevel of QoS. Two examples of such requirements arecompleting an application by a certain deadline andensuring a minimum number of CPUs for executingan application. These QoS requirements increase thechallenge of application scheduling due to a numberof reasons. First, the requirements of differentapplications can conflict with one another, therebyrendering the system unable to satisfy all users. Second,deadline conditions and fixed requirements of CPUscan induce fragmentation in application queues, whichreduces system utilization and leads to poor usersatisfaction. Finally, a meta-scheduler not only hasto take into account these problems, but also has tocontend with the changing conditions of grid resourcesthat are spread across different administrative domains.

Historically, meta-schedulers have given priority toimproving system-centric performance metrics such as

The Computer Journal, Vol. ??, No. ??, ????

2 Garg et. al

utilization, average load and turnaround time for userapplications [5]. They were not designed to caterfor the sophisticated QoS needs of an application,particularly when the demand for resources exceed thesupply. In recent years, a number of researchers haveexplored applying well-known economic mechanismssuch as markets to address user requirements in meta-scheduling [6][7][8]. In a grid with variable resourceavailability, it is difficult to determine accurate resourceand application valuations to take advantage of market-based mechanisms. The parallel applications, that haverigid processor requirements, are not comparable toeach other and cannot be commodified easily so asto be used in auction mechanisms. From the users’perspective, it is difficult to come up with a valuationthat ensures that their application is provided withthe required amount of resources and executed by thedeadline. Therefore, we need scheduling mechanismsthat not only ensure effective utilization of Gridresources, but also take into account users’ interest,demand on resources, and allocate resources in a fairmanner such that no application is starved.

In this paper, we present a novel grid meta-scheduling mechanism that takes inspiration fromauction principles in allocating resources to parallelapplications with competing QoS demands. Theobjective of our meta-scheduling mechanism is to satisfythe QoS requirements of the users as well as to ensuremaximal utilisation of resources simultaneously, whileavoiding starvation, and minimising the effect on othermeasures such as waiting time and slowdown. Wehave considered parallel applications having multiplecommunicating processes that are to be allocatedto a rigid number of processors available from asingle computational resource. We have designedvaluation metrics that enable mapping of slots indifferent application queues at grid resources to multipleapplications with fixed processor requirements. Inthis manner, resource shares are commodified sothat the principles of auctions can be used in thedesign of the meta-scheduling mechanism to benefitboth users and resource providers. Then, byusing simulation on real workload traces of parallelapplications from supercomputing centers, we showthat our scheduler performs much better than classicalscheduling mechanisms in similar working conditions.

The rest of the paper is structured as follows. Section2 discusses related scheduling and economy-basedresource management projects. Section 3 presentsthe system model, and the details of our schedulingmechanism are presented in Section 4. Sections 5presents the experimental setup used for performanceevaluation, and discusses the results. Finally, weconclude the paper and present future steps in thisdirection in Section 6.

2. RELATED WORK

The general problem of creating a schedule for a setof jobs to run on distributed resources is called listscheduling and is considered to be NP-complete [9].The simplest strategy, First Come First Served (FCFS),handles jobs in the order of arrival at the schedulerand submits them to the first available resource. Itis considered adequate for the most purposes, and inconjunction with gang scheduling and backfilling, ableto effectively utilise resources [10][11]. Currently, meta-schedulers such as GridWay [4] and gLite WorkloadManagement System [3] use FCFS in operation.Moab also has a FCFS batch scheduler with easy-backfilling policy [2]. Condor-G [12] uses eitherFCFS or matchmaking with priority sort as schedulingpolicies [13]. However, FCFS is a simple heuristicsthat can only be used for traditional system metricssuch as system utilisation and application waitingtime. Additionally, even though these schedulersgive the administrator the capability to integrate anymechanism, they do not have mechanisms to handleconflicts between concurrent users with overlappingQoS requirements. Hence, we propose a new meta-scheduler that uses mechanisms from established areassuch as economics to handle such requirements.In recent years, many economy-based scheduling

mechanisms have been proposed to handle concurrentuser requirements. A framework for auction protocolsis suggested by Chard et al. [ Chard presented a novelarchitecture for a Virtual Organization (VO) baseddistributed economic metascheduler in which membersof the VO collaboratively allocated Grid resources inREXEC [14] and Tycoon [7] are proportional share

systems in which a task is allocated a share of theresource depending on the proportion of its bid (price)to the total sum of the bids of all tasks executing onthat server. Wieczorek et al. [15] proposed a Gridresource allocation model based on Continuous DoubleAuctions (CDAs) to schedule workflow applications onGrid resources. Pourebrahimi et al. [16] presentedsome research on applying CDAs in resource allocationon the Grid. They proposed an agent-based Gridmodel encompassing three types of agents: buyer,seller, and auctioneer agent. They introduced fourtunable parameters which can be used to modifydifferent behavior of Grid participants. Kant etal. [17] proposed and compared various types ofdouble auctions to investigate its efficiency for resourceallocation in Grid. In addition, Tan et al. [18]proposed the stable CDA to overcome the problemof high volatility. Vanmechelen, et al. [19] developedcentralised and decentralised algorithms for economicresource management using futures market to maximizerealized consumer value. DRIVE [20] is a meta-scheduling framework to enable secure auction in Gridenvironment. LibraSLA [21] prioritises users on thebasis of application deadlines and the user-specified


Double Auction-Inspired Meta-Scheduling of Parallel Applications on Global Grids 3

penalties for not meeting them. Bellagio [6] is anauction-based system that seeks to allocate resources fordistributed computing infrastructure in an economicallyefficient fashion to maximise aggregate end-user utility.Aforementioned market-based systems and mecha-

nisms primarily aim either to improve the profitabilityand utilisation of the resource providers or utility sat-isfaction of the users, but not both at the same time.These systems use only the application valuations pro-vided by the users. However, users cannot be expectedto provide accurate valuations as they lack complete in-formation about resource availability in a dynamic en-vironment such as Grids [8]. As mentioned before, theproblem is also to design valuation metrics that com-moditise resource requirements and availability so as totake advantage of the efficiency of market-based mech-anisms [22]. Moreover, the application of the two sidedmarket based mechanism, such as double auction, tocomputational resource allocation has focused mainlyon those cases where services can be easily traded ascommodities, which is clearly not the case in Gridswhere applications can require simultaneously multipleresources which are difficult to commoditise. Therefore,a variant of the double auction is designed in our con-text.Xiao et al. [23] present an incentive-based scheduling

scheme, which employs a peer-to-peer decentralisedscheduling framework, to maximise the success rate ofapplication which require one processor for execution,and minimise fairness deviation among resourceproviders. However, the incentive-mechanisms in thiscase is primarily profit-based which tries to ensurethat each provider has an equal chance of attractingjobs. However, there is a need to ensure that system-centric objectives such as minimising waiting time forapplications are met as well.Scheduling of non-malleable parallel jobs [24] on

grid resource sites is still in its infancy. Mostof research in this area has been concentrated onsingle supercomputing systems [25]. A survey ofscheduling parallel jobs on the computational Grid byFeitelson et al. [26] classify the scheduling on the Gridinto single-site (non-co-allocation) and multi-site (co-allocation). Our work focuses on scheduling rigid single-site jobs on Grids. Many Genetic Algorithm (GA)-based solutions have been proposed to improve theperformance of parallel job scheduling mechanisms [27,28, 29, 30]. As GAs require a long time to execute,they are not suitable for a dynamic environmentsuch as Grids where schedules have to be recomputedregularly since resource availability changes rapidly.Abawajy et. al [31] presented an online dynamicscheduling policy that manages multiple job streamsacross both single and multiple cluster computingsystems with the objectives of improving the meanresponse time and system utilization. This workassumes that parallel jobs are moldable and thus thenumber of processors can be changed. Sabin et.al [32]

presented an algorithm to optimize turnaround timeand utilization by submitting jobs simultaneously onmultiple heterogeneous resources. Similar model isconsidered by Tchernykh et. al [33] who theoreticallyanalyzed the meta-scheduling problem for multi-siteenvironments. The designed scheduling policies inthese works are for multi-cluster environments whichare generally under single administrative domain butour work is for the Grids that extend across multipleadministrative domains. Moreover, none of these worksconsidered user QoS requirements such as deadlinewhile scheduling parallel jobs.In our previous publication [34], we have presented

a double auction-based meta-scheduling mechanismto increase fairness and user satisfaction for theBag-of-Task applications. This paper extends theapplication model to parallel applications with rigidprocessor requirements. In this paper, therefore, wefocus on designing a meta-scheduling mechanism usingauction principles for parallel applications having rigidprocessor requirements to benefit both the user, byaiming to meet the QoS requirements of applications,and the resources, by balancing load across them.The scheduling of such applications on Grid resourcesis a complex 0 − 1 Knapsack Problem that is morechallenging than traditional scheduling on parallelsystems due to: fixed number of processors requiredby the application; dynamic availability of resourceswith different capabilities in different administrativedomains; and continuous arrival of applications at themeta-scheduler [35].Hence, the contributions of this paper are: 1)

a meta-scheduling mechanism for parallel applicationsthat takes advantage of both auctions and system-based schedulers to meet the needs of users as wellas balance load across resources to ensure effectiveutilisation, 2) A new valuation metrics for modelingboth user and resource requirements. The valuationmetrics commodify both the available resource shareand the users’ application requirements such that theycan be compared and matched using principles fromdouble auctions and 3) Detailed performance analysis ofmeta-scheduling algorithms using different performancemetrics.

3. SYSTEM COMPONENTS

The meta-scheduler, considered in this work, follows themodel commonly found in large computing installationsacross educational and research institutions [36] asshown in Figure 1. In this model, resources aremanaged at different sites by administrators who haveto cater for their local users’ needs. Batch schedulingsystems that manage these resources are generallyorganised as a collection of user-accessible job queueswhere a queue may only allow the submission ofspecific applications that meet certain criteria (e.g.within a maximum application size) [37]. The resource


4 Garg et. al

xxxxxx

xx

xxxx

x

xx

xx

`xxxxx x

xxxx

x

x x Users x x Local Scheduler

xxMeta-Scheduler

xx

x x Resource Providers

xx xxxx xx

xxxx

xxxxxx

xx

xx

xx

xx

xx

xx

xx

xxxx

xxxx

xx

xxxx

xx

xxxx

xxxx

xx

1. Job request from

users with deadline

6. Acknowledge user

about

Resource match

2. Resource Queue

parameters sent by

provider

3. Meta-Scheduling mechanism

to match jobs to resource queues

4. Meta-scheduler send

jobs to local scheduler

for execution

5. Scheduling of

job for matched

time slot

FIGURE 1. Interactions between a meta-scheduler, users and local schedulers at the resource sites

management system (local scheduler) at a site mayemploy policies such as easy or conservative backfillingin order to improve the utilisation and responsivenessfor small applications [38]. The pre-emption ofexecuting applications may not be allowed. The meta-scheduler uses the information supplied by providersand users to match applications to the appropriatequeues on the resources. The meta-scheduler runs thescheduling algorithm at periodic intervals to satisfyboth the users’ and the resource providers’ objectives.It may have control over allocation to some or all ofthe processors in a resource or may only be allowed toaccess certain queues within a resource. After matchingapplications to resource queues, the meta-schedulertransfers the user applications to local schedulers of theresource site for execution. Therefore, other than themeta-scheduler, there are two principal participants inthis system, namely, the resource sites and the users.

• Resource Sites: We consider a grid with m

resource sites, R1, R2...Rm with k job queues.Resource sites supply information about availableslots, load and waiting times of each queue to themeta-scheduler at regular intervals. A slot is aunit of resource allocation which is described by

a start time, a finish time, and the number ofprocessors available for that duration. A resourcesite also supplies an initial valuation for running anapplication in its queue slots. This initial valuationmay be based on the processors provided to thequeue and the load expected to be generated on theresource by the jobs in that queue. The objectiveof the meta-scheduler is to distribute jobs acrossall the resource sites to ensure effective utilisationthrough load balancing, and minimal slowdownand waiting time for applications.

• User: In this work, we consider the user applica-tion to follow a compute-intensive parallel appli-cation model composed of multiple communicatingprocesses. An application has a rigid number ofprocessor requirements that need to be satisfied ata single resource site. Most parallel applicationsare of this nature, as they are sensitive to latencyof message passing, unless they have been explicitlydesigned to be executed across multiple resources.The objective of the users is to have their applica-tions completed by a deadline. It is assumed thatdeadlines are hard, i.e. a user will benefit onlyif his/her application is executed by its deadline.



Users will also provide an initial valuation of theapplication to the meta-scheduler. This valuationcan be based on the importance of the applicationto the user. The estimated execution time of anapplication by the user is considered to be accu-rate [39] in order to facilitate comparison betweenthe algorithms proposed in this paper and thosecited in Section 2 .

4. DOUBLE AUCTION-INSPIRED META-SCHEDULER (DAM)

The Double Auction-inspired Meta-scheduling processproposed in this paper, hereafter referred to as DAM,is a sequence of three broad stages as shown inFigure 2. The first stage (collection) consists ofgathering information about resources and applicationssuch as queue slot availability and waiting time for theformer, and processor and QoS requirements for thelatter. In the next stage (valuation), the valuationsare computed for all applications and resources bythe meta-scheduler. It is important to note that thevaluation is private to the meta-scheduler and hiddenfrom both users and resource providers. Finally, thelast stage of scheduling is to perform matching ofuser applications to the available resources based onthese valuations. Those applications that are notmatched are then retained at the meta-scheduler. Inthe next scheduling cycle, resources and applicationvaluations are recomputed using new information andthe matching is carried out again.As described in Section 2, auction-based mechanisms

have been the subject of many previous studies. Grosuet al. [40] have compared resource allocation protocolsusing First-Price, Second-Price Vickrey and DoubleAuction (DA), and have concluded that DA favoursboth users and resources, while the first-price auctionis biased towards resources and the Vickrey auctiontowards users. Therefore, we have opted to useprinciples of DA, also known as Call auction, withinthe meta-scheduler.

4.1. Call Auction

In a typical Call auction, sellers and buyers submitoffers (asks:aj , 1 ≤ j ≤ m) and requests (bids:bi, 1 ≤

i ≤ n) respectively to an auctioneer who continuallyranks them from highest to lowest in order to generatedemand and supply profiles. The ordering of asks andbids after sorting is the following:

a1 < a2 < . . . aj . . . < amb1 > b2 > . . . bi . . . > bn

A seller is allowed to participate in the matchprocess if am < bn. From the profiles, the maximumquantity exchanged can be determined by matchingasks, starting with the lowest price and moving up, withthe bids, starting with the highest price and movingdown. This format allows buyers to make offers, and

sellers to accept those offers at any particular moment.After matching process, the auctioneer decides theamount of payment received by the provider from theuser based on the ask and bid values.Call auctions present an efficient framework for

general resource allocation, especially if carried out overa long period of time. However, the meta-schedulerin this paper carries out the matching internallywithout any explicit involvement of buyers and sellers.Also, the applications considered for scheduling aredifferent enough, that they cannot be commoditisedand compared using single values. The same holds forresources as well. Therefore, call auctions cannot bedirectly applied to this scenario. However, we havedesigned a valuation mechanism that allows us to takeadvantage of the efficiency of call auctions for ourmatching process.

4.2. Valuation Mechanism

In call auctions, the maximum bid is matched to theminimum ask. In the meta-scheduler, the applicationthat has the earliest deadline (and thus, the mosturgent) must be matched to the fastest queue thathas enough processors available. Therefore, we haveconstructed our valuation mechanism such that theresource valuations are the asks and the applicationor job valuations are the bids. The valuations ofthe resources and the applications depend on manyfactors such as supply and demand, resource loads anduser deadlines. We have to reconcile these multiplemeasures to a single value that can be used forcomparison of valuations. To this end, we apply Multi-Attribute Utility Theory (MAUT) [41][42][43], whichprovides a logical, consistent and tractable approach forquantifying an individual’s preferences by consolidatingthem into a single objective. This theory first addressesthe identification of attributes and the desirabilityfunction for each attribute and, then the aggregationof these desirability functions into a single scalar utilityvalue.

Resource Valuation (Ask) Batch job schedulers thatmanage resources are generally constructed as sets ofqueues with different attributes. The load of a queueis defined as the ratio of the number of processorsoccupied by jobs to the total number of processorsavailable to the queue. In order to balance load acrossindependent grid sites, the meta-scheduler tries to givepreference to the least loaded queues while submittingapplications. Also, the most urgent application mustbe matched to the fastest queue. Hence, the valuationmetric of resource queues should be such that the queuewith the least waiting time should have the least askvalue.The valuation metric should also take into account

the initial valuation given by the resource provider, andalso the demand and supply levels of resources in the


6 Garg et. al

FIGURE 2. Auction-based Meta-Scheduling

0.15

0.2

0.25

0.3

d D

evia

tio

n

0

0.05

0.1Lo

ad

Time (Seconds)Additive Form Multiplicative Form

FIGURE 3. Comparison of Multiplicative and Additive forms of Valuation Metrics

system (denoted as Demand and Supply). Demand

is the sum of the processors required by applicationsto be allocated and Supply is the total number ofprocessors available at all resources. Let lk,t be the

load on the resource queue k at time t. Let ck,t be theinitial resource valuation given by the provider. Letwk,t be the application waiting time for queue k attime t. Thus, the desirability functions are proportional



to wk,t, ck,t, Demand, Supply, and lk,t. Since eachof these attributes are preferentially independent [43],thus the valuation metric can be formed by aggregatingthese attributes either in multiplicative or additiveform. When we compare the effects of each metric ondistribution of load on the resource sites, we observedthat in the case of additive form, the deviation in load1

across the resource sites is much more than that for themultiplicative form as shown in Figure 3. In addition,when more than two attributes are involved, anaccurate additive aggregation requires the exact trade-offs between attributes, which may not be apparent.Thus, we have used a multiplicative aggregation to forma valuation metric from normalised attributes, wherecmax,t is the maximum initial valuation given by aresource provider and lmax,t is the maximum load onthe resources. This valuation metric ak(t) of queue k attime t is given by following equation:

ak(t) = Ok × wk,t ×ck,t

cmax,t×

lk,t

lmax,t×

Demand

Supply, (1)

where Ok is a proportionality constant which is takento be 1 for simulation purposes.

Job Valuation (Bid) Similar to resources, a userapplication has also many attributes such as the numberof CPUs required, deadline and run time. The valuationmetric of application i at time t is designed in such away that the maximum value is assigned to applicationswith urgent deadline. Also, if an application hasnot been mapped in the previous scheduling cycle, itscorresponding bid (valuation) should be increased. Thisis to reduce the possibility of this application gettingstarved of resources by others with higher valuations(bids) that have arrived at the meta-scheduler in themeanwhile. The (di − t) indicates how urgently userwant application i to be executed, where di is the user-supplied deadline for application i. Let sti be thesubmission time of the application. Similar to that forresources, the application metric should also take intoaccount the initial valuation of the application given bythe user, and also demand and supply levels of resourcesin the system (denoted as Demand and Supply). Letvi be the initial application valuation given by the user.Thus, the desirability functions are proportional to vi,sti, Demand, Supply, and (di− t). Let vmax, dmax andstmin be the highest initial valuation, largest deadlineand earliest start time, respectively. The resultantvaluation metric for application i at time t, formedby multiplying all the normalized attributes, is thefollowing:

1The experiment is conducted with the same configurations asgiven in the Section 5. The“load deviation” is used as a metric forcomparison which indicates the standard deviation of the resourceload across the grid from average.

bi(t) = Hi×vi

vmax×dmax − t

di − t×Demand

Supply×

(t+ 1− sti)

t+ 1− stmin,

(2)where Hi is a proportionality constant which is taken

to be 1 for simulation purposes, and di 6= t.

4.3. The Meta-Scheduling Algorithm

As noted in the previous sections, the commodity unit,matched on behalf of the resource site, is a slot where aset of processors are bounded by start and finish times.Figure 4 demonstrates the method using which the slotscan be generated within a Grid site. At time t, the

!

"

#

!

"

!

"

$"

$!

$#

%&

%'

(%"%!

%#

%) %*

!"##$%&'()*$'!"#

()*$

+"$"$,

'$%

%+

,!

,#

,*

,)

,"

FIGURE 4. Available queue slots.

local job scheduler can provide the time slots that arecurrently free, so applications can be scheduled by themeta-scheduler to fill up the queue to a specific timehorizon TH . The slots start from the first available timeand contain as many processors that are not occupiedfor a specific duration. In Figure 4, the slots S1 to S5

are examples of such free slots. The list of availableslots can be generated by the local job scheduler usingthe estimated execution time of the applications. Asshown in Figure 4, after current time t, two processorsare free in queue Q3 upto time TH since jobs J6 andJ7 are estimated to finish before time t. S5 is therefore,an available time slot with available time = TH − t. Inthe case of Queue Q2, jobs J4 and J3 are finishing aftertime t and also estimated to complete at different times.Thus, the simultaneously free processors after time t

result in two time slots S2 and S4, each with differentnumber of processors and available time unit. Similarly,another time slot S1, consisting of one processor on thequeue Q1, is available as well. This approach, whereslot can be of different sizes, was also used by Singh et.al [30]. In this case, the local scheduler at the resourcesite can use backfilling to minimise the fragmentationin the schedule such that execution of applications inthe queue does not get delayed.


8 Garg et. al

1 while current time < next schedule time do

2 RecvResourcePublish(P);// P contains information about providers

3 RecvJobQoS(Q) ;// Q contains QoS information about users

4 Add information of pending applications from previousscheduling cycle to RecvJobQoS;

5 Calculate the Demand and Supply for resources;6 Update the value of bids and asks using eqn. 2 and 1;7 Sort asks in ascending order ;8 Sort bids in descending order;9 while no all applications are assigned to resource queues do

10 i = 1, j = 1;11 if bid bi is greater than ask aj then

12 if QueueWaitingT ime(j) +ExecT ime(i) <Deadline(i) then

13 if check processor availability on resource j

then

14 Schedule the application i to the resource j;15 add application with matched resource site

in Schedule List (Schd List) ;16 update the value of available time slots

from resource j;17 i++;

18

19 else

20 add user application to pending application list;

21 j ++;

22 foreach element ∈ Schd List do

23 notifyuser();

Algorithm 1: Double Auction-inspired Meta-scheduling in each scheduling cycle

The meta-scheduler schedules the applications inregular time intervals, therefore in the beginning ofa scheduling cycle, it gets the updated availablequeue slots using the above approach from local jobschedulers. Our proposed scheduling algorithm isshown in Algorithm 1. In each scheduling cycle, themeta-scheduler schedules the parallel applications aftercollecting all users’ requests and resource performanceinformation such as queue waiting times and freetime slots (Line 1-3). At the end of each schedulingcycle, the meta-scheduler computes the demand forresources and their supply (Line 4). Then, based onthe information collected from users and resources, themeta-scheduler assigns valuation to user applications(bids) and resource’s queue (asks) using the valuationmechanisms presented in the previous section (Line 6).From the sorted bid list, the bid (bi) with the

highest value will be matched to the resource queuewith the minimum ask (aj) that can satisfy the user’srequirement. Whether user application i will bescheduled to resource queue j (corresponding to askaj) depends on the applications’ processor and deadlinerequirements. Thus, the first deadline of applicationi is checked using waiting time of resource queue j

(Line 12), and then processor availability is checkedon resource queue j (Line 13). If there is an ask thatsatisfies the application’s QoS requirements, then thebid is matched to ask; and the matched user and the

resource provider are informed of the decision. Theapplication i is then scheduled on to resource queuej (Line 14) and then added to the schedule list (Line15). The available time slots (commodity units) onresource queue j are updated correspondingly (Line16). If the deadline requirement of application i can besatisfied by resource queue j, then no other ask can bematched to the application’s bid (Line 20). Therefore,the application is removed from the bids list in thatscheduling cycle. If the required number of processorsis not available on resource queue j, bid bi is matchedwith the next ask in the sequence (Line 21). Matchingfor other bids and asks which are in pending list willbe done in the next scheduling cycle with new requests.All the users whose applications have been scheduledare notified by the meta-scheduler about the matching(Line 23). In each scheduling cycle, the process isrepeated with recalculation of ask and bid values untileither all bids or asks are matched. It should be alsonoted that the valuation process is such that unmatchedbids will get higher value in the next scheduling cycleto avoid starvation of jobs.

5. PERFORMANCE EVALUATION

In this section, we evaluate the performance ofour proposed algorithm in various experimentalscenarios. Therefore, we present the details ofexperiments conducted to compare the performance ofDouble Auction-Inspired Meta-Scheduler (DAM) withcurrent state-of-art meta-scheduling algorithms andapproximate theoretical model of the meta-scheduler.

5.1. Comparison of DAM with State-of-ArtMeta-scheduling Algorithms

5.1.1. Experimental ConfigurationFor our experiments, Feitelson’s Parallel WorkloadArchive (PWA) [35] is used to model the parallel appli-cation workload. Since this paper focuses on studyingthe HPC parallel applications of users, the PWA meetsour objective by providing the necessary characteristicsof real parallel applications collected from supercom-puting centers. To avoid the effect of initial setupphase of the HPC center, our experiments utilize thetraces from the second week of the Lawrence LivermoreNational Laboratory (LLNL) Thunder supercomputer(January 2007 to June 2007). The LLNL Thundertrace is chosen due to its highest resource utilizationof 87.6% among available traces to ideally model aheavy workload scenario. From this trace, we obtainthe submit time, requested number of processors, andactual run time of applications. The characteristicsof the traces are given in Table 1. The submissiontime of a parallel application is divided by 1000 toincrease the number of applications submitted perschedule interval as per the methodology presented byFeitelson and Wiseman [44]. Since the workload trace



TABLE 1. Workload CharacteristicsMean Inter-Arrival Time 16.176 sec.

Average Job Runtime 2328.911 sec.

Standard Deviation for JobRuntime

7790.11 sec.

Average CPU requirement 44 CPUs

does not contain any information about the user’sdeadline and initial valuation, these are generatedsynthetically. For a user application with a runtimer, the deadline is generated from a uniform randomdistribution between r and 3r. The trace data of utilitygrid applications are currently not released and sharedby any commercial utility grid providers, thus thisinformation also has to be generated using a randomdistribution. The average initial valuation of applica-tions is chosen randomly between 90000 and 160000currency units using uniform distribution, so that it isalways greater than application execution cost. Theapplication execution cost can be computed by using re-source initial value*execution time*number of CPUs.The user valuations are assigned so that at least halfof users can afford to execute their application onthe resources with the highest valuation. The gridmodelled in our simulation contains 10 resource sitesspread across five countries derived from EuropeanData Grid (EDG) testbed [45]. The configurationsassigned to the resources in the testbed for the simu-lation are listed in Table 2. The configuration of eachresource is decided such that the modelled testbedreflects the heterogeneity of platforms and capabilitiesthat is normally the characteristic of such installations.Each of the resources are simulated as a cluster thatemploys a multi-partition Easy backfilling policy forlocal resource allocation [46].The processors associated with each cluster in Table 2

are exclusively managed by the meta-scheduler (i.e.all users are going through meta-scheduler). We havesub-divided the allocated processors of each clusterinto 3 queues in ratio of 1:2:3 of the total number ofprocessors in the cluster. The processing capabilities ofthe processors are rated in terms of Million Instructionsper second (MIPS) so that the application requirementscan be modelled in Million Instructions (MI). An initialvaluation, randomly computed between 4.5 and 9.5currency units per processor per second, is assigned toeach resource.We have compared our double auction-inspired meta-

scheduling algorithm against five other well-knowntraditional and market based algorithms listed below:

• Shortest Job First (SJF): In this algorithm,applications are prioritized on the basis ofestimated runtime. This is a very commonalgorithm used in cluster management.

• First Come First Serve (FCFS): An applica-tion is assigned to the first available queue. This

TABLE 2. Simulated EDG Testbed ResourcesSite name (location) Number of

processorsSingleproces-sor rating(MIPS)

RAL (UK) 2050 1140Imperial College (UK) 2600 1330NorduGrid (Norway) 650 1176NIKHEF (Netherlands) 540 1166Lyon (France) 600 1320Milano (Italy) 350 1000Torina(Italy) 200 1330Catania (Italy) 250 1200Padova (Italy) 650 1000Bologna (Italy) 1000 1140

is a common algorithm employed by many meta-schedulers such as GridWay [4].

• Earliest Deadline First-Fastest Queue (EDF-FQ): In this algorithm, the applications with theearliest deadline are scheduled on to the resourcequeue slot with the least waiting time (FastestQueue (FQ)).

• Highest Valuation to Fastest Queue(HVFQ): In this algorithm, the applicationwith the highest user valuation is assigned to thequeue slot with the least waiting time. This algo-rithm is generally used in auction mechanism suchas Vickrey auction. Vickrey auction is used inresource management systems such as Spawn [47]and Bellagio [6].

• FairShare or Proportional Share: In thisalgorithm, each application is assigned queue slotsproportional to the ratio of its valuation to thecombined valuation of all the applications. Thisalgorithm is employed in REXEC [14].

The following criteria are used to compare fairnessand user satisfaction provided by these algorithms:

• Urgency vs. Success Ratio: The user’s urgencyto get their application completed, is defined as:

u =deadline− start time

execution time− 1 (3)

where start time and execution time are at-tributes of the application. The deadline is con-sidered very urgent when u < 0.25, urgent when0.25 < u < 0.5, intermediate when 0.5 < u < 0.75,relaxed when 0.75 < u < 1 and very relaxed whenu > 1. “Success Ratio” is ratio of number of appli-cations finished successfully before deadline to thetotal number of applications in a particular urgencygroup. This criterion relates to how the schedulerdeals with users with different demands on time.

• Valuation vs. Success Ratio: The valuationprovided by the user for an application is dividedby the required number of processors in order tonormalize it. We examine how the schedulers


10 Garg et. al

allocate resources fairly among different users withdifferent application valuations. If (u < 0) foran application then the application will not bescheduled by the meta-scheduler.

• Number of deadlines missed with increase innumber of user applications. We use this criterionto examine how the scheduling algorithms areable to cope with user requests when demand forresources exceeds supply.

• Load Deviation: The load of a resource is theratio of the number of processors occupied to totalnumber of processors available at the resource site.We average the load over the grid resources andmeasure the standard deviation. This informsabout how well the scheduling mechanism was ableto balance load across the grid.

5.1.2. Analysis of ResultsIn this section, we discuss the results of our evaluation.

Benefit for Users: This section shows how our meta-scheduler benefits users by not only completing themost number of applications with different QoS needsbut also benefiting every user in different urgency andbudget groups.

• Effect of User Urgency: Figure 5(a) showsthe percentage of total applications completedsuccessfully against the users’ urgency values.Figure 5(a) shows that DAM has scheduled a largernumber of applications than other algorithmsin every urgency group. For example, in theintermediate group (0.5 − 0.75), DAM schedules12% more applications than its closest competitor(FairShare). This is in contrast to the performanceof FCFS and SJF which is the worst in almostevery case. This is due to the fact that DAM isdesigned to increase an application’s value withurgency, while in others this is not considered. Theperformance of FairShare is very close to DAM andit has scheduled an number of applications almostequal to that of DAM when deadline urgency is lessthan 0.25. This is because DAM tries to reducethe waiting time of applications with relaxeddeadline by increasing their valuation. Thus,when the deadline urgency is greater than 1, thenDAM schedules about 12% of more applicationsthan FairShare. Jobs with relaxed deadlinesprogressively gain in valuation (or, float to the topof the bid list) when they are held at the schedulerover time in DAM, and are therefore not starved.This can be seen by comparing the performanceof DAM with EDF-FQ, which prioritizes urgentapplications but performs poorly with relaxeddeadlines. Since the users’ objective is to completetheir applications by the deadline, delaying anapplication at the scheduler is appropriate as longas the deadline is met.

• Effect of User Valuation: From Figure 5(b),we can see that DAM completes more numberof applications across all valuations than theother algorithms. Even though Fareshare againperformed very close to DAM for medium valuationgroups, DAM outperforms FairShare for allother groups by scheduling atleast 10% moreapplications. For applications with very lowvaluation (< 1000), the DAM has managed toschedule about 20% of the applications whencompared to 11% for FairShare which performs aswell as DAM, when the application valuations aremedium. This is because the latter assigns thelowest proportion of resources to the users with thelowest valuation. Therefore, in this case, most ofthe parallel applications fail to execute due to lackof sufficient processors. It is also interesting to notethat HVFQ, which is supposed to favour users withhigh budget, has scheduled almost 4% less numberof applications than DAM for Budget > 33000.This is because HVFQ does not consider otherrequirements of applications such as deadline.

• Number of deadlines missed: From Fig-ure 5(c), we can clearly see that as the demandfor resources (number of applications) increases,the number of applications that missed their dead-line also increases due to the scarcity of resources.But DAM is able to complete around 8% to 15%more applications than other algorithms. As SJFresulted in better packing of jobs at resource sites,SJF performs relatively better than the other algo-rithms such as EDF-FQ, HVFQ and FCFS. FCFSperforms the worst as it does not consider the effectof deadlines and queue waiting times.

• Variation in user’s urgency and arrivalrate: We conducted this experiment to understandthe effects of various valuation parameters onour algorithm’s performance. Figure 6(a) and6(b) shows how user valuation parameters, suchas urgency and arrival rate, effect successfulcompletion of applications. The deadlines of allusers is varied from high to low which is calculatedin terms of ‘Urgency’ level (Figure 6(a)). Thus,for a particular experimental scenario, all usershave same urgency level. Similarly, to vary thesubmission time of jobs arriving for scheduling, jobarrival rate is changed from ‘very low’ to ‘veryhigh’(Figure 6(b)).In Figure 6(a), the decrease in success ratio incase of all the algorithms is due to limited numberof resources which cannot execute all the urgentjobs simultaneously. Similarly, with increase in jobarrival rate, except DAM and FairShare, all thealgorithms resulted in lower success ratio. It isclear from the experiments that schedules obtainedfrom DAM always results in the highest successfulcompletion of applications (almost 15% more) in allcases. It shows that by using our proposed meta-



xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

0.3

0.4

0.5

0.6

uc

ce

ss

Ra

tio

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx





0

0.1

0.2

<0.25 0.25-0.5 0.5-0.75 0.75-1 >1

Su

Deadline Urgency

xxxx

FCFSxxxx

FairSharexxxxxx

SJFxxxxxx

HVFQxxxx

EDF-FQxxxxxx

DAMxxxx FCFS

xxxxFairShare

xxxxxxSJF

xxxxxxHVFQ

xxxxEDF-FQ

xxxxxxDAM

(a) Effect of user urgency

0.7

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

0.6

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

0.5

Ra

tio

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0.3

0.4

uc

ce

ss

R


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx





0.2

Su







0.1

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx



xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx



0

<1000 1000-11000 11000-22000 22000-33000 >33000

Application Valuation/Number of Applications

xxxx

FCFSxxxxxx

FairSharexxxxxx

SJFxxxxxx

HVFQxxxx

EDF-FQxxxxxx

DAMxxxx FCFS

xxxxxxFairShare

xxxxxxSJF

xxxxxxHVFQ

xxxxEDF-FQ

xxxxxxDAM

(b) Effect of user valuation

40

50

60

70

80

90

s M

iss

ed

De

ad

lin

e

0

10

20

30

1000 2000 3000 4000 5000

% o

f J

ob

s

Number of Jobs Submitted

FCFS FairShare SJF HVFQ EDF-FQ DAMFCFS FairShare SJF HVFQ EDF-FQ DAM

(c) Percentage of Deadlines Missed

FIGURE 5. Benefit for Users

0.3

0.4

0.5

0.6

ccess R

ati

o

0

0.1

0.2

1 2 3 4 5 6

Su

c

Urgency Level (low to high)

FCFS FairShare SJF HVFQ EDF-FQ DAM

(a) Effect of variation in urgency

0.4

0.5

0.6

0.7

0.8

0.9

cess R

ati

o

0

0.1

0.2

0.3

0.4

1 2 3 4

Su

cc

Arrival Rate (Very low to Very high)


(b) Effect of variation in arrival rate

FIGURE 6. Benefit for Users by Varing Valuation Parameters

scheduling algorithm, users have higher chance ofjob completion.

Benefit for Resources: Simulation results in Figure 7show how DAM affects the load on different resources.The figure shows the standard deviation in resourceloads against the time period of the execution. It can


12 Garg et. al

0.25

0.2

0.15tio

n0.15

d D

evia

ti

0.1

Lo

ad

0.05

0

Time (Seconds)


FIGURE 7. Variation in Load across Resources.

be noted that while the deviation across resources forother algorithms is steadily increasing, on average DAMhas kept the load deviation, consistently, almost close to0. This implies that the DAM was able to successfullybalance the load across all the resource sites. Thisis due to the fact that the resource queue’s valuationis increased when its load is increased and therefore,heavily loaded queues are sorted to the bottom of theask list. The performance of the EDF-FQ algorithm,which is closest to that of DAM, also results in onaverage five times more Load Deviation than DAM.Moreover, from Figures 5(a) and 5(b), it can be seenthat EDF-FQ does not schedule as many applicationsas DAM, even though EDF-FQ also tries to balance loadacross the resources by submitting according to queuewaiting time. However, FairShare algorithm whichbenefits users in similar way as DAM, has resulted inthe maximum load imbalance which is even worst thanHVFQ. Thus, DAM is not only providing benefit tousers but also providing benefit to resource providersby equally dividing load between them.

5.2. Comparison of DAM with TheoreticalResults

In order to understand the generalized behavior of ourproposed algorithm, we compared the performance ofDAM with theoretical results from a queueing model ofmeta-scheduler in terms of other system-based metricssuch as slowdown and waiting time. An application’sslowdown is its waiting time divided by its executiontime. Mean slowdown is considered because users

generally desire that their application delay should beproportional to its execution time [48][49]. For instance,users with lighter processor requirements will generallyprefer to wait for lesser time than those with heavierrequirements. The details of queueing model of meta-scheduler are given in Appendix A.

The optimal mean waiting time and mean slowdownis calculated using the approximate queuing modelfor meta-scheduling, by solving Equation A.24 andA.25 using the NMinimize function in Mathematica.This is achieved by finding the ri values in eachinstance that produce the local minima for expectedwaiting time (E(W)) and slowdown (E(S)). Sincethe analytical model proposed in Appendix A is anapproximation to the meta-scheduling problem, thusthe solution obtained is actually near-optimal for themeta-scheduling problem at steady state.

Experimental Methodology: Li et al. [50]analyzed various Grid workloads and found that theWeibull distribution is best fitted to model runtimeof applications. Hence, the application runtime isgenerated using a Weibull distribution (α, β) [50].The arrival rate application is assumed to be Poissondistribution with parameter λ. Since our aim is tocompare both analytical and simulation results, theprobability distributions for arrival rate and runtimeof applications are the same in both analytical andsimulation experiments. However, since the analyticalmodel is an approximation of the real meta-schedulingsystems, thus the ri values can differ between theanalytical model (where they are numerically solved)



��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

��DAM Analytical ModelDAM Analytical Model

(a)

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

(b)

��

��

�

��

��

��

��

��

��

��

��

��

��

� ��

�

� ��

��

��

(c)

��

��

��

��

��

��

��

��, ��

�

��

��

��

��

��

��DAM Analytical ModelDAM Analytical Model

(d)

FIGURE 8. Comparison of DAM with analytical results

and the simulated real systems (where the schedulingis performed using heuristics). The arrival rate of ith

priority class applications (λi) depends on probabilitypi. The pi is obtained during the valuation process ofDAM through simulation. Each resource considered isassumed to have same number of processors to make thesimulation scenario as close as possible to the analyticalmodel. To make the solution of the analytical modeltractable, we have considered five Grid resources with128 processors each and four priority class applications.

A large range of λ values are considered demonstrat-ing a wide spectrum of load and arrival rate. The per-formance metrics are computed for these arrival rateseach with different mean application runtime (repre-sented by the combination of values of α and β). Twoset of results with different mean application runtimeare presented in Figure 8. The different mean in twoscenario is obtained by scaling up the value of β whichresults in jobs with longer average runtime and withlarge variance.

In order to obtain steady state results, we haveignored scheduling of the first 5000 applications and

measure mean waiting time and slowdown for the next5000 applications. For each value of λ, experiment isrepeated 30 times, and average of their results is usedfor comparison.

Discussion of Results: Figure 8 shows that thesimulation results of DAM follow similar increasingtrend as the analytical model. This not only validatesthe correctness of the DAM heuristic but also indicatesthat the performance of DAM is near optimal interms of metrics such as mean waiting time and meanslowdown. DAM gives consistently lower values forwaiting time and slowdown than the optimal valuesobtained from analytical model. There is a significantgap between DAM and analytical model results, forexample, when β = 25, the gap between DAM andanalytical model is about 25% to 30%. The reason forthe gap is that the analytical model is an approximationof real meta-scheduling systems and thus it does notmodel the backfilling policies used by the local schedulerof resources which decreases the slowdown and waitingtime of applications more than the optimal solutionobtained from the analytical model.


14 Garg et. al

6. CONCLUSIONS AND FUTURE WORKS

In this paper, we have presented a meta-schedulerfor allocating parallel applications with rigid processorrequirements on distributed resources within a grid.The resource sites are organised as a collection of queueswith different capabilities and waiting times. The goalsof the scheduler are to benefit the users by taking intoaccount their deadlines and the value attached to theirapplications, and to benefit the resources by allocatingthe applications such that the load is balanced acrossthe Grid. The scheduler also has to benefit all users bypreventing starvation of applications that have relaxeddeadlines or low valuation, while keeping the effect onother crucial system metrics such as mean waiting timeand slowdown to a minimum.Given many objectives of the scheduler, we adopted

the principles of an efficient double auction protocolto design the core mechanism in the scheduler.We have met the challenge of designing valuationmetrics that commodify user applications with differentrequirements and resource queues with different waitingtimes to bids and asks respectively, so that they can bematched and cleared in the Double Auction-InspiredMeta-Scheduler (DAM).Experimental evaluation of the proposed mechanism

against common algorithms such as SJF, HVFQ, EDF-FQ, FCFS, and FairShare used in other meta-schedulershas showed that DAM is able to benefit both users andresource providers across all the target metrics. Thedouble auction mechanism is not only able to scheduleabout 8% to 15% more user applications thanothers but also has the highest success ratio in almostall the groups for applications with different deadlinesand different valuations. For users with the lowestbudget (< 1000), the success ratio of their applicationsis increased by almost 10%. Similarly, DAM alsobenefitted resource providers by equally distributing theworkload according to capacity of resource sites. Thus,DAM is able to improve the balancing of load across theconstituent resources of the Grid with almost zero loaddeviation. While FairShare satisfies users at the samelevel as DAM, the difference in resource loads was thehighest for the former. This means FairShare resultedin the schedule where resources were disadvantagedhighest. The key reason for the better performance ofDAM over others is that the valuation metrics capturethe information which is important to both users andresource providers, and therefore resulted in effectivescheduling. Thus, we demonstrate that by the inclusionof both system metrics and market-parameters, we canget more effective scheduling that will satisfy both usersand resource providers. We have also demonstrated howclassical economic mechanisms, adapted suitably, candeal with multiple QoS requirements of the users moreeffectively than the state-of-the art algorithms used intoday’s schedulers. This motivates further enquiry intoexploration of adapting other economic mechanisms to

solve particular problems in job scheduling.In the future, we will integrate DAM in an actual

meta-scheduler implementation such as GridWay andapply it to real-world scenarios. We also intendto experiment with different valuation methods, andexamine the applicability of the mechanism to otherdistributed application models. In the future, we planto extend this work in the context of Cloud computingfor developing new dynamic pricing and market models.In addition, our work is particularly relevant to notionof brokering across multiple Clouds which is becomingpopular in the form of InterClouds.

ACKNOWLEDGEMENTS

We would like to thank Adam Barker, MukkadimPathan, Kapil Gupta, Professor Rao Kotagiri andMarco Netto for their comments and suggestions onthis paper. This research is funded by the AustralianDepartment of Innovation, Industry, Science, Research(DIISR) and Australian Research Council (ARC).Wewould also like to thank reviewers of the journal for theircomments and suggestions that helped in improving thepaper substantially.

REFERENCES

[1] Henderson, R. (1995) Job Scheduling Under thePortable Batch System. Proceedings of the 1995International Workshop on Job Scheduling Strategiesfor Parallel Processing, Santa Barbara, USA.

[2] Bode, B. et al. (2000) The Portable Batch Schedulerand the Maui Scheduler on Linux Clusters. Proceedingsof the 4th Annual Linux Showcase and Conference,Atlanta, GA, USA.

[3] Laure, E. et al. (2006) Programming the Gridwith gLite. Computational Methods in Science andTechnology, 12, 33–45.

[4] Huedo, E., Montero, R., and Llorente, I. (2004) Aframework for adaptive execution in grids. SoftwarePractice and Experience, 34, 631–651.

[5] Sabin, G., Sahasrabudhe, V., and Sadayappan,P. (2005) Assessment and enhancement of meta-schedulers for multi-site job sharing. Proceedingsof 14th IEEE International Symposium on HighPerformance Distributed Computing, Research TrianglePark, NC, USA.

[6] AuYoung, A., Chun, B., Snoeren, A., and Vahdat,A. (2004) Resource allocation in federated distributedcomputing infrastructures. Proceedings of the 1stWorkshop on Operating System and ArchitecturalSupport for the On-demand IT InfraStructure, Boston,USA.

[7] Lai, K., Rasmusson, L., Adar, E., Zhang, L., andHuberman, B. (2005) Tycoon: An implementation ofa distributed, market-based resource allocation system.Multiagent and Grid Systems, 1, 169–182.

[8] Broberg, J., Venugopal, S., and Buyya, R. (2008)Market-oriented Grids and Utility Computing: Thestate-of-the-art and future directions. Journal of GridComputing, 6, 255–276.



[9] Braun, T. D., Siegel, H. J., Beck, N., Boloni,L. L., Maheswaran, M., Reuther, A. I., Robertson,J. P., Theys, M. D., Yao, B., Hensgen, D., andFreund, R. F. (2001) A comparison of eleven staticheuristics for mapping a class of independent tasks ontoheterogeneous distributed computing systems. Journalof Parallel and Distributed Computing, 61, 810–837.

[10] Feitelson, D. and Jettee, M. (1997) Improved utilizationand responsiveness with gang scheduling. Proceedingsof the 1997 International Workshop on Job SchedulingStrategies for Parallel Processing, Geneva, Switerland.

[11] Schwiegelshohn, U. and Yahyapour, R. (1998) Anal-ysis of first-come-first-serve parallel job scheduling.Proceedings of the Ninth Annual ACM-SIAM Sympo-sium on Discrete algorithms, San Francisco, California,United States.

[12] Frey, J., Tannenbaum, T., Livny, M., Foster, I., andTuecke, S. (2002) Condor-G: A computation manage-ment agent for multi-institutional Grids. Cluster Com-puting, 5, 237–246.

[13] Raman, R., Livny, M., and Solomon, M. (2000) Re-source management through multilateral matchmaking.Proceedings of the Ninth IEEE Symposium on HPDC,Pittsburgh, Pennsylvania.

[14] Chun, B. and Culler, D. (2002) User-centric Perfor-mance Analysis of Market-based Cluster Batch Sched-ulers. Proceedings of the 2nd IEEE International Sym-posium on Cluster Computing and the Grid, Berlin,Germany.

[15] Wieczorek, M., Podlipnig, S., Prodan, R., andFahringer, T. (2008) Applying double auctions forscheduling of workflows on the Grid. Proceedings ofthe 2008 ACM/IEEE Conference on Supercomputing,Austin, TX.

[16] Pourebrahimi, B., Bertels, K., Kandru, G., and Vas-siliadis, S. (2006) Market-based resource allocation ingrids. Proceedings of Second IEEE International Con-ference on e-Science and Grid Computing, Amsterdam,The Netherlands.

[17] Kant, U. and Grosu, D. (2005) Double auction proto-cols for resource allocation in grids. Proceedings of theInternational Conference on Information Technology:Coding and Computing, Washington, USA.

[18] Tan, Z. and Gurd, J. (2007) Market-based gridresource allocation using a stable continuous doubleauction. Proceedings of 8th IEEE/ACM InternationalConference on Grid Computing, Austin, Texas, USA.

[19] Vanmechelen, K., Depoorter, W., and Broeckhove, J.(2008) Economic Grid resource management for CPUbound applications with hard deadlines. Proceedingsof the 2008 Eighth IEEE International Symposium onCluster Computing and the Grid, Lyon, France.

[20] Chard, K. and Bubendorfer, K. (2008) A distributedeconomic meta-scheduler for the grid. Proceedings ofthe Eighth IEEE International Symposium on ClusterComputing and the Grid, Lyon, France.

[21] Yeo, C. and Buyya, R. (2005) SLA based allocation ofcluster resources: Handling penalty to enhance utility.Proceedings of the 7th IEEE International Conferenceon Cluster Computing, Boston, USA.

[22] Shneidman, J., Ng, C., Parkes, D., AuYoung, A.,Snoeren, A., Vahdat, A., and Chun, B. (2005) Why

markets could (but don’t currently) solve resourceallocation problems in systems. Proceedings of the10th USENIX Workshop on Hot Topics in OperatingSystems, Santa Fe, NM, USA.

[23] Xiao, L., Zhu, Y., Ni, L., and Xu, Z. (2008) Incentive-based scheduling for market-like computational grids.IEEE Transactions on Parallel and Distributed Sys-tems, 19, 903–913.

[24] Shmueli, E. and Feitelson, D. (2005) Backfilling withlookahead to optimize the packing of parallel jobs.Journal of Parallel and Distributed Computing, 65,1090–1107.

[25] Bucur, A. and Epema, D. (2000) The Influence of theStructure and Sizes of Jobs on the Performance ofCo-allocation. Proceedings of 2000 Workshop on JobScheduling Strategies for Parallel Processing, Cancun,Mexico

[26] Feitelson, D., Rudolph, L., and Schwiegelshohn,U. (2004) Parallel job schedulinga status report.Proceedings of 2004 Workshop on Job SchedulingStrategies for Parallel Processing (JSSPP), pp. 1–16.Springer.

[27] England, D. and Weissman, J. (2004) Costs andbenefits of load sharing in the computational grid.Proceedings of 10th International Worskhop on JobScheduling Strategies for Parallel Processing, NewYork, NY, USA.

[28] Di Martino, V. and Mililotti, M. (2004) Sub optimalscheduling in a grid using genetic algorithms. ParallelComputing, 30, 553–565.

[29] He, L., Jarvis, S., Spooner, D., Chen, X., and Nudd, G.(2004) Dynamic scheduling of parallel jobs with QoSdemands in multiclusters and grids. Proceedings ofthe 5th International Workshop on Grid Computing,Pittsburgh, PA, USA.

[30] Singh, G., Kesselman, C., and Deelman, E. (2007)A provisioning model and its comparison with best-effort for performance-cost optimization in grids.Proceedings of the 16th International Symposium onHigh Performance Distributed Computing, Monterey,CA, USA.

[31] Abawajy, J. H. and Dandamudi, S. P. (2003)Parallel job scheduling on multicluster computingsystem. Proceedings of 2003 IEEE InternationalConference on Cluster Computing (Cluster 2003),Boston, Massachusetts, USA.

[32] Sabin, G., Kettimuthu, R., Rajan, A., and Sadayappan,P. (2003) Scheduling of parallel jobs in a heterogeneousmulti-site environment. Job Scheduling Strategies forParallel Processing, pp. 87–104. Springer.

[33] Tchernykh, A., Ramırez, J., Avetisyan, A., Kuzjurin,N., Grushin, D., and Zhuk, S. (2006) Two level job-scheduling strategies for a computational grid. ParallelProcessing and Applied Mathematics , ?, 774–781.

[34] Garg, S. K., Venugopal, S., and Buyya, R.(2008) A meta-scheduler with auction based resourceallocation for global grids. Proceedings of the 14thInternational Conference on Parallel and DistributedSystems (ICPADS), Melbourne, Australia.

[35] Zhang, W., Cheng, A., and Hu, M. (2006) Multisite co-allocation algorithms for computational grid. Proceed-ings of the 20th International Parallel and DistributedProcessing Symposium, Rhodes Island, Greece.


16 Garg et. al

[36] Huedo, E., Montero, R., Llorente, I., Thain, D.,Livny, M., van Nieuwpoort, R., Maassen, J., Kielmann,T., Bal, H., Kola, G., et al. (2005) The GridWayframework for adaptive scheduling and execution ongrids. Software-Practice and Experience, 6.

[37] Pacifici, G., Spreitzer, M., Tantawi, A., and Youssef,A. (2003) Performance management for web services.Research Report RC22676, IBM TJ Watson ResearchCenter, Yorktown Heights, NY,USA, 10598.

[38] Lifka, D. (1995) The ANL/IBM SP Scheduling System.Proceedings of the 1995 International Workshop onJob Scheduling Strategies for Parallel Processing, SantaBarbara, USA.

[39] Feitelson, D., Rudolph, L., Schwiegelshohn, U., Sevcik,K., and Wong, P. (1997) Theory and practice in paralleljob scheduling. Proceedings of 1997 InternationalWorkshop on Job Scheduling Strategies for ParallelProcessing, Geneva, Switzerland.

[40] Grosu, D. and Das, A. (2004) Auction-based resourceallocation protocols in grids. Proceedings of the 16thInternational Conference on Parallel and DistributedComputing and Systems, Cambridge, USA.

[41] Butler, J., Morrice, D., and Mullarkey, P. (2001) Amultiple attribute utility theory approach to rankingand selection. Management Science, 47, 800–816.

[42] Wallenius, J., Dyer, J., Fishburn, P., Steuer, R., Zionts,S., and Deb, K. (2008) Multiple Criteria DecisionMaking, Multiattribute Utility Theory: RecentAccomplishments and What Lies Ahead. ManagementScience, 54, 1336.

[43] Keeney, R. and Raiffa, H. (1993) Decisions withmultiple objectives: Preferences and value tradeoffs.Cambridge University Press.

[44] Wiseman, Y. and Feitelson, D. (2003) Paired GangScheduling. IEEE Transcatiosn on Parallel andDistributed Systems, 14, 581–592.

[45] Hoschek, W., Jaen-Martinez, J., Samar, A., Stockinger,H., and Stockinger, K. (2000) Data management inan International Data Grid Project. Proceedingsof 1st International Workshop on Grid Computing.Bangalore, India.

[46] de Assuncao, M. and Buyya, R. (2009) Performanceanalysis of allocation policies for interGrid resourceprovisioning. Information and Software Technology,51, 42–55.

[47] Waldspurger, C. A., Hogg, T., Huberman, B. A.,Kep hart, J. O., and Stornetta, W. S. (1992)Spawn: A Distributed Computational Economy. IEEETransactions on Software Engineering, 18, 103–117.

[48] Rudolph, L. and Smith, P. (2000) Valuation of Ultra-scale Computing Systems. Proceedings of 2000 JobScheduling Strategies for Parallel Processing, Cancun,Mexico.

[49] Harchol-Balter, M. and Downey, A. (1997) Exploit-ing process lifetime distributions for dynamic load bal-ancing. ACM Transactions on Computer Systems(TOCS), 15, 253–285.

[50] Li, H., Groep, D., and Wolters, L. (2004) Work-load characteristics of a multi-cluster supercomputer.Proceedings of the 10th International Workshop onJob Scheduling Strategies for Parallel Processing, NewYork, NY, USA.

[51] Kleinrock, L. and Gail, R. (1976) Queueing systems.Wiley New York.

[52] Mu’alem, A. and Feitelson, D. (2001) Utilization,predictability, workloads, and user runtime estimatesinscheduling the IBM SP2 with backfilling. IEEETransactions on Parallel and Distributed Systems, 12,529–543.

[53] Cobham, A. (1954) Priority assignment in waiting lineproblems. Journal of the Operations Research Societyof America, 2, 70–76.

[54] Harchol-Balter, M. (2002) Task assignment withunknown duration. Journal of the ACM (JACM), 49,260–288.

[55] Broberg, J., Tari, Z., and Zeephongsekul, P. (2006)Task assignment with work-conserving migration.Parallel Computing, 32, 808–830.

[56] Nelson, R. and Philips, T. (1989) An approximation tothe response time for shortest queue routing. ACMSIGMETRICS Performance Evaluation Review, 17,181–189.

[57] Nelson, R. and Philips, T. (1993) An approximation forthe mean response time for shortest queue routing withgeneral interarrival and service times. PerformanceEvaluation, 17, 123–139.

[58] Nozaki, S. and Ross, S. (1978) Approximations in finite-capacity multi-server queues with Poisson arrivals.Journal of Applied Probability, 15, 826–834.

[59] Harchol-Balter, M., Crovella, M., and C.D., M. (1999)On choosing a task assignment policy for a distributedserver system. Journal of Parallel and DistributedComputing, 59, 204–228.

APPENDIX A. FORMULATION OF QUEU-ING NETWORK MODEL

In this section we present the detailed derivation ofthe meta-scheduler’s queueing network model. Theanalytical model of the meta-scheduler is based onqueuing theory which has been used extensively formodelling scheduling problems in distributed systems.Queuing network model provides a powerful stochasticand probabilistic approach to analytically model themechanisms of scheduling policies [51]. Using thismodel, we obtain measures for metrics, such as meanwaiting time and mean slowdown of applications, whichis used to compare the performance of our proposedmeta-scheduling algorithm with the (near-)optimal.We can model the meta-scheduling system under

consideration as a network of three queues (Figure A.1)to get the optimal bound for various system parameters.In the meta-scheduler, each application is assigneda priority or valuation and matched to resourcesaccording to its priority. This component of the meta-scheduler can be modelled by a priority queuing system.Since processing time of each application may not beexponential, we have chosen M/G/1 priority queuesystem to analyze this part of the meta-scheduler.An application is then assigned and dispatched to aresource to balance load across all queues in the system.This component of the meta-scheduler is analyzed using



)(1

Tλ )(TN

λ. . .

Request Arrival-Poisson Process

1

n1

2R1

1

nz

2Rz

Rm

DispatcherPriority

Queue

Priority M/G/1

Queue

Text

M/G/m Queue

(least work

remaining policy)

p2

p1

TextM/G/1 Queue

Meta-Scheduler

Site 1

Site z

Site m

z

p2

p1

Z= /m.nz

FIGURE A.1. Queuing Network Model of Meta-scheduling

a comparable queueing system i.e. central queueingsystem with least work remaining policy (M/G/m),where m is the number of Grid sites.

At each resource, an application is executed onmore than one machine at the same time. The localscheduler at the resource end also uses the backfillingpolicies and different types of queues to increase theirutilization [52]. To analyze this part of meta-scheduling,we have taken an approximate model of real localscheduling systems. We have divided the processors in aresource into multiple partitions, where each partitionacts as a M/G/1 queue. This resource with multiplepartition is an approximation of real local schedulingsystems. The CPU/machine processing requirementsof each application also follows a general distribution(denoted as G). Let there be Nz servers/processors atresource site z (1 ≤ z ≤ m), which are divided betweennz queues (partitions). Each partition f on resourcesite z is initially assigned rzf processors.

Thus, in meta-scheduling, each application goesthrough three queuing systems before it starts

executing. Hence, by analyzing the combination of thefollowing sequential network of three queuing systems,we get an approximate analytical model for meta-scheduling system:

• Valuation or prioritization (M/G/1 Queue withPriority)

• Matching or Dispatching (M/G/m Queue policy)• Scheduling at a resource for execution (M/G/1

Queues)

The applications’ arrival at the meta-schedulerfollows a Poisson distribution with mean λ. Letthe service time is random variable X by a generaldistribution. The mean service time of the applicationsis E(X) and the second moment of service time is σ orE(X2). The distribution of processor requirement byeach application is given by C. Let the number of theresource sites be m.


18 Garg et. al

Appendix A.1. Valuation or prioritization(M/G/1 Queue with Priority)

In the first part of DAM, applications are assignedvaluations so that they can be re-sequenced forscheduling. Thus, this scenario can be modelled as asingle server queuing system as shown in Figure A.1.There are K priority classes of applications. The meanservice time of priority class j is 1

µjand the second

moment of service time is σ′

j . The overall secondmoment of service rate of applications by the server isσ

′

. Since the time taken to assign priorities is verysmall in DAM algorithm, the mean service rates ofapplications will also be very small in compare to theiractual processing time. Thus, for experiments, we havemodelled variance σ

′

j very low to keep service time forall the jobs in the first queuing system to be almostequal and very low.Let pj be the probability with which jth priority class

applications arrived at the server. Then, the meanarrival rate (λj) of the j

th priority applications is givenby:

λj = pj × λ (A.1)

Since the service time of all priority classes is 1µj, thus

the system load due to jth priority class applications isgiven by:

ρj =λj

µj(A.2)

Let E(wj) is expected waiting time for theapplications with priority level j. Then using theclassic result on non-preemptive priority queue by A.Cobham [53] we obtain the mean waiting time andslowdown (E(sj)) of class-j applications:

E(wj) =λ×σ

′

2

(1−∑j−1

i=1 ρi)(1 −∑j

i=1 ρi)(A.3)

Thus, the total mean waiting time and slowdown ofapplications in the system is given by:

W1 =

K∑j=1

[(pj × (E(wj) +1

µj)) (A.4)

S1 =

K∑j=1

(pj × (E(wj) +1

µj))× E(X−1

j )] (A.5)

Appendix A.2. Matching or Dispatching(M/G/m Queue)

After applications are served by the above queuingsystem based on priority, applications in the outgoingqueue will be served by the second queuing serveron a FCFS basis. It can be considered as thirdcomponent of the meta-scheduler i.e. matching. As

an approximation, the applications arriving into thesecond queueing system are assumed to follow a Poissonprocess. This is true if the service distribution isexponential, but is an approximation otherwise. Thisapproximation is commonly used in many advancedqueuing models due to the difficulty of modeling aGeneral (G) arrival/departure process, and solving anyresultant models [54][55]. In reality, the arrival intothe meta-scheduler would be less bursty (i.e. moreuniformly random) than those of a Poisson process, butthis has a minimal effect on the overall model. For theassignment of these applications to the resource sites,we have used the central queue with m servers policy.This policy has been proven to be equivalent to least-work-remaining allocation policy, which is claimed tobe optimal by Nelson et. al [56][57]. This policy is notanalytically tractable under M/G/m queueing system.Nonetheless, several good approximations exist in theliterature, many of which are empirical. In this study,we use the approximation given by Nozaki et. al [58]which is also adopted in several other works [59] [49].The approximation for mean queue length is stated as:

E(NM/G/m) = E(NM/M/m)E(X2)

E(X)2, (A.6)

where X: Service Requirement and N: QueueLength.The load of system is given by:

ρ = λ× E(X) (A.7)

Let E(WM/M/m) be the average waiting time in aM/M/m queueing system and ρ be the system load.Then, using well known Pollaczek-Khinchin formula inqueuing theory, the average queue length and waitingtime for M/M/m queueing system is given by:

E(NM/M/m) =PNρ

1− ρ, (A.8)

where PN = [

m−1∑z

(mρ)z

z!+

mmρm

m!1− ρ]−1

E(WM/M/m) =E(NM/M/m)

λ(A.9)

Thus, using Equation A.6, A.8 and A.9, the meanwaiting time and slowdown in the queue by using centralqueue policy is given by:

W2 = E(WM/G/m) = E(WM/M/m)σ

E(X)2(A.10)

S2 = W2E(X−1) (A.11)

Appendix A.3. Scheduling at a Resource forExecution (M/G/1 Queues)

After the application is assigned to the resource queue,the application is needed to be scheduled on multipleservers. Unlike the most of the commonly used queuing



systems where an application requires only one serverfor execution, here each application needs more thanone server (processors in our context) at the same time.Moreover, the local scheduler at a resource site usesdifferent backfilling policies to decrease slowdown ofapplications [52]. Since, it is analytically intractableto solve this system, the designed analytical systemfor this component of meta-scheduling differ slightlyfrom real systems. We divided the processors of theresource into multiple disjoint partitions, with onequeue per partition. For example, let the total numberof processors at a site are ‘6’ and we divided theminto 3 partition with each partition having 1, 2, and3 processors respectively. Thus, application from eachqueue will be executed on one of these partition. Ifapplication requires one processor, it will be executed onpartition 1. If application requires processors betweenrange (1,2) or (2,3), then application will be executedon partition 2 or partition 3.Thus, formally, each partition f on resource z is

initially assigned 1 ≤ rzf ≤ Nz processors. Each ofthese queues processes the applications, which requireprocessors within range of (rz(f−1), rzf ), on FCFS basis.Let there be Nz servers/processors at resource site z,which are divided between nz queues. Thus:

rz0 + rz1 + rz2 + rz3...rzf + ..rz(nz−1) = Nz (A.12)

Since the total number of resource sites is m, thearrival rate of applications at resource site z is givenby:

λz =λ

m(A.13)

Let uzf be the probability that an application requiresprocessor between rz(f−1) and rzf , and thus is processedby queue f of resource site z. Let C be theprobability distribution of processor requirements foran application, this probability is given by:

uzf =

∫ azf

az(f−1)

C(x)dx (A.14)

Thus, the fraction of applications arriving at queue fof the resource site z is given by:

λzf = λzuzf (A.15)

The load shared by each queue, when E(X) is meanservice time, is given by:

ρzf = λzfE(X) (A.16)

Since each queue partition has M/G/1 FCFS queuesystem behavior, thus we can directly use the sameresults for the average waiting time. This is given by:

E(wzf ) =λzfσ

2(1− ρzf )(A.17)

The total expected waiting time at a resource site isthe average of all waiting time at each queue partition,which is given by:

E(wz) =1

nz

nz∑f=1

E(wzf ) (A.18)

Let W3 and S3 be the expected waiting time andslowdown of all resource sites, respectively. Thus, theyare given by:

W3 =λ

m

m∑z=1

E(wz) (A.19)

S3 = W3 × E(X−1) (A.20)

The overall expected waiting time and slowdownmeasures are given by combining waiting time andslowdown of all three queueing system, i.e., EquationsA.4-A.5, A.10-A.11 and A.19-A.20:

Waiting Time = E(W ) = W1 +W2 +W3 (A.21)

Slowdown = E(S) = S1 + S2 + S3 (A.22)

(A.23)

Where Wi and Si, iε[1, 3] is waiting time and slowdownof ith queueing system.Thus, the queueing theory based analytical model

for our meta-scheduling mechanism is given by thefollowing equations:

Minimize(E(W )) subject to (A.24)nz∑k

rzk = Nz, 1 < z < m

Minimize(E(S)) subject to (A.25)nz∑k

rzk = Nz, 1 < z < m

The above model gives an approximation for realmeta-scheduling systems. Thus, to predict theperformance of our meta-scheduling policy, we canobtain optimal expected waiting time and slowdown.Thus, Equations A.24 and A.25 for expected waitingtime and slowdown are needed to be solved fordifferent values of nz using optimization tools suchas Mathematica [Wolfram Research 2008]. Theseanalytical values are used to ascertain the performanceof DAM.


DoubleAuction-Inspired Meta-SchedulingofParallel ... · Grids are composed of distributed...

Documents

Transcript of DoubleAuction-Inspired Meta-SchedulingofParallel ... · Grids are composed of distributed...