MapReduce-Simulator: Matchmaking and Scheduling Algorithm Norman Lim Dept. of Systems and Computer...

MapReduce-Simulator: Matchmaking and

Scheduling Algorithm

Norman LimDept. of Systems and Computer Engineering

Carleton UniversityOttawa, ON, [email protected]

SYSC 5807: Resource Management on Distributed Systems

Overview of Matchmaking and Scheduling (Mapping) Algorithm

1. When a job j arrives, it is placed in a queue depending on the scheduling policy selected.o FIFO: uses first-in first-out queueo EDF: uses earliest deadline first queue

2. Job/Task Mapping algorithm is invoked. o Retrieves the first job in the queue and maps each of the job’s

tasks onto the resources.o Maps each task t onto a resource r that can execute t at its

earliest possible time (at or after its earliest start time).o For a particular job, the tasks in that job with higher execution time

are mapped first.

2

Overview of Matchmaking and Scheduling (Mapping) Algorithm Cont.

EDF Policy Only 3. If a job j is not able to be scheduled to complete

executing before its deadline, then the Job/Task Re-mapping algorithm is invoked.o Remaps j and the set of jobs (called S) that have caused j to miss

its deadline. • S includes jobs that are scheduled to start at, or complete within

the interval: [start time of job j, deadline of job j].o j and the jobs in S are re-ordered according to earliest deadline

first.o The Job/Task Mapping algorithm (see previous slide) is invoked to

map j and the jobs in S.

3

Example: Jobs and Resources

4

Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s

t1

2t2

5t3

8

t4

10t5

11


t1

4t2

5

t3

10

Resource 1 (r1)Map Capacity = 1Reduce Capacity = 1

Resource 2 (r2)Map Capacity = 1Reduce Capacity = 1

EST = earliest start time

Example: FIFO

5

FIFO Queue

r1

r2

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

j1, t3

j1, t2 j1, t1

j1, t5

j1, t4

j2, t2

j2, t1

j2, t3

j2 will miss its deadline.

reduce

map

reduce

map



Example: EDF (Case 1)

6

EDF Queue


r1

r2

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

j1, t3

j1, t2 j1, t1

j1, t5

j1, t4


j2, t2

j2, t1

j2, t3

reduce

map

reduce

map

Example: EDF (Case 2)

7

EDF Queue

r1

r2

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

j1, t3

j1, t2 j1, t1

j1, t5

j1, t4

j2, t2

j2, t1

j2, t3reduce

map

reduce

map



j2 will miss its deadline.



j1, t3

j1, t2 j1, t1

j1, t5

j1, t4

j2, t2

j2, t1

j2, t3

MapReduce-Simulator: Matchmaking and Scheduling Algorithm Norman Lim Dept. of Systems and Computer...

Documents

Transcript of MapReduce-Simulator: Matchmaking and Scheduling Algorithm Norman Lim Dept. of Systems and Computer...