7/25/2019 Hadoop Summit Fair Scheduler (1)

1/30

UC Berkeley

Job Scheduling with theFairand Capacity Schedulers

Matei Zaharia

Wednesday, June 10, 2009

Santa Clara Marriott

7/25/2019 Hadoop Summit Fair Scheduler (1)

2/30

Motiation

!roide "ast response ti#es to s#all $obs in

a shared %adoop cluster

proe utili'ation and data locality oerseparate clusters and %adoop on (e#and

7/25/2019 Hadoop Summit Fair Scheduler (1)

3/30

%adoop at Faceboo)

*00+node cluster running %ie

200 $obs-day .0/ users

pps statistical reports, spa# detection,ad opti#i'ation,

7/25/2019 Hadoop Summit Fair Scheduler (1)

4/30

Faceboo) Job 3ypes

!roduction $obs data i#port, hourly

reports, etc

S#all ad+hoc $obs %ie 4ueries, sa#pling

5ong e6peri#ental $obs #achinelearning, etc

785 "ast response ti#es "or s#all $obs,

guaranteed serice leels "or production$obs

7/25/2019 Hadoop Summit Fair Scheduler (1)

5/30

8utline

Fair scheduler basics

Conguring the "air scheduler

Capacity scheduler

:se"ul lin)s

7/25/2019 Hadoop Summit Fair Scheduler (1)

6/30

F&F8 Scheduling

Job Queue

7/25/2019 Hadoop Summit Fair Scheduler (1)

7/30

F&F8 Scheduling

Job Queue

7/25/2019 Hadoop Summit Fair Scheduler (1)

8/30

F&F8 Scheduling

Job Queue

7/25/2019 Hadoop Summit Fair Scheduler (1)

9/30

Fair Scheduling

Job Queue

7/25/2019 Hadoop Summit Fair Scheduler (1)

10/30

Fair Scheduling

Job Queue

7/25/2019 Hadoop Summit Fair Scheduler (1)

11/30

Fair Scheduler ;asics

7roup $obs into

7/25/2019 Hadoop Summit Fair Scheduler (1)

12/30

!ools

(eter#ined "ro# a congurable $ob

property (e"ault in 0=20 user=na#e >one pool per

user?

!ools hae properties

Mini#u# #ap slots

Mini#u# reduce slots

5i#it on @ o" running $obs

7/25/2019 Hadoop Summit Fair Scheduler (1)

13/30

A6a#ple !ool llocations

entirecluster100slots

#atei $eB ads

min share =40

to#min share =

30

$ob 215

slots

$ob 15

slots

$ob 130

slots

$ob 40

slots

7/25/2019 Hadoop Summit Fair Scheduler (1)

14/30

Scheduling lgorith#

Split each poolDs #in share a#ong its

$obs Split each poolDs total share a#ong its$obs

When a slot needs to be assigned

&" there is any $ob below its #in share,schedule it

Alse schedule the $ob that weDe been#ost un"air to >based on

7/25/2019 Hadoop Summit Fair Scheduler (1)

15/30

Scheduler (ashboard

7/25/2019 Hadoop Summit Fair Scheduler (1)

16/30

Scheduler (ashboard

Changepriority

ChangepoolF&F8 #ode >"or

testing?

7/25/2019 Hadoop Summit Fair Scheduler (1)

17/30

dditional Features

Weights "or une4ual sharing

Job weights based on priority >each leel 26?

Job weights based on si'e

!ool weights

5i#its "or @ o" running $obs

!er user

!er pool

7/25/2019 Hadoop Summit Fair Scheduler (1)

18/30

&nstalling the Fair Scheduler

;uild it

ant pac)age

!lace it on the classpath

cp build-contrib-"airscheduler-G=$ar lib

7/25/2019 Hadoop Summit Fair Scheduler (1)

19/30

Conguration Files

%adoop cong >con"-#apred+site=6#l?

Contains scheduler options, pointer to poolsle

!ools le >pools=6#l?

Contains #in share allocations and li#its onpools

Heloaded eery 1. seconds at runti#e

7/25/2019 Hadoop Summit Fair Scheduler (1)

20/30

Mini#al hadoop+site=6#l

Iproperty

Ina#e#apred=$obtrac)er=tas)SchedulerI-na#eIalueorg=apache=hadoop=#apred=FairSchedulerI-alue

I-property

Iproperty

Ina#e#apred="airscheduler=allocation=leI-na#e

Ialue-path-to-pools=6#lI-alue

I-property

7/25/2019 Hadoop Summit Fair Scheduler (1)

21/30

Mini#al pools=6#l

IK6#l ersionL1=0LK

IallocationsI-allocations

7/25/2019 Hadoop Summit Fair Scheduler (1)

22/30

Conguring a !ool

IK6#l ersionL1=0LK

Iallocations Ipool na#eLadsL

I#inMaps10I-#inMaps

I#inHeduces.I-#inHeduces

I-pool

I-allocations

7/25/2019 Hadoop Summit Fair Scheduler (1)

23/30

Setting Hunning Job 5i#its

IK6#l ersionL1=0LK

Iallocations

Ipool na#eLadsL

I#inMaps10I-#inMaps

I#inHeduces.I-#inHeduces

I#a6HunningJobsI-#a6HunningJobs

I-pool

Iuser na#eL#ateiL

I#a6HunningJobs1I-#a6HunningJobs

I-user

I-allocations

7/25/2019 Hadoop Summit Fair Scheduler (1)

24/30

(e"ault !er+:ser Hunning Job 5i#it

IK6#l ersionL1=0LK

Iallocations

Ipool na#eLadsL

I#inMaps10I-#inMaps

I#inHeduces.I-#inHeduces

I#a6HunningJobsI-#a6HunningJobs

I-pool

Iuser na#eL#ateiL

I#a6HunningJobs1I-#a6HunningJobs

I-user

IuserMa6Jobs(e"ault10I-userMa6Jobs(e"aultI-allocations

7/25/2019 Hadoop Summit Fair Scheduler (1)

25/30

8ther !ara#eters

#apred="airscheduler=assign#ultiple

ssign a #ap and a reduce on eachheartbeat i#proes ra#p+up speed andthroughput reco##endation set to true

7/25/2019 Hadoop Summit Fair Scheduler (1)

26/30

8ther !ara#eters

#apred="airscheduler=poolna#eproperty

Which JobCon" property sets what pool a $obis in

- (e"ault user=na#e >one pool per user?

- Can #a)e up your own, e=g=

7/25/2019 Hadoop Summit Fair Scheduler (1)

27/30

:se"ul Setting

Iproperty

Ina#e#apred="airscheduler=poolna#epropertyI-na#e

Ialuepool=na#eI-alue

I-property

Iproperty

Ina#epool=na#eI-na#e

IalueNOuser=na#ePI-alue

I-property

Ma)e pool=na#e

de"ault touser=na#e

7/25/2019 Hadoop Summit Fair Scheduler (1)

28/30

Future !lans

!ree#ption >)illing tas)s? i" a $ob is stared

o" its #in or "air share "or so#e ti#e>%(88!+**.?

7lobal scheduling opti#i'ation >%(88!+

**Q?

F&F8 pools >%(88!+R0, %(88!+.1R*?

7/25/2019 Hadoop Summit Fair Scheduler (1)

29/30

Capacity Scheduler

8rgani'es $obs into 4ueues

ueue shares as TDs o" cluster

F&F8 scheduling within each 4ueue

Supports pree#ption

http--hadoop=apache=org-core-docs-current-

capacityUscheduler=ht#l

7/25/2019 Hadoop Summit Fair Scheduler (1)

30/30

3han)sV

Fair scheduler included in %adoop 0=19/ and

in ClouderaDs (istribution "or %adoop

Fair scheduler "or %adoop 0=1Q and 0=1Rhttp--issues=apache=org-$ira-browse-%(88!+Q*

Capacity scheduler included in %adoop 0=19/

(ocs

http--hadoop=apache=org-core-docs-current

My e#ail#ateicloudera=co#