Adaptive MapReduce using Situation-Aware Mappers

Adaptive MapReduce using Situation-AwareMappers

Rares Vernica1 (HP Labs),Andrey Balmin, Kevin S. Beyer, Vuk Ercegovac (IBM Research)

1Work done at IBM Research.

15th International Conference on Extending Database Technology,March 26-30 2012

Rares Vernica (HP Labs) Adaptive MapReduce EDBT 2012 1 / 25

Outline

1 Motivation

2 Problem Statement

3 Situation-Aware MappersAdaptive MappersAdaptive CombinersAdaptive Sampling and Partitioning

4 Summary

MapReduce Review

map (k,v) → list(k,v);reduce (k,list(v)) → list(k,v).

DFSINPUT 1/3

INPUT 3/3

INPUT 2/3

Input:(k,v)

MAPREDUCE

Output:list(k,v)

REDUCE

SHUFFLE

Input:(k, list(v))

DFSOUTPUT 1/2

OUTPUT 2/2

Output:list(k,v)

combine (k,list(v)) → list(k,v).

MapReduce Review

map (k,v) → list(k,v);reduce (k,list(v)) → list(k,v).

DFSINPUT 1/3

INPUT 3/3

INPUT 2/3

Input:(k,v)

MAPREDUCE

Output:list(k,v)

REDUCE

SHUFFLE

Input:(k, list(v))

DFSOUTPUT 1/2

OUTPUT 2/2

Output:list(k,v)

combine (k,list(v)) → list(k,v).

Motivation: MapReduce Issues

MapReduceParallel data-processing frameworkOpen-source implementation (Hadoop)Simple programming environment

MapReduce: “simplicity over performance”Limited choice of execution strategies:

Mappers checkpoint after every splitMap outputs are sorted and written to fileReducer read statically predetermined partitions

Solutions to MapReduce Issues

MapReduce-inspired alternativesDryad (Microsoft)Spark (UC Berkeley)Hyracks (UC Irvine)Nephele (TU Berlin)

Have more choices in runtime execution

Our Solution: Adaptive MapReduce

Make MapReduce (Hadoop) more flexibleLeverage existing investment in:

Framework (Hadoop)Query processing systems (Jaql, Pig, Hive)

Techniques for:Dynamic checkpoint intervals (Map)Best-effort hash-based aggregation (Combine)Dynamic, sample-based, partitioning (Reduce)

Performance tuning:Cardinality and cost estimation (due to UDFs)Adaptive to runtime environment

Problem Statement: Adaptive MapReduce

GoalsImprove MapReduce (Hadoop) performance by:

New runtime optionsAdaptive to runtime environment

Preserve Hadoop’sFault-toleranceScalabilityProgramability

Outline

1 Motivation

2 Problem Statement

3 Situation-Aware MappersAdaptive MappersAdaptive CombinersAdaptive Sampling and Partitioning

4 Summary

Situation-Aware Mappers

Main ideaMake MapReduce more dynamic

Mappers:

Aware of the global state of the jobCommunicate through a distributed meta-data storeBreak assumption: isolation

Main ideaMake MapReduce more dynamicMappers:

Aware of the global state of the job

Communicate through a distributed meta-data storeBreak assumption: isolation

Aware of the global state of the jobCommunicate through a distributed meta-data store

Break assumption: isolation

Adaptive MapReduce

MAPMAPMAP

REDUCEREDUCE

MAPMAPMAP

REDUCEREDUCE

DMDSDFS

MAPMAPMAP

REDUCEREDUCE

Adaptive TechniquesAM: Adaptive MappersAC: Adaptive CombinersAS: Adaptive SamplingAP: Adaptive Partitioning

Adaptive MapReduce

MAPMAPMAP

REDUCEREDUCE

MAPMAPMAP

REDUCEREDUCE

MAPMAPMAP

REDUCEREDUCE

Distributed Meta-Data StoreDistributed read/writeTransactionale.g., ZooKeeper

Adaptive MapReduce

MAPMAPMAP

REDUCEREDUCE

MAPMAPMAP

REDUCEREDUCE

DMDSDFS

MAPMAPMAP

REDUCEREDUCE

Adaptive Mappers Motivation

Input data is divided into splitsOne-to-one correspondence of mappers and splitsAM decouple # splits from # mappers

: Startup cost, e.g., scheduling, loading ref. data

, : Split processing cost

Small splits Large startup cost Balanced workload

Large splits Small startup cost Inbalanced workload

Adaptive Mappers Small startup cost Balanced workload

Adaptive Mappers Motivation

Input data is divided into splitsOne-to-one correspondence of mappers and splitsAM decouple # splits from # mappers

Adaptive Mappers Small startup cost Balanced workload

Adaptive Mappers Algorithm

JobID locations Host1 [Split1, Split2, ... ] Host2 ...

MapReduce Client

Root1ZooKeeper

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

4 assigned Split1{Map2}

Split1

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split15

OK/Fail

Store meta-data inZooKeeperImplemented as a newInputFormat

MapReduce Client

Root1ZooKeeper

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split1

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split15

OK/Fail

MapReduce Client

Root1ZooKeeper

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split1

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split15

OK/Fail

MapReduce Client

Root1ZooKeeper

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split1

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split15

OK/Fail

MapReduce Client

Root1ZooKeeper

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split1

MapReduce Client

Root1ZooKeeper

Map1Init

Map2Init

Split15

OK/Fail

Additional FeaturesProcess local splits first, then remote splitsFault tolerance

Restated task unlocks splitsSplit reprocessing is shared

Scheduler aware (FIFO, FAIR, and FLEX)

Experimental Setting

Hardware40-node IBM Systemx iDataPlex dx340Two quad-core Intel Xeon E5540 64-bit 2.83GHz32GB RAMFour SATA disks160 map and 160 reduce slots

SoftwareUbuntu Linux, kernel 2.6.32-24 64-bit server editionJava 1.6 64-bit server editionHadoop 0.20.2ZooKeeper 3.3.1

Start-up Cost vs. ZooKeeper Overhead

20 200 2000

Number of Splits

020406080

100120140

280300

Regular MappersAdaptive Mappers 2000 1-byte records

Sleep 1s/record5 nodes, 20 map slots20-2000 Reg. Mappers20 Adaptive Mappers

Small ZooKeeperoverheadLarge Map startupcost ∼2s/map

Adaptive Mappers Workloads

1 Set-Similarity Join [Vernica et al., 2010]Publication datasetsDBLP: 1.2M records, 310MBCITESEERX: 1.3M records, 1,750MBIncreased to ×10 and ×100

2 JOINSingle dataset (“fact” table), Sort Benchmark data generatorFan-out coefficient (“dimension” table)average join fan-out 1 : 30TERASORT: 1B records, 93GB

Adaptive Mappers Experiments - Set-Similarity Join

2048102451225612864 32 AM

Split Size (MB)

Regular MappersAdaptive Mappers

Stage 3:One-Phase Record JoinBroadcast join equivalentDBLP and CITESEERX ×10Single wave of AM

×3 speedup over defaultHadoop split size (64MB)Optimal with no tuning

Adaptive Mappers Experiments - JOIN

102451225612864 32 16 8 AM

Split Size (MB)

Regular MappersAdaptive Mappers

Map-only job1B TERASORT recordsModels a skewed joinSingle wave of AM

Regular Mappers:Large split: data skewSmall split: schedulingand start-up overhead

Optimal with no tuning

Adaptive MapReduce

DFS DFS

REDUCE

AS REDUCE

Adaptive Combiners

Main ideaReplace sort with hashingReduce serialization, sort, and IO

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSort MergeMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Hash-group and Combine

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSort

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSort

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

CombineSortMap

Regular Combiners

Sort Buffer

: User code: Data

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Regular Combiners

Sort Buffer

: User code: Data

CombineSort Merge

Adaptive Combiners

Adaptive Combiners Details

“Best-effort” aggregationNever spill to diskHash-table replacement policies:

No-Replacement (NR)Least-Recently-Used (LRU)

Implemented as:Library for HadoopOptimization choice for Jaql

Adaptive Combiners Experiments

GROUP-BYSynthetic dataset with 3 dimensions (A1, A2, and A3) and 1 factGroup records and apply aggregation functionTWL: 10B records, 120GB

AM AC AM, AC

Regular CombinersAdaptive Combiners NRAdaptive Combiners LRU

GROUP-BY on A1×2.5 speedup

AM 1 25 100

Cache Size (K)

Regular CombinersAdaptive Combiners NRAdaptive Combiners LRUMiss Ratio NRMiss Ratio LRU

GROUP-BY on A1 and A2×3 speedup

Adaptive MapReduce

DFS DFS

REDUCE

AS REDUCE

Adaptive Sampling and Partitioning

Step 1 Compute and publishlocal histogram

Step 2 Collect localhistograms andcompute partitioningfunction

Step 3 Broadcast partitioningfunction

MAPREDUCE

Summary

Adaptive runtime techniques for MapReduceSituation-Aware MappersMake MapReduce more dynamic

Up to ×3 speedup for well-tuned jobsOrders of magnitude speedup for badly tuned jobsNever hurt performanceConfigure themselvesPart of IBM InfoSphere BigInsights

Vernica, R., Carey, M., and Li, C. (2010).Efficient parallel set-similarity joins using MapReduce.In SIGMOD Conference.

Adaptive MapReduce using Situation-Aware Mappers

Documents

Transcript of Adaptive MapReduce using Situation-Aware Mappers

Pipelined-MapReduce an Improved MapReduce

LEEN: Locality/Fairness - Aware Key Partitioning for ...salsahpc.indiana.edu/CloudCom2010/slides/PDF/LEEN...LEEN: Locality/Fairness - Aware Key Partitioning for MapReduce in the Cloud

Introduction to MapReduce | MapReduce Architecture | MapReduce Fundamentals

Energy Aware Resource Management for MapReduce Jobs · Energy Aware Resource Management for MapReduce Jobs by Adam Gregory A thesis submitted to the Faculty of Graduate and Postdoctoral

Resource-aware Adaptive Scheduling for MapReduce Clusters

Enterprise PHP: mappers, models and services

MappERS – C the app for citizens users’ guidelines · Funded by the EU ISIG .eu MAPPERS – C THE APP FOR CITIZENS USERS’ GUIDELINES MAppERS - Mobile Applications for Emergency

Dynamic mappers of NGS reads - Institut Gaspard Mongeigm.univ-mlv.fr/AlgoB/slides/Brinda_SeqBio_2014.pdf · · 2014-11-18Dynamic mappers of NGS reads Karel Břinda ... (* character),

Lecture 3 – Hadoop Technical IntroductionData Distribution Implicit in design of MapReduce! All mappers are equivalent; so map whatever data is local to a particular node in HDFS

MapReduce-MPI Library Users Manualmapreduce.sandia.gov/doc/Manual.pdf · MapReduce-MPI WWW Site - MapReduce-MPI Documentation What is a MapReduce? The canonical example of a MapReduce

Computations Incremental Incoop: MapReduce for › presentation › 5252 › ... · Incremental map/reduce and contraction phase Memoization-aware scheduler. Memoization Scheduling

02 12 07 Global Warming Mind Mappers eBook

On Trafﬁc-Aware Partition and Aggregation in MapReduce for …cssongguo/papers/mapreduce15.pdf · On Trafﬁc-Aware Partition and Aggregation in MapReduce for Big Data Applications

Availability and Network-Aware MapReduce Task Scheduling over … · 2017. 1. 27. · Availability and Network-aware MapReduce Task Scheduling over the Internet Bing Tang1, Qi Xie2,

MapReduce. MapReduce Outline MapReduce Architecture MapReduce Internals MapReduce Examples JobTracker Interface.

Processing with What is MapReduce? Hadoop/MapReduce ...

Cloud Computing using MapReduce, Hadoop, Spark Computing using MapReduce, Hadoop, Spark ... – Used by Yahoo!, Facebook, Amazon, ... • Mappers save outputs to local disk before

Indel Mappers

Location-aware MapReduce in Virtual Cloud 2011 IEEE computer society International Conference on Parallel Processing Yifeng Geng1,2, Shimin Chen3, YongWei.

Financial Mappers Overview · 2020-04-09 · Financial Mappers and Financial Mappers Pro Overview Glenis Phillips Designer of Financial Mappers Director Plencore Wealth Ltd glenis.phillips@financialmappers.com.au