EMR AWS Demo
29
ELASTIC MAP REDUCE OF AWS Rim Moussa ENI-Carthage University of Carthage 2017/2018 1
-
Upload
rim-moussa -
Category
Education
-
view
47 -
download
1
Transcript of EMR AWS Demo
DEMO OUTLINE
Simple Storage Service: S3
Job
Jar: Map Reduce code
Input: input data files
Output: output data files
All data must be on S3 including jar and input data
Create Hadoop Cluster
Size: number of workers
Hardware configuration
Stat the job
2
FIND OUT
29
S3
How to upload data from a terminal to S3
Scenario where data is some where in the net
Hadoop Master
Compile the job on the master
Submit the job from a terminal on the master
Performance Tuning
Hadoop cluster configuration
RAM allocated to each Mapper, Reducer
Data Compression
Code
Input Split Size
Adjust the number of reducers