Installing HortonWork's Hadoop on AWS EC2
-
Upload
rohit-ghatol -
Category
Documents
-
view
202 -
download
1
description
Transcript of Installing HortonWork's Hadoop on AWS EC2
![Page 1: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/1.jpg)
Installing Hortonwork’s Hadoop on AWS
Rohit GhatolDirector of Engineering @ Synerzip
http://www.linkedin.com/in/rohitghatol @rohitghatol http://rohitghatol.com
BY
![Page 2: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/2.jpg)
Software Stack
![Page 3: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/3.jpg)
EC2
Hadoop
Hadoop
Hadoop
Hadoop
Hadoop
Hadoop
Controls &
Monitors
Hadoop Services• HDFS• YARN• Hbase• Hive• Pig• Sqoop• Oozie• Zookeeper• Nagios• Ganglia
Ambari
Apache Ambari
![Page 4: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/4.jpg)
Step 1 – Create Base AMI
Community Red hat AMI Base AMI
6
![Page 5: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/5.jpg)
SSHDownload PEM Key
Step 2 – Password Less SSH AccessLaptop accessing EC2 Instance EC2 Instance accessing other
EC2 Instances
AccessUsingSSH
Upload PEM Key
EC2(Ambari Server)
EC2
EC2
EC2
EC2
AMI
![Page 6: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/6.jpg)
EC2
EC2
EC2
EC2
AMI
EC2
EC2
EC2
Step 3 – Launch 6 Instances using Base Image
Launch
6
Note the Private DNS names of all these machines• ip-10-23-12-33• ip-10-23-11-23• ip-10-23-32-54• ip-10-23-44-14• ip-10-23-65-73• ip-10-23-37-47
![Page 7: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/7.jpg)
Install Ambari Server
EC2
EC2EC2
Step 4 – Install Ambari Server on EC2
EC2
Supply the earlier recorded private DNS to Ambari• ip-10-23-12-33• ip-10-23-11-23• ip-10-23-32-54• ip-10-23-44-14• ip-10-23-65-73• ip-10-23-37-47
Supply list of private DNS
EC2 EC2 EC2
EC2 EC2 EC2
Ambari Server Installs Ambari Agent
![Page 8: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/8.jpg)
EC2
EC2
EC2
EC2
EC2
EC2
EC2
Installsservices
Hadoop Services• HDFS• YARN• Hbase• Hive• Pig• Sqoop• Oozie• Zookeeper• Nagios• Ganglia
Step 5 – Install X service on Y machine
EC2 withAmbari
![Page 9: Installing HortonWork's Hadoop on AWS EC2](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485c1adb4af9f640d8b4ee1/html5/thumbnails/9.jpg)
Step 6 – Launch Ambari Web Consolehttp://<<ambari-server>>:8080