Hadoop Installation Guide
Transcript of Hadoop Installation Guide
Hadoop installation guide
This current tutorial is all about installation of Hadoop and setting up a single node cluster
on Linux Ubuntu environment.it is a made easy short tutorial when compared with the existing hefty
tutorials on the web...anyway here we go...
What we want to do
1. Install sun java jdk
2. Install hadoop-0.20
3. Install hue (a file browser and job designer on hdfs)
Prerequisites
The following process was tested on Ubuntu 10.04LTS and also 11.04...but regarding stability i would
personally prefer 10.04LTS..
Download 10.04 or 11.04 from this link:http://www.ubuntu.com/download/ubuntu/download
Once done with the installation of Ubuntu, sun java must be installed. Let’s get into it...
Open terminal (applications>terminal)...the following steps(highlighted command) must be copied one
by one paste it to terminal interface hit enter after every step..
Step 1: we create a canonical partner repository...to do so paste the below code to terminal.
sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
Step 2: we update the source list
sudo apt-get update
Step 3: now we Install sun-java6-jdk
sudo apt-get install sun-java6-jdk
Step 4: we make a quick check whether Sun’s JDK is correctly set up
java -version
Now we head to the installation of Hadoop...
Step 1: Ubuntu 10.04 is a lucid system...hit the below link and save the file in the downloads folder,
http://archive.cloudera.com/one-click-install/lucid/cdh3-repository_1.0_all.deb
Step 2: now we install the downloaded package with the following command
sudo dpkg -i Downloads/cdh3-repository_1.0_all.deb
Step 3: now we update the installed package
sudo apt-get update
Step 4: now we are to the final Hadoop installation
sudo apt-get install hadoop-0.20-conf-pseudo
We are done with the installation of Hadoop...Now we start the daemons and verify whether all the
components are working fine. With the below command
for service in /etc/init.d/hadoop-0.20-*; do sudo $service start; done
now we say Hadoop accomplished, if ur well used to the commands of Hadoop then u can start
working with Hadoop, if u want to say adios to the command UI or if ur a newbie to Hadoop i would
prefer using hue-a file browser and job designer on hdfs...
Now we head to the installation of hue(Hadoop user environment)
Step 1: To install cloudera Hue
sudo apt-get install hue
Once done with the installation of hue. Follow the next command
Step 2:open a new terminal and type the command
sudo gedit
Step 3: now head to the file location as places>computer>filesystem>>etc>hive>conf>hive-site.xml file
(to edit drag the file to gedit interface) and then save it after editing the below code...
Change the location of the tables by changing the value of the javax.jdo.option.ConnectionURL
property:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=/usr/share/hue/metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
Step 4: now once again open the gedit editor to provide a secret key
sudo gedit
step 5:Open the /etc/hue/hue.ini configuration file(head to the location and drag the file to gedit editor
interface).In the [desktop] section, enter a long series of random characters (30 to 60 characters is
recommended) and save the file after editing with the below code..
[desktop]
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
Step 6: Start Hue by running
sudo /etc/init.d/hue start
Step 7: To use Hue, open a web browser and go to: http://localhost:8088/
If error occurs stop hue followed by stopping hadoop nodes as:
sudo /etc/init.d/hue stop
Now stop the Hadoop daemons as (error occurs but will rectify the misconfiguration in a while):
for service in /etc/init.d/hadoop-0.20-*; do sudo $service stop; done
Now restart Hadoop daemons as:
for service in /etc/init.d/hadoop-0.20-*; do sudo $service start; done
Restart hue as:
sudo /etc/init.d/hue start
Now the following plugins must be installed to avoid misconfiguration in hue browser
Step 8: installing flume
sudo apt-get install flume
Step 9: installing hbase
sudo apt-get install hadoop-hbase
step 10:installing Hadoop-pig
sudo apt-get install hadoop-pig
Now we cross check whether the entire configuration is working well or not…open hue in a browser
using http://localhost:8088/
To check whether all the daemons are working fine. Hit the file browser and job browser from the dock
provided. If it works well then we are done...
----THE END---- -Sriram