Hadoop Fully Distributed Cluster.docx

7/28/2019 Hadoop Fully Distributed Cluster.docx

1/5


2/5

Requirements

Java 1.6 Hadoop 0.20.x.

Installing Hive from a Stable Release

Start by downloading the most recent stable release of Hive from one of the Apache downloadmirrors :

http://hive.apache.org/releases.html

We can directly download from site or issue command through terminsl to download asfollows :

wgethttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gz

After downloading complete issue following command :

tar -xzf hive-0.8.1.tar.gz

open hive-0.8.1/conf/hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
http://hive.apache.org/releases.htmlhttp://hive.apache.org/releases.htmlhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://hive.apache.org/releases.html


3/5

export HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf

# Hive Configuration Directory can be controlled by:

export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf

# Folder containing extra ibraries required for hive compilation/execution can be controlled by:export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-0.20.205/lib:/home/shashwat/Hadoop/hbase-0.90.4/lib:/home/shashwat/Hadoop/hive-

0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar


4/5

Then open hive-site.xml, in this file you will find a property called hive.aux.jars.path asfollows :

hive.aux.jars.path

file:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar

Version compatibility with hadoop and hive :

Hive

Version

Hadoop

VersionConfiguration Parameters

0.4 0.18 --hadoop-version 0.18

0.5 0.20--hadoop-version 0.20 --hive-versions 0.5 --

ami-version 1.0

0.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7


5/5

Hive

Version

Hadoop

VersionConfiguration Parameters

--ami-version 1.0

0.7 0.20--hadoop-version 0.20 --hive-versions 0.7 --

ami-version 1.0

0.7.1 0.20

--hadoop-version 0.20 --hive-versions 0.7.1 -

-ami-version 1.0

0.7.1 0.20.205--hadoop-version 0.20 --hive-versions 0.7.1 -

-ami-version 2.0

0.7.1.1 0.20.205--hadoop-version 0.20.205 --hive-versions

0.7.1.1 --ami-version 2.0







After configuration complete issue these commmands from terminal :

bin/hive - -service hiveserver : it will start the thrift server :

then open new terminal and issue commmand

bin/hive : this will give you hive terminal where you can execute commands like

create table

show tables etc.

More configuration options

Hive Configuration Variables used to interact with HadoopVariable Name Description Default Value

hadoop.bin.path

The location of hadoop script

which is used to submit jobs tohadoop when submitting through aseparate jvm.

$HADOOP_HOME/bin/hadoop

hadoop.config.dirThe location of the configurationdirectory of the hadoop installation

$HADOOP_HOME/conf

Hadoop Fully Distributed Cluster.docx

Documents

Transcript of Hadoop Fully Distributed Cluster.docx