Post on 03-Apr-2018
7/28/2019 Hadoop Fully Distributed Cluster.docx
1/5
7/28/2019 Hadoop Fully Distributed Cluster.docx
2/5
Requirements
Java 1.6 Hadoop 0.20.x.
Installing Hive from a Stable Release
Start by downloading the most recent stable release of Hive from one of the Apache downloadmirrors :
http://hive.apache.org/releases.html
We can directly download from site or issue command through terminsl to download asfollows :
wgethttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gz
After downloading complete issue following command :
tar -xzf hive-0.8.1.tar.gz
open hive-0.8.1/conf/hive-env.sh
# Set HADOOP_HOME to point to a specific hadoop install directory
http://hive.apache.org/releases.htmlhttp://hive.apache.org/releases.htmlhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://hive.apache.org/releases.html7/28/2019 Hadoop Fully Distributed Cluster.docx
3/5
export HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf
# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf
# Folder containing extra ibraries required for hive compilation/execution can be controlled by:export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-0.20.205/lib:/home/shashwat/Hadoop/hbase-0.90.4/lib:/home/shashwat/Hadoop/hive-
0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar
7/28/2019 Hadoop Fully Distributed Cluster.docx
4/5
Then open hive-site.xml, in this file you will find a property called hive.aux.jars.path asfollows :
hive.aux.jars.path
file:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar
Version compatibility with hadoop and hive :
Hive
Version
Hadoop
VersionConfiguration Parameters
0.4 0.18 --hadoop-version 0.18
0.5 0.20--hadoop-version 0.20 --hive-versions 0.5 --
ami-version 1.0
0.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7
7/28/2019 Hadoop Fully Distributed Cluster.docx
5/5
Hive
Version
Hadoop
VersionConfiguration Parameters
--ami-version 1.0
0.7 0.20--hadoop-version 0.20 --hive-versions 0.7 --
ami-version 1.0
0.7.1 0.20
--hadoop-version 0.20 --hive-versions 0.7.1 -
-ami-version 1.0
0.7.1 0.20.205--hadoop-version 0.20 --hive-versions 0.7.1 -
-ami-version 2.0
0.7.1.1 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.1 --ami-version 2.0
0.7.1.2 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.2 --ami-version 2.0
0.7.1.3 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.3 --ami-version 2.0
0.7.1.4 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.4 --ami-version 2.0
After configuration complete issue these commmands from terminal :
bin/hive - -service hiveserver : it will start the thrift server :
then open new terminal and issue commmand
bin/hive : this will give you hive terminal where you can execute commands like
create table
show tables etc.
More configuration options
Hive Configuration Variables used to interact with HadoopVariable Name Description Default Value
hadoop.bin.path
The location of hadoop script
which is used to submit jobs tohadoop when submitting through aseparate jvm.
$HADOOP_HOME/bin/hadoop
hadoop.config.dirThe location of the configurationdirectory of the hadoop installation
$HADOOP_HOME/conf