What Is Salesforce? | Salesforce Training - What Does Salesforce Do? | Salesforce Tutorial | Edureka
Edureka VM_ Updated
-
Upload
sreenivas-thota -
Category
Documents
-
view
168 -
download
17
description
Transcript of Edureka VM_ Updated
Big Data and Hadoop
Version 2.0
www.edureka.co/big-data-and-hadoop
Importing Edureka VM A guide to setup Edureka VM
© Brain4ce Education Solutions Pvt. Ltd.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 1
www.edureka.co/big-data-and-hadoop
Edureka VM
A guide to setup Edureka VM
Table of Contents
Install Virtual Box .................................................................................................................................... 2
Install Edureka VM ................................................................................................................................ 11
Commonly Faced Issues: ....................................................................................................................... 26
Size Compatibility Issue: ....................................................................................................................... 31
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 2
www.edureka.co/big-data-and-hadoop
Install Virtual Box
Prerequisites:
• Minimum 4 GB RAM
• Dual Core Processor or above.
• Needed 20 GB* free Hard Disk Space to run this VM Smoothly.
* It may also run with below 20 GB but in future you may face “size compatibility" issue.
If your system does not meet the above pre-requisites, we would suggest you to use our
Remote Server.
To access our Remote Server, please refer to the document "Remote Login Using Putty -
Hadoop 2.2.0” present in LMS in the Module "Edureka VM Installation" as in the below
screenshot.
You may also refer to "Remote Login Using Putty - Hadoop 2.2.0” present in the Module
"Edureka VM Installation” of your LMS to access our remote server as in below screenshot.
FIGURE 1-0
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 3
www.edureka.co/big-data-and-hadoop
Step 1: Download Virtual Box from below link based on your Operating System.
http://www.oracle.com/technetwork/server-storage/virtualbox/downloads/index.html Here, we have shown installation for VirtualBox-4.3.20, same steps you can follow for the updated versions. FIGURE 1-1
For Windows
For Ubuntu
For Mac OS
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 4
www.edureka.co/big-data-and-hadoop
Step 2: Run the setup.
FIGURE 1-2
Step 3: Click “Next”.
FIGURE 1-3
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 5
www.edureka.co/big-data-and-hadoop
Step 4: Select the way you want your features to be installed and click “Next”. You can also
change the location as per your will.
FIGURE 1-4
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 6
www.edureka.co/big-data-and-hadoop
Step 5: Choose all the options and click “Next”.
FIGURE 1-5
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 7
www.edureka.co/big-data-and-hadoop
Step 6: Click “Yes” to install VM Virtual Box 4.3.20
FIGURE 1-6
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 8
www.edureka.co/big-data-and-hadoop
Step 7: Click “Install” to begin the installation.
FIGURE 1-7
FIGURE 1-7.1
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 9
www.edureka.co/big-data-and-hadoop
Step 8: Click “Install” on security popup.
FIGURE 1-8
FIGURE 1-8.1
With this screen, your Oracle VM Virtual Box Manager has been downloaded and
installed successfully.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 10
www.edureka.co/big-data-and-hadoop
Note: If you unable to install Virtual Box on Windows, install VMware Player
which will serve the same purpose.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 11
www.edureka.co/big-data-and-hadoop
Install Edureka VM
Step 1: Download Edureka VM from- http://share.edureka.co/pydio/data/public/hadoop
Note: The file size of Edureka VM is 4.5 GB.
1. If you are not able to download the complete file because of internet speed, please refer the below
link for the Split files of Edureka VM.
https://edureka.wistia.com/medias/f5k5ibsucm/download?media_file_id=48883291
2. We suggest you to use the Download Manager while downloading Edureka VM to avoid any
network issues that may occur. You can download it from
http://www.speedbit.com/dap/download/ for different platforms which is an open source tool.
3. By default the Virtual Box is installed on the C Drive, in case the C Drive has insufficient
space and you have free space (20 GB) in any other drive, then to refer the further steps
Click Here
Step 2: On Import Virtual Appliance box click on the file menu to import Open Virtualization
format file (.ova) downloaded. Go to “File” menu of Virtual Box Manager and click on “Import Appliance”. FIGURE 2-1
Note: If you are not getting File option, please make sure the virtual box is in full screen mode. FIGURE 2-2
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 12
www.edureka.co/big-data-and-hadoop
Step 3: Select “Edureka_VM” and click on “Open”.
FIGURE 2-3
Select the location where you
have Edureka_VM.ova file
downloaded
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 13
www.edureka.co/big-data-and-hadoop
Step 4: After selecting the .ova file click on “Next”.
FIGURE 2-4
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 14
www.edureka.co/big-data-and-hadoop
Step 5: Click “Import” on Appliance settings box.
FIGURE 2-5
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 15
www.edureka.co/big-data-and-hadoop
Note: After importing the .ova file in your virtual box, check the settings of virtual box.
1) Refer the screen shot below:
At bottom, if you are getting invalid setting detected, make changes in the base memory.
The cursor range should be within the limit of green line.
Note: Assign around 25-35% RAM to your virtual box of total RAM, not more than that.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 16
www.edureka.co/big-data-and-hadoop
2) Check the network settings:
Check adapter 1:
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 17
www.edureka.co/big-data-and-hadoop
Check adapter 2:
Click OK and try to start the VM.
Note: If you face the below error:
Make change in both adapter as NAT.
Here, we have imported the Edureka VM successfully
and changed the needed settings!!!
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 18
www.edureka.co/big-data-and-hadoop
Step 6: Once it got imported, you find the below image. Select “Edureka_VM” and Click”
Start”.
FIGURE 2-6
Step 7: If you get error like below, Click on “Change Network Settings”
FIGURE 2-7
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 19
www.edureka.co/big-data-and-hadoop
Step 8: Don’t do any changes, just click “OK”
FIGURE 2-8
Step 9: Edureka VM will start on Oracle VM Virtual Box. You will have to write edureka on
password field.
FIGURE 2-9
Note: Oozie is a dummy user. There is no configuration done in that user. Password for
Oozie User is oozie
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 20
www.edureka.co/big-data-and-hadoop
Step 10: The VM will open. On Desktop you will find LMS directory and readme file, please
go them. LMS directory has all the practical files and codes, readme file gives the information
about the VM.
FIGURE 2-10
Step 11: Open terminal and Check your hostname in terminal, and it should be in host file.
If it is not there, follow the below steps:
First Check the hostname: In my case --> localhost.locadomain
Open the host name file: (Enter password, if asked)
Note: If your host name is already in host file, close the file otherwise please add hostname
at the last as mentioned in IMAGE below:
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 21
www.edureka.co/big-data-and-hadoop
(In my case, hostname is already there)
Note: Before you start working with Edureka VM, check if all daemons are running or not,
by using below command:
sudo jps
Output must contain:
If any of the above is missing, try following commands:
sudo service hadoop-master stop
sudo service hadoop-master start
hadoop dfsadmin -safemode leave
sudo jps
Note: Please type the command in terminal, don't copy it. It may take hidden symbols.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 22
www.edureka.co/big-data-and-hadoop
Note: If you have installed VMWare Player on your machine, please find the below steps to
import the Edureka VM.
Step 12: To import the Edureka VM, start the VMPlayer and click on Open a Virtual
Machine as shown in the below image
FIGURE 2-12
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 23
www.edureka.co/big-data-and-hadoop
Step 13: Select the location where you have ova file of Edureka VM and click on open
FIGURE 2-13
Step 14: Select the location where you have ova file of Edureka VM and click on open
FIGURE 2-14
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 24
www.edureka.co/big-data-and-hadoop
Step 15: You will find the below screen
FIGURE 2-15
Step 16: If you are receiving the below message please click on retry
FIGURE 2-16
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 25
www.edureka.co/big-data-and-hadoop
FIGURE 2-17
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 26
www.edureka.co/big-data-and-hadoop
Commonly Faced Issues:
1. If you get Intel VT-x or AMD-v issue, follow the steps in the document present in below link. https://edureka.wistia.com/medias/0hliot0nh5/download?media_file_id=46964037
FIGURE 1
2. https://edureka.wistia.com/medias/0hliot0nh5 3. If you get Intel VT-x or AMD-v issue , follow the steps in the document present in below
link. https://edureka.wistia.com/medias/0hliot0nh5
FIGURE 3
4. When you are trying to access HDFS, you get “NameNode is in SafeMode” , just like below
snapshot.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 27
www.edureka.co/big-data-and-hadoop
2. When you are trying to access HDFS, you may get “Name node is in SafeMode”, just like below
snapshot.
FIGURE 2
Solution: Go to terminal and give the command “ hadoop dfsadmin -safemode leave “ . Now
go and check your HDFS.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 28
www.edureka.co/big-data-and-hadoop
3. Command: oozie job -oozie http://localhost:11000/oozie -config
/home/edureka/Desktop/LMS/Oozie/WordCountTest/job.properties -run
Error: E0501 : E0501: Could not perform authorization operation, User: edureka is not
allowed to impersonate edureka
Solution: Firstly, stop oozie if it’s running.
Command: cd /usr/lib/oozie-4.0.0/
Command: ./bin/oozie-stop.sh
Three changes needs to be done.
Change 1
Edit hadoop’s core-site.xml
Command: sudo gedit /usr/lib/hadoop-2.2.0/etc/hadoop/core-site.xml
Remove oozie and put edureka as mentioned in below document, save the file and close it.
Restart the cluster.
Command: sudo service hadoop-master stop
Command: sudo service hadoop-master start
Command: hadoop dfsadmin -safemode leave
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 29
www.edureka.co/big-data-and-hadoop
Change 2
Edit your job.properties and workflow.xml files. Use jobTracker port as 8032 in both the files and
oozie.wf.application.path as ${nameNode}/WordCountTest as mentioned in below snapshots.
Command: sudo gedit Desktop/LMS/Oozie/WordCountTest/job.properties
Command: sudo gedit Desktop/LMS/Oozie/WordCountTest/workflow.xml
Now you need to transfer the WordCountTest directory on hdfs ( / ).
Command: hadoop dfs -put Desktop/LMS/Oozie/WordCountTest /
Change 3
Giving permissions to Oozie directory.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 30
www.edureka.co/big-data-and-hadoop
Command: sudo chmod -R 777 /usr/lib/oozie-4.0.0
Command: sudo chown -R edureka /usr/lib/oozie-4.0.0
Now change the directory to Oozie and start it.
Command: cd /usr/lib/oozie-4.0.0/
Command: ./bin/oozie-start.sh
Run the oozie command.
Command: oozie job -oozie http://localhost:11000/oozie -config
/home/edureka/Desktop/LMS/Oozie/WordCountTest/job.properties -run
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 31
www.edureka.co/big-data-and-hadoop
Size Compatibility Issue: To run the Edureka image, it needs 20 GB free space.
If you are not having enough space in C drive (where you have installed virtual box), then
while importing the Edureka_VM image, please follow the following procedure.
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 32
www.edureka.co/big-data-and-hadoop
Since, you are not having enough space in C Drive, then you need to create a new folder in
another Drive.
Here, I have created Edureka folder in D drive and paste the path as mentioned, don’t remove
the last file name.
D:\Edureka\EdurekaVM_32-disk1.vmdk
Big Data and Hadoop
© B r a i n 4 c e E d u c a t i o n S o l u t i o n s P v t . L t d
Page 33
www.edureka.co/big-data-and-hadoop
Click Here to continue with next step