Hadoop Installation
Hadoop Installation Step By Step
1. Open Terminal (Cntrl + Shift + T)
2. Install the software using terminal
a. Update the drivers in Linux by redirecting to Super User
sudo su Enter the password (hadoop)
b. Update the repositories sudo apt-get update
c. Install eclipse sudo apt-get install eclipse
d. Install MySql Server and MySql Client sudo apt-get install mysql-server mysql-client
e. Install Java sudo apt-get install openjdk-7-jdk
f. Install SSH(Security Shell) sudo apt-get install ssh
3. Create a folder a e Work u der Ho e folder.
Note: It is ot a dator that folder a e should e Work , it a ea a e.
4. Cop the hadoop-1.1.2.tar.gz file a d paste it i to Work folder a d e tra t it.
Hadoop Installation
5. Below are the configuration related to parti ular l’s file to ru o Psuedo ode.
core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/Work/hadoopdata/tmp</value>
</property>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/Work/hadoopdata/dfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/Work/hadoopdata/dfs/data</value>
</property>
</configuration>
Hadoop Installation
mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8021</value>
</property>
<property>
<name>mapred.local.dir</name>
<value>/home/hadoop/Work/hadoopdata/mapred/local</value>
</property>
<property>
<name>mapred.system.dir</name>
<value>/mapred/system</value>
</property>
</configuration>
6. Place the environment variables in the bashrc file
sudo gedit ~/.bashrc
go to e d of the file, after fi li k e ter a d e ter the elow state e ts to set the
environment variables
# exporting the environment varaibles
export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk
export HADOOP_HOME=/home/hadoop/Work/hadoop-1.1.2
export HIVE_HOME=/home/hadoop/Work/hive-0.10.0
export DERBY_HOME=/home/hadoop/Work/db-derby-10.10.1.1-bin
export PIG_HOME=/home/hadoop/Work/pig-0.11.1
export HBASE_HOME=/home/hadoop/Work/hbase-0.94.8
export ZOOKEEPER_HOME=/home/hadoop/Work/zookeeper-3.4.5
export SQOOP_HOME=/home/hadoop/Work/sqoop-1.4.3.bin__hadoop-1.0.0
export FLUME_HOME=/home/hadoop/Work/apache-flume-1.4.0-bin
export OOZIE_HOME=/home/hadoop/Work/oozie-3.3.2
Hadoop Installation
export
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$DERBY_HOME/bin:$P
IG_HOME/bin:$HBASE_HOME/bin:$OOZIE_HOME/bin:$ZOOKEEPER_HOME/bin:$SQOOP
_HOME/bin:$FLUME_HOME/bin:$PATH
Note: In the above mentioned path “/home/hadoop” here “hadoop” is my
username, type “whoami” command in terminal and copy your username and
replace with your username in the bashrc file.
7. Once the above step completes, close the bashrc file and close the existing terminal to reflect
the changes of bashrc file or you can refresh the bashrc file using refresh command to reflect
the saved changes from existing terminal.
Refresh Bashrc File commands:
source ~/.bashrc
or
. ~/.bashrc
Type the below commands to verify the variables which we have set earlier in bashrc file
echo $JAVA_HOME
echo $HADOOP_HOME
8. Disable the SSH by executing the below 3 commands one by one
a. ssh localhost
b. ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
c. cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
9. Format the Name Node for initial start up
hadoop namenode –format
10. Open Terminal and start hadoop
start-all.sh
11. Verify all the hadoop 5 process are started
jps
