本篇只簡單介紹安裝步驟
1. 角色分配
10.11.84.4 web-crawler--1.novalocal master/slave10.11.84.5 web-crawler--2.novalocal slave10.11.84.6 web-crawler--3.novalocal slave
2.安裝目錄 /usr/local/hadoop （HADOOP_HOME）
3,創建group hadoop
groupadd hadoop
4.創建用戶hadoop
useradd -d /home/hadoop -g hadoop hadoopchown -R hadoop:hadoop /usr/local/hadoop
5. 每臺機器安裝jdk（安裝在相同的目錄）
1)查看java 版本，可以看到jdk為openjdk，最好更換為sun jdk
[hadoop@hadoop2 ~]$ java -versionopenjdk version "1.8.0_131"OpenJDK Runtime Environment (build 1.8.0_131-b12)OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)[hadoop@hadoop2 ~]$ rpm -qa | grep javajava-1.8.0-openjdk-headless-1.8.0.131-11.b12.el7.x86_64javapackages-tools-3.4.1-11.el7.noarchtzdata-java-2017b-1.el7.noarchjava-1.7.0-openjdk-headless-1.7.0.141-2.6.10.5.el7.x86_64java-1.7.0-openjdk-1.7.0.141-2.6.10.5.el7.x86_64java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64python-javapackages-3.4.1-11.el7.noarch
2）卸載openjdk
rpm -qa | grep java | xargs yum -y remove
3）安裝sunjdk
/usr/local 目錄下解壓jdk-8u181-linux-i586.tar.gz 并命名為jdk
配置環境變量如下
vim /etc/profile 末尾添加如下內容
export JAVA_HOME=/usr/local/jdkexport JRE_HOME=${JAVA_HOME}/jreexport PATH=${JAVA_HOME}/bin:${PATH}export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:${CLASSPATH} 4）查看jdk是否安裝成功
[root@hadoop2 local]# java -versionjava version "1.8.0_151"Java(TM) SE Runtime Environment (build 1.8.0_151-b12)Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
6. master 到 slave ssh 免密碼登錄
在每臺機器hadoop用戶下執行如下命令
[hadoop@hadoop1 ~]$ ssh-keygen -t rsa -P '' #一路直接回車發送master 機器~/.ssh/id_rsa.pub 到所有slave節點Scp id_rsa.pub hadoop@10.11.84.5:~Scp id_rsa.pub hadoop@10.11.84.6:~讓后在每臺slave節點hadoop 用戶下執行如下命令Cd ~/.sshCat ../id_rsa.pub >> authorized_keysChmod 644 authorized_keys
7.關閉所有機器防火墻、selinux
1）centos 7 以下
防火墻
查看防火墻狀態
service iptables status
臨時關閉防火墻
service iptables stop
永久關閉防火墻
chkconfig iptables off
selinux
setenforce 0 #臨時關閉
永久關閉需要修改如下文件
vim /etc/selinux/config
將SELINUX=enforcing改為SELINUX=disabled
2)centos 7
firewall-cmd --reload #重啟firewall
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall開機啟動
firewall-cmd --state #查看默認防火墻狀態（關閉后顯示notrunning，開啟后顯示running）
selinux 同上
8.修改hadoop配置文件
Core-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?><!-- Put site-specific property overrides in this file. --><configuration><property><name>hadoop.tmp.dir</name><value>/home/hadoop/tmp</value><description>Abase for other temporary directories.</description></property><property><name>fs.default.name</name><value>hdfs://web-crawler--1.novalocal:9000</value></property></configuration>hadoop-env.sh 添加java_home export JAVA_HOME=/usr/local/jdkhdfs-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?><!-- Put site-specific property overrides in this file. --><configuration><property><name>dfs.replication</name><value>2</value></property><property><name>dfs.name.dir</name><value>/home/hadoop/name</value></property><property><name>dfs.data.dir</name><value>/home/hadoop/data</value></property></configuration>mapred-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?><!-- Put site-specific property overrides in this file. --><configuration> <property> <name>mapred.job.tracker</name> <value>http://web-crawler--1.novalocal:9001</value>;;; </property><property> <name>mapred.system.dir</name> <value>/home/hadoop/mapred/system</value> </property><property> <name>mapred.local.dir</name> <value>/home/hadoop/mapred/local</value> </property></configuration>
5)masters
web-crawler--1.novalocal
6)slaves
web-crawler--1.novalocalweb-crawler--2.novalocal web-crawler--3.novalocal
9. 修改環境變量，添加HADOOP_HOME 和JAVA_HOME
Vim /etc/profile (root)export ANT_HOME=/usr/local/antexport PATH=${PATH}:${ANT_HOME}/binexport HADOOP_HOME=/usr/local/hadoop-1.2.1export JAVA_HOME=/usr/local/jdkexport PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATHexport CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATHexport PATH=/usr/local/lib/phantomjs/bin:$PATHexport ANT_HOME=/usr/local/antexport PATH=${PATH}:${ANT_HOME}/binexport PATH=$PATH:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin10.啟動hadoop
1）在master上format namenode
hadoop namenode -format
2)啟動hadoop
sh /user/local/hadoop/bin/start-all.sh
11.進行簡單測試
Hadoop fs -mkdir -p /user/hadoop/private/inHadoop fs -put /user/local/hadoop/bin/start-all.sh /user/hadoop/private/inCd /usr/local/hadoopHadoop jar hadoop-examples-1.2.1.jar wordcount /user/hadoop/private/in /user/hadoop/private/out
總結
以上所述是小編給大家介紹的centos6.5 安裝hadoop1.2.1的教程詳解，希望對大家有所幫助，如果大家有任何疑問請給我留言，小編會及時回復大家的。在此也非常感謝大家對VEVB武林網網站的支持！
新聞熱點
疑難解答
圖片精選
