NN1 | NN2 | DN | ZK | ZKFC | JNN | RS | NM | |
---|---|---|---|---|---|---|---|---|
NODE01 | ? | ? | ? | |||||
NODE02 | ? | ? | ? | ? | ? | ? | ||
NODE03 | ? | ? | ? | ? | ? | |||
NODE04 | ? | ? | ? | ? |
1. 配置mapred-site.xml
[root@node01 hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@node01 hadoop]# vi mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
2. 配置yarn-site.xml
[root@node01 hadoop]# vi yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>cluster1</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>node03</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>node04</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>node02:2181,node03:2181,node04:2181</value>
</property>
3. 將上述兩個文件分發(fā)到其他Hadoop節(jié)點
[root@node01 hadoop]# scp mapred-site.xml yarn-site.xml node02:`pwd`
4. 讓兩個ResourceManager之間互相免密鑰
[root@node03 .ssh]# ssh-keygen -t dsa -P '' -f ./id_dsa
[root@node03 .ssh]# cat id_dsa.pub >> authorized_keys
[root@node03 .ssh]# scp id_dsa.pub node04:`pwd`/node03.pub
[root@node04 .ssh]# cat node03.pub >> authorized_keys
另一個節(jié)點執(zhí)行相同步驟
5. 啟動
啟動順序:
- zookeeper,zkServer.sh start
- hdfs (注意廊散,有一個腳本不要用桑滩,start-all)start-dfs.sh
如果nn 和 nn2沒有啟動,需要在node01允睹,node02分別手動啟動:
hadoop-daemon.sh start namenode - start-yarn.sh (啟動nodemanager)
- 在03,04節(jié)點分別執(zhí)行腳本: yarn-daemon.sh start resourcemanager
6. 啟動后各節(jié)點角色進(jìn)程
[root@node01 ~]# jps
Jps
NameNode
DFSZKFailoverController
JournalNode
[root@node02 ~]# jps
DFSZKFailoverController
NodeManager
NameNode
DataNode
JournalNode
Jps
QuorumPeerMain
[root@node03 ~]# jps
DataNode
JournalNode
QuorumPeerMain
Jps
ResourceManager
NodeManager
[root@node04 ~]# jps
NodeManager
DataNode
Jps
QuorumPeerMain
ResourceManager
7. http訪問端口:8088
8. 停止
node01: stop-dfs.sh
node01: stop-yarn.sh (停止nodemanager)
node03,node04: yarn-daemon.sh stop resourcemanager (停止resourcemanager)
ZooKeeper:zkServer.sh stop