YARN的HA搭建

YARN的HA搭建

選擇192.168.230.10(node1),192.168.230.14(node5)兩臺主機做爲ResourceManager
datanode與nodemanager必定要放在一個機器上

YARN的體系架構

一、配置/opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml:

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

二、配置/opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml:

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

<!-- 啓用 -->
 <property>
   <name>yarn.resourcemanager.ha.enabled</name>
   <value>true</value>
 </property>
 <!-- 集羣ID號 -->
 <property>
   <name>yarn.resourcemanager.cluster-id</name>
   <value>cluster1</value>
 </property>
 <property>
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>rm1,rm2</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm1</name>
   <value>node1</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm2</name>
   <value>node5</value>
 </property>
 <property>
   <name>yarn.resourcemanager.zk-address</name>
   <value>node2:2181,node3:2181,node5:2181</value>
 </property>

三、將在node1主機上配置好的yarn-site.xml，mapred-site.xml文件發送到node二、node三、node5主機上

scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node2:/opt/modules/hadoop-2.5.1/etc/hadoop/
scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node2:/opt/modules/hadoop-2.5.1/etc/hadoop/

scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node3:/opt/modules/hadoop-2.5.1/etc/hadoop/
scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node3:/opt/modules/hadoop-2.5.1/etc/hadoop/

scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node5:/opt/modules/hadoop-2.5.1/etc/hadoop/
scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node5:/opt/modules/hadoop-2.5.1/etc/hadoop/

四、在node1主機上啓動yarn

[root@node1 hadoop-2.5.1]# ./sbin/start-yarn.sh

[root@node1 hadoop-2.5.1]# jps

五、啓動單節點ResourceManager

[root@node5 hadoop-2.5.1]# ./sbin/yarn-daemon.sh start resourcemanager

[root@node5 hadoop-2.5.1]# jps

六、在瀏覽器地址欄上訪問

http://node1:8088/cluster/nodes

http://node5:8088/cluster/nodes，頁面會直接跳轉到node1的頁面

6.一、殺死node1主機的resourcemanager進程，http://node1:8088/cluster訪問不成功

[root@node1 hadoop-2.5.1]# kill -9 8145

6.二、查看http://node5:8088/cluster發現node5節點的ResourceManager成爲Active

6.三、在node1主機從新單節點啓動ResourceManager，發現node1的ResourceManager變成了Standby

[root@node1 hadoop-2.5.1]# ./sbin/yarn-daemon.sh start resourcemanager