大數據技術之Hadoop3.1.2版本HA模式
大數據技術之Hadoop3.1.2版本HA模式html
做者：尹正傑java
版權聲明：原創做品，謝絕轉載！不然將追究法律責任。node
一.Hadoop的HA特色git
1>.主備NameNodegithub
2>.解決單點故障（屬性，位置）web
1.主NameNode對外提供服務，備NameNode同步主NameNode元數據一帶切換； 2.全部DataNode同時向兩個NameNode彙報數據塊信息（位置）； 3.JNN：集羣（屬性）； 4.Standby：備，完成了edits.log文件的合併產生新的image，推送回ANN；
3>.兩種切換選擇ajax
1.手動切換 經過命令實現主備之間的切換，能夠用HDFS升級等場合。 2.自動切換 基於Zookeeper實現。
4>.基於zookeeper自動切換方案sql
1.Zookeeper Failover Controller進程用於監控NameNode監控狀態； 2.並向zookeeper註冊NameNode； 3.NameNode掛掉後，ZKFC爲NameNode競爭鎖，得到ZKFC鎖的NameNode變爲active；
5>.配置Hadoop3.1.2的官方文檔（http://hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html）shell
二.zookeeper徹底分佈式集羣搭建apache
1>.下載zookeeper軟件（下載地址：https://www.apache.org/dyn/closer.cgi/zookeeper/）
[root@node101.yinzhengjie.org.cn ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz --2019-04-12 08:48:40-- https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz Resolving mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.8.193, 2402:f000:1:408:8100::1 Connecting to mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.8.193|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 37676320 (36M) [application/x-gzip] Saving to: ‘zookeeper-3.4.14.tar.gz’ 100%[============================================================================================================================================>] 37,676,320 7.57MB/s in 4.5s 2019-04-12 08:48:45 (7.94 MB/s) - ‘zookeeper-3.4.14.tar.gz’ saved [37676320/37676320] [root@node101.yinzhengjie.org.cn ~]#
2>.加壓zookeeper並配置環境變量
[root@node101.yinzhengjie.org.cn ~]# tar -zxf zookeeper-3.4.14.tar.gz -C /yinzhengjie/softwares/ [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/ total 1672 drwxr-xr-x 2 2002 2002 202 Mar 6 09:09 bin -rw-rw-r-- 1 2002 2002 97426 Mar 6 08:50 build.xml drwxr-xr-x 2 2002 2002 77 Mar 6 09:09 conf drwxr-xr-x 2 2002 2002 4096 Mar 6 09:10 dist-maven -rw-rw-r-- 1 2002 2002 1709 Mar 6 08:50 ivysettings.xml -rw-rw-r-- 1 2002 2002 10742 Mar 6 08:50 ivy.xml drwxr-xr-x 4 2002 2002 308 Mar 6 09:09 lib -rw-rw-r-- 1 2002 2002 11970 Mar 6 08:50 LICENSE.txt -rw-rw-r-- 1 2002 2002 3132 Mar 6 08:50 NOTICE.txt -rw-rw-r-- 1 2002 2002 31622 Mar 6 08:50 pom.xml -rw-rw-r-- 1 2002 2002 1765 Mar 6 08:50 README.md -rw-rw-r-- 1 2002 2002 1770 Mar 6 08:50 README_packaging.txt drwxr-xr-x 3 2002 2002 22 Mar 6 08:50 src -rw-rw-r-- 1 2002 2002 1515359 Mar 6 08:50 zookeeper-3.4.14.jar -rw-rw-r-- 1 2002 2002 836 Mar 6 09:10 zookeeper-3.4.14.jar.asc -rw-rw-r-- 1 2002 2002 33 Mar 6 08:50 zookeeper-3.4.14.jar.md5 -rw-rw-r-- 1 2002 2002 41 Mar 6 08:50 zookeeper-3.4.14.jar.sha1 drwxr-xr-x 3 2002 2002 47 Mar 6 09:09 zookeeper-client drwxr-xr-x 12 2002 2002 4096 Mar 6 09:09 zookeeper-contrib drwxr-xr-x 7 2002 2002 4096 Mar 6 09:09 zookeeper-docs drwxr-xr-x 3 2002 2002 35 Mar 6 09:09 zookeeper-it drwxr-xr-x 4 2002 2002 46 Mar 6 09:09 zookeeper-jute drwxr-xr-x 5 2002 2002 176 Mar 6 09:09 zookeeper-recipes drwxr-xr-x 3 2002 2002 32 Mar 6 09:09 zookeeper-server [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tail -3 /etc/profile #ADD ZOOKEEPER_HOME PATH ZOOKEEPER_HOME=/yinzhengjie/softwares/zookeeper-3.4.14 PATH=$PATH:$ZOOKEEPER_HOME/bin [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# source /etc/profile [root@node101.yinzhengjie.org.cn ~]#
3>.配置zookeeper的配置文件
[root@node101.yinzhengjie.org.cn ~]# cp /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo_sample.cfg /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/conf/ total 16 -rw-rw-r-- 1 2002 2002 535 Mar 6 08:50 configuration.xsl -rw-rw-r-- 1 2002 2002 2161 Mar 6 08:50 log4j.properties -rw-r--r-- 1 root root 922 Apr 12 08:55 zoo.cfg -rw-rw-r-- 1 2002 2002 922 Mar 6 08:50 zoo_sample.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg # 滴答，計時的基本單位，默認是2000毫秒，即2秒。它是zookeeper最小的時間單位，用於丈量心跳時間和超時時間等，一般設置成默認2秒便可。 tickTime=2000 # 初始化限制是10滴答，默認是10個滴答，即默認是20秒。指定follower節點初始化是連接leader節點的最大tick次數。 initLimit=5 # 數據同步的時間限制，默認是5個滴答，即默認時間是10秒。設定了follower節點與leader節點進行同步的最大時間。與initLimit相似，它也是以tickTime爲單位進行指定的。 syncLimit=2 # 指定zookeeper的工做目錄，這是一個很是重要的參數，zookeeper會在內存中在內存只能中保存系統快照，並按期寫入該路徑指定的文件夾中。生產環境中須要注意該文件夾的磁盤佔用狀況。 dataDir=/home/yinzhengjie/zookeeper # 監聽zookeeper的默認端口。zookeeper監聽客戶端連接的端口，通常設置成默認2181便可。 clientPort=2181 # 這個操做將限制鏈接到 ZooKeeper 的客戶端的數量，限制併發鏈接的數量，它經過 IP 來區分不一樣的客戶端。此配置選項能夠用來阻止某些類別的 Dos 攻擊。將它設置爲 0 或者忽略而不進行設置將會取消對併發鏈接的限制。 #maxClientCnxns=60 # 在上文中已經提到，3.4.0及以後版本，ZK提供了自動清理事務日誌和快照文件的功能，這個參數指定了清理頻率，單位是小時，須要配置一個1或更大的整數，默認是0，表示不開啓自動清理功能。 #autopurge.purgeInterval=1 # 這個參數和上面的參數搭配使用，這個參數指定了須要保留的文件數目。默認是保留3個。 #autopurge.snapRetainCount=3 #server.x=[hostname]:nnnnn[:nnnnn]，這裏的x是一個數字，與myid文件中的id是一致的。右邊能夠配置兩個端口，第一個端口用於F和L之間的數據同步和其它通訊，第二個端口用於Leader選舉過程當中投票通訊。 server.101=node101.yinzhengjie.org.cn:2888:3888 server.102=node102.yinzhengjie.org.cn:2888:3888 server.103=node103.yinzhengjie.org.cn:2888:3888 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/java.env #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #指定JDK的安裝路徑 export JAVA_HOME=/yinzhengjie/softwares/jdk #指定zookeeper的heap內存大小 export JVMFLAGS="-Xms256m -Xmx256m $JVMFLAGS" [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
4>.編寫zookeeper的啓動腳本
[root@node101.yinzhengjie.org.cn ~]# vi /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cat /usr/local/bin/xzk.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判斷用戶是否傳參 if [ $# -ne 1 ];then echo "無效參數，用法爲: $0 {start|stop|restart|status}" exit fi #獲取用戶輸入的命令 cmd=$1 #定義函數功能 function zookeeperManger(){ case $cmd in start) echo "啓動服務" remoteExecution start ;; stop) echo "中止服務" remoteExecution stop ;; restart) echo "重啓服務" remoteExecution restart ;; status) echo "查看狀態" remoteExecution status ;; *) echo "無效參數，用法爲: $0 {start|stop|restart|status}" ;; esac } #定義執行的命令 function remoteExecution(){ for (( i=101 ; i<=103 ; i++ )) ; do tput setaf 2 echo ========== node${i}.yinzhengjie.org.cn zkServer.sh $1 ================ tput setaf 9 ssh node${i}.yinzhengjie.org.cn "source /etc/profile ; zkServer.sh $1" done } #調用函數 zookeeperManger [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh -rw-r--r-- 1 root root 1125 Apr 12 09:03 /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# chmod +x /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh -rwxr-xr-x 1 root root 1125 Apr 12 09:03 /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
5>.建立myid文件並寫入服務器編號
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'mkdir /home/yinzhengjie/zookeeper/' [WARNING]: Consider using file module with state=directory rather than running mkdir node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# for (( i=101;i<=103;i++ )) do ssh node${i}.yinzhengjie.org.cn "echo -n $i > /home/yinzhengjie/zookeeper/myid" ;done [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'cat /home/yinzhengjie/zookeeper/myid' node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 103 node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 102 node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 101 [root@node101.yinzhengjie.org.cn ~]#
6>.分發配置文件
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node102.yinzhengjie.org.cn:/etc/ profile 100% 2127 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node103.yinzhengjie.org.cn:/etc/ profile 100% 2127 1.4MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/ ........ log.42 100% 184 128.3KB/s 00:00 log.63b 100% 48KB 22.7MB/s 00:00 snapshot.0 100% 296 277.6KB/s 00:00 snapshot.272 100% 55KB 26.5MB/s 00:00 snapshot.273 100% 55KB 24.3MB/s 00:00 snapshot.639 100% 140KB 27.2MB/s 00:00 snapshot.83f 100% 4824 3.5MB/s 00:00 minikdc-krb5.conf 100% 1103 1.3MB/s 00:00 minikdc.ldiff 100% 1630 2.2MB/s 00:00 log.100000001 100% 2401KB 50.2MB/s 00:00 log.100001bf0 100% 1001KB 59.6MB/s 00:00 snapshot.100000000 100% 73 17.1KB/s 00:00 snapshot.100001bec 100% 2507KB 61.6MB/s 00:00 check_compatibility.py 100% 7212 4.7MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 6.5MB/s 00:00 checkstyle.xml 100% 7489 7.3MB/s 00:00 findbugsExcludeFile.xml 100% 5027 6.6MB/s 00:00 log4j.properties 100% 2712 1.9MB/s 00:00 test-patch.properties 100% 855 930.7KB/s 00:00 test-github-pr.sh 100% 24KB 9.7MB/s 00:00 test-patch.sh 100% 24KB 19.4MB/s 00:00 pom.xml 100% 10KB 7.6MB/s 00:00 LICENSE.txt 100% 12KB 5.9MB/s 00:00 NOTICE.txt 100% 3132 1.2MB/s 00:00 README.md 100% 1765 1.4MB/s 00:00 README_packaging.txt 100% 1770 2.5MB/s 00:00 build.xml 100% 95KB 39.5MB/s 00:00 ivy.xml 100% 10KB 8.2MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 8.9MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 41.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 25.6KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 60.7KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/ ...... snapshot.273 100% 55KB 14.8MB/s 00:00 snapshot.639 100% 140KB 21.7MB/s 00:00 snapshot.83f 100% 4824 4.4MB/s 00:00 minikdc-krb5.conf 100% 1103 1.0MB/s 00:00 minikdc.ldiff 100% 1630 1.4MB/s 00:00 log.100000001 100% 2401KB 53.2MB/s 00:00 log.100001bf0 100% 1001KB 46.5MB/s 00:00 snapshot.100000000 100% 73 40.2KB/s 00:00 snapshot.100001bec 100% 2507KB 57.4MB/s 00:00 check_compatibility.py 100% 7212 1.4MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 4.4MB/s 00:00 checkstyle.xml 100% 7489 5.0MB/s 00:00 findbugsExcludeFile.xml 100% 5027 4.2MB/s 00:00 log4j.properties 100% 2712 2.5MB/s 00:00 test-patch.properties 100% 855 376.7KB/s 00:00 test-github-pr.sh 100% 24KB 6.3MB/s 00:00 test-patch.sh 100% 24KB 13.5MB/s 00:00 pom.xml 100% 10KB 6.5MB/s 00:00 LICENSE.txt 100% 12KB 4.9MB/s 00:00 NOTICE.txt 100% 3132 2.8MB/s 00:00 README.md 100% 1765 1.9MB/s 00:00 README_packaging.txt 100% 1770 1.6MB/s 00:00 build.xml 100% 95KB 27.3MB/s 00:00 ivy.xml 100% 10KB 10.1MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 13.6MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 42.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 38.8KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 29.3KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 690.2KB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
7>.啓動zookeeper集羣
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10062 Jps node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 9720 Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 9759 Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start 啓動服務 ========== node101.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node102.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node103.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status 查看狀態 ========== node101.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower ========== node102.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader ========== node103.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10177 QuorumPeerMain 10279 Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10209 QuorumPeerMain 10317 Jps node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10765 QuorumPeerMain 10925 Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-12 09:23:04,270 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-12 09:23:04,272 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-12 09:23:04,273 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-12 09:23:04,277 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-12 09:23:04,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-12 09:23:04,365 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-12 09:23:04,395 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x6500007f14d00000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper] [zk: localhost:2181(CONNECTED) 1] ls /zookeeper [quota] [zk: localhost:2181(CONNECTED) 2] ls /zookeeper/quota [] [zk: localhost:2181(CONNECTED) 3] quit Quitting... 2019-04-12 09:23:13,101 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x6500007f14d00000 closed 2019-04-12 09:23:13,102 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x6500007f14d00000 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
三.部署hadoop 3.1.2 HA案例實操（切記，3個namenode之間應該相互配置免密登陸喲！）
1>.備份原有的hadoop徹底分佈式的配置目錄
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1.2/etc/ total 4 drwxr-xr-x 3 1001 1002 4096 Apr 11 09:27 hadoop [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cp -r /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/ /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop-full [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1.2/etc/ total 8 drwxr-xr-x 3 1001 1002 4096 Apr 11 09:27 hadoop drwxr-xr-x 3 root root 4096 Apr 12 07:15 hadoop-full [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
2>.編輯hadoop的環境變量配置文件，主要修改各個服務的所屬用戶
[root@node101.yinzhengjie.org.cn ~]# vi /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hadoop-env.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# tail -6 /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hadoop-env.sh #Add Hadoop Path by yinzhengjie export JAVA_HOME=/yinzhengjie/softwares/jdk export HDFS_NAMENODE_USER=root export HDFS_DATANODE_USER=root export HDFS_ZKFC_USER=root export HDFS_JOURNALNODE_USER=root [root@node101.yinzhengjie.org.cn ~]#
3>.編輯core-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://yinzhengjie-hdfs</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/data/hadoop/ha</value> </property> <property> <name>hadoop.http.staticuser.user</name> <value>root</value> </property> <property> <name>ha.zookeeper.quorum</name> <value>node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181</value> </property> </configuration> <!-- core-site.xml配置文件的做用： 用於定義系統級別的參數，如HDFS URL、Hadoop的臨時目錄以及用於rack-aware集羣中的配置文件的配置等，此中的參數定義會覆蓋core-default.xml文件中的默認配置。 fs.defaultFS 參數的做用： #聲明namenode的地址，至關於聲明hdfs文件系統。咱們能夠指定某個ip地址，在ha模式中咱們一般指定hdfs集羣的邏輯名稱 hadoop.tmp.dir 參數的做用： #聲明hadoop工做目錄的地址。 hadoop.http.staticuser.user 參數的做用： #在網頁界面訪問數據使用的用戶名。默認值是一個不真實存在的用戶，此用戶權限很小，不能訪問不一樣用戶的數據。這保證了數據安全。也能夠設置爲hdfs和hadoop等具備較高權限的用戶，但會致使可以登錄網頁界面的人能看到其它用戶數據。實際設置請綜合考慮。如無特殊需求。使用默認值就好. ha.zookeeper.quorum 參數的做用： #指定zookeeper集羣的地址，讓zookeeper幫咱們管理hdfs集羣。千萬別手動在後面加zookeeper的根目錄喲，不然在使用hdfs進行zookeeper格式化會報錯！ --> [root@node101.yinzhengjie.org.cn ~]#
4>.編輯hdfs-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.nameservices</name> <value>yinzhengjie-hdfs</value> </property> <property> <name>dfs.ha.namenodes.yinzhengjie-hdfs</name> <value>namenode1,namenode2,namenode3</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode2</name> <value>node102.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode3</name> <value>node103.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode2</name> <value>node102.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode3</name> <value>node103.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://node101.yinzhengjie.org.cn:8485;node102.yinzhengjie.org.cn:8485;node103.yinzhengjie.org.cn:8485/yinzhengjie-hdfs</value> </property> <property> <name>dfs.client.failover.proxy.provider.yinzhengjie-hdfs</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property> <property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/root/.ssh/id_rsa</value> </property> <property> <name>dfs.journalnode.edits.dir</name> <value>/data/hadoop/ha/journalnode</value> </property> <property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property> </configuration> <!-- hdfs-site.xml 配置文件的做用： #HDFS的相關設定，如文件副本的個數、塊大小及是否使用強制權限等，此中的參數定義會覆蓋hdfs-default.xml文件中的默認配置. dfs.replication 參數的做用： #爲了數據可用性及冗餘的目的，HDFS會在多個節點上保存同一個數據塊的多個副本，其默認爲3個。而只有一個節點的僞分佈式環境中其僅用 保存一個副本便可，這能夠經過dfs.replication屬性進行定義。它是一個軟件級備份。 dfs.nameservices 參數的做用： #指定一個邏輯名稱，該名稱會自動去幫咱們找打真實NameNode節點，這和我們的keepalive的VIP有點相似！這個名稱我們能夠自定義！可是咱們正在引用時千萬別引用錯了！ dfs.ha.namenodes.yinzhengjie-hdfs 參數的做用： #看到沒，上面咱們定義了dfs.nameservices的名稱，如今咱們就用到了，咱們把定義的邏輯名稱指向了各個namenode的別名 dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1 參數的做用： #定義遠程主機調用的服務器地址。 dfs.namenode.http-address.yinzhengjie-hdfs.namenode1 參數的做用： #上面咱們定義了集羣的邏輯名稱，也定義節點的邏輯名稱，可是真正的ip地址咱們始終沒有指定，所以咱們這一步驟就是指定dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1對應的IP地址。固然你寫主機名稱也是能夠的，只不過嘛你得在/etc/hosts文件作映射或者配置DNS中有對應的A記錄喲！ dfs.namenode.shared.edits.dir 參數的做用： #配置JournalNode集羣的地址。 dfs.client.failover.proxy.provider.yinzhengjie-hdfs 參數的做用： #配置故障轉移的代理類，這是HDFS客戶端找到active NameNode類名的一個代理類，若是你不配置這個項的話，那麼客戶端鏈接active namenode地址時會報錯！實際上故障轉移就是經過這個代理類來實現的，咱們只須要默認官網的配置便可！不須要手動修改！ dfs.ha.fencing.methods 參數的做用： #指定ha出現故障時的隔離方法！ dfs.ha.fencing.ssh.private-key-files 參數的做用： #指定隔離主機的私鑰PATH。 dfs.journalnode.edits.dir 參數的做用： #指定JN節點存放編輯日誌的目錄。 dfs.ha.automatic-failover.enabled 參數的做用： #開啓自動故障轉移功能. -> [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
5>.查看works配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/workers node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn [root@node101.yinzhengjie.org.cn ~]#
5>.將修改的配置文件分發到其餘節點中
[root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cd /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node102.yinzhengjie.org.cn:`pwd` core-site.xml 100% 1804 1.0MB/s 00:00 hdfs-site.xml 100% 4635 5.4MB/s 00:00 hadoop-env.sh 100% 16KB 12.5MB/s 00:00 [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]#
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node103.yinzhengjie.org.cn:`pwd` core-site.xml 100% 1804 227.6KB/s 00:00 hdfs-site.xml 100% 4635 2.2MB/s 00:00 hadoop-env.sh 100% 16KB 13.3MB/s 00:00 [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]#
6>.啓動journalnode服務
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 11082 Jps 10765 QuorumPeerMain node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10336 Jps 10177 QuorumPeerMain node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10209 QuorumPeerMain 10373 Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /data/hadoop/ total 0 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node101.yinzhengjie.org.cn ~]# jps 15733 JournalNode 10765 QuorumPeerMain 15775 Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# jps 10209 QuorumPeerMain 13607 Jps 13566 JournalNode [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# jps 10177 QuorumPeerMain 13605 Jps 13564 JournalNode [root@node103.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode ERROR: Cannot set priority of journalnode process 15496 [root@node101.yinzhengjie.org.cn ~]# 舒適提示： 若是遇到上述問題的話，請檢查你的配置文件是否有誤！多一個空格都不行，若是你用個人博客的配置文件的話，建議不要把註釋拷貝進去，只拷貝有效的XML代碼便可！
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:34 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:34 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:34 journalnode /data/hadoop/ha/journalnode: total 0 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:45 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:45 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:45 journalnode /data/hadoop/ha/journalnode: total 0 [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:45 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:45 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:45 journalnode /data/hadoop/ha/journalnode: total 0 [root@node103.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node103.yinzhengjie.org.cn ~]#
7>.格式化HDFS文件系統（目的是生成初始的元數據信息，生成的信息會存放在咱們hdfs-core-site.xml配置文件中指定的工做目錄下，注意生成的信息和上面的JN的VERSION之間的關係！）
[root@node101.yinzhengjie.org.cn ~]# hdfs namenode -format 2019-04-15 07:53:21,960 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 07:53:21,972 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 07:53:22,069 INFO namenode.NameNode: createNameNode [-format] Formatting using clusterid: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 2019-04-15 07:53:22,666 INFO namenode.FSEditLog: Edit logging is async:true 2019-04-15 07:53:22,682 INFO namenode.FSNamesystem: KeyProvider: null 2019-04-15 07:53:22,683 INFO namenode.FSNamesystem: fsLock is fair: true 2019-04-15 07:53:22,684 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE) 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: supergroup = supergroup 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: isPermissionEnabled = true 2019-04-15 07:53:22,693 INFO namenode.FSNamesystem: Determined nameservice ID: yinzhengjie-hdfs 2019-04-15 07:53:22,693 INFO namenode.FSNamesystem: HA Enabled: true 2019-04-15 07:53:22,739 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2019-04-15 07:53:22,755 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000 2019-04-15 07:53:22,756 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true 2019-04-15 07:53:22,763 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000 2019-04-15 07:53:22,763 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Apr 15 07:53:22 2019-04-15 07:53:22,765 INFO util.GSet: Computing capacity for map BlocksMap 2019-04-15 07:53:22,765 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,766 INFO util.GSet: 2.0% max memory 839.5 MB = 16.8 MB 2019-04-15 07:53:22,766 INFO util.GSet: capacity = 2^21 = 2097152 entries 2019-04-15 07:53:22,784 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false 2019-04-15 07:53:22,791 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: defaultReplication = 2 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxReplication = 512 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: minReplication = 1 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: encryptDataTransfer = false 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000 2019-04-15 07:53:22,841 INFO namenode.FSDirectory: GLOBAL serial map: bits=24 maxEntries=16777215 2019-04-15 07:53:22,856 INFO util.GSet: Computing capacity for map INodeMap 2019-04-15 07:53:22,856 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,856 INFO util.GSet: 1.0% max memory 839.5 MB = 8.4 MB 2019-04-15 07:53:22,856 INFO util.GSet: capacity = 2^20 = 1048576 entries 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: ACLs enabled? false 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: XAttrs enabled? true 2019-04-15 07:53:22,857 INFO namenode.NameNode: Caching file names occurring more than 10 times 2019-04-15 07:53:22,862 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536 2019-04-15 07:53:22,867 INFO snapshot.SnapshotManager: SkipList is disabled 2019-04-15 07:53:22,873 INFO util.GSet: Computing capacity for map cachedBlocks 2019-04-15 07:53:22,873 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,873 INFO util.GSet: 0.25% max memory 839.5 MB = 2.1 MB 2019-04-15 07:53:22,873 INFO util.GSet: capacity = 2^18 = 262144 entries 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25 2019-04-15 07:53:22,885 INFO namenode.FSNamesystem: Retry cache on namenode is enabled 2019-04-15 07:53:22,886 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 2019-04-15 07:53:22,888 INFO util.GSet: Computing capacity for map NameNodeRetryCache 2019-04-15 07:53:22,889 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,889 INFO util.GSet: 0.029999999329447746% max memory 839.5 MB = 257.9 KB 2019-04-15 07:53:22,889 INFO util.GSet: capacity = 2^15 = 32768 entries 2019-04-15 07:53:26,352 INFO namenode.FSImage: Allocated new BlockPoolId: BP-692446373-172.30.1.101-1555340006352 2019-04-15 07:53:26,365 INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted. 2019-04-15 07:53:26,657 INFO namenode.FSImageFormatProtobuf: Saving image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression 2019-04-15 07:53:26,779 INFO namenode.FSImageFormatProtobuf: Image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 391 bytes saved in 0 seconds . 2019-04-15 07:53:26,792 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 2019-04-15 07:53:26,976 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at node101.yinzhengjie.org.cn/172.30.1.101 ************************************************************/ [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs /data/hadoop/ha/dfs: total 0 drwxr-xr-x 3 root root 21 Apr 15 07:53 name /data/hadoop/ha/dfs/name: total 0 drwxr-xr-x 2 root root 112 Apr 15 07:53 current /data/hadoop/ha/dfs/name/current: total 16 -rw-r--r-- 1 root root 391 Apr 15 07:53 fsimage_0000000000000000000 -rw-r--r-- 1 root root 62 Apr 15 07:53 fsimage_0000000000000000000.md5 -rw-r--r-- 1 root root 2 Apr 15 07:53 seen_txid -rw-r--r-- 1 root root 216 Apr 15 07:53 VERSION [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/ total 0 drwxr-xr-x 3 root root 30 Apr 15 07:53 journalnode [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/ total 0 drwxr-xr-x 3 root root 30 Apr 15 07:53 journalnode [root@node103.yinzhengjie.org.cn ~]#
8>.啓動NameNode並同步數據
[root@node101.yinzhengjie.org.cn ~]# jps Jps JournalNode QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start namenode [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps JournalNode NameNode Jps QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs namenode -bootstrapStandby 2019-04-15 08:08:00,871 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = node103.yinzhengjie.org.cn/172.30.1.103 STARTUP_MSG: args = [-bootstrapStandby] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 08:08:00,890 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 08:08:01,007 INFO namenode.NameNode: createNameNode [-bootstrapStandby] 2019-04-15 08:08:01,297 INFO ha.BootstrapStandby: Found nn: namenode1, ipc: node101.yinzhengjie.org.cn/172.30.1.101:8020 2019-04-15 08:08:01,297 INFO ha.BootstrapStandby: Found nn: namenode2, ipc: node102.yinzhengjie.org.cn/172.30.1.102:8020 ===================================================== About to bootstrap Standby ID namenode3 from: Nameservice ID: yinzhengjie-hdfs Other Namenode ID: namenode1 Other NN's HTTP address: http://node101.yinzhengjie.org.cn:9870 Other NN's IPC address: node101.yinzhengjie.org.cn/172.30.1.101:8020 Namespace ID: 1130551675 Block pool ID: BP-692446373-172.30.1.101-1555340006352 Cluster ID: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 Layout version: -64 isUpgradeFinalized: true ===================================================== 2019-04-15 08:08:02,595 INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted. 2019-04-15 08:08:02,643 INFO namenode.FSEditLog: Edit logging is async:true 2019-04-15 08:08:02,756 INFO namenode.TransferFsImage: Opening connection to http://node101.yinzhengjie.org.cn:9870/imagetransfer?getimage=1&txid=0&storageInfo=-64:1130551675:1555340006352:CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6&bootstrapstandby=true 2019-04-15 08:08:03,014 INFO common.Util: Combined time for file download and fsync to all disks took 0.01s. The file download took 0.01s at 0.00 KB/s. Synchronous (fsync) write to disk of /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 took 0.00s. 2019-04-15 08:08:03,014 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 391 bytes. 2019-04-15 08:08:03,076 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at node103.yinzhengjie.org.cn/172.30.1.103 ************************************************************/ [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs /data/hadoop/ha/dfs: total 0 drwxr-xr-x 3 root root 21 Apr 15 08:08 name /data/hadoop/ha/dfs/name: total 0 drwxr-xr-x 2 root root 112 Apr 15 08:08 current /data/hadoop/ha/dfs/name/current: total 16 -rw-r--r-- 1 root root 391 Apr 15 08:08 fsimage_0000000000000000000 -rw-r--r-- 1 root root 62 Apr 15 08:08 fsimage_0000000000000000000.md5 -rw-r--r-- 1 root root 2 Apr 15 08:08 seen_txid -rw-r--r-- 1 root root 216 Apr 15 08:08 VERSION [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 08:08:02 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node103.yinzhengjie.org.cn ~]#
四.zookeeper格式化、啓動ha及測試
1>.初始化hdfs在zookeeper註冊節點目錄
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:15:10,366 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:15:10,369 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:15:10,369 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:15:10,374 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:15:10,406 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:15:10,476 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:15:10,497 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper] [zk: localhost:2181(CONNECTED) 1] [zk: localhost:2181(CONNECTED) 1] quit Quitting... 2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed 2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs zkfc -formatZK 2019-04-15 08:22:35,041 INFO tools.DFSZKFailoverController: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DFSZKFailoverController STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101 STARTUP_MSG: args = [-formatZK] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 08:22:35,061 INFO tools.DFSZKFailoverController: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 08:22:35,387 INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at node101.yinzhengjie.org.cn/172.30.1.101:8020 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:22:35,537 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar 2019-04-15 08:22:35,543 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/yinzhengjie/softwares/hadoop-3.1.2/lib/native 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.name=root 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.home=/root 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root 2019-04-15 08:22:35,546 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@3c407114 2019-04-15 08:22:35,585 INFO zookeeper.ClientCnxn: Opening socket connection to server node103.yinzhengjie.org.cn/172.30.1.103:2181. Will not attempt to authenticate using SASL (unknown error) 2019-04-15 08:22:35,590 INFO zookeeper.ClientCnxn: Socket connection established to node103.yinzhengjie.org.cn/172.30.1.103:2181, initiating session 2019-04-15 08:22:35,601 INFO zookeeper.ClientCnxn: Session establishment complete on server node103.yinzhengjie.org.cn/172.30.1.103:2181, sessionid = 0x67000f716ed80001, negotiated timeout = 10000 2019-04-15 08:22:35,613 INFO ha.ActiveStandbyElector: Session connected. 2019-04-15 08:22:35,633 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/yinzhengjie-hdfs in ZK. 2019-04-15 08:22:35,638 INFO zookeeper.ZooKeeper: Session: 0x67000f716ed80001 closed 2019-04-15 08:22:35,696 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x67000f716ed80001 2019-04-15 08:22:35,698 INFO tools.DFSZKFailoverController: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DFSZKFailoverController at node101.yinzhengjie.org.cn/172.30.1.101 ************************************************************/ [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:24:07,198 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:24:07,201 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:24:07,201 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:24:07,206 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:24:07,207 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:24:07,230 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:24:07,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:24:07,322 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200001, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper, hadoop-ha] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha [yinzhengjie-hdfs] [zk: localhost:2181(CONNECTED) 2] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs [] [zk: localhost:2181(CONNECTED) 3] [zk: localhost:2181(CONNECTED) 3] quit Quitting... 2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed 2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
2>.啓動hadoop集羣
[root@node101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:15:05 PDT 2019 from 172.30.1.1 on pts/1 node101.yinzhengjie.org.cn: namenode is running as process 16023. Stop it first. Starting datanodes Last login: Mon Apr 15 08:28:26 PDT 2019 on pts/1 Starting journal nodes [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:28:29 PDT 2019 on pts/1 node101.yinzhengjie.org.cn: journalnode is running as process 15733. Stop it first. node102.yinzhengjie.org.cn: journalnode is running as process 13735. Stop it first. node103.yinzhengjie.org.cn: journalnode is running as process 13723. Stop it first. Starting ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:28:50 PDT 2019 on pts/1 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 16023 NameNode 17228 Jps 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# jps 10209 QuorumPeerMain 13939 DataNode 13735 JournalNode 14136 Jps [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps 10177 QuorumPeerMain 14209 Jps 13723 JournalNode 13995 DataNode 13917 NameNode [root@node103.yinzhengjie.org.cn ~]#
3>.啓動hadoop集羣成功後，咱們再次查看zookeeper集羣管理hadoop的節點信息
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:31:09,342 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:31:09,346 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:31:09,346 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:31:09,361 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:31:09,361 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:31:09,365 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:31:09,399 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:31:09,507 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:31:09,531 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200002, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls /hadoop-ha/yinzhengjie-hdfs [ActiveBreadCrumb, ActiveStandbyElectorLock] [zk: localhost:2181(CONNECTED) 1] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb [] [zk: localhost:2181(CONNECTED) 2] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock [] [zk: localhost:2181(CONNECTED) 3] [zk: localhost:2181(CONNECTED) 3] get /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #咱們這裏能夠看到註冊節點成功的爲node101.yinzhengjie.org.cn
cZxid = 0x800000011 ctime = Mon Apr 15 08:28:56 PDT 2019 mZxid = 0x800000011 mtime = Mon Apr 15 08:28:56 PDT 2019 pZxid = 0x800000011 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 4] [zk: localhost:2181(CONNECTED) 4] get /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #注意，這裏註冊節點成功的爲node101.yinzhengjie.org.cn cZxid = 0x800000010 ctime = Mon Apr 15 08:28:56 PDT 2019 mZxid = 0x800000010 mtime = Mon Apr 15 08:28:56 PDT 2019 pZxid = 0x800000010 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x66000f71a2960001 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 5]
4>.查看node101.yinzhengjie.org.cn節點的Web UI信息
5>.查看node103.yinzhengjie.org.cn節點的Web UI信息
6>.停用node101.yinzhengjie.org.cn的namenode進程停用，觀察現象（注意檢查多個namenode之間是否配置了免密碼登陸）
[root@node101.yinzhengjie.org.cn ~]# jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 16023 NameNode 17255 ZooKeeperMain 17305 Jps 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop namenode [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 17361 Jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 17255 ZooKeeperMain 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop zkfc [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 18928 ZooKeeperMain 18370 DataNode 19106 Jps 18617 JournalNode 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
相關文章
- 1. 大數據技術之Hadoop3.1.2版本HA模式
- 2. 大數據技術之Hadoop（HDFS）第8章 HDFS HA高可用
- 3. 大數據技術之HBase
- 4. 大數據技術之Linux
- 5. 大數據技術之Kafka
- 6. 大數據技術之Java
- 7. 大數據技術之Zookeeper
- 8. 大數據技術之Shell
- 9. 大數據技術之Oozie
- 10. 大數據技術之HDFS
- 更多相關文章...
- • XML 相關技術 - XML 教程
- • 數據庫涉及到哪些技術？ - MySQL教程
- • 委託模式
- • Flink 數據傳輸及反壓詳解
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. 大數據技術之Hadoop3.1.2版本HA模式
- 2. 大數據技術之Hadoop（HDFS）第8章 HDFS HA高可用
- 3. 大數據技術之HBase
- 4. 大數據技術之Linux
- 5. 大數據技術之Kafka
- 6. 大數據技術之Java
- 7. 大數據技術之Zookeeper
- 8. 大數據技術之Shell
- 9. 大數據技術之Oozie
- 10. 大數據技術之HDFS