首先確保集羣配置了SSH免密登陸。(詳見5)bash
(a)在/home/atguigu目錄下建立bin目錄(/home/atguigu/bin),並在bin目錄下xsync建立文件,文件內容以下:ssh
[atguigu@hadoop102 ~]$ mkdir bin
[atguigu@hadoop102 ~]$ cd bin/
[atguigu@hadoop102 bin]$ touch xsync
[atguigu@hadoop102 bin]$ vi xsync
在該文件中編寫以下代碼oop
#!/bin/bash #1 獲取輸入參數個數,若是沒有參數,直接退出 pcount=$# if((pcount==0)); then echo no args; exit; fi #2 獲取文件名稱 p1=$1 fname=`basename $p1` echo fname=$fname #3 獲取上級目錄到絕對路徑 pdir=`cd -P $(dirname $p1); pwd` echo pdir=$pdir #4 獲取當前用戶名稱 user=`whoami` #5 循環 for((host=103; host<105; host++)); do echo ------------------- hadoop$host -------------- rsync -rvl $pdir/$fname $user@hadoop$host:$pdir done
(b)修改腳本 xsync 具備執行權限ui
[atguigu@hadoop102 bin]$ chmod 777 xsyncspa
(c)調用腳本形式:xsync 文件名稱日誌
[atguigu@hadoop102 bin]$ xsync /home/atguigu/bincode
在/home/atguigu/bin建立start-cluster.sh,若是zookeeper不能啓動,在zkEnv.sh加上server
start-cluster.shblog
#!/bin/bash user=`whoami` echo "=============== 開始啓動全部節點服務 ===============" echo "=============== 正在啓動Zookeeper...... ===============" for((host=102; host<=104; host++)); do echo "--------------- hadoop$host Zookeeper...... ----------------" ssh $user@hadoop$host '/opt/module/zookeeper-3.4.10/bin/zkServer.sh start' done echo "================ 正在啓動HDFS ===============" ssh $user@hadoop102 '/opt/module/hadoop-2.7.2/sbin/start-dfs.sh' echo "================ 正在啓動YARN ===============" ssh $user@hadoop103 '/opt/module/hadoop-2.7.2/sbin/start-yarn.sh' echo "================ hadoop102正在啓動JobHistoryServer ===============" ssh $user@hadoop102 '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh start historyserver' done
(b)修改腳本 具備執行權限進程
[atguigu@hadoop102 bin]$ chmod 777 start-cluster.sh
在/home/atguigu/bin建立stop-cluster.sh
內容爲
#!/bin/bash user=`whoami` echo "================ 開始中止全部節點服務 ===============" echo "================ hadoop102正在中止JobHistoryServer ===============" ssh $user@hadoop102 '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh stop historyserver' echo "================ 正在中止YARN ===============" ssh $user@hadoop103 '/opt/module/hadoop-2.7.2/sbin/stop-yarn.sh' echo "================ 正在中止HDFS ===============" ssh $user@hadoop102 '/opt/module/hadoop-2.7.2/sbin/stop-dfs.sh' echo "=============== 正在中止Zookeeper...... ===============" for((host=102; host<=104; host++)); do echo "--------------- hadoop$host Zookeeper...... ----------------" ssh $user@hadoop$host '/opt/module/zookeeper-3.4.10/bin/zkServer.sh stop' done
修改腳本 具備執行權限
[atguigu@hadoop102 bin]$ chmod 777 stop-cluster.sh
最後使用xsync 分發到其它集羣上(切記要改變權限)
在/home/atguigu/bin建立util.sh
內容爲
#!/bin/bash for ip in hadoop102 hadoop103 hadoop104 do echo "------------------------------[ jps $ip ]-------------------------" ssh atguigu@$ip "source /etc/profile;jps" done
無密鑰配置
(1)免密登陸原理,如圖所示
(2)生成公鑰和私鑰:
[atguigu@hadoop102 .ssh]$ ssh-keygen -t rsa
而後敲(三個回車),就會生成兩個文件id_rsa(私鑰)、id_rsa.pub(公鑰)
(3)將公鑰拷貝到要免密登陸的目標機器上
[atguigu@hadoop102 .ssh]$ ssh-copy-id hadoop102
[atguigu@hadoop102 .ssh]$ ssh-copy-id hadoop103
[atguigu@hadoop102 .ssh]$ ssh-copy-id hadoop104
在/home/用戶名/bin下,建立zkstart.sh
#!/bin/bash user=`whoami` echo "=============== 正在啓動Zookeeper...... ===============" for((host=102; host<=104; host++)); do echo "--------------- hadoop$host Zookeeper...... ----------------" ssh $user@hadoop$host '/opt/module/zookeeper-3.4.10/bin/zkServer.sh start' done
建立zkStop.sh
#!/bin/bash user=`whoami` echo "=============== 正在中止Zookeeper...... ===============" for((host=102; host<=104; host++)); do echo "--------------- hadoop$host Zookeeper...... ----------------" ssh $user@hadoop$host '/opt/module/zookeeper-3.4.10/bin/zkServer.sh stop' done
修改權限 chmod 777
在kafka目錄下建立startkafka.sh
nohup bin/kafka-server-start.sh config/server.properties > kafka.log 2>&1 &
修改權限 chmod 777
./startkafka.sh 便可執行
只須要把startkafka.sh分發到各個機器,再獨自啓動便可
【注:
a. >kafka.log 將運行的日誌寫到kafka中, 2>&1 的意思就是將標準錯誤重定向到標準輸出。
b. &:後臺運行。當你只使用「&」時,關閉終端,進程會關閉。因此當你要讓程序在後臺不掛斷運行時,須要將nohup和&一塊兒使用。
c. 啓動命令首位加上nohup,即便停掉crt,kafka、flume依然能夠在後臺執行,這樣就不用每次登錄,從新運行啓動命令了。若是須要停掉服務,只需運行 kill -9 [程序運行的號便可]
】