hadoop(六)rsync遠程同步|xsync集羣分發(徹底分佈式準備三)|8

前置環境準備：
centos7克隆ip|機器名|映射關係|別名配置（hadoop徹底分佈式準備一）
scp命令copy文件和配置(hadoop徹底分佈式準備二)

rsync遠程同步工具

優勢

rsync主要用於備份和鏡像。具備速度快、避免複製相同內容和支持符號連接的優勢。
rsync和scp區別:用 rsync作文件的複製要比scp的速度快, rsync只對差別文件作更新。scp是把全部文件都複製過去。

基本語法

rsync  -rvl      $pdir/$fname     $user(@hadoop$host: $pdir/$fname

命令 選項參數 要拷貝的文件路徑/名稱 目的用戶@主機:目的路徑名稱
選項參數說明
選項 功能
-r 遞歸
-v 顯示覆制過程
-l Copy符號連接

案例

將hadoop101機器上的/opt/software 同步到hadoop102的服務器上的對應目錄下。

[shaozhiqi@hadoop101 ~]$ rsync  -rvl /opt/software shaozhiqi@hadoop102:/opt/ shaozhiqi@hadoop102's password: \sending incremental file list software/hadoop-3.1.2.tar.gz software/jdk-8u211-linux-x64.tar.gz sent 527488768 bytes received 51 bytes 14066368.51 bytes/sec total size is 527424191 speedup is 1.00 [shaozhiqi@hadoop101 ~]$ 

查看102機器目錄同步成功

[shaozhiqi@hadoop102 software]$ ll
total 515068 -rw-rw-r--. 1 shaozhiqi shaozhiqi 332433589 Jun 23 04:55 hadoop-3.1.2.tar.gz -rw-rw-r--. 1 shaozhiqi shaozhiqi 194990602 Jun 23 04:56 jdk-8u211-linux-x64.tar.gz [shaozhiqi@hadoop102 software]$ 

xsync集羣分發腳本

(1)需求:循環複製文件到全部節點的相同目錄下
(2)需求分析:
在 home/shaozhiqi/bin這個目錄下存放的腳本, shaozhiqi用戶能夠在系統任何地方直接執行。
可用案例：
若是咱們修改了其中一臺hadoop的某個xml的，須要在其餘機器上都同步過去就能夠這樣作

需求實現
在home/shao目錄下建立bin目錄,並在bin目錄下 xsync建立文件,文件
腳本實現

#!/bin/bash # $#：表示傳遞給腳本或函數的參數個數。 #1 獲取輸入參數個數，若是沒有參數，直接退出 pcount=$# if((pcount==0)); then echo no args; exit; fi #2 獲取文件名稱 p1=$1 fname=`basename $p1` echo fname=$fname #3 獲取上級目錄到絕對路徑 pdir=`cd -P $(dirname $p1); pwd` echo pdir=$pdir #4 獲取當前用戶名稱 user=`whoami` #5循環 for (( host=103; host<105; host++)); do echo -------hadoop$host ----------------------- rsync -rvl $pdir/$fname $user@hadoop$host:$pdir done 

basename shell 語法

[shaozhiqi@hadoop102 opt]$ basename module/hadoop-3.1.2/etc/hadoop/core-site.xml core-site.xml [shaozhiqi@hadoop102 opt]$ basename core-site.xml core-site.xml 

dirname shell語法

[shaozhiqi@hadoop102 opt]$ dirname module/hadoop-3.1.2/etc/hadoop/core-site.xml module/hadoop-3.1.2/etc/hadoop [shaozhiqi@hadoop102 opt]$ 

whoami

[shaozhiqi@hadoop102 opt]$ whoami
shaozhiqi
[shaozhiqi@hadoop102 opt]$

賦權限

[shaozhiqi@hadoop102 bin]$ chmod 777 testxsync [shaozhiqi@hadoop102 bin]$ ll total 4 -rwxrwxrwx. 1 shaozhiqi shaozhiqi 520 Jun 23 05:48 testxsync 

執行前查看103

[shaozhiqi@hadoop103 ~]$ ll
total 0 

執行前查看104

[shaozhiqi@hadoop104 ~]$ ll
total 0 

執行腳本將bin同步到103.104

[shaozhiqi@hadoop102 bin]$ testxsync bin/
fname=bin
dirname: missing operand Try 'dirname --help' for more information. pdir=/home/shaozhiqi -------hadoop103 ----------------------- shaozhiqi@hadoop103's password: sending incremental file list bin/ bin/.testxsync.swo bin/.testxsync.swp bin/testxsync sent 25281 bytes received 73 bytes 2668.84 bytes/sec total size is 25052 speedup is 0.99 -------hadoop104 ----------------------- shaozhiqi@hadoop104's password: Permission denied, please try again. shaozhiqi@hadoop104's password: sending incremental file list bin/ bin/.testxsync.swo bin/.testxsync.swp bin/testxsync sent 25281 bytes received 73 bytes 2414.67 bytes/sec total size is 25052 speedup is 0.99 [shaozhiqi@hadoop102 bin]$ 

執行後查看10三、104腳本執行成功

[shaozhiqi@hadoop103 ~]$ ll
total 0 drwxrwxr-x. 2 shaozhiqi shaozhiqi 64 Jun 23 06:13 bin [shaozhiqi@hadoop103 ~]$ [shaozhiqi@hadoop104 ~]$ ll total 0 drwxrwxr-x. 2 shaozhiqi shaozhiqi 64 Jun 23 06:13 bin [shaozhiqi@hadoop104 ~]$ 

說明咱們[shaozhiqi@hadoop103 ~]$ mkdir 的文件都會在 /home/shaozhiqi/bin在 home/shaozhiqi/bin這個目錄下存放的腳本, shaozhiqi用戶能夠在系統任何地方直接執行。若是有意外，致使沒法識別，那就把它放在、usr/local/bin下