unison + inotify 實現文件實時雙向同步部署步驟
unison + inotify 實現文件實時雙向同步部署步驟html
一. Unison簡介
Unison是Windows、Linux以及其餘Unix平臺下均可以使用的文件同步工具，它能使兩個文件夾（本地或網絡上的）保持內容的一致。Unison擁有與其它一些同步工具或文件系統的相同的特性，但也有自身的特色：linux
1. 跨平臺使用；git
2. 對內核和用戶權限沒有特別要求；github
3. Unison是雙向的，它能自動處理兩分拷貝中更新沒有衝突的部分，有衝突的部分將會顯示出來讓用戶選擇更新策略；web
4. 只要是能連通的兩臺主機，就能夠運行unison，能夠直接使用socket鏈接或安全的ssh鏈接方式，對帶寬的要求不高，使用相似rsync的壓縮傳輸協議。vim
二. 編譯安裝Unison
unison各類版本下載地址：tomcat
http://www.seas.upenn.edu/~bcpierce/unison//download.html安全
unison編譯器下載地址：bash
http://caml.inria.fr/pub/distrib服務器
從以上地址能夠下載各類平臺,各類版本的unison,有基於源碼安裝的,有二進制的,我下載的是二進制的,能夠直接使用.這裏介紹源碼安裝:
1.實驗環境：
txzlkhdweb1 192.168.16.1 同步目錄 /root/test
txzlkhdweb2 192.168.16.2 同步目錄 /root/test
2.建立共享文件目錄
txzlkhdweb1 :
[root@txzlkhdweb1 /]# mkdir /root/test
txzlkhdweb2:
[root@txzlkhdweb2 ~]# mkdir /root/test
3.安裝Objective Caml compiler
Linux下從源碼包編譯安裝，須要一個叫作Objective Caml compiler的工具，版本至少3.0.7，能夠從這裏下載：http://caml.inria.fr/
[tomcat@txzlkhdweb1 ~]$ cd /usr/local/src/
[tomcat@txzlkhdweb1 ~]$ wget http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
--2013-05-03 20:23:02-- http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
正在解析主機 caml.inria.fr... 128.93.11.35
Connecting to caml.inria.fr|128.93.11.35|:80... 已鏈接。
已發出 HTTP 請求，正在等待迴應... 200 OK
長度：3660473 (3.5M) [application/x-gzip]
Saving to: `ocaml-3.12.1.tar.gz'
100%[======================================>] 3,660,473 150K/s in 44s
2013-05-03 20:23:49 (81.5 KB/s) - `ocaml-3.12.1.tar.gz' saved [3660473/3660473]
[root@txzlkhdweb1 src]# tar -zxf ocaml-3.12.1.tar.gz
[root@txzlkhdweb1 src]# cd ocaml-3.12.1
[root@txzlkhdweb1 ocaml-3.12.1]# ./configure
Configuring for a x86_64-unknown-linux-gnu ...
gcc found
........................................
options for linking ...... -lX11
The "labltk" library: not supported
** Objective Caml configuration completed successfully **
[root@txzlkhdweb1 ocaml-3.12.1]# make world opt
make coldstart
make[1]: Entering directory `/usr/local/src/ocaml-3.12.1'
cd byterun; make all
make[2]: Entering directory `/usr/local/src/ocaml-3.12.1/byterun'
sed -n -e '/^ /s/ \([A-Z]\)/ \&\&lbl_\1/gp' \
-e '/^}/q' instruct.h > jumptbl.h
......................................................
../ocamlcompopt.sh -nostdlib -c -g -annot -warn-error A -w L -w R -w Z -I ../otherlibs/unix -I ocamlbuild -I stdlib -o ocamlbuild/ocamlbuild_unix_plugin.cmx ocamlbuild/ocamlbuild_unix_plugin.ml
../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx ocamlbuild/ocamlbuild_executor.cmx ocamlbuild/ocamlbuild_unix_plugin.cmx -o ocamlbuild/ocamlbuildlib.cmxa
../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx -o ocamlbuild/ocamlbuildlightlib.cmxa
make[1]: Leaving directory `/usr/local/src/ocaml-3.12.1'
[root@txzlkhdweb1 ocaml-3.12.1]# make install
if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi
if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi
if test -d /usr/local/lib/ocaml/stublibs; then : ; else mkdir -p /usr/local/lib/ocaml/stublibs; fi
..................................
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild_executor.o
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo
install /usr/local/man/man1/ocamlbuild.1
4.編譯安裝Unison
[root@txzlkhdweb1 src]# wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz
--2013-05-03 22:29:49-- http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz
正在解析主機 www.seas.upenn.edu... 158.130.68.91, 2607:f470:8:64:5ea5::9
Connecting to www.seas.upenn.edu|158.130.68.91|:80... 已鏈接。
已發出 HTTP 請求，正在等待迴應... 200 OK
長度：2694761 (2.6M) [application/x-gzip]
Saving to: `unison-2.40.102.tar.gz'
100%[======================================================>] 2,694,761 113K/s in 34s
2013-05-03 22:30:26 (76.8 KB/s) - `unison-2.40.102.tar.gz' saved [2694761/2694761]
[root@txzlkhdweb1 ocaml-3.12.1]# make install
if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi
if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi
............................
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo
install /usr/local/man/man1/ocamlbuild.1
[root@txzlkhdweb1 src]# cd unison-2.40.102
[root@txzlkhdweb1 unison-2.40.102]# make UISTYLE=text
ocamlc -o mkProjectInfo unix.cma str.cma mkProjectInfo.ml
./mkProjectInfo > Makefile.ProjectInfo
UISTYLE = text
Building for Unix
NATIVE = true
THREADS = false
STATIC = false
OSTYPE =
OSARCH =
ocamlopt: ubase/rx.mli ---> ubase/rx.cmi
ocamlopt -I lwt -I ubase -I system -I system/generic -I lwt/generic -c /usr/local/src/unison-2.40.102/ubase/rx.mli
.........................................
make[1]: Entering directory `/usr/local/src/unison-2.40.102'
if [ -f `which etags` ]; then \
etags *.mli */*.mli *.ml */*.ml */*.m *.c */*.c *.txt \
; fi
make[1]: Leaving directory `/usr/local/src/unison-2.40.102'
[root@txzlkhdweb1 unison-2.40.102]# make install
mv /root/bin//unison /tmp/unison-15209
mv: 沒法 stat 「/root/bin//unison」: 沒有那個文件或目錄
make: [doinstall] 錯誤 1 (忽略)
cp unison /root/bin/
cp: 沒法建立通常文件「/root/bin/」: 是一個目錄
make: *** [doinstall] 錯誤 1
#出現錯誤的緣由在與Unison默認是將文件Copy到/root/bin目錄，但Linux默認是沒有該目錄的，所以咱們須要將生成的可執行文件unison複製到系統的PATH目錄。
[root@txzlkhdweb1 unison-2.40.102]# cp unison /usr/local/bin
5.經過SSH登錄到遠程主機txzlkhdweb2 ，再將unison複製到slave的PATH目錄
[root@txzlkhdweb1 unison-2.40.102]# scp unison root@192.168.16.2:/usr/local/bin/unison
The authenticity of host '192.168.16.2 (192.168.16.2)' can't be established.
RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.16.2' (RSA) to the list of known hosts.
root@192.168.16.2s password:
unison 100% 1473KB 1.4MB/s 00:00
三．配置雙機ssh信任
因爲unison在遠程同步文件夾要登錄遠程服務器,所以要配置兩機互相信任
本例假設本地機爲：192.168.16.1(txzlkhdweb1)
遠程機：192.168.16.2(txzlkhdweb2)
1. 在兩臺機器上建立 RSA密鑰
如下操做要在本地機和遠程機上都執行一遍
（1）以 root 用戶登陸
（2）在 root 用戶的 主目錄內建立.ssh 目錄並設置正確的權限
[root@txzlkhdweb1 ~]# mkdir ~/.ssh
[root@txzlkhdweb1 ~]# chmod 700 ~/.ssh
[root@txzlkhdweb1 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
20:4e:83:71:d4:12:e1:b1:a3:d0:fe:b1:8e:9e:ed:a8 root@txzlkhdweb1
[root@txzlkhdweb1 ~]# ssh 192.168.16.1cat /root/.ssh/id_rsa.pub >> authorized_keys
The authenticity of host '192.168.16.1(192.168.16.1)' can't be established.
RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.16.1(RSA) to the list of known hosts.
root@192.168.16.1's password:
[root@txzlkhdweb1 ~]# ssh 192.168.16.2cat /root/.ssh/id_rsa.pub >> authorized_keys
root@192.168.16.2s password:
[root@txzlkhdweb1 ~]# scp authorized_keys 192.168.16.2:/root/.ssh/
root@192.168.16.2's password:
authorized_keys 100% 785 0.8KB/s 00:00
#重啓兩臺服務器的SSH服務
[root@txzlkhdweb1 ~]# /etc/init.d/sshd restart
[root@txzlkhdweb2~]# /etc/init.d/sshd restart
四．unison的使用
1.unison遠程使用
使用方法：
# unison <本地目錄> ssh://remotehostname(IP)/<遠程目錄的絕對路徑>
例如:
[root@txzlkhdweb1 ~]# /usr/local/bin/unison /root/test ssh://root@192.168.16.1//root/test
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
Nothing to do: replicas have not changed since last sync.
表示將本機的目錄/txzlkhdweb1/root/test和遠端主機的/txzlkhdweb2/root/test進行同步。通常的，須要兩臺機能ssh鏈接。
注意 在主機和目錄間又多加了一個 "/"
2.修改參數說明
[root@txzlkhdweb1 .unison]# vim ~/.unison/default.prf
#!/bin/bash
# this script is created by yourname.
# Unison preferences file
#root = /root/test
#root = ssh://root@192.168.16.2//root/test
#force =
#ignore =
batch = true
#repeat = 1
#retry = 3
owner = true
group = true
perms = -1
fastcheck = false
rsync = false
sshargs = -C
xferbycopying = true
log = true
logfile = /root/.unison/unison.log
若是服務器端 unison 可執行文件不在默認目錄下，甚至沒有 unison 命令（須要你編譯一個上傳到服務器），則須要使用 -servercmd 參數告訴要執行的服務器 unison 命令位置。
使用 -testserver 參數，則成功連接即退出，也不會去執行目錄的比較等後續操做。
• -servercmd xxx
告訴 unison， 服務器端的 unison 命令是什麼。參見上面的示例。
• -auto
接受缺省的動做，而後等待用戶確認是否執行。
• -batch
batch mode, 全自動模式，接受缺省動做，並執行。
• -ignore xxx
增長 xxx 到忽略列表中
• -ignorecase [true|false|default]
是否忽略文件名大小寫
• -follow xxx
是否支持對符號鏈接指向內容的同步
• owner = true (保持同步過來的文件屬主)
• group = true (保持同步過來的文件組信息)
• perms = -1 (保持同步過來的文件讀寫權限)
• repeat = 1 (間隔1秒後,開始新的一次同步檢查)
• retry = 3 (失敗重試)
• sshargs = -C (使用ssh的壓縮傳輸方式)
• xferbycopying = true
• -immutable xxx
不變目錄，掃描時能夠忽略
• -silent
安靜模式
• -times
同步修改時間
• -path xxx 參數
只同步 -path 參數指定的子目錄以及文件，而非整個目錄。-path 能夠屢次出現,例如
unison /home/username ssh://remotehost//home/username \
-path shared \
-path pub \
-path .netscape/bookmarks.html
五．安裝inotify 軟件
1.查看當前系統是否支持inotify
[root@txzlkhdweb1 test]# ls -l /proc/sys/fs/inotify
總計 0
-rw-r--r-- 1 root root 0 05-04 00:56 max_queued_events
-rw-r--r-- 1 root root 0 05-04 00:56 max_user_instances
-rw-r--r-- 1 root root 0 05-04 00:56 max_user_watches
2.下載inotify 源碼包
[root@txzlkhdweb1 ~]# wget https://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
正在解析主機 github.com... 192.30.252.129
Connecting to github.com|192.30.252.129|:443... 已鏈接。
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV CA-1':
Issued certificate not yet valid.
已發出 HTTP 請求，正在等待迴應... 302 Found
位置：http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz [跟隨至新的 URL]
--2013-05-04 01:23:55-- http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
正在解析主機 cloud.github.com... 54.240.164.110, 54.230.113.195, 54.240.164.29, ...
Connecting to cloud.github.com|54.240.164.110|:80... 已鏈接。
已發出 HTTP 請求，正在等待迴應... 200 OK
長度：358772 (350K) [null]
Saving to: `inotify-tools-3.14.tar.gz'
100%[=================================================>] 358,772 356K/s in 1.0s
2013-05-04 01:23:56 (356 KB/s) - `inotify-tools-3.14.tar.gz' saved [358772/358772]
3.編譯安裝inotify
[root@txzlkhdweb1 ~]# tar zxf inotify-tools-3.14.tar.gz
[root@txzlkhdweb1 ~]# cd inotify-tools-3.14
[root@txzlkhdweb1 inotify-tools-3.14]# make && make install
make all-recursive
make[1]: Entering directory `/root/inotify-tools-3.14'
.......
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/root/inotify-tools-3.14'
make[1]: Leaving directory `/root/inotify-tools-3.14'
4.編寫inotify 實時監控腳本
[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches
5000000000
[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events
327679
#事件監控腳本：
[root@txzlkhdweb1 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh
#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
LOCAL_DIR="/root/test/"
REMOTE_DIR="/root/test/"
REMOTE_USER="root"
INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"
UNISION="/usr/local/bin/unison"
#目前iplist.txt 文件中就一個地址 192.168.16.2
IP_LIST_FILE="/usr/local/inotify/iplist.txt"
ATTRIBUTES="modify,delete,create,attrib"
${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files
do
for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`
do
${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"
done
Done
[root@txzlkhdweb1 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh
[root@txzlkhdweb1 inotify-tools-3.14]# cd /usr/local/inotify
5.在txzlkhdweb2 上安裝一樣的inotify工具，步驟同上，此處省略
6.編寫同步腳本
[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches
5000000000
[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events
327679
#事件監控腳本：
[root@txzlkhdweb2 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh
#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
LOCAL_DIR="/root/test/"
REMOTE_DIR="/root/test/"
REMOTE_USER="root"
INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"
UNISION="/usr/local/bin/unison"
#目前iplist.txt 文件中就一個地址 192.168.16.1
IP_LIST_FILE="/usr/local/inotify/iplist.txt"
ATTRIBUTES="modify,delete,create,attrib"
${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files
do
for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`
do
${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"
done
Done
[root@txzlkhdweb2 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh
[root@txzlkhdweb2 inotify-tools-3.14]# cd /usr/local/inotify
[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &
7. 執行inotify 監控腳本
[root@txzlkhdweb1 inotify-tools-3.14]# nohup ./unison.sh &
[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &
8. 查看命令腳本執行程序是否在執行
[root@txzlkhdweb1 ~]# ps -ef | grep inotify
root 31194 31193 0 May18 pts/0 00:00:00 /usr/local/inotify-tools-3.14/bin/inotifywait -mrq --timefmt %d/%m/%y %H:%M --format %T %w%f -e modify,delete,create,attrib /root/test/
root 31469 24614 0 01:36 pts/0 00:00:00 grep inotify
9. 手工建立文件測試
[root@txzlkhdweb1 ~]# touch /root/test/f1.txt
[root@txzlkhdweb1 ~]# Contacting server...
Contacting server...
Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2//root/test]
Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2 //root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
new file ----> f1.txt
local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0
slave : absent
Propagating updates
UNISON 2.40.102 started propagating changes at 00:22:59.08 on 19 May 2013
[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test
Looking for changes
Waiting for changes from server
Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf
[END] Copying f1.txt
UNISON 2.40.102 finished propagating changes at 00:22:59.09 on 19 May 2013
Saving synchronizer state
Synchronization complete at 00:22:59 (1 item transferred, 0 skipped, 0 failed)
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
new file ----> f1.txt
local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0
slave : absent
Propagating updates
UNISON 2.40.102 started propagating changes at 00:23:00.09 on 19 May 2013
[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test
Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf
Failed: Destination updated during synchronization
The file f1.txt has been created
100% 00:00 ETAFailed [f1.txt]: Destination updated during synchronization
The file f1.txt has been created
UNISON 2.40.102 finished propagating changes at 00:23:00.09 on 19 May 2013
Saving synchronizer state
Synchronization incomplete at 00:23:00 (0 items transferred, 0 skipped, 1 failed)
failed: f1.txt
#這裏同步屢次問題暫時不知道如何解決，但願高手指點
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Reconciling changes
Nothing to do: replicas have not changed since last sync.
Looking for changes
Waiting for changes from server
Reconciling changes
Nothing to do: replicas have been changed only in identical ways since last sync.
參考網址：
http://ixdba.blog.51cto.com/2895551/584334
http://sofar.blog.51cto.com/353572/1271608
- 1. 採用 unison + inotify 實現文件實時雙向同步
- 2. inotify+unison雙向同步環境部署
- 3. centos Unison+Inotify雙向同步
- 4. unison+inotify實現目錄數據實時雙向同步
- 5. inotify+unison實現數據雙向實時同步
- 6. unison_inotify實現文件雙向同步
- 7. Rsync+unison雙向文件同步
- 8. Rsync + inotify 實現文件實時同步
- 9. inotify+rsync實現實時同步部署
- 10. UNISON+inotify數據的雙向實時同步
- 更多相關文章...
- • MySQL下載步驟詳解 - MySQL教程
- • PHP 實例 - AJAX 實時搜索 - PHP教程
- • ☆基於Java Instrument的Agent實現
- • Docker容器實戰(七) - 容器眼光下的文件系統
-
每一个你不满意的现在，都有一个你没有努力的曾经。