以前在IDC部署了Jumpserver堡壘機環境,做爲登錄線上服務器的統一入口。後面運行一段時間後,發現Jumpserver服務器的CPU負載使用率高達80%以上,主要是python程序對CPU的消耗比較大,因爲是單機部署,處於安全考慮,急須要部署一套Jumpserver雙機高可用環境,實現LB+HA的下降負載和故障轉移的目的。如下記錄了環境部署的過程:html
以下進行調整後,以前的jumpserver用戶名、祕鑰、密碼等信息都不會變,只須要將ssh鏈接的地址改成ssh端口負載均衡的vip地址便可! 也就是說對於用戶來講,只須要修改登陸ip地址,其餘的都不受影響! 1)環境準備 192.168.10.20 以前的單機版jumpserver,做爲master主機 192.168.10.21 新加的jumpserver,做爲slave從機 jumpserver機器的ssh端口統一調整爲8888 web訪問的80端口負載是7層負載,經過Nginx+keepalived實現,域名爲jump.kevin-inc.com ssh端口的負載是4層負載,也能夠經過nginx的stream實現,(我在線上用的nginx+keepalived負載層並無安裝stream模塊,爲了避免影響線上業務,另配置了lvs+keepalived) 2)部署jumpserver備機(192.168.10.21)的jumpserver環境 參考:http://www.cnblogs.com/kevingrace/p/5570279.html 3)配置jumpserver主機和備機的mysql主主同步環境(先將master主機的jumpserver庫數據同步到slave主機的mysql裏面) 參考這篇文章中的mysql主主同步配置:http://www.cnblogs.com/kevingrace/p/6710136.html 4)同步文件,使用rsync+inotify實時同步,或使用rsync+crontab短期定時同步(須要提早作192.168.10.20和192.168.10.21兩臺機器的ssh無密碼登錄的信任關係) 同步系統文件/etc/passwd、/etc/shaow、/etc/group文件 同步jumpserver相關用戶以及key文件:jumpserver/keys 同步用戶家目錄的home目錄 注意:爲了防止文件被強行覆蓋掉,這裏只能作單方向的文件同步,不能作雙向同步,不然會出現:在其中一臺機器的jumpserver界面裏建立好用戶後,可是在jumpserver服務器上的 /etc/passwd文件裏卻沒有該用戶信息,由於被對方機器的同步強行覆蓋掉了。 正確的作法: 在192.168.10.20機器上作rsync+crontab同步(10秒同步一次),另外一臺機器192.168.10.21不作同步; 登錄http://192.168.10.20的jumpserver界面建立用戶,這樣用戶信息很快就會被同步到另外一臺機器上了(注意:建立用戶要在http://192.168.10.20的jumpserver界面裏建立) [root@jumpserver01 ~]# crontab -l ......... * * * * * /usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 10;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 20;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 30;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 40;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 50;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/passwd root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * /usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 10;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 20;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 30;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 40;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 50;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/shadow root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * /usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 10;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 20;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 30;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 40;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * sleep 50;/usr/bin/rsync -e "ssh -p8888" -avpgolr /etc/group root@192.168.10.21:/etc/ > /dev/null 2>&1 * * * * * /usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * sleep 10;/usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * sleep 20;/usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * sleep 30;/usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * sleep 40;/usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * sleep 50;/usr/bin/rsync -e "ssh -p8888" -avpgolr /data/jumpserver/keys/ 192.168.10.21:/data/jumpserver/keys/ > /dev/null 2>&1 * * * * * /usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 * * * * * sleep 10;/usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 * * * * * sleep 20;/usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 * * * * * sleep 30;/usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 * * * * * sleep 40;/usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 * * * * * sleep 50;/usr/bin/rsync -e "ssh -p8888" -avpgolr /home/ 192.168.10.21:/home/ > /dev/null 2>&1 而後重啓兩臺機器的jumpserver服務。 5)web訪問的80端口負載均衡配置。訪問地址是http://jump.kevin-inc.com 參考:http://www.cnblogs.com/kevingrace/p/6138185.html [root@inner-lb01 ~]# cat /data/nginx/conf/vhosts/jump.kevin-inc.com.conf upstream jump-inc { server 192.168.10.20:80 max_fails=3 fail_timeout=10s; server 192.168.10.21:80 max_fails=3 fail_timeout=10s; } server { listen 80; server_name jump.kevin-inc.com; access_log /data/nginx/logs/jump.kevin-inc.com-access.log main; error_log /data/nginx/logs/jump.kevin-inc.com-error.log; location / { proxy_pass http://jump-inc; proxy_redirect off ; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header REMOTE-HOST $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_connect_timeout 300; proxy_send_timeout 300; proxy_read_timeout 600; proxy_buffer_size 256k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; proxy_temp_file_write_size 256k; proxy_next_upstream error timeout invalid_header http_502 http_503 http_504; proxy_max_temp_file_size 128m; #proxy_cache mycache; #proxy_cache_valid 200 302 1h; #proxy_cache_valid 301 1d; #proxy_cache_valid any 1m; } } 6)ssh登錄的8888端口的負載均衡配置 lvs+keepalived的配置參考:http://www.cnblogs.com/kevingrace/p/5570500.html 兩臺lvs配置以下(vip爲10.0.8.24) [root@jump-lvs01 ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id LVS_Master } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.24 } } virtual_server 192.168.10.24 8888 { delay_loop 6 lb_algo wrr lb_kind DR #nat_mask 255.255.255.0 persistence_timeout 600 protocol TCP real_server 192.168.10.20 8888 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 8888 } } real_server 192.168.10.21 8888 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 8888 } } } [root@jump-lvs02 ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id LVS_Backup } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.24 } } virtual_server 192.168.10.24 8888 { delay_loop 6 lb_algo wrr lb_kind DR #nat_mask 255.255.255.0 persistence_timeout 600 protocol TCP real_server 192.168.10.20 8888 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 8888 } } real_server 192.168.10.21 8888 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 8888 } } } 在xshell客戶端登錄堡壘機,堡壘機的地址能夠是192.168.10.20、192.168.10.2一、192.168.10.24,三個地址均可以。