Keepalived+Nginx+Tomcat 實現高可用Web集羣(轉)

時間  2019-11-09
標籤 keepalived+nginx+tomcat keepalived nginx tomcat 實現 可用 web 集羣 欄目 負載均衡 简体版
原文   原文鏈接

Keepalived+Nginx+Tomcat 實現高可用Web集羣

96 
溯水心生 
2018.01.08 20:28* 字數 1382 閱讀 965評論 1
 
集羣規劃圖片

1、Nginx的安裝過程

1.下載Nginx安裝包，安裝依賴環境包

(1)安裝 C++編譯環境

yum  -y install gcc   #C++

(2)安裝pcre

yum  -y install pcre-devel

(3)安裝zlib

yum  -y install  zlib-devel

(4)安裝Nginx

定位到nginx 解壓文件位置，執行編譯安裝命令nginx

[root@localhost nginx-1.12.2]# pwd /usr/local/nginx/nginx-1.12.2 [root@localhost nginx-1.12.2]# ./configure && make && make install

(5)啓動Nginx

安裝完成後先尋找那安裝完成的目錄位置web

[root@localhost nginx-1.12.2]# whereis nginx nginx: /usr/local/nginx [root@localhost nginx-1.12.2]#

進入Nginx子目錄sbin啓動Nginxvim

[root@localhost sbin]# ls nginx [root@localhost sbin]# ./nginx & [1] 5768 [root@localhost sbin]#

查看Nginx是否啓動tomcat


 
Niginx啓動成功截圖

或經過進程查看Nginx啓動狀況ruby

[root@localhost sbin]# ps -aux|grep nginx root 5769 0.0 0.0 20484 608 ? Ss 14:03 0:00 nginx: master process ./nginx nobody 5770 0.0 0.0 23012 1620 ? S 14:03 0:00 nginx: worker process root 5796 0.0 0.0 112668 972 pts/0 R+ 14:07 0:00 grep --color=auto nginx [1]+ 完成 ./nginx [root@localhost sbin]#

到此Nginx安裝完成並啓動成功。bash

(6)Nginx快捷啓動和開機啓動配置

編輯Nginx快捷啓動腳本【注意Nginx安裝路徑，須要根據本身的NGINX路徑進行改動】服務器

[root@localhost init.d]# vim /etc/rc.d/init.d/nginx 
  1 #!/bin/sh
  2 #
  3 # nginx - this script starts and stops the nginx daemon
  4 #
  5 # chkconfig: - 85 15
  6 # description: Nginx is an HTTP(S) server, HTTP(S) reverse \
  7 # proxy and IMAP/POP3 proxy server
  8 # processname: nginx
  9 # config: /etc/nginx/nginx.conf
 10 # config: /usr/local/nginx/conf/nginx.conf
 11 # pidfile: /usr/local/nginx/logs/nginx.pid
 12  
 13 # Source function library.
 14 . /etc/rc.d/init.d/functions
 15  
 16 # Source networking configuration.
 17 . /etc/sysconfig/network
 18  
 19 # Check that networking is up.
 20 [ "$NETWORKING" = "no" ] && exit 0
 21 nginx="/usr/local/nginx/sbin/nginx"
 22 prog=$(basename $nginx)
 23 NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"
 24 [ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
 25 lockfile=/var/lock/subsys/nginx
 26  
 27 make_dirs() {
 28     # make required directories
 29     user=`$nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -`
 30     if [ -z "`grep $user /etc/passwd`" ]; then
 31     useradd -M -s /bin/nologin $user
 32     fi
 33     options=`$nginx -V 2>&1 | grep 'configure arguments:'`
 34     for opt in $options; do
 35     if [ `echo $opt | grep '.*-temp-path'` ]; then
 36     value=`echo $opt | cut -d "=" -f 2`
 37     if [ ! -d "$value" ]; then
 38     # echo "creating" $value
 39     mkdir -p $value && chown -R $user $value
 40     fi
 41     fi
 42     done
 43 }
 44  
 45 start() {
 46     [ -x $nginx ] || exit 5
 47     [ -f $NGINX_CONF_FILE ] || exit 6
 48     make_dirs
 49     echo -n $"Starting $prog: "
 50     daemon $nginx -c $NGINX_CONF_FILE
 51     retval=$?
 52     echo
 53     [ $retval -eq 0 ] && touch $lockfile
 54     return $retval
 55 }
 56  
 57 stop() {
 58     echo -n $"Stopping $prog: "
 59     killproc $prog -QUIT
 60     retval=$?
 61     echo
 62     [ $retval -eq 0 ] && rm -f $lockfile
 63     return $retval
 64 }
 65  
 66 restart() {
 67     #configtest || return $?
 68     stop
 69     sleep 1
 70     start
 71 }
 72  
 73 reload() {
 74     #configtest || return $?
 75     echo -n $"Reloading $prog: "
 76     killproc $nginx -HUP
 77     RETVAL=$?
 78     echo
 79 }
 80  
 81 force_reload() {
 82     restart
 83 }
 84  
 85 configtest() {
 86     $nginx -t -c $NGINX_CONF_FILE
 87 }
 88  
 89 rh_status() {
 90     status $prog
 91 }
 92  
 93 rh_status_q() {
 94     rh_status >/dev/null 2>&1
 95 }
 96  
 97 case "$1" in
 98 start)
 99 rh_status_q && exit 0
100 $1
101 ;;
102 stop)
103  
104 rh_status_q || exit 0
105 $1
106 ;;
107 restart|configtest)
108 $1
109 ;;
110 reload)
111 rh_status_q || exit 7
112 $1
113 ;;
114 force-reload)
115 force_reload
116 ;;
117 status)
118 rh_status
119 ;;
120 condrestart|try-restart)
121 rh_status_q || exit 0
122 ;;
123 *)
124 echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
125 exit 2
126 esac

 

爲啓動腳本受權 並加入開機啓動負載均衡

[root@localhost init.d]# chmod -R 777 /etc/rc.d/init.d/nginx [root@localhost init.d]# chkconfig nginx

啓動Nginxjsp

[root@localhost init.d]# ./nginx start

將Nginx加入系統環境變量oop

[root@localhost init.d]# echo 'export PATH=$PATH:/usr/local/nginx/sbin'>>/etc/profile && source /etc/profile

Nginx命令 [ service nginx (start|stop|restart) ]

[root@localhost init.d]# service nginx start Starting nginx (via systemctl): [ 肯定 ]

Tips:快捷命令

service nginx (start|stop|restart)

2、KeepAlived安裝和配置

1.安裝Keepalived依賴環境

yum install -y popt-devel     
yum install  -y ipvsadm
yum install -y libnl*
yum install -y libnf*
yum install -y openssl-devel

2.編譯Keepalived並安裝

[root@localhost keepalived-1.3.9]# ./configure [root@localhost keepalived-1.3.9]# make && make install

3.將Keepalive 安裝成系統服務

[root@localhost etc]# mkdir /etc/keepalived [root@localhost etc]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

手動複製默認的配置文件到默認路徑

[root@localhost etc]# mkdir /etc/keepalived [root@localhost etc]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ [root@localhost etc]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

爲keepalived 建立軟連接

[root@localhost sysconfig]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/

設置Keepalived開機自啓動

[root@localhost sysconfig]# chkconfig keepalived on 注意：正在將請求轉發到「systemctl enable keepalived.service」。 Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service

啓動Keepalived服務

[root@localhost keepalived]# keepalived -D -f /etc/keepalived/keepalived.conf

關閉Keepalived服務

[root@localhost keepalived]# killall keepalived

3、集羣規劃和搭建

 
集羣規劃圖片

環境準備：

CentOS 7.2 

Keepalived   Version 1.4.0 - December 29, 2017 

Nginx           Version: nginx/1.12.2 

Tomcat         Version:8

集羣規劃清單

虛擬機 IP 說明
Keepalived+Nginx1[Master] 192.168.43.101 Nginx Server 01
Keeepalived+Nginx[Backup] 192.168.43.102 Nginx Server 02
Tomcat01 192.168.43.103 Tomcat Web Server01
Tomcat02 192.168.43.104 Tomcat Web Server02
VIP 192.168.43.150 虛擬漂移IP

1.更改Tomcat默認歡迎頁面，用於標識切換Web

更改TomcatServer01 節點ROOT/index.jsp 信息，加入TomcatIP地址，並加入Nginx值，即修改節點192.168.43.103信息以下：

<div id="asf-box"> <h1>${pageContext.servletContext.serverInfo}(192.168.224.103)<%=request.getHeader("X-NGINX")%></h1> </div>

更改TomcatServer02 節點ROOT/index.jsp信息，加入TomcatIP地址，並加入Nginx值，即修改節點192.168.43.104信息以下：

<div id="asf-box"> <h1>${pageContext.servletContext.serverInfo}(192.168.224.104)<%=request.getHeader("X-NGINX")%></h1> </div>

2.啓動Tomcat服務，查看Tomcat服務IP信息，此時Nginx未啓動，所以request-header沒有Nginx信息。

 
Tomcat啓動信息

3.配置Nginx代理信息

1.配置Master節點[192.168.43.101]代理信息

upstream tomcat {
   server 192.168.43.103:8080 weight=1; server 192.168.43.104:8080 weight=1; } server{ location / { proxy_pass http://tomcat; proxy_set_header X-NGINX "NGINX-1"; } #......其餘省略 }

2.配置Backup節點[192.168.43.102]代理信息

upstream tomcat {
    server 192.168.43.103:8080 weight=1; server 192.168.43.104:8080 weight=1; } server{ location / { proxy_pass http://tomcat; proxy_set_header X-NGINX "NGINX-2"; } #......其餘省略 }

3.啓動Master 節點Nginx服務

[root@localhost init.d]# service nginx start Starting nginx (via systemctl): [ 肯定 ]

此時訪問 192.168.43.101 能夠看到103和104節點Tcomat交替顯示，說明Nginx服務已經將請求負載到了2臺tomcat上。


 
Nginx 負載效果

4.同理配置Backup[192.168.43.102] Nginx信息，啓動Nginx後，訪問192.168.43.102後能夠看到Backup節點已起到負載的效果。

 
Backup負載效果

4.配置Keepalived 腳本信息

1.在Master節點和Slave節點 /etc/keepalived目錄下添加check_nginx.sh 文件，用於檢測Nginx的存活情況，添加keepalived.conf文件

check_nginx.sh文件信息以下：

#!/bin/bash #時間變量，用於記錄日誌 d=`date --date today +%Y%m%d_%H:%M:%S` #計算nginx進程數量 n=`ps -C nginx --no-heading|wc -l` #若是進程爲0，則啓動nginx，而且再次檢測nginx進程數量， #若是還爲0，說明nginx沒法啓動，此時須要關閉keepalived if [ $n -eq "0" ]; then /etc/rc.d/init.d/nginx start n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi

添加完成後，爲check_nginx.sh 文件受權，便於腳本得到執行權限。

[root@localhost keepalived]# chmod -R 777 /etc/keepalived/check_nginx.sh

2.在Master 節點 /etc/keepalived目錄下，添加keepalived.conf 文件，具體信息以下：

vrrp_script chk_nginx {  
 script "/etc/keepalived/check_nginx.sh" //檢測nginx進程的腳本 interval 2 weight -20 } global_defs { notification_email { //能夠添加郵件提醒 } } vrrp_instance VI_1 { state MASTER #標示狀態爲MASTER 備份機爲BACKUP interface ens33 #設置實例綁定的網卡(ip addr查看，須要根據我的網卡綁定) virtual_router_id 51 #同一實例下virtual_router_id必須相同 mcast_src_ip 192.168.43.101 priority 250 #MASTER權重要高於BACKUP 好比BACKUP爲240 advert_int 1 #MASTER與BACKUP負載均衡器之間同步檢查的時間間隔，單位是秒 nopreempt #非搶佔模式 authentication { #設置認證 auth_type PASS #主從服務器驗證方式 auth_pass 123456 } track_script { check_nginx } virtual_ipaddress { #設置vip 192.168.43.150 #能夠多個虛擬IP，換行便可 } }

3.在Backup節點 etc/keepalived目錄下添加 keepalived.conf 配置文件

信息以下：

vrrp_script chk_nginx {  
 script "/etc/keepalived/check_nginx.sh" //檢測nginx進程的腳本 interval 2 weight -20 } global_defs { notification_email { //能夠添加郵件提醒 } } vrrp_instance VI_1 { state BACKUP #標示狀態爲MASTER 備份機爲BACKUP interface ens33 #設置實例綁定的網卡(ip addr查看) virtual_router_id 51 #同一實例下virtual_router_id必須相同 mcast_src_ip 192.168.43.102 priority 240 #MASTER權重要高於BACKUP 好比BACKUP爲240 advert_int 1 #MASTER與BACKUP負載均衡器之間同步檢查的時間間隔，單位是秒 nopreempt #非搶佔模式 authentication { #設置認證 auth_type PASS #主從服務器驗證方式 auth_pass 123456 } track_script { check_nginx } virtual_ipaddress { #設置vip 192.168.43.150 #能夠多個虛擬IP，換行便可 } }

Tips:關於配置信息的幾點說明

  • state - 主服務器需配成MASTER，從服務器需配成BACKUP
  • interface - 這個是網卡名，我使用的是VM12.0的版本，因此這裏網卡名爲ens33
  • mcast_src_ip - 配置各自的實際IP地址
  • priority - 主服務器的優先級必須比從服務器的高，這裏主服務器配置成250，從服務器配置成240
  • virtual_ipaddress - 配置虛擬IP（192.168.43.150）
  • authentication - auth_pass主從服務器必須一致，keepalived靠這個來通訊
  • virtual_router_id - 主從服務器必須保持一致

5.集羣高可用(HA)驗證

  • Step1 啓動Master機器的Keepalived和 Nginx服務
[root@localhost keepalived]# keepalived -D -f /etc/keepalived/keepalived.conf [root@localhost keepalived]# service nginx start

查看服務啓動進程

[root@localhost keepalived]# ps -aux|grep nginx root 6390 0.0 0.0 20484 612 ? Ss 19:13 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 6392 0.0 0.0 23008 1628 ? S 19:13 0:00 nginx: worker process root 6978 0.0 0.0 112672 968 pts/0 S+ 20:08 0:00 grep --color=auto nginx

查看Keepalived啓動進程

[root@localhost keepalived]# ps -aux|grep keepalived root 6402 0.0 0.0 45920 1016 ? Ss 19:13 0:00 keepalived -D -f /etc/keepalived/keepalived.conf root 6403 0.0 0.0 48044 1468 ? S 19:13 0:00 keepalived -D -f /etc/keepalived/keepalived.conf root 6404 0.0 0.0 50128 1780 ? S 19:13 0:00 keepalived -D -f /etc/keepalived/keepalived.conf root 7004 0.0 0.0 112672 976 pts/0 S+ 20:10 0:00 grep --color=auto keepalived

使用 ip add 查看虛擬IP綁定狀況，如出現192.168.43.150 節點信息則綁定到Master節點

[root@localhost keepalived]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:91:bf:59 brd ff:ff:ff:ff:ff:ff inet 192.168.43.101/24 brd 192.168.43.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.43.150/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::9abb:4544:f6db:8255/64 scope link valid_lft forever preferred_lft forever inet6 fe80::b0b3:d0ca:7382:2779/64 scope link tentative dadfailed valid_lft forever preferred_lft forever inet6 fe80::314f:5fe7:4e4b:64ed/64 scope link tentative dadfailed valid_lft forever preferred_lft forever 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000 link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000 link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff
  • Step 2 啓動Backup節點Nginx服務和Keepalived服務，查看服務啓動狀況，如Backup節點出現了虛擬IP，則Keepalvied配置文件有問題，此狀況稱爲腦裂。
[root@localhost keepalived]# clear [root@localhost keepalived]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:14:df:79 brd ff:ff:ff:ff:ff:ff inet 192.168.43.102/24 brd 192.168.43.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::314f:5fe7:4e4b:64ed/64 scope link valid_lft forever preferred_lft forever 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000 link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000 link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff
  • Step 3 驗證服務

    瀏覽並屢次強制刷新地址： http://192.168.43.150 ,能夠看到103和104屢次交替顯示，並顯示Nginx-1,則代表 Master節點在進行web服務轉發。
  • Step 4 關閉Master keepalived服務和Nginx服務，訪問Web服務觀察服務轉移狀況
[root@localhost keepalived]# killall keepalived [root@localhost keepalived]# service nginx stop

此時強制刷新192.168.43.150發現 頁面交替顯示103和104並顯示Nginx-2 ，VIP已轉移到192.168.43.102上，已證實服務自動切換到備份節點上。

  • Step 5 啓動Master Keepalived 服務和Nginx服務

    此時再次驗證發現，VIP已被Master從新奪回，並頁面交替顯示 103和104，此時顯示Nginx-1

4、Keepalived搶佔模式和非搶佔模式

keepalived的HA分爲搶佔模式和非搶佔模式，搶佔模式即MASTER從故障中恢復後，會將VIP從BACKUP節點中搶佔過來。非搶佔模式即MASTER恢復後不搶佔BACKUP升級爲MASTER後的VIP。 

非搶佔模式配置：

  • 1> 在vrrp_instance塊下兩個節點各增長了nopreempt指令，表示不爭搶vip
  • 2> 節點的state都爲BACKUP兩個keepalived節點都啓動後，默認都是BACKUP狀態，雙方在發送組播信息後，會根據優先級來選舉一個MASTER出來。因爲二者都配置了nopreempt，因此MASTER從故障中恢復後，不會搶佔vip。這樣會避免VIP切換可能形成的服務延遲。
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程