nginx+keepalived+tomcat配置高可用web集羣

時間 2020-01-16

標籤 nginx+keepalived+tomcat nginx keepalived tomcat 配置可用 web 集羣欄目 Nginx 简体版

原文原文鏈接

nginx+keepalived+tomcat配置高可用web集羣javascript

基本架構：php

角色	ip	安裝軟件	做用	主機名
nginx主	192.168.247.129	nginx+keepalived	反向代理	nginxmaster.com
nginx備	192.168.247.130	nginx+keepalived	反向代理	nginxsalve.com
tomcat1	192.168.247.128	tomcat	web服務器	tomcat1.com
tomcat2	192.168.247.131	tomcat	web服務器	tomcat2.com
nfs主	192.168.247.132	nfs	低端共享存儲	nfsmaster.com
nfs備	192.168.247.133	nfs	低端共享存儲	nfsslave.com
虛擬ip	192.168.247.150

集羣部署：
css

tomcat安裝部署：
html

編譯安裝tomcat：（tomcat1，tomcat2）
前端

安裝JDK，運行java環境：（tomcat1，tomcat2）java

卸載centos自帶的JDK：node

rpm -qa | grep java
yum -y remove java-1.6.0-openjdk-1.6.0.0-1.7.b09.el5

下載安裝JDK：
linux

##下載地址：http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html 
##下載最新的（64位）：jdk-7u79-linux-x64.rpm

##在/usr下創建一個java目錄，以備將java程序安裝在此目錄下
mkdir /usr/java
cd /usr/local/src
rpm -ivh jdk-7u79-linux-x64.rpm        ##安裝後，在/usr/java目錄下就會生成一個新的目錄jdk1.7.0_79，該目錄下存放的是安裝後的文件

##設置環境變量：
vi /etc/profile  ##在最後添加：
export JAVA_HOME=/usr/java/jdk1.7.0_79
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

source /etc/profile

##驗證是否安裝成功：
java -version         ##出現以下則正常
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

安裝tomcat7：（tomcat1，tomcat2）nginx

軟件：apache-tomcat-7.0.29.tar.gz
cd /usr/local/src/
tar -zxf apache-tomcat-7.0.42.tar.gz
cp -a apache-tomcat-7.0.42 /usr/local/tomcat7/

/usr/local/tomcat7/bin/startup.sh       ##啓動tomcat，出現以下則正常
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar

##設置開機自動啓動

##訪問下：    ip:8080   看是否正常

配置tomcat：（tomcat1，tomcat2）
c++

配置server.xml文件：

cd /usr/local/tomcat7/conf/
vi server.xml
##找到：
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
##將其註釋掉

##添加以下（已優化）：
    <Connector  port="8080"   
  
                protocol="HTTP/1.1"  
  
                maxHttpHeaderSize="8192"  
  
                maxThreads="1000"  
  
                minSpareThreads="100"  
  
                maxSpareThreads="1000"  
  
                minProcessors="100"  
  
                maxProcessors="1000"  
  
                enableLookups="false"  
  
                compression="on"  
  
                compressionMinSize="2048"  
  
                compressableMimeType="text/html,text/xml,text/javascript,text/css,text/plain"  
  
                connectionTimeout="20000"  
  
                URIEncoding="utf-8"  
  
                acceptCount="1000"  
  
                redirectPort="8443"  
  
                disableUploadTimeout="true"/>
##配置優化： 
            優化JVM堆內存：
                vi /usr/local/tomcat7/bin/catalina.sh
                添加：JAVA_OPTS='-server -Xms1024m -Xmx2048m -XX:PermSize=256M -XX:MaxNewSize=256m -XX:MaxPermSize=256m'
		
	    修改日誌目錄：/data/tomcat7/logs
	        vi /usr/local/tomcat7/conf/server.xml
			<Valve className="org.apache.catalina.valves.AccessLogValve" directory="/data/tomcat7/logs"
			
		vi /usr/local/tomcat7/conf/logging.properties
			1catalina.org.apache.juli.FileHandler.level = FINE
			1catalina.org.apache.juli.FileHandler.directory = /data/tomcat7/logs
			1catalina.org.apache.juli.FileHandler.prefix = catalina.

			2localhost.org.apache.juli.FileHandler.level = FINE
			2localhost.org.apache.juli.FileHandler.directory = /data/tomcat7/logs
			2localhost.org.apache.juli.FileHandler.prefix = localhost.

			3manager.org.apache.juli.FileHandler.level = FINE
			3manager.org.apache.juli.FileHandler.directory = /data/tomcat7/logs
			3manager.org.apache.juli.FileHandler.prefix = manager.

			4host-manager.org.apache.juli.FileHandler.level = FINE
			4host-manager.org.apache.juli.FileHandler.directory = /data/tomcat7/logs
			4host-manager.org.apache.juli.FileHandler.prefix = host-manager.
		
		vi /usr/local/tomcat7/bin/catalina.sh
			if [ -z "$CATALINA_OUT" ] ; then
				CATALINA_OUT=/data/tomcat7/logs/catalina.out
			fi
	
	    修改日誌輸出格式：
	        vi /usr/local/tomcat7/conf/server.xml
			pattern="%{X-Real-IP}i %l %u %t &quot;%r&quot; %s %b" />            #nginx作反向代理，%{X-Real-IP}i 獲取用戶的真實ip              

                
                    ##找到：Valve標籤
                    ##將pattern參數的值修改成："%{X-Real-IP}i %l %u %t &quot;%r&quot; %s %b"        ##{X-Real-IP}i，獲取的是真實客戶端ip，不是代理機ip
                    ##每一個參數的含義：http://twb.iteye.com/blog/182100
                    ##nginx前端代理tomcat取真實客戶端IP：http://www.cnphp6.com/archives/60908

##從新啓動tomcat：
[root@tomcat1 ~]# /usr/local/tomcat7/bin/shutdown.sh 
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar
[root@tomcat1 ~]# /usr/local/tomcat7/bin/startup.sh 
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar

##tomcat性能優化：
參考：http://www.cnblogs.com/ggjucheng/archive/2013/04/16/3024731.html

反向代理的部署：

編譯安裝nginx：（主，備）

安裝nginx的依賴包：

yum -y install gcc gcc-c++ autoconf automake
yum -y install zlib zlib-devel openssl openssl-devel pcre-devel

編譯安裝pcre，使nginx支持rewrite重寫：

cd /usr/local/src/
wget http://sourceforge.net/projects/pcre/files/pcre/8.35/pcre-8.35.tar.gz
tar -zxf pcre-8.35.tar.gz
cd pcre-8.35
./configure
make
make install

安裝nginx：

cd /usr/local/src/
wget http://nginx.org/download/nginx-1.6.2.tar.gz
tar -zxf nginx-1.6.2.tar.gz 
cd nginx-1.6.2
./configure  --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_gzip_static_module --with-http_flv_module --with-http_gunzip_module --with-http_realip_module
make
make install
###注意--with-http_ssl_module，後面要用ssl

配置nginx：（主，備）

建立nginx用戶：

useradd -s /sbin/nologin nginx

修改文件打開數：

ulimit -n 65535
##設置開機自動啓動

生成ssl證書（模擬）（忽略）

##建立證書存放目錄：
cd /usr/local/nginx/
mkdir ca
cd ca
##建立服務器私鑰：
openssl genrsa -des3 -out server.key 1024
##建立簽名請求的證書（CSR）：
openssl req -new -key server.key -out server.csr
##在加載SSL支持的Nginx並使用上述私鑰時除去必須的口令
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
##最後標記證書使用上述私鑰和CSR：
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

生成ssl證書（模擬）

##建立證書存放目錄：
mkdir /usr/local/nginx/ca
cd /usr/local/nginx/ca
##生成密鑰文件key和csr文件：
openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr
##生成服務器認證文件crt：
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

修改配置文件：

# mv /usr/local/nginx/conf/nginx.conf /usr/local/nginx/conf/nginx.conf.old
# vi /usr/local/nginx/conf/nginx.conf
user nginx nginx;
worker_processes  2;
error_log  logs/error.log;
pid        logs/nginx.pid;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" $http_x_forwarded_for';
    
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;

    sendfile       on;

    #tcp_nopush     on;
    tcp_nodelay    on;

    client_header_timeout 30;
    client_body_timeout 30;
    send_timeout   30;
    client_max_body_size 100M;

    keepalive_timeout  60;

    proxy_connect_timeout  100;
    proxy_send_timeout  100;
    proxy_read_timeout  100;
    proxy_buffer_size  16k;
    proxy_buffers  4 32k;
    proxy_busy_buffers_size  64k;
    
    gzip  on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 2;
    gzip_types text/plain application/x-javascript text/css application/xml text/javascript;
    gzip_vary on;

    upstream web1{
      ip_hash;
      server 192.168.247.128:8080 max_fails=3 fail_timeout=30s;
      server 192.168.247.131:8080 max_fails=3 fail_timeout=30s;
    }


    server {
        listen       443;
        server_name  www.scj.com;
        root /opt/nginx/www/www.scj.com;
        index index.html index.htm index.jsp;

        ssl on;
        ssl_certificate /usr/local/nginx/ca/server.crt;
        ssl_certificate_key /usr/local/nginx/ca/server.key;

        access_log  logs/www.scj.com/access.log main;
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        location ~ .*\.jsp$ {
          proxy_pass http://web1;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_next_upstream http_502 http_504 error timeout invalid_header;
        }

        location ~ .*\.(gif|jpg|png|swf|bmp|jpeg)$ {
            expires 10d;
        }

        location ~ .*\.(js|css)?$ {
            expires 1h;
        }
    }

    #server {
       #listen 80 default;
       #server_name _;
       #return 500;
    #}
}

##建立日誌目錄和根目錄
mkdir /usr/local/nginx/logs/www.scj.com        ##日誌目錄
mkdir -p /opt/nginx/www/www.scj.com            ##建立根目錄

##啓動nginx：
# /usr/local/nginx/sbin/nginx

# ps -ef | grep nginx
root       7890      1  0 08:00 ?        00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
nginx      7891   7890  0 08:00 ?        00:00:00 nginx: worker process      
nginx      7892   7890  0 08:00 ?        00:00:00 nginx: worker process      
root       7896   7798  0 08:00 pts/0    00:00:00 grep nginx

# netstat  -tlnpa | grep nginx
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN      7890/nginx

##設置開機自動啓動

部署keepalived，實現自動切換：

安裝keepalived：（主，備）

cd /usr/local/src/
wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz
tar -zxf keepalived-1.2.15.tar.gz
cd keepalived-1.2.15
./configure --prefix=/usr/local/keepalived
make
make install

拷貝文件：（主，備）

cp -a /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
cp -a /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp -a /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp -a /usr/local/keepalived/sbin/keepalived /usr/sbin/

##注意： /etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 的路徑必定要正確，
##由於在執行/etc/init.d/keepalived這個啓動腳本時，會讀取/etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 這兩個文件

修改配置文件：

nginx主：

[root@nginxmaster ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.old
[root@nginxmaster ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
      732233048@qq.com
    }
    notification_email_from root@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id nginx_ha
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    #nopreempt

    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.247.150
    }
}

virtual_server 192.168.247.150 443 {
    delay_loop 6
    #lb_algo wrr
    #lb_kind DR
    #persistence_timeout 50
    protocol TCP
    real_server 192.168.247.129 443 {
        #weight 3
        notify_down /etc/keepalived/killkeepalived.sh
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 443
        }
    }
}

nginx備：

[root@nginxslave ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.old
[root@nginxslave ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
      732233048@qq.com
    }
    notification_email_from root@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id nginx_ha
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    #nopreempt

    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.247.150
    }
}

virtual_server 192.168.247.150 443 {
    delay_loop 6
    #lb_algo wrr
    #lb_kind DR
    #persistence_timeout 50
    protocol TCP
    real_server 192.168.247.130 443 {
        #weight 3
        notify_down /etc/keepalived/killkeepalived.sh
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 443
        }
    }
}

建立/etc/keepalived/killkeepalived.sh腳本：（主，備）

vi /etc/keepalived/killkeepalived.sh
#!/bin/bash
# check nginx server status

nginx_status=`netstat  -tlnpa | grep 443 | wc -l`

if [ $nginx_status -eq 0 ];then
  /usr/local/nginx/sbin/nginx
  sleep 1
  nginx_status=`netstat  -tlnpa | grep 443 | wc -l`
  if [ $nginx_status -eq 0 ];then
    /etc/init.d/keepalived stop
  fi
fi

chmod 755 /etc/keepalived/killkeepalived.sh

修改keepalived的日誌文件：（主，備）

參考：http://chenwenming.blog.51cto.com/327092/745316

說明：

##centos6.3以後的syslog更名叫rsyslog了，默認在 /etc/rsyslog.conf

修改/etc/sysconfig/keepalived：

vi /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp               -P    Only run with VRRP subsystem.
# --check              -C    Only run with Health-checker subsystem.
# --dont-release-vrrp  -V    Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs  -I    Dont remove IPVS topology on daemon stop.
# --dump-conf          -d    Dump the configuration data.
# --log-detail         -D    Detailed log messages.
# --log-facility       -S    0-7 Set local syslog facility (default=LOG_DAEMON)
#

#KEEPALIVED_OPTIONS="-D"
KEEPALIVED_OPTIONS="-D -d -S 0"               ##在最後添加此行

修改/etc/rsyslog.conf:

vi /etc/rsyslog.conf
##在最後添加此行：
local0.*                                                /var/log/keepalived.log

## /etc/init.d/rsyslog restart
Shutting down system logger:                               [  OK  ]
Starting system logger:                                    [  OK  ]

啓動keepalived：

nginx主：

[root@nginxmaster ~]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]

##查看日誌：
Sep 23 20:16:27 nginxmaster Keepalived[2909]: Starting Keepalived v1.2.15 (09/23,2015)
Sep 23 20:16:27 nginxmaster Keepalived[2910]: Starting Healthcheck child process, pid=2912
Sep 23 20:16:27 nginxmaster Keepalived[2910]: Starting VRRP child process, pid=2913
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Netlink reflector reports IP 192.168.247.129 added
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Netlink reflector reports IP fe80::20c:29ff:fee0:d236 added
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering Kernel netlink reflector
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering Kernel netlink command channel
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering gratuitous ARP shared channel
Sep 23 20:16:27 nginxmaster kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Sep 23 20:16:27 nginxmaster kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Sep 23 20:16:27 nginxmaster kernel: IPVS: ipvs loaded.
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP 192.168.247.129 added
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP fe80::20c:29ff:fee0:d236 added
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Registering Kernel netlink reflector
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Registering Kernel netlink command channel
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Configuration is using : 63272 Bytes
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Using LinkWatch kernel netlink reflector...
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Configuration is using : 11705 Bytes
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: IPVS: Scheduler or persistence engine not found
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: IPVS: No such process
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Using LinkWatch kernel netlink reflector...
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Activating healthchecker for service [192.168.247.129]:443
Sep 23 20:16:28 nginxmaster kernel: IPVS: Scheduler module ip_vs_ not found
Sep 23 20:16:31 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 23 20:16:32 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP 192.168.247.150 added
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 23 20:16:37 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150


##設置開機自動啓動
chkconfig keepalived on

nginx備：

[root@nginxslave ~]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]

##查看日誌：
Sep 24 09:43:29 nginxslave Keepalived[9481]: Starting Keepalived v1.2.15 (09/24,2015)
Sep 24 09:43:29 nginxslave Keepalived[9482]: Starting Healthcheck child process, pid=9484
Sep 24 09:43:29 nginxslave Keepalived[9482]: Starting VRRP child process, pid=9485
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Netlink reflector reports IP 192.168.247.130 added
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Netlink reflector reports IP fe80::20c:29ff:fe1e:8eab added
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering Kernel netlink reflector
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering Kernel netlink command channel
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering gratuitous ARP shared channel
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Configuration is using : 63262 Bytes
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Using LinkWatch kernel netlink reflector...
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Netlink reflector reports IP 192.168.247.130 added
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Netlink reflector reports IP fe80::20c:29ff:fe1e:8eab added
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Registering Kernel netlink reflector
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Registering Kernel netlink command channel
Sep 24 09:43:29 nginxslave kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Sep 24 09:43:29 nginxslave kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Sep 24 09:43:29 nginxslave kernel: IPVS: ipvs loaded.
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Configuration is using : 11695 Bytes
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: IPVS: Scheduler or persistence engine not found
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: IPVS: No such process
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Using LinkWatch kernel netlink reflector...
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Activating healthchecker for service [192.168.247.130]:443
Sep 24 09:43:29 nginxslave kernel: IPVS: Scheduler module ip_vs_ not found


##設置開機自動啓動
chkconfig keepalived on

查看vip綁定在哪臺機器上：

[root@nginxmaster ~]#ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:e0:d2:36 brd ff:ff:ff:ff:ff:ff
    inet 192.168.247.129/24 brd 192.168.247.255 scope global eth0
    inet 192.168.247.150/32 scope global eth0
    inet6 fe80::20c:29ff:fee0:d236/64 scope link 
       valid_lft forever preferred_lft forever

[root@nginxslave ~]#ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:8e:ab brd ff:ff:ff:ff:ff:ff
    inet 192.168.247.130/24 brd 192.168.247.255 scope global eth0
    inet6 fe80::20c:29ff:fe1e:8eab/64 scope link 
       valid_lft forever preferred_lft forever

##由上發現vip150綁定在nginxmaster，正常

測試：

測試一：把nginx主的nginx服務stop掉

[root@nginxmaster ~]# /usr/local/nginx/sbin/nginx -s stop

##查看日誌：
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: TCP connection to [192.168.247.129]:443 failed !!!
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Executing [/etc/keepalived/killkeepalived.sh] for service [192.168.247.129]:443 in VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Lost quorum 1-0=1 > 0 for VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: SMTP alert successfully sent.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: TCP connection to [192.168.247.129]:443 success.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Adding service [192.168.247.129]:443 to VS [192.168.247.150]:443
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Gained quorum 1+0=1 <= 1 for VS [192.168.247.150]:443
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: SMTP alert successfully sent

##keepalived會先把129移除，而後執行/etc/keepalived/killkeepalived.sh腳本，nginx服務正常啓動，又把129加入到集羣中

測試二：把nginx主的keepalived服務stop掉

[root@nginxmaster ~]# /etc/init.d/keepalived stop
Stopping keepalived:                                       [  OK  ]

##查看nginx主的日誌：
Sep 23 23:41:05 nginxmaster Keepalived[3988]: Stopping Keepalived v1.2.15 (09/23,2015)
Sep 23 23:41:05 nginxmaster Keepalived_vrrp[3991]: VRRP_Instance(VI_1) sending 0 priority
Sep 23 23:41:05 nginxmaster Keepalived_vrrp[3991]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: Netlink reflector reports IP 192.168.247.150 removed
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: IPVS: No such service

##查看nginx備日誌：
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: TCP connection to [192.168.247.130]:443 success.
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Adding service [192.168.247.130]:443 to VS [192.168.247.150]:443
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: IPVS: Service not defined
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Gained quorum 1+0=1 <= 1 for VS [192.168.247.150]:443
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: SMTP alert successfully sent.
Sep 24 12:58:05 nginxslave dhclient[906]: DHCPREQUEST on eth0 to 192.168.247.254 port 67 (xid=0x10a73226)
Sep 24 12:58:05 nginxslave dhclient[906]: DHCPACK from 192.168.247.254 (xid=0x10a73226)
Sep 24 12:58:07 nginxslave dhclient[906]: bound to 192.168.247.130 -- renewal in 705 seconds.
Sep 24 13:05:23 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 24 13:05:24 nginxslave Keepalived_healthcheckers[10012]: Netlink reflector reports IP 192.168.247.150 added
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:05:29 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150

##nginx備變成master，綁定vip 150

測試三：把nginx主的nginx服務stop，並再也不讓它成功啓動

##刪掉日誌目錄，nginx便不能啓動了
[root@nginxmaster logs]# rm -rf /usr/local/nginx/logs/www.scj.com/

##關閉nginx服務
[root@nginxmaster ~]# /usr/local/nginx/sbin/nginx -s stop

##查看nginx主日誌：
Sep 23 23:44:33 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: TCP connection to [192.168.247.129]:443 failed !!!
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: IPVS: Service not defined
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Executing [/etc/keepalived/killkeepalived.sh] for service [192.168.247.129]:443 in VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Lost quorum 1-0=1 > 0 for VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: SMTP alert successfully sent.
Sep 23 23:44:39 nginxmaster Keepalived[4076]: Stopping Keepalived v1.2.15 (09/23,2015)
Sep 23 23:44:39 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) sending 0 priority
Sep 23 23:44:39 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 23 23:44:39 nginxmaster Keepalived_healthcheckers[4078]: Netlink reflector reports IP 192.168.247.150 removed
Sep 23 23:44:39 nginxmaster Keepalived_healthcheckers[4078]: IPVS: No such service

##nginx服務不能啓動了，而後把keepalived服務關掉

##查看nginx備日誌
Sep 24 13:08:57 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:08:58 nginxslave Keepalived_healthcheckers[10012]: Netlink reflector reports IP 192.168.247.150 added
Sep 24 13:09:03 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:09:52 nginxslave dhclient[906]: DHCPREQUEST on eth0 to 192.168.247.254 port 67 (xid=0x10a73226)
Sep 24 13:09:52 nginxslave dhclient[906]: DHCPACK from 192.168.247.254 (xid=0x10a73226)
Sep 24 13:09:54 nginxslave dhclient[906]: bound to 192.168.247.130 -- renewal in 800 seconds.

##nginx備變爲master，並綁定vip 150

測試四：把nginx備的nginx服務stop

##查看nginx備的nginx服務會不會又正常啓動

測試五：把nginx備的keepalived服務stop

##nginx主不會有任何變化

測試六：把nginx備的nginx服務stop掉，並再也不讓它正常啓動

##nginx備的keepalived服務會被關掉
##nginx主不會有任何變化

注意：

修復後，必定要記得把keepalived服務啓動起來

nfs部署：

http://732233048.blog.51cto.com/9323668/1669417

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。