天涯LVS部署

項目地址 https://code.google.com/p/tianyalvs/

添加LVS互爲主備修改版：http://pan.baidu.com/s/1dEbHI0L

示意圖：

           +---------+

         |      |  

           |      | LAMP + certmaster master + func master + rsync + ssh從

           |      |

           +---------+

              |

         +--------------+     

      +----------+  +----------+     

      |certmaster|  |certmaster|

      |  func  |  |  func  |

      |  minion |  |  minion |

      | rsync主 |  | rsync主 |

      +----------+  +----------+

php的版本要求5.2

軟件下載：

wget http://sourceforge.net/projects/mcrypt/files/Libmcrypt/2.5.8/libmcrypt-2.5.8.tar.gz

wget http://sourceforge.net/projects/mcrypt/files/MCrypt/2.6.8/mcrypt-2.6.8.tar.gz

wget http://sourceforge.net/projects/mhash/files/mhash/0.9.9.9/mhash-0.9.9.9.tar.gz

架構：

-- LVS web(func minion、certmaster master、rsync slave)

軟件安裝：LAMP、func、certmaster、rsync、keepavlied（主要是使用genhash命令）、mysql-python

    192.168.1.211

    --hostname：func-master

--LVS (func slave、certmaster minion、rsync master)

軟件安裝：lvs、keepavlied、func、certmaster、rsync

    192.168.1.13

    --hostname：lvs1

    192.168.1.14

    --hostname：lvs2

--realserer（func slave）

軟件安裝：func、certmaster、rsync

    192.168.1.200

    --hostname：lvs1

本次環境：CentOS 6.6 x64

1、LVS web

一、LAMP安裝

yum -y install mysql mysql-server mysql-devel

1.1 apache:

useradd -M -s /sbin/nologin apache

./configure --prefix=/usr/local/apache --with-mysql=/usr --enable-cgi --enable-ssl --enable-so --enable-rewrite --enable-modules=all --enable-mods-shared=all --enable-cache --enable-disk-cache --enable-mem-cache --enable-deflate=shared

make; make install

1.2 安裝php

安裝依賴：

cd /usr/local/src

tar -zxvf libmcrypt-2.5.8.tar.gz

cd libmcrypt-2.5.8

./configure; make; make install

tar -zxvf mhash-0.9.9.9.tar.gz; cd mhash-0.9.9.9; ./configure; make; make install

tar zxvf mcrypt-2.6.8.tar.gz; cd mcrypt-2.6.8; export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH; ./configure; make; make install

mkdir /usr/lib/mysql;  ln -s /usr/lib64/mysql/libmysqlclient_r.so /usr/lib/mysql/ 

./configure --prefix=/usr/local/php --with-config-file-path=/usr/local/php/etc --with-config-file-scan-dir=/usr/local/php/etc/php.d --with-mysql=/usr --with-mysqli=/usr/bin/mysql_config-64 --with-apxs=/usr/local/apache/bin/apxs --enable-mbstring --with-free

make; make install

centos 7.2 + mariadb + httpd-2.2.31下php5.2編譯

ln -s /usr/lib64/mysql/libmysqlclient* /usr/lib/mysql/

./configure --prefix=/usr/local/php --with-config-file-path=/usr/local/php/etc --with-config-file-scan-dir=/usr/local/php/etc/php.d --with-mysql=/usr --with-mysqli=/usr/bin/mysql_config --with-apxs2=/usr/local/apache/bin/apxs --enable-mbstring --with-free

php.ini：

short_open_tag = On

date.timezone = "aisa/shanghai"

二、安裝certmaster、func依賴

yum -y install gcc gcc-g++ openssl openssl-devel pytho python-devel rrdtool rsync bc  keepalived

安裝MySQL-python

wget https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz

wget https://pypi.python.org/packages/source/s/setuptools/setuptools-19.2.tar.gz

tar -zxvf setuptools-19.2.tar.gz; cd setuptools-19.2; python setup.py install; cd ../

tar -zxvf pip-7.1.2.tar.gz; cd pip-7.1.2; python setup.py install;

pip install MySQL-python

三、安裝certmaster、func

tar -zxvf pyOpenSSL-0.9.tar.gz

cd pyOpenSSL-0.9

/usr/local/bin/python setup.py install

tar -zxvf certmaster-0.25.tar.gz

cd certmaster-0.25

/usr/local/bin/python setup.py install

tar -zxvf func-0.25.tar.gz

cd func-0.25

/usr/local/bin/python setup.py install

certmaster配置：

vi /etc/certmaster/certmaster.conf

[main]

autosign = no

listen_addr =

#證書交換通信端口

listen_port = 1998

cadir = /etc/pki/certmaster/ca

cert_dir = /etc/pki/certmaster

certroot = /var/lib/certmaster/certmaster/certs

csrroot = /var/lib/certmaster/certmaster/csrs

cert_extension = cert

sync_certs = True

func配置：

vi /etc/func/minion.conf

[main]

log_level = DEBUG

acl_dir = /etc/func/minion-acl.d

listen_addr =

#(Func通信端口

listen_port = 1999

minion_name =

防火牆開放1998端口

啓動服務

service certmaster start

chown -R apache:apache /etc/pki/certmaster/ca  apache用戶須要可讀

四、等LVS的certmaster、func環境安裝好後，可以使用下面經常使用操做：

certmaster-ca --list //能夠查看未簽名的計算機名。

certmaster-ca --sign lvs1 //對slave服務器進行簽名(證書交換)。

certmaster-ca --sign `certmaster-ca --list ` //如證書請求的服務器比較多，能夠這樣一會兒搞定。

certmaster-ca -c lvs1 //刪除該主機證書

func "*" call --forks="5" command run "date" //啓用5個進程來執行date命令。

func '*' ping

五、rsync + ssh 從服務器配置

cd /home/ && mkdir rsync && cd rsync

ssh-keygen -t dsa -b 1024 -f rsync-key

//生成了：rsync-key rsync-key.pub兩個文件，其中rsync-key爲私鑰,rnync-key.pub是公鑰，要傳到主服務器作認證的

六、部署天涯LVS管理平臺管理端

tar -zxvf TianyaLVS.tar.gz

mkdir /www

mv TianyaLVS /www

chown -R apache:apache /www/TianyaLVS/LVS /www/TianyaLVS/p_w_picpaths/LVS /www/TianyaLVS/TianyaLVSsystem/logs    //運行apache的用戶對這三個目錄要有寫入的權限

6.2 mysql建立數據庫並導入數據

CREATE DATABASE IF NOT EXISTS tianyalvs DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

grant all privileges on tianyalvs.* to tylvs@'localhost' identified by 'tylvs_passwd';

flush privileges;

use tianyalvs;

source tianyalvs_v2.sql;

6.3 apache配置vhost

httpd.conf配置中，去掉 Include conf/extra/httpd-vhosts.conf 的註釋，並修改運行apache的用戶和組

apache添加vhost，這裏以lvsman.lvs.com爲例，要求ssl加密，訪問http://lvsman.lvs.com自動跳轉到https://lvsman.lvs.com

vi /usr/local/apache/conf/extra/httpd-vhosts.conf

NameVirtualHost *:80

<VirtualHost *:80>

        ServerAdmin songyanlin@tuandai.com

        DocumentRoot "/www/TianyaLVS/"

        ServerName lvsman.lvs.com

        ServerAlias lvsman.lvs.com

        ErrorLog "logs/lvsman.lvs.com-error_log"

        CustomLog "logs/lvsman.lvs.com-access_log" common

        RewriteEngine        on

        #RewriteCond          %{HTTPS} !=on  

        RewriteRule          ^(.*)  https://%{SERVER_NAME}$1 [L,R]

        <Directory "/www/TianyaLVS">

                AllowOverride All

                allow from all

        </Directory>

</VirtualHost>

apache ssl配置請參照http://tianshili.blog.51cto.com/5050423/1735515

httpd-ssl.conf配置：

Listen 443

AddType application/x-x509-ca-cert .crt

AddType application/x-pkcs7-crl    .crl

SSLCipherSuite HIGH:MEDIUM:!MD5:!RC4

SSLProxyCipherSuite HIGH:MEDIUM:!MD5:!RC4

SSLHonorCipherOrder on 

SSLProtocol all -SSLv2 -SSLv3

SSLProxyProtocol all -SSLv2 -SSLv3

SSLPassPhraseDialog  builtin

SSLSessionCache        "shmcb:/usr/local/apache/logs/ssl_scache(512000)"

SSLSessionCacheTimeout  300

SSLMutex  "file:/usr/local/apache/logs/ssl_mutex"

<VirtualHost _default_:443>

DocumentRoot "/www/TianyaLVS"

ServerName lvsman.lvs.com:443

ServerAdmin songyanlin@tuandai.com

ErrorLog "/usr/local/apache/logs/error_log"

TransferLog "/usr/local/apache/logs/lvsman.lvs.com.access_log"

        <Directory "/www/TianyaLVS">

                AllowOverride All

                #allow from all

                #Order                deny,allow

                #Deny                 from all

                #Allow                192.168.1.0/24

                Allow                192.168.1.20

                Allow                192.168.1.21

                # HTTP基本認證

                AuthType             basic

                AuthName             "Protected Intranet Area"

                AuthBasicProvider    file

                AuthUserFile         conf/auth_passwd

                Require              valid-user

        </Directory>

SSLEngine on

SSLCertificateFile "/usr/local/apache2/conf/server.crt"

SSLCertificateKeyFile "/usr/local/apache2/conf/server.key"

<FilesMatch "\.(cgi|shtml|phtml|php)$">

    SSLOptions +StdEnvVars

</FilesMatch>

<Directory "/usr/local/apache2/cgi-bin">

    SSLOptions +StdEnvVars

</Directory>

BrowserMatch "MSIE [2-5]" \

         nokeepalive ssl-unclean-shutdown \

         downgrade-1.0 force-response-1.0

CustomLog "/usr/local/apache2/logs/ssl_request_log" \

          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

修改項

cd TianyaLVS/TianyaLVSsystem/application/config

vi config.php

#######################

//訪問URL

$config['base_url']  = "http://localhost/";

//站點目錄位置

$config['webroot']  = '/www/TianyaLVS';

//管理端應用程序位置

$config['adminroot']  = '/home/LVS';

//python PATH 路徑

$config['pythonroot']  = '/usr/bin/python';

//genhash bin PATH 路徑

$config['genhash_bin'] = '/usr/bin/genhash';

vi database.php

#########################

$db['default']['hostname'] = "localhost";

$db['default']['username'] = "dbuser";

$db['default']['password'] = "dbpass";

$db['default']['database'] = "tianyalvs";

$db['default']['dbdriver'] = "mysql";

tar -zxvf LVS_shell.tar.gz

mv LVS /home; cd /home/LVS

#探測real server func狀態

check.sh '*'

#並生成圖

graph-lvs.sh

根據實際狀況修改參數

引用

WORKDIR="/www/TianyaLVS/LVS/$3/performance"

RRDTOOL="/usr/bin/rrdtool"

GRAPHS="/www/TianyaLVS/p_w_picpaths/LVS/$3"

＃獲取、分析rrd

LVSgraph.py

根據實際狀況修改參數，與database.php保持一致

LVSpath=/home/LVS

import MySQLdb

host='localhost'

user='dbuser'

passwd='dbpass'

db='tianyilvs'

＃func主模塊

ModuleClass.py

＃同步配置LB腳本

Production_realserver.sh

根據實際狀況修改參數

realserversh=/home/LVS

#LB腳本模板

rrealserver.sh.model

#同步master的rrd文件

rsync.sh

根據實際狀況修改參數

KEY=/home/rsync/rsync-key

RUSER=rsyncuser

RPATH=/home/lvs-rrd/performance

LPATH=/www/TianyaLVS/LVS/$1

python /home/LVS/LVSgraph.py lvs1  //如有key要保持，則輸入 yes，下同

python /home/LVS/LVSgraph.py lvs2

添加計劃任務 vi /etc/crontab，去LVS服務器同步rrdtool數據過來，生效圖表

#lVS Performance

*/1 * * * *     root    /usr/bin/python /home/LVS/LVSgraph.py

訪問http://LVS-WEB/，完成系統安裝、配置

2、LVS

lvs1：

一、安裝 lvs keepalived

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz

ln -s /usr/src/kernels/`uname -r`/ /usr/src/linux

tar zxvf ipvsadm-1.26.tar.gz

cd ipvsadm-1.26

make && make install

yum -y install keepalived

二、安裝certmaster、func

yum -y install gcc gcc-g++ openssl openssl-devel pytho python-devel rrdtool rsync

tar -zxvf pyOpenSSL-0.9.tar.gz

cd pyOpenSSL-0.9

/usr/local/bin/python setup.py install

tar -zxvf certmaster-0.25.tar.gz

cd certmaster-0.25

/usr/local/bin/python setup.py install

tar -zxvf func-0.25.tar.gz

cd func-0.25

/usr/local/bin/python setup.py install

certmaster配置：

vi /etc/certmaster/certmaster.conf

[main]

autosign = no

listen_addr =

#與master端口保持一致

listen_port = 1998

cadir = /etc/pki/certmaster/ca

cert_dir = /etc/pki/certmaster

certroot = /var/lib/certmaster/certmaster/certs

csrroot = /var/lib/certmaster/certmaster/csrs

cert_extension = cert

sync_certs = False

vi /etc/certmaster/minion.conf

[main]

certmaster = func-master.server.com

#與master端口保持一致

certmaster_port = 1998

log_level = DEBUG

cert_dir = /etc/pki/certmaster

func配置：

vi /etc/func/minion.conf

[main]

log_level = DEBUG

acl_dir = /etc/func/minion-acl.d

listen_addr =

#與master端口保持一致

listen_port = 1999

#slave主機名

minion_name =lvs1

啓動服務

chkconfig --level 345 certmaster on

service certmaster start

chkconfig --level 345 funcd on

service funcd start

防火牆對LVS-WEB開放1998(certmaster)、1999(func)端口

//安裝完畢後必定要重啓服務器操做系統，否則服務器端有時看不到該主機的證書請求。

三、rsync + ssh 主服務器配置

#useradd rsyncuser

#su rsyncuser

#cd /home/rsyncuser

#if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi

#mv rsync-key.pub(從服務器生成的公鑰) .ssh/

#cd .ssh/

SSH1: #cat rsync-key.pub >> authorized_keys

SSH2: #ssh-keygen -X -f rsync-key.pub >> authorized_keys2

chmod 600 authorized_keys //這一步必定要作，特殊權限要求。

我用的是SSH2，但用SSH2的方式有一個提示長度越界的錯誤，那就用SSH1的方法也沒問題。

四、配置 rrdtool產生數據

tar -zxvf lvs-rrd.tar.gz

mv lvs-rrd /home

肯定 /home/lvs-rrd/lvs.rrd.update裏的路徑

RRDTOOL="/usr/bin/rrdtool"

IPVSADM="/sbin/ipvsadm"

WORKDIR="/home/lvs-rrd/performance"

最後添加計劃任務

vi /etc/crontab

*/2 * * * *     root    /home/lvs-rrd/lvs.rrd.update 2> /dev/null > /dev/null

lvs2的安裝配置參照lvs1