SaltStack實踐案例一

    經過SaltStack的配置管理來實現一個「中小型web架構」的自動化部署和配置管理，主要包括如下功能和服務：

    系統初始化

    Haproxy服務

    Keepalived服務

    Nginx服務

    PHP（FastCGI）服務

    Memcached服務

    按照本案例的思路，咱們將按照系統初始化、功能模塊化、業務模塊這樣的設計思路來進行設計和實施：

    系統初始化：指操做系統安裝完畢以後，須要使用到的初始配置，好比安裝監控代理、調整內核參數、設置域名解析等

    功能模塊：指的是生產用到的應用，好比Nginx、PHP、Haproxy、Keepalived等這類應用服務的安裝和管理，每個功能完美建立一個目錄來存放，咱們把這個目錄的集合稱之爲「功能模塊」

    業務模塊：在功能模塊中咱們編寫了大量基礎的功能狀態，在業務層面直接進行引用，因此功能模塊就是儘量的全、並且獨立。而業務模塊，不一樣的業務類型就能夠在Include功能模塊裏面的安裝和部署，每一個業務使用本身獨特的配置文件等。最終在top.sls裏面咱們只須要給某個Minion指定一個業務的狀態便可。

1、環境規劃

    環境規劃包含實驗環境規劃SaltStack環境。

    1.實驗環境：

    salt-master-1.example.com    10.0.241.122    Master

    salt-minion-1.example.com    10.0.241.123    Minion、Haproxy＋Keepalived、Nginx＋PHP

    salt-minion-2.example.com    10.0.241.124    Minion、Memcached、Haproxy＋Keepalived、Nginx＋PHP

    2.SaltStack環境配置

    本例子有兩個環境base和prod，base環境用來存放初始化的功能。prod環境用於放置生產的配置管理功能：

[root@salt-master-1 ~]# vim /etc/salt/master
file_roots:
  base:
    - /srv/salt/base
  prod:
    - /srv/salt/prod
    
pillar_roots:
  base:
    - /srv/pillar/base
  prod:
    - /srv/pillar/prod
    
[root@salt-master-1 ~]# mkdir -p /srv/salt/{base,prod}
[root@salt-master-1 ~]# mkdir -p /srv/pillar/{base,prod}
[root@salt-master-1 ~]# systemctl restart salt-master.service

2、系統初始化

    1.DNS配置

[root@salt-master-1 ~]# cat /srv/salt/base/init/dns.sls
/etc/resolv.conf:
    file.managed:
        - source: salt://init/files/resolv.conf
        - user: root
        - group: root
        - mode: 644
# 把準備好的resolv.conf放置在/srv/salt/base/init/files/目錄下

   2.History記錄時間

[root@salt-master-1 ~]# cat /srv/salt/base/init/history.sls
/etc/profile:
    file.append:
        - text:
            - export HISTTIMEFORMAT="%F %T `whoami` "

   3.命令操做審計

[root@salt-master-1 ~]# cat /srv/salt/base/init/audit.sls
/etc/bashrc:
    file.append:
        - text:
            - export PROMPT_COMMAND='{ msg=$(history 1 | { read x y; echo $y; });logger "[euid=$(whoami)]":$(who am i):['prod']"$msg"; }'

   4.內核參數優化

[root@salt-master-1 ~]# cat /srv/salt/base/init/sysctl.sls
net.ipv4.ip_local_port_range:
    sysctl.present:
        - value: 10000 65000
fs.file_max:
    sysctl.present:
        - value: 2000000
net.ipv4.ip_forward:
    sysctl.present:
        - value: 1
vm.swappiness:
    sysctl.present:
        - value: 0

   5.epel倉庫

[root@salt-master-1 ~]# cat /srv/salt/base/init/epel.sls
yum_repo_release:
    pkg.installed:
        - sources:
            - epel-release: http://mirrors.aliyun.com/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
        - unless: rpm -qa | grep epel-release-7-5

   6.zabbix_agentd安裝

    經過使用pillar來設置zabbix server的ip地址：

[root@salt-master-1 ~]# cat /srv/salt/base/init/top.sls
base:
    '*':
        - zabbix
[root@salt-master-1 ~]# cat /srv/pillar/base/zabbix.sls
zabbix-agent:
    Zabbix_Server: 10.0.241.122

    安裝並啓動zabbix agent：

[root@salt-master-1 ~]# cat  /srv/salt/base/init/zabbix_agent.sls
zabbix-agent:
    pkg.installed:
        - name: zabbix22-agent

    file.managed:
        - name: /etc/zabbix_agentd.conf
        - source: salt://zabbix/files/zabbix_agentd.conf
        - template: jinja
        - defaults:
            Server: {{ pillar['zabbix-agent']['Zabbix_Server'] }}
        - require:
            - pkg: zabbix-agent
    service.running:
        - enable: True
        - watch:
            - pkg: zabbix-agent
            - file: zabbix-agent

[root@salt-master-1 ~]# cat /srv/salt/base/init/env_init.sls
include:
    - init.dns
    - init.history
    - init.audit
    - init.sysctl
    - init.epel
    - init.zabbix_agent
[root@salt-master-1 ~]# cat  /srv/salt/base/top.sls
base:
    '*':
    - init.env_init

# 在服務器上執行
[root@salt-master-1 ~]# salt 'salt-minion-1' state.highstate test=True

3、Haproxy配置管理

    Haproxy是一個開源的高性能的反向代理項目，支持四層和七層的負載均衡，多種負載均衡算法和健康檢查等。

    Keepalived是一個高可用集羣的項目，它是VRRP協議的完美實現，咱們經過Keepalived來管理Haproxy上面的VIP。當主Haproxy發生故障時，將VIP漂移到備用的Haproxy上來繼續提供服務。

[root@salt-master-1 ~]# mkdir /srv/salt/prod/pkg -p
[root@salt-master-1 ~]# mkdir /srv/salt/prod/haproxy/files -p
[root@salt-master-1 ~]# mkdir /srv/salt/prod/keepalived/files -p
# 在每一個服務的目錄下面均建立一個files目錄用來存放源碼包和須要的相關啓動腳本、配置文件等。

    1.pkg配置

[root@salt-master-1 ~]# cat /srv/salt/prod/pkg/pkg-init.sls
pkg-init:
    pkg.installed:
        - pkgs:    # 注意
            - gcc
            - gcc-c++
            - glibc
            - make
            - autoconf
            - openssl
            - openssl-devel

    2.Haproxy服務配置

[root@salt-master-1 ~]# cd /usr/local/src/ && wget http://www.haproxy.org/download/1.6/src/haproxy-1.6.2.tar.gz && tar zxf haproxy-1.6.2.tar.gz && cd haproxy-1.6.2/examples/
[root@salt-master-1 examples]# sed -i 's/\/usr\/sbin\/'\$BASENAME'/\/usr\/local\/haproxy\/sbin\/'\$BASENAME'/g' haproxy.init
# 修改haproxy的啓動腳本
[root@salt-master-1 examples]# cp haproxy.init /srv/salt/prod/haproxy/files/

    編寫Haproxy代碼以下：

[root@salt-master-1 examples]# cat /srv/salt/prod/haproxy/install.sls
include:
    - pkg.pkg-init
haproxy-install:
    file.managed:
        - name: /usr/local/src/haproxy-1.6.2.tar.gz
        - source: salt://haproxy/files/haproxy-1.6.2.tar.gz
        - mode: 755
        - user: root
        - group: root
    cmd.run:
        - name: cd /usr/local/src/ && tar zxf haproxy-1.6.2.tar.gz && make TARGET=linux26 PREFIX=/usr/local/haproxy && make install PREFIX=/usr/local/haproxy
        - unless: test -d /usr/local/haproxy
        - require:
            - pkg: pkg-init
            - file: haproxy-install
/etc/init.d/haproxy:
    file.managed:
        - source: salt://haproxy/files/haproxy.init
        - mode: 755
        - user: root
        - group: root
        - require:
            - cmd: haproxy-install
net.ipv4.ip_nolocal_bind:
    sysctl.present:
        - value: 1
haproxy-config-dir:
    file.directory:
        - name: /etc/haproxy
        - mode: 755
        - user: root
        - group: root
haproxy-init:
    cmd.run:
        - name: chkconfig --add haproxy
        - unless: chkconfig --list | grep haproxy
        - require:
            - file: /etc/init.d/haproxy

    管理haproxy的配置文件有兩種方法：

    1).直接在須要使用haproxy的地方引用haproxy的安裝，而後加入haproxy的配置文件管理和服務管理。優勢：簡單明瞭；缺點：不夠靈活通用。

    2).使用jinja模版，將haproxy的基礎配置編寫完成後，其餘的配置經過Pillar來進行自動生成。優勢：很是靈活通用；缺點：因爲須要使用大量的if、for等jinja模版語法，並且須要配置Pillarlai實現配置，比較複雜，有難度，容易出錯。

    3.Haproxy業務引用

    咱們如今切換功能服務配置外，編寫一個業務模塊Cluster，而後調用Haproxy來完成配置管理。這樣作的好處是把基礎服務的配置管理和業務分開。

    建立cluster目錄，而且在cluster目錄建立files目錄，用來存放配置文件：

[root@salt-master-1 ~]# mkdir -p /srv/salt/prod/cluster/files
[root@salt-master-1 ~]# cat /srv/salt/prod/cluster/files/haproxy-outside.cfg
global
maxconn 100000
chroot /usr/local/haproxy
uid 99
gid 99
daemon
nbproc 1
pidfile /usr/local/haproxy/logs/haproxy.pid
log 127.0.0.1 local3 info

# 默認參數設置
defaults
option http-keep-alive
maxconn 100000
mode http
timeout connect 5000ms
timeout client 5000ms
timeout server 5000ms

# 開啓Haproxy Status狀態監控，增長驗證
listen stats
mode http
bind 0.0.0.0:8888
stats enable
stats uri /haproxy-status
stats auth haproxy:saltstack

# 前端設置
frontend frontend_www_example_com
bind 10.0.241.123:80
mode http
option httplog
log global
    default_backend backend_www_example_com
# 後端設置
backend backend_www_example_com
option forwardfor header X-REAL-IP
option httpchk HEAD / HTTP/1.0
balance source
server web-node1 10.0.241.123:8080 check inter 2000 rise 30 fall 15
server web-node1 10.0.241.124:8080 check inter 2000 rise 30 fall 15

    編寫haproxy的服務管理：

[root@salt-master-1 ~]# cat /srv/salt/prod/cluster/haproxy-outside.sls
include:
    - haproxy.install

haproxy-service:
    file.managed:
        - name: /etc/haproxy/haproxy.cfg
        - source: salt://cluster/files/haproxy-outside.cfg
        - user: root
        - group: root
        - mode: 644

    service.running:
        - name: haproxy
        - enable: True
        - reload: True
        - require:
            - cmd: haproxy-init
        - watch:
            - file: haproxy-service

    4.執行Haproxy狀態

[root@salt-master-1 ~]# cat /srv/salt/base/top.sls
base:
    '*':
        - init.env_init
prod:
    '*':
        - cluster.haproxy-outside

#[root@salt-master-1 prod]# salt 'salt-minion-1' state.highstate test=True

4、Keepalived配置管理

    首先放置源碼包、Keepalived的啓動腳本、sysconfig配置文件在/srv/salt/prod/keepalived/files/目錄下。啓動腳本和配置文件均可以從源碼包中獲取到。

    1.軟件包準備

[root@salt-master-1 ~]# cd /usr/local/src/ && wget  &&  cp keepalived-1.2.19.tar.gz /srv/salt/prod/keepalived/files/ && tar zxf keepalived-1.2.19.tar.gz && cd keepalived-1.2.19/ && cp keepalived/etc/init.d/keepalived.init /srv/salt/prod/keepalived/files/ && cp keepalived/etc/init.d/keepalived.sysconfig /srv/salt/prod/keepalived/files/

[root@salt-master-1 keepalived-1.2.19]# vim /srv/salt/prod/keepalived/files/keepalived.init
將daemon keepalived ${KEEPALIVED_OPTIONS} 修改成 daemon /usr/local/keepalived/sbin/keepalived ${KEEPALIVED_OPTIONS}

    2.編寫Keepalived安裝sls

[root@salt-master-1 keepalived]# cat install.sls
keepalived-install:
    file.managed:
        - name: /usr/local/src/keepalived-1.2.19.tar.gz
        - source: salt://keepalived/files/keepalived-1.2.19.tar.gz
        - mode: 755
        - user: root
        - group: root
    cmd.run:
        - cmd: cd /usr/local/src/ && tar zxf keepalived-1.2.19.tar.gz && cd keepalived-1.2.19 && ./configure --prefix=/usr/local/keepalived --disable-fwmark && make install
        - unless: test -d /usr/local/keepalived
        - require:
            file: keepalived-install
# Keepalived的sysconfig配置文件
/etc/sysconfig/keepalived:
    file.managed:
        - source: salt://keepalived/files/keepalived.sysconfig
        - mode: 644
        - user: root
        - group: root

# Keepalived的服務管理腳本
/etc/init.d/keepalived:
    file.managed:
        - source: salt://keepalived/files/keepalived.init
        - mode: 755
        - user: root
        - group: root

# Keepalived加入系統服務管理
keepalived-init:
    cmd.run:
        - name: chkconfig --add keepalived
        - unless: chkconfig --list | grep keepalived
        - require:
            - file: /etc/init.d/keepalived

# keepalived的配置文件目錄以下
/etc/keepalived:
    file.directory:
        - user: root
        - group: root

   3.Keepalived業務引用

    首先和Haproxy同樣，咱們須要有一個Keepalived的配置文件，不過此次配置文件和Haproxy稍有不一樣，由於keepalived分爲主、備節點，一些配置在主節點和備節點上是不一樣的。咱們須要使用jinja模版來完成配置文件的管理。

[root@salt-master-1 keepalived]# cat /srv/salt/prod/cluster/files/haproxy-outside-keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
        saltstack@example.com
    }
    notification_email_from keepalived@example.com
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    route_id {{ ROUTEID }}
}

vrrp_instance haproxy_ha {
state {{ STATEID }}
interface eth0
    virtual_router_id 36
priority {{ PRIORITYID }}
    advert_int 1
authentication {
    auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.241.123
    }
}

    Cluster業務目錄下編寫Haproxy使用Keepalived作高可用的sls：

[root@salt-master-1 keepalived]# cat  /srv/salt/prod/cluster/haproxy-outside-keepalived.sls
include:
    - keepalived.install
keepalived-server:
    file.managed:
        - name: /etc/keepalived/keepalived.conf
        - source: salt://cluster/files/haproxy-outside-keepalived.conf
        - mode: 644
        - user: root
        - group: root
        - template: jiaja
        {% if grains['fqdn'] == 'salt-minion-1.example.com' %}
        - ROUTEID: haproxy_ha
        - STATEID: MASTER
        - PRIORITYID: 150

        {% elif grains['fqdn'] == 'salt-minion-2.example.com' %}
        - ROUTEID: haproxy_ha
        - STATEID: BACKUP
        - PRIORITYID: 100
        {% endif %}
    service.running:
        - name: keepalived
        - enable: True
        - watch:
            - file: keepalived-server

    4.執行keepalived狀態

[root@salt-master-1 keepalived]# cat  /srv/salt/base/top.sls
base:
    '*':
        - init.env_init
        - pkg-init
prod:
    '*':
        - cluster.haproxy-outside
        - cluster.haproxy-outside-keepalived