centos7部署postgresql集羣高可用 patroni + etcd 之patroni篇

時間 2019-11-09

標籤 centos7 centos 部署 postgresql 集羣可用 patroni etcd 欄目 Tomcat 简体版

原文原文鏈接

實驗環境：centos7.4純淨版html

postgres版本: 9.6.15node

etcd版本:3.3.11python

patroni版本:1.6.0git

patroni介紹可參考：https://github.com/zalando/patronigithub

ip規劃sql

192.168.216.130 node1 master數據庫

192.168.216.132 node2 slavebootstrap

192.168.216.134 node3 slavecentos

etcd集羣部署請看上一篇文章:https://www.cnblogs.com/caidingyu/p/11408389.htmlapi

postgres部署參考文章：https://www.cnblogs.com/virtulreal/p/9921978.html

修改node1中postgresql.conf配置以下

max_connections = '100'
max_wal_senders = '10'
port = '5432'
listen_addresses = '0.0.0.0'
synchronous_commit = on
full_page_writes = on
wal_log_hints = on
synchronous_standby_names = '*'
max_replication_slots = 10
wal_level = replica

修改node1中pg_hba.conf配置以下

[root@localhost data]# more pg_hba.conf|grep -v ^#|grep -v ^$
local   all             all                                     peer
host    all             all             127.0.0.1/32            md5
host    all             postgres        127.0.0.1/32            md5
host    all             all             192.168.216.0/24      md5
host    all             all             ::1/128                 md5
local   replication     replicator                                peer
host    replication     replicator        127.0.0.1/32            md5
host    replication     replicator        ::1/128                 md5
host    replication     replicator        192.168.216.130/32      md5
host    replication     replicator        192.168.216.132/32      md5
host    replication     replicator        192.168.216.134/32      md5

node1上建立複製槽，相當重要，patroni 用到了這個玩意

postgres=# create user replicator replication login encrypted password '1qaz2wsx';
postgres=# alter user postgres with password '1qaz2wsx'; 
postgres=# select * from pg_create_physical_replication_slot('pgsql96_node1'); 
postgres=# select * from pg_create_physical_replication_slot('pgsql96_node2'); 
postgres=# select * from pg_create_physical_replication_slot('pgsql96_node3');

　node2 配置stream replication

systemctl stop postgresql-9.6
su - postgres
cd /var/lib/pgsql/9.6/data
rm -rf ./*
/usr/pgsql-9.6/bin/pg_basebackup -h 192.168.216.130 -D /var/lib/pgsql/9.6/data -U replicator -v -P -R

vi recovery.conf
recovery_target_timeline = 'latest'
standby_mode = 'on'
primary_conninfo = 'host=192.168.216.130 port=5432 user=replicator password=1qaz2wsx'
primary_slot_name = 'pgsql96_node2'
trigger_file = '/tmp/postgresql.trigger.5432'

執行exit返回root用戶
systemctl start postgresql-9.6

　node3 配置stream replication　

systemctl stop postgresql-9.6
su - postgres
cd /var/lib/pgsql/9.6/data
rm -rf ./*
/usr/pgsql-9.6/bin/pg_basebackup -h 192.168.216.130 -D /var/lib/pgsql/9.6/data -U replicator -v -P -R

vi recovery.conf
recovery_target_timeline = 'latest'
standby_mode = 'on'
primary_conninfo = 'host=192.168.216.130 port=5432 user=replicator password=1qaz2wsx'
primary_slot_name = 'pgsql96_node3'
trigger_file = '/tmp/postgresql.trigger.5432'

執行exit返回root用戶
systemctl start postgresql-9.6

在node1上鍊接數據庫，查看複製狀態

select client_addr,pg_xlog_location_diff(sent_location, write_location) as write_delay,pg_xlog_location_diff(sent_location, flush_location) as flush_delay,pg_xlog_location_diff(sent_location, replay_location) as replay_delay from pg_stat_replication;

下載、安裝 patroni,如遇網絡問題能夠多執行幾回pip install或者切換其餘pip源

yum install gcc
yum install python-devel.x86_64
cd /tmp
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
pip install psycopg2-binary
pip install patroni[etcd,consul]

　驗證是否安裝成功

which patroni
patronictl --help

node1上patroni 配置文件以下，該配置文件須要手動建立

mkdir -p /usr/patroni/conf
 cd /usr/patroni/conf/
cat /usr/patroni/conf/patroni_postgresql.yml
scope: pgsql96
namespace: /pgsql/
name: pgsql96_node1

restapi:
  listen: 192.168.216.130:8008
  connect_address: 192.168.216.130:8008

etcd:
  host: 192.168.216.130:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        listen_addresses: "0.0.0.0"
        port: 5432
        wal_level: logical
        hot_standby: "on"
        wal_keep_segments: 1000
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"
#        archive_mode: "on"
#        archive_timeout: 1800s
#        archive_command: gzip < %p > /data/backup/pgwalarchive/%f.gz
#      recovery_conf:
#        restore_command: gunzip < /data/backup/pgwalarchive/%f.gz > %p

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 192.168.216.130:5432
  data_dir: /var/lib/pgsql/9.6/data
  bin_dir: /usr/pgsql-9.6/bin
#  config_dir: /etc/postgresql/9.6/main
  authentication:
    replication:
      username: replicator
      password: 1qaz2wsx
    superuser:
      username: postgres
      password: 1qaz2wsx

#watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

　node2上patroni 配置文件以下

[root@localhost etcd]# cat /usr/patroni/conf/patroni_postgresql.yml
scope: pgsql96
namespace: /pgsql/
name: pgsql96_node2

restapi:
  listen: 192.168.216.132:8008
  connect_address: 192.168.216.132:8008

etcd:
  host: 192.168.216.132:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        listen_addresses: "0.0.0.0"
        port: 5432
        wal_level: logical
        hot_standby: "on"
        wal_keep_segments: 1000
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"
#        archive_mode: "on"
#        archive_timeout: 1800s
#        archive_command: gzip < %p > /data/backup/pgwalarchive/%f.gz
#      recovery_conf:
#        restore_command: gunzip < /data/backup/pgwalarchive/%f.gz > %p

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 192.168.216.132:5432
  data_dir: /var/lib/pgsql/9.6/data
  bin_dir: /usr/pgsql-9.6/bin
#  config_dir: /etc/postgresql/9.6/main
  authentication:
    replication:
      username: replicator
      password: 1qaz2wsx
    superuser:
      username: postgres
      password: 1qaz2wsx

#watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

node3上patroni 配置文件以下

scope: pgsql96
namespace: /pgsql/
name: pgsql96_node3

restapi:
  listen: 192.168.216.134:8008
  connect_address: 192.168.216.134:8008

etcd:
  host: 192.168.216.134:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        listen_addresses: "0.0.0.0"
        port: 5432
        wal_level: logical
        hot_standby: "on"
        wal_keep_segments: 1000
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"
#        archive_mode: "on"
#        archive_timeout: 1800s
#        archive_command: gzip < %p > /data/backup/pgwalarchive/%f.gz
#      recovery_conf:
#        restore_command: gunzip < /data/backup/pgwalarchive/%f.gz > %p

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 192.168.216.134:5432
  data_dir: /var/lib/pgsql/9.6/data
  bin_dir: /usr/pgsql-9.6/bin
#  config_dir: /etc/postgresql/9.6/main
  authentication:
    replication:
      username: replicator
      password: 1qaz2wsx
    superuser:
      username: postgres
      password: 1qaz2wsx

#watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

手動啓動 patroni,注意這裏須要切換到postgres用戶下啓動

node一、node二、node3 三個節點依次啓動

patroni /usr/patroni/conf/patroni_postgresql.yml

查看 patroni 集羣狀態　

克隆一個窗口，執行patronictl -c /usr/patroni/conf/patroni_postgresql.yml list

查看 etcd 的信息

etcdctl ls /pgsql/pgsql96

etcdctl get /pgsql/pgsql96/members/pgsql96_node1

爲了方便開機自啓，故配置成 patroni.service，3個node都須要進行配置，配置好patroni.service後就能夠直接在root用戶下切換Leader以及重啓postgres節點等操做

[root@localhost data]# vi /etc/systemd/system/patroni.service
[root@localhost data]# cat /etc/systemd/system/patroni.service
[Unit]
Description=patroni - a high-availability PostgreSQL
Documentation=https://patroni.readthedocs.io/en/latest/index.html
After=syslog.target network.target etcd.target
Wants=network-online.target

[Service]
Type=simple
User=postgres
Group=postgres
PermissionsStartOnly=true
ExecStart=/usr/bin/patroni /usr/patroni/conf/patroni_postgresql.yml
ExecReload=/bin/kill -HUP $MAINPID
LimitNOFILE=65536
KillMode=process
KillSignal=SIGINT
Restart=on-abnormal
RestartSec=30s
TimeoutSec=0

[Install]
WantedBy=multi-user.target

禁止 postgresql 的自啓動，經過 patroni 來管理 postgresql

systemctl status patroni
systemctl start patroni
systemctl enable patroni

systemctl status postgresql
systemctl disable postgresql

systemctl status etcd
systemctl enable etcd

如何手動切換Leader

執行patronictl -c /usr/patroni/conf/patroni_postgresql.yml switchover

多執行patronictl -c /usr/patroni/conf/patroni_postgresql.yml list進行刷新，能夠看到Leader由node2切換爲node1

能夠嘗試模擬node節點斷電，即手動中止Leader所在節點的patroni服務，此時查詢集羣狀態，能夠看到Leader自動切換到其餘node節點上

目前能夠看到Leader位於132節點上，

在132節點上中止patroni服務

再次查看集羣狀態，能夠看到132被stop，Leader自動被轉移到134上

參考：https://blog.csdn.net/ctypyb2002/article/details/81007990

https://www.cnblogs.com/ctypyb2002/p/9792939.html