多節點bigchaindb集羣部署

文章比較的長，安裝下來大概4個小時左右，我我的使用的服務器，速度會快一點。

安裝環境

ostname	ip	os
node-admin	192.168.237.130	ubuntu 18.04.2 desktop
node1	192.168.237.131	ubuntu 18.04.2 server
node2	192.168.237.132	ubuntu 18.04.2 server
node3	192.168.237.133	ubuntu 18.04.2 server
node4	192.168.237.134	ubuntu 18.04.2 server
node5	192.168.237.135	ubuntu 18.04.2 server

其中node-admin用來使用ansible來執行遠程命令（爲了部署便利），node1-node5爲bigchain DB節點。全部機器用戶均爲root

須要首先安裝ansible，

sudo apt-add-repository ppa:ansible/ansible

sudo apt-get update

sudo apt-get install ansible

首先在全部節點上安裝openssh-server，並容許root用戶ssh。具體方式爲修改/etc/ssh/sshd_config，修改成PermitRootLogin yes，而後重啓ssh服務。

安裝與配置ansible

 接下來在node-admin上安裝與配置ansible。

sudo apt-add-repository ppa:ansible/ansible
apt-get update && apt-get upgrade
apt-get install ansible

而後修改/etc/ansible/hosts:

root@node-admin:~# grep -Ev  "^$|#" /etc/ansible/hosts
[bigchain]
192.168.237.131 ansible_ssh_pass=123
192.168.237.132 ansible_ssh_pass=123
192.168.237.133 ansible_ssh_pass=123
192.168.237.134 ansible_ssh_pass=123
192.168.237.135 ansible_ssh_pass=123
root@node-admin:~#

設置爲首次連接不須要key認證

sed -i "s/^#\(host_key_checking\).*/\1 = False/g" /etc/ansible/ansible.cfg

設置好了，能夠測試一下：

ansible bigchain -m ping

 

 若是這裏測試失敗，那就是安裝ansible失敗，最好，每個節點單獨安裝一下。

安裝bigchainDB

默認是在node-admin上執行

• 同步時鐘

# 測試是否能ping通cn.pool.ntp.org
ansible bigchain -m command -a "ping cn.pool.ntp.org -c 4"
# 同步
ansible bigchain -m command -a "ntpdate cn.pool.ntp.org"

 這裏須要在每個節點安裝ntpdate

sudo apt-get install ntpdate

• 安裝mongoDB

編輯sources.list

echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

備用 清華的源  https://mirrors.tuna.tsinghua.edu.cn/help/mongodb/

echo "deb [ arch=amd64 ] https://mirrors.tuna.tsinghua.edu.cn/mongodb/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

建立mongod.yml，內容以下：

vim mongod.yml

---

- hosts: bigchain
  remote_user: root
  # invoke setup module to gather facts before executing tasks
  gather_facts: true

  tasks:
  - name: debug
    debug: msg="myhostname={{ansible_hostname}}"

  - name: apt-key
    command: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6

  - name: sources.list
    template: 
      src: mongodb-org-3.4.list
      dest: /etc/apt/sources.list.d/mongodb-org-3.4.list
      owner: root
      group: root
      mode: 0644

  - name: update
    command: apt-get update -y

  - name: install packages
    apt:
      name: "{{item}}"
      force: yes
    with_items:
      - mongodb-org
      - g++ 
      - python3-dev
      - libffi-dev
      - python3-pip

  - name: setuptools
    command: pip3 install --upgrade pip setuptools

  - name: db
    command: mkdir -p /data/db

運行yml(耗時很長，中間須要等待全部的安裝)

ansible-playbook mongod.yml

• 啓動mongod

# 啓動mongo
ansible bigchain -m command -a "mongod --replSet bigchain --fork --logpath /var/log/mongodb/mongod.log"

• 建立副本集

 在node1執行，其它節點就會同步。

# 進入mongodb 命令行模式下
mongo
config = {_id: 'bigchain', members: [{
    "_id": 0,
    "host":"192.168.237.131:27017"
    }]
    }

    rs.initiate(config);
    rs.add("192.168.237.132:27017")
    rs.add("192.168.237.133:27017")
    rs.add("192.168.237.134:27017")
    rs.add("192.168.237.135:27017")

• 安裝bigchainDB

ansible bigchain -m command -a "pip3 install --upgrade bigchaindb"
ansible bigchain -m command -a "bigchaindb -y configure mongodb"

• 修改bigchainDB配置

容許接收全部地址信息

ansible bigchain -m raw -a "sed -i 's/\(\"bind\": \"\)localhost:9984\"/\192.168.237.0:9984\"/g' /root/.bigchaindb"

修改replSet名稱爲bigchain（以前mongod的replset名稱）：

ansible bigchain -m raw -a "sed -i 's/\(\"replicaset\": \"\).*/\1bigchain\",/g' /root/.bigchaindb"

還須要修改keyring選項，使之存儲除本節點以外的全部其餘的節點的keyring的公鑰。先來獲取全部節點的keyring的公鑰

ansible bigchain -m raw -a "cat .bigchaindb | grep public | awk -F\\\" '{print \$4}'"

輸出以下：

 

這裏的keyring是須要記錄下來，下面會用到。

在/root下建立conf.py文件

import sys
import json

keyring = {
    "192.168.237.131": "7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ",
    "192.168.237.132": "GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv",
    "192.168.237.134": "Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv",
    "192.168.237.133": "8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv",
    "192.168.237.135": "GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH"
}

rets = []
for key, value in keyring.items():
    if key != sys.argv[1]:
        rets.append(value)

conf = json.load(open("/root/.bigchaindb"))
conf['keyring'] = rets

json.dump(conf, open("/root/.bigchaindb", "w"), indent=2)

繼續在/root下建立bigchain.yml，用來分發該腳本，並執行該腳本

--

- hosts: bigchain
  remote_user: root
  # invoke setup module to gather facts before executing tasks
  gather_facts: true

  tasks:
  - name: debug
    debug: msg="my ip of eth0 is {{ansible_eth0.ipv4.address}}"

  - name: copy file
    template:
      src: conf.py
      dest: /root/conf.py
      owner: root
      group: root
      mode: 0644

  - name: modify configuration
    command: python conf.py {{ansible_eth0.ipv4.address}}

在node-admin下執行：

ansible-playbook bigchain.yml

執行成功後，能夠看到node1-node5的.bigchaindb裏keyring均成功寫入。

• 啓動bigchainDB

任選一節點（如node1上）執行：

bigchaindb init

而後啓動全部節點的bigchaindb

bigchaindb start

# 或者後臺啓動
nohup bigchaindb start > /dev/null 2>&1 &

# 啓動所有
ansible bigchain -m shell -a "nohup bigchaindb start > /dev/null 2>&1 &"

# kill所有
ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep bigchaindb | awk 'NR>1{print p}{p=\$2}')"

使用bigchaindb

 首先安裝bigchaindb的python driver

•  git安裝方式：

ansible bigchain -m command -a "apt-get install git -y"

ansible bigchain -m command -a "apt-get install libssl-dev -y"

ansible bigchain -m command -a "pip3 install --process-dependency-links git+https://github.com/bigchaindb/bigchaindb-driver.git"

• pip3安裝方式：https://pypi.org/project/bigchaindb-driver/#install

ansible bigchain -m command -a "pip3 install bigchaindb-driver"

 測試實例能夠參考官網：https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html