基於Salt Event系統構建Master端returner

前置閱讀

環境說明

開工

前置配置

編寫returner

測試

SaltStack 的 returner 是由minion端主動鏈接returner完成執行結果的存儲, 在部分場景下並不能知足需求. 因爲Salt底層已經構建了一套 Event系統, 全部的操做均會產生event. 所以基於Salt Event系統構建Master端returner成爲一種可能.

以前已經完成了 SaltStack Event系統監聽events測試, 本文將基於Salt Event系統構建Master端returner.

前置閱讀

SaltStack Event系統: http://docs.saltstack.com/en/latest/topics/event/index.html

SaltStack Event系統監聽events測試: http://pengyao.org/saltstack_event_system_listen_events.html

環境說明

測試結構: Master/Minions結構, 共一臺minion, 對應id爲: salt-minion-01.example.com

Salt Version: 2014.1.1

本次測試結果將存放在MySQL中, 爲了方便測試, 已經在Master本地部署了MySQL Server

開工

前置配置

安裝MySQLdb依賴

yum -y install MySQL-python

配置本次測試須要使用的數據庫及用戶

# 建立salt數據庫

mysql -e 'create database salt'

# 建立用於鏈接salt數據庫的用戶

mysql -e '"grant all on salt.* to salt@localhost identified by "salt_pass';

# 將數據庫配置添加至master配置文件中

echo -e "\n\n# MySQL\nmysql.host: 'localhost'\nmysql.user: 'salt'\nmysql.pass: 'salt_pass'\nmysql.db: 'salt'\nmysql.port: 3306" >> /etc/salt/master

爲了與salt自帶的 mysql returner 兼容, 本次直接使用mysql retuner對應的數據庫表結構:

USE `salt`;

--

-- Table structure for table `jids`

--

DROP TABLE IF EXISTS `jids`;

CREATE TABLE `jids` (

  `jid` varchar(255) NOT NULL,

  `load` mediumtext NOT NULL,

  UNIQUE KEY `jid` (`jid`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

--

-- Table structure for table `salt_returns`

--

DROP TABLE IF EXISTS `salt_returns`;

CREATE TABLE `salt_returns` (

  `fun` varchar(50) NOT NULL,

  `jid` varchar(255) NOT NULL,

  `return` mediumtext NOT NULL,

  `id` varchar(255) NOT NULL,

  `success` varchar(10) NOT NULL,

  `full_ret` mediumtext NOT NULL,

  `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

  KEY `id` (`id`),

  KEY `jid` (`jid`),

  KEY `fun` (`fun`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

編寫returner

salt_event_to_mysql.py

#!/bin/env python

#coding=utf8

# Import python libs

import json

# Import salt modules

import salt.config

import salt.utils.event

# Import third party libs

import MySQLdb

__opts__ = salt.config.client_config('/etc/salt/master')

# Create MySQL connect

conn = MySQLdb.connect(host=__opts__['mysql.host'], user=__opts__['mysql.user'], passwd=__opts__['mysql.pass'], db=__opts__['mysql.db'], port=__opts__['mysql.port'])

cursor = conn.cursor()

# Listen Salt Master Event System

event = salt.utils.event.MasterEvent(__opts__['sock_dir'])

for eachevent in event.iter_events(full=True):

    ret = eachevent['data']

    if "salt/job/" in eachevent['tag']:

        # Return Event

        if ret.has_key('id') and ret.has_key('return'):

            # Igonre saltutil.find_job event

            if ret['fun'] == "saltutil.find_job":

                continue

            sql = '''INSERT INTO `salt_returns`

                (`fun`, `jid`, `return`, `id`, `success`, `full_ret` )

                VALUES (%s, %s, %s, %s, %s, %s)'''

            cursor.execute(sql, (ret['fun'], ret['jid'],

                                 json.dumps(ret['return']), ret['id'],

                                 ret['success'], json.dumps(ret)))

            cursor.execute("COMMIT")

    # Other Event

    else:

        pass

運行本returner:

python salt_event_to_mysql.py

測試

新開啓一個終端, 運行Salt指令:

salt '*' test.ping

輸出爲:

salt-minion-01.example.com:

    True

檢查mysql數據庫, 查詢salt_returns表數據:

mysql salt -e "select * from salt_returns\G" 

輸出爲:

*************************** 1. row ***************************

    fun: test.ping

    jid: 20140417161103569310

    return: true

    id: salt-minion-01.example.com

    success: 1

    full_ret: {"fun_args": [], "jid": "20140417161103569310", "return": true, "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2014-04-17T16:11:03.584859", "fun": "test.ping", "id": "salt-minion-01.example.com"}

    alter_time: 2014-04-17 16:11:03

入庫成功