【Open-Falcon】Linux下安裝Open-Falcon

 

1、Open-Falcon組件簡述

【Open-Falcon繪圖相關組件】

• Agent:  部署在目標機器採集機器監控項
• Transfer : 數據接收端，轉發數據到後端Graph和Judge
• Graph：操做rrd文件存儲監控數據
• Query：查詢各個Graph數據，提供統一http查詢接口
• Dashboard：查詢監控歷史趨勢圖的web端
• Task：負責一些定時任務，索引全量更新、垃圾索引清理、自身組件監控等

【Open-Falcon報警相關組件】

• Sender ：報警發送模塊，控制併發度，提供發送的緩衝queue
• UIC(FE)：用戶組管理，單點登陸
• Portal：配置報警策略，管理機器分組的web端
• HBS：HeartBeat Server，心跳服務器
• Judge：報警判斷模塊
• Links：報警合併依賴的web端，存放報警詳情
• Alarm：報警時間處理器

【Open-Falcon架構圖】

官網架構圖：

 

 網友：

 

 

2、安裝準備

1.安裝Redis

http://www.cnblogs.com/xialiaoliao0911/p/7523952.html

2.安裝MySQL

http://www.cnblogs.com/xialiaoliao0911/p/7523931.html

3.Open-Falocn下載地址

二進制版本：https://pan.baidu.com/s/1jOb6z-HRJ7i6nSFxf7I5Bg

4. 初始化MySQL表結構

# open-falcon全部組件都無需root帳號啓動，推薦使用普通帳號安裝，提高安全性。此處咱們使用普通帳號：work來安裝部署全部組件 # 固然了，使用yum安裝依賴的一些lib庫的時候仍是要有root權限的。 git clone https://github.com/open-falcon/scripts.git
cd ./scripts/ mysql -h localhost -u root --password="" < db_schema/graph-db-schema.sql mysql -h localhost -u root --password="" < db_schema/dashboard-db-schema.sql mysql -h localhost -u root --password="" < db_schema/portal-db-schema.sql mysql -h localhost -u root --password="" < db_schema/links-db-schema.sql mysql -h localhost -u root --password="" < db_schema/uic-db-schema.sql

5.解壓open-falcon.tar.gz

#新建用戶falcon useadd falcon #新建臨時目錄tmp su - falcon cd /home/falcon mkdir tmp #解壓 tar -zxf of-release-v0.1.0.tar.gz -C ./tmp/
for x in `find ./tmp/ -name "*.tar.gz"`;do \ app=`echo $x|cut -d '-' -f2`; \ mkdir -p $app; \ tar -zxf $x -C $app; \ done

 

3、安裝Open-Falcon繪圖相關組件

1.Agent

每臺機器上，都須要部署agent，agent會自動採集預先定義的各類採集項，每隔60秒，push到transfer。

cd $WORKSPACE/agent/ mv cfg.example.json cfg.json vim cfg.json - 修改 transfer這個配置項的enabled爲 true，表示開啓向transfer發送數據的功能 - 修改 transfer這個配置項的addr爲：["127.0.0.1:8433"] (改地址爲transfer組件的監聽地址, 爲列表形式，可配置多個transfer實例的地址，用逗號分隔) # 默認狀況下（全部組件都在同一臺服務器上），保持cfg.json不變便可 # cfg.json中的各配置項，能夠參考 https://github.com/open-falcon/agent/blob/master/README.md
 # 啓動 ./control start # 查看日誌 ./control tail
 
#啓動完成後，經過瀏覽器進行訪問
 http://192.168.102.141:1988/

【配置文件】

 /home/falcon/tmp/agent/cfg.json

[falcon@open-falcon-demo agent]$ more cfg.json { "debug": false, "hostname": "open-falcon-demo", "ip": "192.168.102.141", "plugin": { "enabled": false, "dir": "./plugin", "git": "https://github.com/open-falcon/plugin.git", "logs": "./logs" }, "heartbeat": { "enabled": true, "addr": "127.0.0.1:6030", "interval": 60, "timeout": 1000 }, "transfer": { "enabled": true, "addrs": [ "127.0.0.1:8433", "127.0.0.1:8433" ], "interval": 60, "timeout": 1000 }, "http": { "enabled": true, "listen": ":1988", "backdoor": false }, "collector": { "ifacePrefix": ["eth", "em"] }, "ignore": { "cpu.busy": true, "df.bytes.free": true, "df.bytes.total": true, "df.bytes.used": true, "df.bytes.used.percent": true, "df.inodes.total": true, "df.inodes.free": true, "df.inodes.used": true, "df.inodes.used.percent": true, "mem.memtotal": true, "mem.memused": true, "mem.memused.percent": true, "mem.memfree": true, "mem.swaptotal": true, "mem.swapused": true, "mem.swapfree": true } }

經過瀏覽器打開後的界面：

 

 

2.aggregator

cd $WORKSPACE/aggregator/ mv cfg.example.json cfg.json

【配置文件】

/home/falcon/tmp/aggregator/cfg.json

[falcon@open-falcon-demo aggregator]$ more cfg.json { "debug": false, "http": { "enabled": true, "listen": "0.0.0.0:6055" }, "database": { "addr": "root:mysql@tcp(127.0.0.1:3306)/falcon_portal?loc=Local&parseTime=true", "idle": 10, "ids": [1, -1], "interval": 55 }, "api": { "hostnames": "http://127.0.0.1:5050/api/group/%s/hosts.json", "push": "http://127.0.0.1:6060/api/push", "graphLast": "http://127.0.0.1:9966/graph/last" } }

 

3.Transfer

transfer默認監聽在:8433端口上，agent會經過jsonrpc的方式來push數據上來。

cd $WORKSPACE/transfer/ mv cfg.example.json cfg.json # 默認狀況下（全部組件都在同一臺服務器上），保持cfg.json不變便可 # cfg.json中的各配置項，能夠參考 https://github.com/open-falcon/transfer/blob/master/README.md
# 若有必要，請酌情修改cfg.json # 啓動transfer ./control start # 校驗服務,這裏假定服務開啓了6060的http監聽端口。檢驗結果爲ok代表服務正常啓動。 curl -s "http://127.0.0.1:6060/health" #查看日誌 ./control tail # 中止transfer ./control stop

[falcon@open-falcon-demo transfer]$ more cfg.json { "debug": false, "minStep": 30, "http": { "enabled": true, "listen": "0.0.0.0:6060" }, "rpc": { "enabled": true, "listen": "0.0.0.0:8433" }, "socket": { "enabled": false, "listen": "0.0.0.0:4444", "timeout": 3600 }, "judge": { "enabled": true, "batch": 200, "connTimeout": 1000, "callTimeout": 5000, "maxConns": 32, "maxIdle": 32, "replicas": 500, "cluster": { "judge-00" : "127.0.0.1:6080" } }, "graph": { "enabled": true, "batch": 200, "connTimeout": 1000, "callTimeout": 5000, "maxConns": 32, "maxIdle": 32, "replicas": 500, "cluster": { "graph-00" : "127.0.0.1:6070" } }, "tsdb": { "enabled": false, "batch": 200, "connTimeout": 1000, "callTimeout": 5000, "maxConns": 32, "maxIdle": 32, "retry": 3, "address": "127.0.0.1:8088" } }

 

4.Graph

graph組件是存儲繪圖數據、歷史數據的組件。transfer會把接收到的數據，轉發給graph。

cd $WORKSPACE/graph/ mv cfg.example.json cfg.json
mkdir -p /home/falcon/data/6070 #新建graph數據存儲目錄 # 默認狀況下（全部組件都在同一臺服務器上），保持cfg.json不變便可 # cfg.json中的各配置項，能夠參考 https://github.com/open-falcon/graph/blob/master/README.md
 # 啓動 ./control start # 查看日誌 ./control tail # 校驗服務,這裏假定服務開啓了6071的http監聽端口。檢驗結果爲ok代表服務正常啓動。 curl -s "http://127.0.0.1:6071/health"

[falcon@open-falcon-demo graph]$ more cfg.json { "pid": "/home/falcon/open-falcon/graph/var/app.pid", #修改成本機實際的目錄 "log": "info", "debug": false, "http": { "enabled": true, "listen": "0.0.0.0:6071" }, "rpc": { "enabled": true, "listen": "0.0.0.0:6070" }, "rrd": { "storage": "/home/falcon/data/6070" #graph數據存儲目錄,須要手動創建 }, "db": { "dsn": "root:mysql@tcp(127.0.0.1:3306)/graph?loc=Local&parseTime=true", #標記紅色的爲MySQL數據的root密碼 "maxIdle": 4 }, "callTimeout": 5000, "migrate": { "enabled": false, "concurrency": 2, "replicas": 500, "cluster": { "graph-00" : "127.0.0.1:6070" } } }

 

5.Query

query組件，繪圖數據的查詢接口，query組件收到用戶的查詢請求後，會從後端的多個graph，查詢相應的數據，聚合後，再返回給用戶。

cd $WORKSPACE/query/ mv cfg.example.json cfg.json
#進入query目錄新建graph_backends.txt文件，並寫入graph相關的內容,內容來源於graph的cfg.json的migrate>cluster
cd /home/falcon/tmp/query
vi graph_backends.txt 
graph-00 127.0.0.1:6070
 # 默認狀況下（全部組件都在同一臺服務器上），保持cfg.json不變便可 # cfg.json中的各配置項，能夠參考 https://github.com/open-falcon/query/blob/master/README.md
 # 啓動 ./control start # 查看日誌 ./control tail

[falcon@open-falcon-demo query]$ more cfg.json { "log_level":  "info", "slowlog": 2000, "debug": "false", "http": { "enabled":  true, "listen":   "0.0.0.0:9966" }, "graph": { "backends": "./graph_backends.txt", "reload_interval": 60, "connTimeout": 1000, "callTimeout": 5000, "maxConns": 32, "maxIdle": 32, "replicas": 500, "cluster": { "graph-00": "127.0.0.1:6070" } }, "api": { "query": "http://127.0.0.1:9966", "dashboard": "http://127.0.0.1:8081", "max": 500 } }

 

6.Dashboard

dashboard是面向用戶的查詢界面，在這裏，用戶能夠看到push到graph中的全部數據，並查看其趨勢圖。

Install dependency #配置EPEL源,安裝virtualenv環境 rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install -y python-pip pip install virtualenv

#根據MySQL實際路徑，新建兩個軟鏈接

 ln -s /usr/local/mysql/lib/libmysqlclient.so.20 /usr/lib/libmysqlclient.so.20
 ln -s /usr/local/mysql/lib/libmysqlclient.so.20 /usr/lib64/libmysqlclient.so.20

#將pip_requirements.txt中的mysql-python這一行去掉，使用easy_install單獨安裝
#進入到virtualenv環境

 [falcon@open-falcon-demo dashboard]$ virtualenv env

 [falcon@open-falcon-demo dashboard]$ source env/bin/activate

  #安裝mysql-python
 (env)[falcon@open-falcon-demo dashboard]$ easy_install mysql-python   

 

 #查看READ.me文件，找到./env/bin/pip install -r pip_requirements.txt -i http://pypi.douban.com/simple這行而後執行

 (env)[falcon@open-falcon-demo dashboard]$ ./env/bin/pip install -r pip_requirements.txt -i http://pypi.douban.com/simple

 

  #啓動Dashboard

  (env)[falcon@open-falcon-demo dashboard]$ ./control start

  #查看Dashboard啓動狀態

  (env)[falcon@open-falcon-demo dashboard]$ ./control status

  #查看日誌

  (env)[falcon@open-falcon-demo dashboard]$ ./control tail

  #退出virtualenv環境

  (env)[falcon@open-falcon-demo dashboard]$  deactivate

  #啓動完成後，可經過瀏覽器進行訪問

   http://192.168.102.141:8081/

 【配置文件】

/home/falcon/tmp/dashboard/rrd/config.py

[falcon@open-falcon-demo rrd]$ more config.py #-*-coding:utf8-*- import os #-- dashboard db config -- DASHBOARD_DB_HOST = "127.0.0.1" DASHBOARD_DB_PORT = 3306 DASHBOARD_DB_USER = "root" DASHBOARD_DB_PASSWD = "mysql" DASHBOARD_DB_NAME = "dashboard" #-- graph db config -- GRAPH_DB_HOST = "127.0.0.1" GRAPH_DB_PORT = 3306 GRAPH_DB_USER = "root" GRAPH_DB_PASSWD = "mysql" GRAPH_DB_NAME = "graph" #-- app config -- DEBUG = True SECRET_KEY = "secret-key" SESSION_COOKIE_NAME = "open-falcon" PERMANENT_SESSION_LIFETIME = 3600 * 24 * 30 SITE_COOKIE = "open-falcon-ck" #-- query config -- QUERY_ADDR = "http://127.0.0.1:9966" #BASE_DIR = "/home/falcon/open-falcon/dashboard/" BASE_DIR="/home/falcon/data/6070" #和graph新建的數據存儲目錄相同 LOG_PATH = os.path.join(BASE_DIR,"log/") try: from rrd.local_config import * except: pass

 

 

7.task

cd /home/falcon/tmp/task mv cfg.example.json cfg.json

#修改配置文件
[falcon@open-falcon-demo task]$ more cfg.json { "debug": false, "http": { "enable": true, "listen": "0.0.0.0:8002" }, "index": { "enable": true, "dsn": "root:mysql@tcp(127.0.0.1:3306)/graph?loc=Local&parseTime=true", #MySQL的root密碼 "maxIdle": 4, "autoDelete": false, "cluster":{ "test.hostname01:6071" : "0 0 0 ? * 0-5", "test.hostname02:6071" : "0 30 0 ? * 0-5" } }, "collector" : { "enable": true, "destUrl" : "http://127.0.0.1:1988/v1/push", "srcUrlFmt" : "http://%s/statistics/all", "cluster" : [ "transfer,test.hostname:6060", "graph,test.hostname:6071", "task,test.hostname:8001" ] } }

#啓動task [falcon@open-falcon-demo task]$ ./control start #查看啓動狀態 [falcon@open-falcon-demo task]$ ./control status #查看日誌 [falcon@open-falcon-demo task]$ ./control tail #重啓 [falcon@open-falcon-demo task]$ ./control restart

 

 

4、安裝Open-Falcon報警相關組件

1.Sender

調用各個公司提供的mail-provider和sms-provider，按照某個併發度，從redis中讀取郵件、短信併發送，alarm生成的報警短信和報警郵件都是直接寫入redis便可，sender來發送。

cd $WORKSPACE/sender/ mv cfg.example.json cfg.json # vi cfg.json # redis地址須要和後面的alarm、judge使用同一個 # queue維持默認 # worker是最多同時有多少個線程玩命得調用短信、郵件發送接口 # api要給出sms-provider和mail-provider的接口地址 ./control start

[falcon@open-falcon-demo sender]$ more cfg.json { "debug": false, "http": { "enabled": true, "listen": "0.0.0.0:6066" }, "redis": { "addr": "127.0.0.1:6379", "maxIdle": 5 }, "queue": { "sms": "/sms", "mail": "/mail" }, "worker": { "sms": 10, "mail": 50 }, "api": { "sms": "http://11.11.11.11:8000/sms", "mail": "http://11.11.11.11:9000/mail" } }

 

2.UIC(FE)

cd $WORKSPACE/fe/ mv cfg.example.json cfg.json # 請基於cfg.example.json 酌情修改相關配置項 # 啓動 ./control start # 查看日誌 ./control tail # 中止服務 ./control stop

[falcon@open-falcon-demo fe]$ more cfg.json { "log": "debug", "company": "MI", "http": { "enabled": true, "listen": "0.0.0.0:1234" }, "cache": { "enabled": true, "redis": "127.0.0.1:6379", "idle": 10, "max": 1000, "timeout": { "conn": 10000, "read": 5000, "write": 5000 } }, "salt": "", "canRegister": true, "ldap": { "enabled": false, "addr": "ldap.example.com:389", "baseDN": "dc=example,dc=com", "bindDN": "cn=mananger,dc=example,dc=com", "bindPasswd": "12345678", "userField": "uid", "attributes": ["sn","mail","telephoneNumber"] }, "uic": { "addr": "root:mysql@tcp(127.0.0.1:3306)/uic?charset=utf8&loc=Asia%2FChongqing", #紅色爲MySQL數據庫root密碼 "idle": 10, "max": 100 }, "shortcut": { "falconPortal": "http://192.168.102.141:5050/", #Portal訪問地址 "falconDashboard": "http://192.168.102.141:8081/", #Dashboard訪問地址 "falconAlarm": "http://192.168.102.141:9912/" #Alarm訪問地址 } }

 

 

3.Portal

portal是用於配置報警策略的地方。

yum install -y python-virtualenv  # run as root cd $WORKSPACE/portal/ virtualenv ./env ./env/bin/pip install -r pip_requirements.txt # vi frame/config.py # 1. 修改DB配置 # 2. SECRET_KEY設置爲一個隨機字符串 # 3. UIC_ADDRESS有兩個，internal配置爲FE模塊的內網地址，portal一般是和UIC在一個網段的， # 內網地址相互訪問速度快。external是終端用戶經過瀏覽器訪問的UIC地址，很重要！ # 4. 其餘配置可使用默認的 ./control start portal默認監聽在5050端口，瀏覽器訪問便可

more  /home/falcon/tmp/portal/frame/config.py # -*- coding:utf-8 -*- __author__ = 'Ulric Qin' # -- app config -- DEBUG = True # -- db config -- DB_HOST = "127.0.0.1" DB_PORT = 3306 DB_USER = "root" DB_PASS = "mysql" #數據庫密碼 DB_NAME = "falcon_portal" # -- cookie config -- SECRET_KEY = "4e.5tyg8-u9ioj" SESSION_COOKIE_NAME = "falcon-portal" PERMANENT_SESSION_LIFETIME = 3600 * 24 * 30 UIC_ADDRESS = { 'internal': 'http://127.0.0.1:1234', 'external': 'http://192.168.102.141:1234', #可經過瀏覽器訪問的地址 } UIC_TOKEN = '' MAINTAINERS = ['root'] CONTACT = 'ulric.qin@gmail.com' COMMUNITY = True try: from frame.local_config import * except Exception, e: print "[warning] %s" % e

 

4.HBS

心跳服務器，只依賴Portal的DB cd $WORKSPACE/hbs/ mv cfg.example.json cfg.json # vi cfg.json 把數據庫配置配置爲portal的db ./control start 若是先安裝的繪圖組件又來安裝報警組件，那應該已經安裝過agent了，hbs啓動以後會監聽一個http端口，一個rpc端口，agent要和hbs通訊，從新去修改agent的配置cfg.json，把heartbeat那項enabled設置爲true，並配置上hbs的rpc地址，./control restart重啓agent，以後agent就能夠和hbs心跳了

[falcon@open-falcon-demo hbs]$ more cfg.json { "debug": true, "database": "root:mysql@tcp(127.0.0.1:3306)/falcon_portal?loc=Local&parseTime=true", "hosts": "", "maxIdle": 100, "listen": ":6030", "trustable": [""], "http": { "enabled": true, "listen": "0.0.0.0:6031" } }

 

5.Judge

報警判斷模塊，judge依賴於HBS，因此得先搭建HBS

 cd $WORKSPACE/judge/ mv cfg.example.json cfg.json # vi cfg.json # remain: 這個配置指定了judge內存中針對某個數據存多少個點，好比host01這個機器的cpu.idle的值在內存中最多存多少個， # 配置報警的時候好比all(#3)，這個#後面的數字不能超過remain-1 # hbs: 配置爲hbs的地址，interval默認是60s，表示每隔60s從hbs拉取一次策略 # alarm: 報警event寫入alarm中配置的redis，minInterval表示連續兩個報警之間至少相隔的秒數，維持默認便可 ./control start

[falcon@open-falcon-demo judge]$ more cfg.json { "debug": true, "debugHost": "nil", "remain": 11, "http": { "enabled": true, "listen": "0.0.0.0:6081" }, "rpc": { "enabled": true, "listen": "0.0.0.0:6080" }, "hbs": { "servers": ["127.0.0.1:6030"], "timeout": 300, "interval": 60 }, "alarm": { "enabled": true, "minInterval": 300, "queuePattern": "event:p%v", "redis": { "dsn": "127.0.0.1:6379", "maxIdle": 5, "connTimeout": 5000, "readTimeout": 5000, "writeTimeout": 5000 } } }

 

 

6.Links

links組件的做用：當多個告警被合併爲一條告警信息時，短信中會附帶一個告警詳情的http連接地址，供用戶查看詳情。

# yum install -y python-virtualenv $ cd $WORKSPACE/links/ $ virtualenv ./env $ ./env/bin/pip install -r pip_requirements.txt

./control start ./control status ./control  tail

 

cd /home/falcon/tmp/links/frame [falcon@open-falcon-demo frame]$ more config.py # -*- coding:utf-8 -*- __author__ = 'Ulric Qin' # -- app config -- DEBUG = True # -- db config -- DB_HOST = "127.0.0.1" DB_PORT = 3306 DB_USER = "root" DB_PASS = "mysql" DB_NAME = "falcon_links" # -- cookie config -- SECRET_KEY = "4e.5tyg8-u9ioj" SESSION_COOKIE_NAME = "falcon-links" PERMANENT_SESSION_LIFETIME = 3600 * 24 * 30

try: from frame.local_config import * except Exception, e: print "[warning] %s" % e

 

7.Alarm

alarm模塊是處理報警event的，judge產生的報警event寫入redis，alarm從redis讀取，這個模塊被業務搞得很糟亂，各個公司能夠根據本身公司的需求重寫.

cd $WORKSPACE/alarm/ mv cfg.example.json cfg.json # vi cfg.json # 把redis配置成與judge同一個 ./control start

注意，alarm當前的版本，highQueues和lowQueues都不能爲空，是個bug，稍候修復。咱們能夠把event:p0~event:p5配置到highQueues，把event:p6配置到lowQueues

[falcon@open-falcon-demo alarm]$ more cfg.json { "debug": true, "uicToken": "", "http": { "enabled": true, "listen": "0.0.0.0:9912" }, "queue": { "sms": "/sms", "mail": "/mail" }, "redis": { "addr": "127.0.0.1:6379", "maxIdle": 5, "highQueues": [ "event:p0", "event:p1", "event:p2", "event:p3", "event:p4", "event:p5" ], "lowQueues": [ "event:p6" ], "userSmsQueue": "/queue/user/sms", "userMailQueue": "/queue/user/mail" }, "api": { "portal": "http://192.168.102.141:5050", "uic": "http://127.0.0.1:1234", "links": "http://192.168.102.141:5090" } }

 

PS：本例安裝open-falcon時是使用falcon用戶安裝的。

falcon用戶的家目錄是:/home/falcon

全部配置好的配置文件的打包在這裏：https://pan.baidu.com/s/1ii6r0-iJYYt4Mn_WzHcfcw

 

【agent】
http://192.168.102.141:1988/
【dashboard】
http://192.168.102.141:8081/
【uic/fe】
http://192.168.102.141:1234/
【Portal】
http://192.168.102.141:5050/
【alarm】
http://192.168.102.141:9912/

手動觸發graphcurl -s "http://127.0.0.1:6071/index/updateAll"