分佈式服務器監控工具NebulaSolarDash

分佈式服務器監控工具NebulaSolarDash  

詳細介紹：https://www.oschina.net/p/nebula-solar-dash

github:   https://github.com/toddlerya/NebulaSolarDash#readme

      工具分爲客戶端和服務端兩部分： 服務端使用了 bottle 來做爲 Web 框架，Echarts 來渲染生成圖表；客戶端使用 Python 原生類庫採集服務器資源。

* 以客戶端採集數據間隔時間120s爲例，單節點24小時會向數據庫寫入大約4MB數據。
* 單個客戶端每次採集發送到服務端寫入數據庫的信息大概在5~6kb左右，請自行結合服務器個數以及監控時長和服務器存儲自行設定監控間隔。

一、下載安裝包NebulaSolarDash並解壓：

[root@nginx1 ~]# unzip toddlerya-NebulaSolarDash-2.0.1-0-g58fe715.zip 

Archive:  toddlerya-NebulaSolarDash-2.0.1-0-g58fe715.zip

58fe71551f72441964ebcb7bb30fc0e436c9868c

   creating: toddlerya-NebulaSolarDash-58fe715/

  inflating: toddlerya-NebulaSolarDash-58fe715/LICENSE  

  inflating: toddlerya-NebulaSolarDash-58fe715/__init__.py  

   creating: toddlerya-NebulaSolarDash-58fe715/assets/

   creating: toddlerya-NebulaSolarDash-58fe715/assets/css/

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/css/bootstrap.min.css  

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/css/ns_tb.css  

   creating: toddlerya-NebulaSolarDash-58fe715/assets/js/

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/js/bootstrap.min.js  

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/js/dark.js  

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/js/echarts.min.js  

   creating: toddlerya-NebulaSolarDash-58fe715/assets/picture/

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/picture/NebulaSolarDash.gif  

  inflating: toddlerya-NebulaSolarDash-58fe715/assets/picture/NebulaSolarDash2.0.gif  

   creating: toddlerya-NebulaSolarDash-58fe715/conf/

  inflating: toddlerya-NebulaSolarDash-58fe715/conf/__init__.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/conf/ns.ini  

  inflating: toddlerya-NebulaSolarDash-58fe715/init_db.py  

   creating: toddlerya-NebulaSolarDash-58fe715/lib/

  inflating: toddlerya-NebulaSolarDash-58fe715/lib/__init__.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/lib/bottle.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/lib/common_lib.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/manager.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/ns_agent.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/ns_server.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/readme.md  

  inflating: toddlerya-NebulaSolarDash-58fe715/release-note.txt  

  inflating: toddlerya-NebulaSolarDash-58fe715/run.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/start_agent.sh  

  inflating: toddlerya-NebulaSolarDash-58fe715/start_insall_app.sh  

  inflating: toddlerya-NebulaSolarDash-58fe715/stop.py  

  inflating: toddlerya-NebulaSolarDash-58fe715/stop_uninstall_app.sh  

  inflating: toddlerya-NebulaSolarDash-58fe715/uninstall_app.sh  

   creating: toddlerya-NebulaSolarDash-58fe715/views/

  inflating: toddlerya-NebulaSolarDash-58fe715/views/agent_info.tpl  

  inflating: toddlerya-NebulaSolarDash-58fe715/views/each_agent_detail.tpl  

二、修改配置文件即設置server與client：

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# pwd

/root/toddlerya-NebulaSolarDash-58fe715

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# vim conf/ns.ini 

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# cat conf/ns.ini 

[server]

; 服務端IP

ip = 172.25.254.130

; 服務端端口號

port = 8081

debug = True

;報警信息閾值,百分比

;舉例：

;cpu_yellow = 80，表明cpu使用率達到80%即提示使用×××標示

;cpu_red = 95，表明cpu使用率達到95%即提示使用×××標示

mem_yellow = 80

mem_red = 95

cpu_yellow = 80

cpu_red = 95

[agent]

; 客戶端採集數據間隔時間, 單位是s

interval = 60

install_path = /home/RunTimeNSDash

;全部須要監控的節點的ip,以英文逗號分隔

[all_agent_ip]

ips = 172.25.254.134,172.25.254.135

三、出現驗證問題，接下來進行無祕鑰操做：

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# python manager.py -install

[+] 開始安裝客戶端到各個節點並自動啓動客戶端以及服務端

[+] 設置安裝目錄成功: /home/RunTimeNSDash

[+] 刪除歷史數據成功

[+] 啓動服務端成功

[+] 這次安裝的節點共計 2 個

[09/06/17 18:38:44] : INFO    : 校驗服務器連通性: 172.25.254.134

[09/06/17 18:38:44] : INFO    : 開始部署

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[09/06/17 18:38:44] : ERROR   : can not logon 172.25.254.134 without passwd.

[09/06/17 18:38:44] : INFO    : 校驗服務器連通性: 172.25.254.135

[09/06/17 18:38:44] : INFO    : 開始部署

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[09/06/17 18:38:44] : ERROR   : can not logon 172.25.254.135 without passwd.

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# ss

ss           ssh          ssh-agent    sshd         ssh-keygen   ssltap       

sserver      ssh-add      ssh-copy-id  sshd-keygen  ssh-keyscan  

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# ssh-keygen 

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa): 

/root/.ssh/id_rsa already exists.

Overwrite (y/n)? n

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# ssh-copy-id root@172.25.254.134

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

root@172.25.254.134's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@172.25.254.134'"

and check to make sure that only the key(s) you wanted were added.

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# ssh-copy-id root@172.25.254.135

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

root@172.25.254.135's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@172.25.254.135'"

and check to make sure that only the key(s) you wanted were added.

四、進行安裝部署操做

運行參數：

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# python manager.py -h

usage: manager.py [-h] [-install] [-uninstall] [-startall] [-stopall]

                  [-start START_ONE] [-stop STOP_ONE]

Manager Tool

optional arguments:

  -h, --help        show this help message and exit

  -install          安裝客戶端到各個節點並自動啓動客戶端以

                    服務端

  -uninstall        中止各個節點的客戶端並中止程序清理安裝

                    件，同時中止服務端

  -startall         啓動各個節點的客戶端並設置crond守護

  -stopall          中止各個節點的客戶端並去除crond守護

  -start START_ONE  啓動一個指定節點的客戶端並設置crond守護

  -stop STOP_ONE    中止一個指定節點的客戶端並去除crond守護

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# python manager.py -install

[+] 開始安裝客戶端到各個節點並自動啓動客戶端以及服務端

[+] 設置安裝目錄成功: /home/RunTimeNSDash

[+] 刪除歷史數據成功

[+] 啓動服務端成功

[+] 這次安裝的節點共計 2 個

[09/06/17 18:39:25] : INFO    : 校驗服務器連通性: 172.25.254.134

[09/06/17 18:39:25] : INFO    : 開始部署

[09/06/17 18:39:27] : INFO    : 校驗服務器連通性: 172.25.254.135

[09/06/17 18:39:27] : INFO    : 開始部署

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# python manager.py -startall

[+] 啓動各個節點的客戶端並設置crond守護

[+] 這次安裝的節點共計 2 個

[09/06/17 18:40:18] : INFO    : 校驗服務器連通性: 172.25.254.134

[09/06/17 18:40:18] : INFO    : 開始部署

[09/06/17 18:40:20] : INFO    : 校驗服務器連通性: 172.25.254.135

[09/06/17 18:40:20] : INFO    : 開始部署

[root@nginx1 toddlerya-NebulaSolarDash-58fe715]# lsof -i:8081

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

python  7588 root    4u  IPv4  42008      0t0  TCP *:tproxy (LISTEN)

五、實驗的驗證

Nebua-Solar服務器資源監控節點列表

序號	主機名	IP地址	內存	CPU
1	host3	172.25.254.134	-109%	0.08%
2	web	172.25.254.135	1%	0.45%

節點基礎信息 -- 各個圖表均可以使用鼠標拖動和滾輪縮放

主機名	IP地址	CPU	內存(MB)	SWAP(MB)	操做系統	內核版本	運行時長	當前時間
host3	172.25.254.134	2 x AMD Athlon(tm) X4 730 Quad Core Processor	977	0	CentOS Linux 7.2.1511 Core	3.10.0-327.el7.x86_64	27 days, 4:51:37	20170901-18:35:42

節點磁盤存儲信息統計

序號	文件系統	總大小	已用	剩餘	使用率	掛載點
1	/dev/mapper/centos-root	18G	2.2G	16G	13%	/
2	devtmpfs	479M	0	479M	0%	/dev
3	tmpfs	489M	0	489M	0%	/dev/shm
4	tmpfs	489M	50M	440M	11%	/run
5	tmpfs	489M	0	489M	0%	/sys/fs/cgroup
6	/dev/sda1	497M	126M	372M	26%	/boot
7	tmpfs	98M	0	98M	0%	/run/user/0

20170901-18:24:26
USAGE(%) : 0.08
NICE(%) : 0
USER(%) : 0.01
SYSTEM(%) : 0.06
IOWAIT(%) : 0.01

0.0668

20170901-18:31:27
平均負載值 : 0

節點基礎信息 -- 各個圖表均可以使用鼠標拖動和滾輪縮放

主機名	IP地址	CPU	內存(MB)	SWAP(MB)	操做系統	內核版本	運行時長	當前時間
web	172.25.254.135	2 x AMD Athlon(tm) X4 730 Quad Core Processor	1823	0	CentOS Linux 7.2.1511 Core	3.10.0-514.26.2.el7.x86_64	41 days, 4:19:10	20170906-19:02:46

節點磁盤存儲信息統計

序號	文件系統	總大小	已用	剩餘	使用率	掛載點
1	/dev/mapper/centos-root	18G	12G	6.1G	66%	/
2	devtmpfs	897M	0	897M	0%	/dev
3	tmpfs	912M	144K	912M	1%	/dev/shm
4	tmpfs	912M	99M	814M	11%	/run
5	tmpfs	912M	0	912M	0%	/sys/fs/cgroup
6	/dev/sda1	497M	190M	307M	39%	/boot
7	tmpfs	183M	32K	183M	1%	/run/user/0
8	/dev/sr0	4.1G	4.1G	0	100%	/run/media/root/CentOS

20170906-18:57:32
USAGE(%) : 0.45
NICE(%) : 0.01
USER(%) : 0.12
SYSTEM(%) : 0.31
IOWAIT(%) : 0

20170906-18:57:32

20170906-18:55:31平均負載值 : 0