運維監控系統之Open-Falcon

時間 2019-11-17

標籤監控系統 open falcon 简体版

原文原文鏈接

1、Open-Falcon介紹

一、監控系統，能夠從運營級別（基本配置便可），以及應用級別（二次開發，經過端口進行日誌上報），對服務器、操做系統、中間件、應用進行全面的監控，及報警，對咱們的系統正常運行的做用很是重要。html

二、基礎監控前端

CPU、Load、內存、磁盤、IO、網絡相關、內核參數、ss 統計輸出、端口採集、核心服務的進程存活信息採集、關鍵業務進程資源消耗、NTP offset採集、DNS解析採集，這些指標，都是open-falcon的agent組件直接支持的。java

Linux運維基礎採集項：http://book.open-falcon.org/zh/faq/linux-metrics.htmlpython

對於這些基礎監控選項所有理解透徹的時刻，也就是對Linux運行原理及命令進階的時刻。mysql

三、第三方監控linux

術業有專攻，運行在OS上的應用甚多，Open-Falcon的開發團隊不可能把全部的第三方應用的監控所有作完，這個就須要開源社區提供更多的插件，當前對於不少經常使用的第三方應用都有相關插件了。ios

四、JVM監控git

對於Java做爲主要開發語言的大多數公司，對於JVM的監控不可或缺。github

每一個JVM應用的參數，好比GC、類加載、JVM內存、進程、線程，均可以上報給Falcon，而這些參數的得到，均可以經過MxBeans實現。golang

使用 Java 平臺管理 bean：http://www.ibm.com/developerworks/cn/java/j-mxbeans/

五、業務應用監控

對於業務須要監控的接口，好比響應時間等。能夠根據業務的須要，上報相關數據到Falcon，並經過Falcon查看結果。

2、Zabbix、Nagios、Open-Falcon這3大開源運維監控工具的比較

3、Open-Falcon編寫的整個腦洞歷程

參考：http://mp.weixin.qq.com/s?__biz=MjM5OTcxMzE0MQ==&mid=400225178&idx=1&sn=c98609a9b66f84549e41cd421b4df74d

小米開源監控系統OpenFalcon應對高併發7種手段：http://h2ex.com/894

官網：

http://open-falcon.org/

https://book.open-falcon.org

https://github.com/open-falcon/falcon-plus/releases

說明：Open-Falcon部署分爲前端（客戶端）和後端（服務端）

4、後端部署（服務端）

參考官方文檔：https://book.open-falcon.org/zh_0_2/quick_install/prepare.html

一、系統環境

[root@open-falcon-server ~]# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

二、系統優化

#安裝下載軟件
yum install wget -y

#更換aliyun源
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

#下載epel源
yum install epel-release.noarch -y
rpm -Uvh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
yum clean all
yum makecache

#下載經常使用軟件
yum install git telnet net-tools tree nmap sysstat lrzsz dos2unix tcpdump ntpdate -y

#配置時間同步
ntpdate cn.pool.ntp.org

#更改主機名
hostnamectl set-hostname open-falcon-server
hostname open-falcon-server

#開啓緩存
sed -i 's#keepcache=0#keepcache=1#g' /etc/yum.conf 
grep keepcache /etc/yum.conf

#關閉selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

#關閉防火牆
systemctl stop firewalld.service
systemctl disable firewalld.service

三、軟件環境準備

#安裝 redis

yum install redis -y

#redis經常使用命令

redis-server     redis 服務端

redis-cli　　　　redis 命令行客戶端

redis-benchmark  redis 性能測試工具

redis-check-aof 　　AOF文件修復工具

redis-check-dump  RDB文件修復工具

redis-sentinel 　　  Sentinel 服務端

#啓動redis

#啓動redis
[root@open-falcon-server ~]# redis-server &
[1] 3308
[root@open-falcon-server ~]# 3308:C 27 Oct 08:37:26.435 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
3308:M 27 Oct 08:37:26.436 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.10 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 3308
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

3308:M 27 Oct 08:37:26.437 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
3308:M 27 Oct 08:37:26.437 # Server started, Redis version 3.2.10
3308:M 27 Oct 08:37:26.437 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
3308:M 27 Oct 08:37:26.438 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
3308:M 27 Oct 08:37:26.438 * The server is now ready to accept connections on port 6379

#安裝mysql

[root@open-falcon-server ~]#wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm  

[root@open-falcon-server ~]# ll
total 12
-rw-------. 1 root root 1262 Oct 27  2017 anaconda-ks.cfg
-rw-r--r--. 1 root root 6140 Nov 12  2015 mysql-community-release-el7-5.noarch.rpm

[root@open-falcon-server ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-release-el7-5    ################################# [100%]
[root@open-falcon-server ~]# ll /etc/yum.repos.d/
total 48
-rw-r--r--. 1 root root 2573 Nov 21  2014 CentOS-Base.repo
-rw-r--r--. 1 root root 1664 Dec  9  2015 CentOS-Base.repo.backup
-rw-r--r--. 1 root root 1309 Dec  9  2015 CentOS-CR.repo
-rw-r--r--. 1 root root  649 Dec  9  2015 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root  290 Dec  9  2015 CentOS-fasttrack.repo
-rw-r--r--. 1 root root  630 Dec  9  2015 CentOS-Media.repo
-rw-r--r--. 1 root root 1331 Dec  9  2015 CentOS-Sources.repo
-rw-r--r--. 1 root root 1952 Dec  9  2015 CentOS-Vault.repo
-rw-r--r--. 1 root root  951 Jun 24 23:04 epel.repo
-rw-r--r--. 1 root root 1050 Jun 24 23:04 epel-testing.repo
-rw-r--r--. 1 root root 1209 Jan 29  2014 mysql-community.repo
-rw-r--r--. 1 root root 1060 Jan 29  2014 mysql-community-source.repo

[root@open-falcon-server ~]#yum install -y mysql-server

#啓動mysql並設置開機自啓動
[root@open-falcon-server ~]# systemctl start mysqld.service
[root@open-falcon-server ~]# systemctl enable mysqld.service

[root@open-falcon-server ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.38 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye

#檢查服務

[root@open-falcon-server ~]# netstat -lntup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      3308/redis-server * 
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1443/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1634/master         
tcp6       0      0 :::3306                 :::*                    LISTEN      3283/mysqld         
tcp6       0      0 :::6379                 :::*                    LISTEN      3308/redis-server * 
tcp6       0      0 :::22                   :::*                    LISTEN      1443/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1634/master         
udp        0      0 0.0.0.0:68              0.0.0.0:*                           2042/dhclient       
udp        0      0 0.0.0.0:4817            0.0.0.0:*                           2042/dhclient       
udp6       0      0 :::20380                :::*                                2042/dhclient

四、初始化MySQL表結構（備註：導入表結構後，注意檢查）

cd /tmp/ && git clone https://github.com/open-falcon/falcon-plus.git 
cd /tmp/falcon-plus/scripts/mysql/db_schema/
mysql -h 127.0.0.1 -u root -p < 1_uic-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 2_portal-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 3_dashboard-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 4_graph-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql
rm -rf /tmp/falcon-plus/

#設置數據庫密碼 (備註：用戶名：root ,密碼: password )

[root@open-falcon-server open-falcon]# mysqladmin -uroot password "password"
Warning: Using a password on the command line interface can be insecure.

#檢查導入的數據庫表

[root@open-falcon-server open-falcon]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 35
Server version: 5.6.38 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| alarms             |
| dashboard          |
| falcon_portal      |
| graph              |
| mysql              |
| performance_schema |
| uic                |
+--------------------+
8 rows in set (0.00 sec)

mysql> exit
Bye

五、配置Go語言開發環境

#安裝go語言開發包（備註：安裝須要epel源）

yum install golang -y

#檢查版本

[root@open-falcon-server ~]# go version
 go version go1.8.3 linux/amd64

#查看go的安裝路徑

[root@open-falcon-server ~]# find / -name go
/etc/alternatives/go
/var/lib/alternatives/go
/usr/bin/go
/usr/lib/golang/src/cmd/go  #須要的是這個路徑
/usr/lib/golang/src/go
/usr/lib/golang/bin/go
/usr/lib/golang/pkg/linux_amd64/go

#設置環境變量GOROOT和GOPATH

export GOROOT=/usr/lib/golang
export GOPATH=/home

六、下載編譯好的二進制版本

https://github.com/open-falcon/falcon-plus/releases

直接下載官方編譯好的二進制版本(x86 64位平臺)。

#開始安裝open-falcon

[root@open-falcon-server ~]# export FALCON_HOME=/home/work
[root@open-falcon-server ~]# export WORKSPACE=$FALCON_HOME/open-falcon
[root@open-falcon-server ~]# mkdir -p $WORKSPACE

#上傳軟件
[root@open-falcon-server ~]# ll
-rw-r--r--. 1 root root 40031472 Oct 29 2017 open-falcon-v0.2.1.tar.gz

#解壓
[root@open-falcon-server ~]# tar -xf open-falcon-v0.2.1.tar.gz -C $WORKSPACE
[root@open-falcon-server ~]# cd $WORKSPACE

#查看解壓結果
[root@open-falcon-server open-falcon]# ll
total 3896
drwxrwxr-x. 7 501 501      67 Aug 15 19:16 agent
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 aggregator
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 alarm
drwxrwxr-x. 6 501 501      51 Aug 15 19:16 api
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 gateway
drwxrwxr-x. 6 501 501      51 Aug 15 19:16 graph
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 hbs
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 judge
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 nodata
-rwxrwxr-x. 1 501 501 3987469 Aug 15 19:16 open-falcon
lrwxrwxrwx. 1 501 501      16 Aug 15 19:16 plugins -> ./agent/plugins/
lrwxrwxrwx. 1 501 501      15 Aug 15 19:16 public -> ./agent/public/
drwxrwxr-x. 5 501 501      40 Aug 15 19:16 transfer

七、修改配置文件cfg.json

坑：

部分模塊依賴鏈接數據庫，由於若是不修改配置文件，aggregator模塊會出現沒法啓動，graph、hbs、nodata、api、alarm模塊會出現開啓不報錯可是狀態爲開啓失敗的狀況。

若是須要每一個模塊都能正常啓動，須要將上面模塊的cfg.json的數據庫信息進行修改。根據本教程的配置，須要修改配置文件所在的目錄：

模塊	配置文件所在路徑
aggregator	/home/work/aggregator/config/cfg.json
graph	/home/work/graph/config/cfg.json
hbs	/home/work/hbs/config/cfg.json
nodata	/home/work/nodata/config/cfg.json
api	/home/work/api/config/cfg.json
alarm	/home/work/alarm/config/cfg.json

1）修改aggregator的配置文件

vim /home/work/aggregator/config/cfg.json