CentOS 7下安裝Logstash ELK Stack 日誌管理系統（下）

[root@elk elk]# firewall-cmd --permanent --add-port=5601/tcp
Success
[root@elk elk]# firewall-cmd --reload
success
[root@elk elk]# firewall-cmd --list-all
public (default, active)
interfaces: eno16777984 eno33557248
sources:
services: dhcpv6-client ssh
ports: 9200/tcp 9300/tcp 5601/tcp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:

3.5 安裝kibana

[root@elk elk]# yum localinstall kibana-4.5.1-1.x86_64.rpm –y
[root@elk elk]# systemctl enable kibana
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /usr/lib/systemd/system/kibana.service.
[root@elk elk]# systemctl start kibana

[root@elk elk]# systemctl status kibana
● kibana.service - no description given
Loaded: loaded (/usr/lib/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2016-05-20 15:49:02 CST; 20s ago
Main PID: 11260 (node)
CGroup: /system.slice/kibana.service
└─11260 /opt/kibana/bin/../node/bin/node /opt/kibana/bin/../src/cli

May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:elasticsearch...May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:kbn_vi...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:markdo...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:metric...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:spyMod...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:status...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["status","plugin:table_...lized"}
May 20 15:49:05 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:05+00:00","tags":["listening","info"],"pi...:5601"}
May 20 15:49:10 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:10+00:00","tags":["status","plugin:elasticsearch...May 20 15:49:14 elk.test.com kibana[11260]: {"type":"log","@timestamp":"2016-05-20T07:49:14+00:00","tags":["status","plugin:elasti...found"}
Hint: Some lines were ellipsized, use -l to show in full.

檢查kibana服務運行（Kibana默認進程名：node ，端口5601）

[root@elk elk]# netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 909/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1595/master
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 11260/node

修改防火牆，對外開放tcp/5601

[root@elk elk]# firewall-cmd --permanent --add-port=5601/tcp
Success
[root@elk elk]# firewall-cmd --reload
success
[root@elk elk]# firewall-cmd --list-all
public (default, active)
interfaces: eno16777984 eno33557248
sources:
services: dhcpv6-client ssh
ports: 9200/tcp 9300/tcp 5601/tcp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:

這時，咱們能夠打開瀏覽器，測試訪問一下kibana服務器http://192.168.30.67:5601/，確認沒有問題，以下圖：

在這裏，咱們能夠修改防火牆，將用戶訪問80端口鏈接轉發到5601上，這樣能夠直接輸入網址不用指定端口了，以下:

[root@elk elk]# firewall-cmd --permanent --add-forward-port=port=80:proto=tcp:toport=5601[root@elk elk]# firewall-cmd --reload[root@elk elk]# firewall-cmd --list-all
public (default, active)
interfaces: eno16777984 eno33557248
sources:
services: dhcpv6-client ssh
ports: 9200/tcp 9300/tcp 5601/tcp
masquerade: no
forward-ports: port=80:proto=tcp:toport=5601:toaddr=
icmp-blocks:
rich rules:

3.6 安裝logstash，以及添加配置文件

[root@elk elk]# yum localinstall logstash-2.3.2-1.noarch.rpm –y

生成證書

[root@elk elk]# cd /etc/pki/tls/[root@elk tls]# lscert.pem certs misc openssl.cnf private

[root@elk tls]# openssl req -subj '/CN=elk.test.com/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out
certs/logstash-forwarder.crt
Generating a 2048 bit RSA private key
...................................................................+++......................................................+++writing new private key to 'private/logstash-forwarder.key'-----

以後建立logstash 的配置文件。以下：

View Code

啓動logstash，並檢查端口，配置文件裏，咱們寫的是5000端口

[root@elk conf.d]# systemctl start logstash
[root@elk elk]# /sbin/chkconfig logstash on
[root@elk conf.d]# netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 909/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1595/master
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 11260/node
tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 618/rsyslogd
tcp6 0 0 :::5000 :::* LISTEN 12819/java
tcp6 0 0 :::3306 :::* LISTEN 1270/mysqld
tcp6 0 0 127.0.0.1:9200 :::* LISTEN 10430/java
tcp6 0 0 ::1:9200 :::* LISTEN 10430/java
tcp6 0 0 127.0.0.1:9300 :::* LISTEN 10430/java
tcp6 0 0 ::1:9300 :::* LISTEN 10430/java
tcp6 0 0 :::22 :::* LISTEN 909/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1595/master
tcp6 0 0 :::514 :::* LISTEN 618/rsyslogd

修改防火牆，將5000端口對外開放。

[root@elk ~]# firewall-cmd --permanent --add-port=5000/tcp
success
[root@elk ~]# firewall-cmd --reload
success
[root@elk ~]# firewall-cmd --list-all
public (default, active)
interfaces: eno16777984 eno33557248
sources:
services: dhcpv6-client ssh
ports: 9200/tcp 9300/tcp 5000/tcp 5601/tcp
masquerade: no
forward-ports: port=80:proto=tcp:toport=5601:toaddr=
icmp-blocks:
rich rules:

3.7 修改elasticsearch 配置文件

查看目錄，建立文件夾es-01（名字不是必須的）,logging.yml是自帶的，elasticsearch.yml是建立的文件，內如見下：

[root@elk ~]# cd /etc/elasticsearch/[root@elk elasticsearch]# tree
.
├── es-01│ ├── elasticsearch.yml
│ └── logging.yml
└── scripts

[root@elk elasticsearch]# cat es-01/elasticsearch.yml
----http:
port: 9200network:
host: elk.test.com
node:
name: elk.test.com
path:
data: /etc/elasticsearch/data/es-01

3.8 重啓elasticsearch、logstash服務。

3.9 將 fiebeat安裝包拷貝到 rsyslog、nginx 客戶端上

[root@elk elk]# scp filebeat-1.2.3-x86_64.rpm root@rsyslog.test.com:/root/elk
[root@elk elk]# scp filebeat-1.2.3-x86_64.rpm root@nginx.test.com:/root/elk
[root@elk elk]# scp /etc/pki/tls/certs/logstash-forwarder.crt rsyslog.test.com:/root/elk
[root@elk elk]# scp /etc/pki/tls/certs/logstash-forwarder.crt nginx.test.com:/root/elk

客戶端部署filebea

filebeat客戶端是一個輕量級的，從服務器上的文件收集日誌資源的工具，這些日誌轉發處處理到Logstash服務器上。該Filebeat客戶端使用安全的Beats協議與Logstash實例通訊。lumberjack協議被設計爲可靠性和低延遲。Filebeat使用託管源數據的計算機的計算資源，而且Beats輸入插件儘可能減小對Logstash的資源需求。

4.1.（node1）安裝filebeat，拷貝證書，建立收集日誌配置文件

[root@rsyslog elk]# yum localinstall filebeat-1.2.3-x86_64.rpm -y
#拷貝證書到本機指定目錄中
[root@rsyslog elk]# cp logstash-forwarder.crt /etc/pki/tls/certs/.
[root@rsyslog elk]# cd /etc/filebeat/[root@rsyslog filebeat]# tree
.
├── conf.d
│ ├── authlogs.yml
│ └── syslogs.yml
├── filebeat.template.json
└── filebeat.yml1 directory, 4 files

修改的文件有3個，filebeat.yml，是定義鏈接logstash 服務器的配置。conf.d目錄下的2個配置文件是自定義監控日誌的，下面看下各自的內容：

filebeat.yml

View Code

authlogs.yml & syslogs.yml

View Code

修改完成後，啓動filebeat服務

[root@rsyslog filebeat]# service filebeat start
Starting filebeat: [ OK ]
[root@rsyslog filebeat]# chkconfig filebeat on

[root@rsyslog filebeat]# netstat -altp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 localhost:25151 *:* LISTEN 6230/python2
tcp 0 0 *:ssh *:* LISTEN 5509/sshd
tcp 0 0 localhost:ipp *:* LISTEN 1053/cupsd
tcp 0 0 localhost:smtp *:* LISTEN 1188/master
tcp 0 0 rsyslog.test.com:51155 elk.test.com:commplex-main ESTABLISHED 7443/filebeat
tcp 0 52 rsyslog.test.com:ssh 192.168.30.65:10580 ESTABLISHED 7164/sshd
tcp 0 0 *:ssh *:* LISTEN 5509/sshd
tcp 0 0 localhost:ipp *:* LISTEN 1053/cupsd
tcp 0 0 localhost:smtp *:* LISTEN 1188/master

若是鏈接不上，狀態不正常的話，檢查下客戶端的防火牆。

4.2. （node2）安裝filebeat，拷貝證書，建立收集日誌配置文件

[root@nginx elk]# yum localinstall filebeat-1.2.3-x86_64.rpm -y
[root@nginx elk]# cp logstash-forwarder.crt /etc/pki/tls/certs/.
[root@nginx elk]# cd /etc/filebeat/[root@nginx filebeat]# tree
.
├── conf.d
│ ├── nginx.yml
│ └── syslogs.yml
├── filebeat.template.json
└── filebeat.yml1 directory, 4 files

修改filebeat.yml 內容以下：

View Code

syslogs.yml & nginx.yml

View Code

修改完成後，啓動filebeat服務，並檢查filebeat進程

[root@nginx filebeat]# service filebeat start
Starting filebeat: [ OK ]
[root@nginx filebeat]# chkconfig filebeat on

[root@nginx filebeat]# netstat -aulpt
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 *:ssh *:* LISTEN 1076/sshd
tcp 0 0 localhost:smtp *:* LISTEN 1155/master
tcp 0 0 *:http *:* LISTEN 1446/nginx
tcp 0 52 nginx.test.com:ssh 192.168.30.65:11690 ESTABLISHED 1313/sshd
tcp 0 0 nginx.test.com:49500 elk.test.com:commplex-main ESTABLISHED 1515/filebeat
tcp 0 0 nginx.test.com:ssh 192.168.30.65:6215 ESTABLISHED 1196/sshd
tcp 0 0 nginx.test.com:ssh 192.168.30.65:6216 ESTABLISHED 1200/sshd
tcp 0 0 *:ssh *:* LISTEN 1076/sshd

經過上面能夠看出，客戶端filebeat進程已經和 elk 服務器鏈接了。下面去驗證。

5、驗證，訪問kibana http://192.168.30.67

查看下兩臺機器的系統日誌：node1的

node2的nginx 訪問日誌

體驗

以前在學習rsyslog +LogAnalyzer，而後又學了這個以後，發現elk 無論從總體系統，仍是體驗都是不錯的，並且更新快。後續會繼續學習，更新相關的監控過濾日誌方法，日誌分析，以及使用kafka 來進行存儲的架構。