elk

ELK之一:ELK基礎及安裝部署

 

一：什麼是ELK？

1.1：ELK又稱爲ELK Stack,是 Elasticsearch、Logstash、Kibana 三個開源軟件的組合,每一個完成不一樣的功能，Elasticsearch 可實現數據的實時全文搜索搜索、支持分佈式可實現高可用、提供API接口，能夠處理大規模日誌數據，好比Nginx、Tomcat、系統日誌等功能，官方地址：https://www.elastic.co/

1.2：Logstash:經過插件實現日誌收集，支持日誌過濾，支持普通log、自定義json格式的日誌解析:

1.3：kibana主要是調用elasticsearch的數據，並進行前端數據可視化的展示：

二：安裝部署

2.1：安裝環境準備：

2.1.1：系統環境部分

 

兩臺服務器: Server1:主機名：elkserver1 IP地址：192.168.0.4 Server2：主機名：elkserver2 IP地址：192.168.0.31 操做系統：Centos 7.2.11 x86_64 Server1： systemctl disable firewalld #開機關閉防火牆 sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config #開機關閉selinux echo "* soft nofile 65536" >> /etc/security/limits.conf #修改進程打開最大文件描述符限制 echo "* hard nofile 65536" >> /etc/security/limits.conf 

 

2.1.2：兩臺服務器分別安裝java運行環境，能夠安裝二進制(須要配置profile環境變量)也能夠安裝rpm包，本文采用下載好的jdk-8u92：

java下載地址：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

[root@elkserver2 tianqi]# ll  jdk-8u92-linux-x64.rpm

[root@elkserver1 tianqi]# yum install jdk-8u92-linux-x64.rpm

2.1.3：軟件包準備,能夠配置yum源安裝也能夠使用rpm包或二進制包，推薦在官網下載rpm包下載：

elasticsearch:官網下載地址：https://www.elastic.co/downloads/elasticsearch ，當前最新版本2.3.5

logstash：官網下載地址：https://www.elastic.co/downloads/logstash，當前最新版本2.3.4

kibana:官網下載地址:https://www.elastic.co/downloads/kibana，當前最新版本4.5.4

2.2：安裝部署Eelasticsearch及集羣：

2.2.1：兩臺服務器分別都執行安裝Eelasticsearch：

2.2.2:編輯elasticsearch配置文件：

Server1：

 

[root@elkserver1 tianqi]# grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml cluster.name: hfelk #集羣名稱，名稱相同即屬於同一個集羣 node.name: elkserver1 #本機在集羣的內的名稱 path.data: /els/data #保存數據的目錄，此目錄空間要大IO要高 path.logs: /els/logs #保存日誌的目錄 bootstrap.mlockall: true #服務啓動的時候鎖定內存，防止寫入swap network.host: 0.0.0.0 #監聽地址 http.port: 9200 #監聽端口 discovery.zen.ping.unicast.hosts: ["192.168.0.4", "192.168.0.31"] #建立集羣的時候組播地址，用於廣播無效的狀況下

Server2：

[root@elkserver2 tianqi]# grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml cluster.name: hfelk node.name: elkserver2 #最大的不一樣就是node名稱不同，其餘都一致 path.data: /els/data path.logs: /els/logs bootstrap.mlockall: true network.host: 0.0.0.0 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.0.4", "192.168.0.31"]

2.2.3：分別在兩臺服務器建立保存數據和日誌的目錄並對elastic用戶受權：

Server1：

[root@elkserver1 tianqi]# mkdir /els/{data,logs} -pv mkdir: created directory ‘/els’ mkdir: created directory ‘/els/data’ mkdir: created directory ‘/els/logs’ [root@elkserver2 tianqi]# chown elasticsearch.elasticsearch /els/ -R

Server2：

[root@elkserver2 tianqi]# mkdir /els/{data,logs} -pv mkdir: created directory ‘/els’ mkdir: created directory ‘/els/data’ mkdir: created directory ‘/els/logs’ [root@elkserver2 tianqi]# chown elasticsearch.elasticsearch /els/ -R

2.2.4：分別啓動elasticsearch服務：

[root@elkserver1 tianqi]# systemctl start elasticsearch [root@elkserver2 tianqi]# systemctl start elasticsearch

#Server1啓動成功的日誌信息，若是啓動不成功則根據日誌進行排錯，檢查是否java不知足環境仍是elasticsearch用戶對數據目錄沒有寫入權限：

[2016-08-22 05:06:28,601][INFO ][node ] [elkserver1] initialized [2016-08-22 05:06:28,601][INFO ][node ] [elkserver1] starting ... [2016-08-22 05:06:28,802][INFO ][transport ] [elkserver1] publish_address {192.168.0.4:9300}, bound_addresses {[::]:9300} [2016-08-22 05:06:28,813][INFO ][discovery ] [elkserver1] hfelk/9gKGXIUfThC6iixl8kXXFQ [2016-08-22 05:06:31,924][INFO ][cluster.service #master選舉爲elkserver2 ] [elkserver1] detected_master {elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300}, added {{elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300},}, reason: zen-disco-receive(from master [{elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300}]) [2016-08-22 05:06:32,040][INFO ][http ] [elkserver1] publish_address {192.168.0.4:9200}, bound_addresses {[::]:9200} [2016-08-22 05:06:32,040][INFO ][node ] [elkserver1] started

#查看端口狀態：

2.2.5：訪問elasticsearch的web界面：

2.3：elasticsearch的插件們：

#插件是爲了完成不一樣的功能，官方提供了一些插件可是是收費的，另外也有一些開發愛好者提供的插件，能夠實現對elasticsearch集羣的狀態監控與管理配置等功能，以下：

2.3.1：安裝head插件：

[root@elkserver1 tianqi]# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head

2.3.2:訪問插件：

http://hfelk.chinacloudapp.cn:9200/_plugin/head/

#https://github.com/mobz/elasticsearch-head

#集羣狀態：

#界面操做

#在插件提交數據：

#在插件獲取數據：

2.4：安裝kopf插件：

[root@elkserver1 tianqi]# /usr/share/elasticsearch/bin/plugin  install lmenezes/elasticsearch-kopf

#github地址：https://github.com/lmenezes/elasticsearch-kopf

2.5：監控集羣狀態，能夠經過訪問集羣狀態接口的返回信息對集羣狀態進行監控，以下:

2.5.1：獲取集羣狀態的命令：

[root@elkserver1 tianqi]# curl  -XGET 「http://192.168.0.4:9200/_cluster/health?pretty=true」

#獲取到的是一個字典格式的返回值，那就能夠經過python對其中的信息進行分析，例如對status進行分析，若是等於green(綠色)就是運行在正常，等於yellow(黃色)表示副本分片丟失，red(紅色)表示主分片丟失

{ "cluster_name" : "hfelk", "status" : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 5, "active_shards" : 10, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 }

2.5.2：腳本內容以下：

 

#!/usr/bin/env python #coding:utf-8 #Author Zhang Shijie import smtplib from email.mime.text import MIMEText from email.utils import formataddr import subprocess body = "" def mail(user,mbody): ret = True msg = MIMEText(mbody, 'plain', 'utf-8') msg['From'] = formataddr(["張傑",'yy@126.com']) msg['To'] = formataddr(["ELS報警郵件",'xx@qq.com']) msg['Subject'] = "主題" server = smtplib.SMTP("smtp.126.com", 25) server.login("yy@126.com", "本身的密碼") server.sendmail('yy@126.com', user, msg.as_string()) server.quit() return ret false="false" obj = subprocess.Popen(("curl -sXGET http://本身的服務器地址:9200/_cluster/health?pretty=true"),shell=True, stdout=subprocess.PIPE) data = obj.stdout.read() data1 = eval(data) status = data1.get("status") if status == "green": mail("xx@qq.com","ELS 服務器綠色") pass elif status == "yellow": mail("xx@qq.com","ELS 服務器黃色") elif status == "yellow": mail("xx@qq.com","ELS 服務器紅色") else: mail("xx@qq.com","ELS服務器可能不在運行")

2.5.3：測試一下腳本：

#打開郵件內容以下：

#腳本內容能夠根據實際狀況修改便可！

三：安裝kibana：

3.1：安裝及配置部分：

3.1.1：安裝：

3.1.2：配置：

[root@elkserver1 tianqi]# vim /opt/kibana/config/kibana.yml

[root@elkserver1 tianqi]# grep 「^[a-Z]」 /opt/kibana/config/kibana.yml 

server.port: 5601 #監聽的端口 server.host: "0.0.0.0" #監聽的地址 elasticsearch.url: "http://192.168.0.4:9200" #elasticsearch服務器的地址，即kibana和elasticsearch能夠不在一個服務器

3.1.3：啓動服務：

[root@elkserver1 tianqi]# systemctl  start kibana
[root@elkserver1 tianqi]# systemctl  enable kibana

3.1.4：訪問web頁面：

http://ELS服務器地址:端口   #這是能夠訪問了，可是不能通過認證，因此誰均可以未經認證訪問，所以將端口關閉改成nginx代理

3.2：使用nginx代理kibana：

3.2.1：編譯安裝一個nginx吧，yum的版本比較低：

[root@elkserver1 tianqi]# rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm [root@elkserver1 yum.repos.d]# cd /usr/local/src/ [root@elkserver1 src]# wget http://nginx.org/download/nginx-1.8.1.tar.gz [root@elkserver1 src]# tar xvf nginx-1.8.1.tar.gz [root@elkserver1 src]# mv nginx-1.8.1 /usr/local/ [root@elkserver1 src]# cd /usr/local/nginx-1.8.1/ [root@elkserver1 nginx-1.8.1]# ./configure --prefix=/usr/local/nginx --sbin-path=/usr/local/nginx/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx/nginx.pid --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --with-http_ssl_module --with-http_stub_status_module --with-http_gzip_static_module --http-client-body-temp-path=/var/tmp/nginx/client/ --http-proxy-temp-path=/var/tmp/nginx/proxy/ --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi --http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre root@elkserver1 nginx-1.8.1]# make && make install [root@elkserver1 nginx-1.8.1]# useradd nginx -s /sbin/nologin [root@elkserver1 nginx-1.8.1]# mkdir -pv /var/tmp/nginx/client [root@elkserver1 nginx-1.8.1]# /usr/local/nginx/sbin/nginx #啓動nginx，若是沒法啓動看日誌，缺什麼補什麼

3.2.2：增長一個nginx配置文件：

[root@elkserver1 ~]# vim /usr/local/nginx/conf/conf.d/hfelk_server.conf

server { listen 80; server_name hfelk.chinacloudapp.cn; #當前主機名 auth_basic "Input User and Password"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陸認證 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

3.2.3：編輯nginx配置主文件導入上一步的配置文件：

[root@elkserver1 ~]# vim /usr/local/nginx/conf/nginx.conf

include /usr/local/nginx/conf/conf.d/*.conf;

3.2.4:生成用戶認證文件：

[root@elkserver1 ~]# htpasswd  -b -c /usr/local/nginx/conf/htpasswd.users zhangjie  123456
Adding password for user zhangjie

[root@elkserver1 ~]# chown  nginx.nginx  /usr/local/nginx/conf/htpasswd.users

[root@elkserver1 ~]# chmod  600 /usr/local/nginx/conf/htpasswd.users

 

3.2.4：重啓nginx：

[root@elkserver1 ~]# /usr/local/nginx/sbin/nginx -t nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful [root@elkserver1 ~]# /usr/local/nginx/sbin/nginx -s reload

3.2.5：再訪問試試：

3.2.5：輸入密碼登陸:

#輸入正確的密碼便可登陸

四：實現https訪問：

4.1：自制CA證書：

4.1.1：生成CA key，這是至關於CA機構的根證書:

[root@elkserver1 ~]# cd /usr/local/nginx [root@elkserver1 nginx]# mkdir key [root@elkserver1 nginx]# cd key/ [root@elkserver1 key]# openssl genrsa -out ca.key 2048 Generating RSA private key, 2048 bit long modulus ....................................................................................+++ ................................................+++ e is 65537 (0x10001) [root@elkserver1 key]# openssl genrsa -des3 -out server.key 2048 Generating RSA private key, 2048 bit long modulus ...+++ ......................+++ e is 65537 (0x10001) Enter pass phrase for server.key: #輸入密碼 Verifying - Enter pass phrase for server.key: #重複輸入一次密碼

4.1.2:生成簽名證書，#這是在nginx服務器執行，是生成一個向CA服務器申請簽名證書的csr證書，CA服務器根據此csr證書發給一個簽名的證書，生成後會是一對兒，即一個公鑰一個私鑰，公鑰用於加密，私鑰用於簽名：

[root@elkserver1 key]# openssl req -new -key server.key -out server.csr  Enter pass phrase for server.key: #必須輸入密碼 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:China string is too long, it needs to be less than  2 bytes long Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:Beijing     Locality Name (eg, city) [Default City]:Beijing Organization Name (eg, company) [Default Company Ltd]:HFAW Organizational Unit Name (eg, section) []:HFAW Common Name (eg, your name or your server's hostname) []:HFELK Email Address []:zhangshijie@weathercn.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:

4.1.3：#刪除服務器私鑰的密碼，其實不刪除也能夠，只是之後重啓nginx都要密碼，其實要密碼更安全：

[root@elkserver1 key]# openssl rsa -in server.key -out server_new.key Enter pass phrase for server.key: #輸入一次建立key的時候的密碼 writing RSA key

#當前目錄的key文件：

4.1.4：對csr文件簽名，會生成一個crt格式的證書：

[root@elkserver1 key]# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt Signature ok subject=/C=CN/ST=Beijing/L=Beijing/O=HFAW/OU=HFAW/CN=HFELK/emailAddress=zhangshijie@weathercn.com Getting Private key Enter pass phrase for server.key: #CA證書的密碼 [root@elkserver1 key]# ll total 20 -rw-r--r-- 1 root root 1675 Aug 23 06:15 ca.key -rw-r--r-- 1 root root 1298 Aug 23 06:36 server.crt #通過CA服務器簽名後的crt證書，就能夠光明正大的在nginx服務器進行使用了 -rw-r--r-- 1 root root 1050 Aug 23 06:21 server.csr -rw-r--r-- 1 root root 1743 Aug 23 06:16 server.key -rw-r--r-- 1 root root 1675 Aug 23 06:32 server_new.key

4.1.5：配置nginx使用證書：

server { listen 443 ssl; ssl_certificate /usr/local/nginx/key/server.crt; #公鑰 ssl_certificate_key /usr/local/nginx/key/server_new.key; #私鑰 server_name hfelk.chinacloudapp.cn; #當前主機名 auth_basic "Input User and Passwowd"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陸認證 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

4.1.6：訪問驗證：

4.1.6.1：https://服務器地址  #默認就是443端口 

4.1.6.2：添加例外：

4.1.6.3：確認安全例外

4.1.6.4:輸入密碼登陸：

4.1.6.5：登陸後的界面：

4.2:配置http訪問轉發至https：

4.2.1:編輯nginx配置文件：

server { listen 80; server_name hfelk.chinacloudapp.cn; rewrite ^(.*)$ https://$server_name$1 permanent; #經過write重寫爲https訪問 } server { listen 443 ssl; ssl_certificate /usr/local/nginx/key/server.crt; ssl_certificate_key /usr/local/nginx/key/server_new.key; server_name hfelk.chinacloudapp.cn; #當前主機名 auth_basic "Input User and Passwowd"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陸認證 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

4.2.2:測試訪問http可否轉發至https:

#瀏覽器訪問：http://hfelk.chinacloudapp.cn/