https://www.unixhot.com/article/59html
對於日誌來講,最多見的需求就是收集、存儲、查詢、展現,開源社區正好有相對應的開源項目:logstash(收集)、elasticsearch(存儲+搜索)、kibana(展現),咱們將這三個組合起來的技術稱之爲ELKStack,因此說ELKStack指的是Elasticsearch、Logstash、Kibana技術棧的結合。通用的架構圖:java
最佳實踐:yum安裝的方式。
node
因yum源都是國外的,須要×××才能使用。本文不作詳細安裝步驟了,請參考上面的連接。
我是將要安裝的軟件都下載到本地,而後使用yum localinstall 「package name‘的方式安裝。python
elasticsearch 依賴 java環境,在這裏咱們用yum安裝便可。linux
[root@elk01-node2 ~]# yum -y install java
[root@elk01-node2 tools]# yum localinstall elasticsearch-2.4.4.rpm
Logstash也依賴java環境,這裏咱們也使用yum安裝java環境便可。nginx
[root@elk01-node2 tools]# yum localinstall logstash-2.3.4-1.noarch.rpm
Kibana 是爲 Elasticsearch 設計的開源分析和可視化平臺。你可使用 Kibana 來搜索,查看存儲在 Elasticsearch 索引中的數據並與之交互。你能夠很容易實現高級的數據分析和可視化,以圖表的形式展示出來。git
[root@elk01-node2 tools]# yum localinstall kibana-4.5.4-1.x86_64.rpm
學習軟件的方法:安裝 - 配置 - 啓動 - 測試github
[root@elk01-node2 elk]# vim /etc/elasticsearch/elasticsearch.yml 修改內容 以下: 17 cluster.name: myes # 集羣名,集羣的時候須要用到。 23 node.name: linux-node1 # 節點名,不能重複 33 path.data: /data/es-data # 數據存放的位置 37 path.logs: /var/log/elasticsearch/ # 日誌存放的位置 43 bootstrap.memory_lock: true # 此配置的意思是,鎖住es佔用的內存分區,防止被交換到swap分區,影響性能。 54 network.host: 10.0.0.204 # 監聽的接口地址,默認監聽的端口是9200 58 http.port: 9200 # 監聽的端口
新建對應的目錄。web
[root@elk01-node2 elk]# mkdir /data/es-data -p [root@elk01-node2 elk]# chown -R elasticsearch. /data/
若是不能正常啓動,看日誌。redis
[root@elk01-node2 elk]# /etc/init.d/elasticsearch start [root@elk01-node2 elk]# netstat -tnlpua|grep 9200 tcp6 0 0 10.0.0.204:9200 :::* LISTEN 2943/java
[root@elk01-node2 elk]# curl http://10.0.0.204:9200 { "name" : "linux-node1", "cluster_name" : "myes", "cluster_uuid" : "CRLwDyWsSX-6q4RC4wRcqA", "version" : { "number" : "2.4.4", "build_hash" : "fcbb46dfd45562a9cf00c604b30849a6dec6b017", "build_timestamp" : "2017-01-03T11:33:16Z", "build_snapshot" : false, "lucene_version" : "5.5.2" }, "tagline" : "You Know, for Search" } [root@elk01-node2 elk]# curl -i XGET 'http://10.0.0.204:9200/_count?' curl: (6) Could not resolve host: XGET; Name or service not known HTTP/1.1 200 OK Content-Type: application/json; charset=UTF-8 Content-Length: 59 {"count":1,"_shards":{"total":5,"successful":5,"failed":0}}
因安裝插件須要×××。從github下載的插件能夠用。
es下載的插件放在/usr/share/elasticsearch/plugins/目錄下面。
安裝方法
/usr/share/elasticsearch/bin/plugin install marvel-agent /usr/share/elasticsearch/bin/plugin install head
git下載的插件安裝
[root@elk01-node2 plugins]# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head ( this plugin is to github download.) [root@elk01-node2 plugins]# /usr/share/elasticsearch/bin/plugin install lmenezes/elasticsearch-kopf
測試安裝的插件
{ "user": "wf", "mesg": "hehe" }
http://10.0.0.204:9200/_plugin/kopf/#!/cluster
(ps:圖是補的)
elk01-node2.damaiche.org-204 配置
[root@elk01-node2 elk]# grep ^[a-z] /etc/elasticsearch/elasticsearch.yml cluster.name: myes node.name: linux-node1 path.data: /data/es-data path.logs: /var/log/elasticsearch/ bootstrap.memory_lock: true network.host: 10.0.0.204 http.port: 9200 discovery.zen.ping.unicast.hosts: ["10.0.0.203", "10.0.0.204"] # 如不能正常加入到集羣中去。10.0.0.204 須要將組播改爲單播。(204 知道203 是它的兄弟,203 不知道無所謂)
10.0.0.203配置
ps: es 安裝方法相同
[root@web01-node1 ~]# grep ^[a-z] /etc/elasticsearch/elasticsearch.yml cluster.name: myes node.name: linux-node2 path.data: /data/es-data path.logs: /var/log/elasticsearch/ bootstrap.memory_lock: true network.host: 10.0.0.203 http.port: 9200
etc/init.d/elasticsearch restart
10.0.0.204的日誌,以今天加入到集羣裏去了。
/var/log/elasticsearch/my-es.log
說明:
監測集羣健康狀態
例子:
https://www.elastic.co/guide/en/elasticsearch/guide/current/_cluster_health.html
[root@web01-node1 ~]# curl -i XGET http://10.0.0.204:9200/_cluster/health?pretty=True curl: (6) Could not resolve host: XGET; Name or service not known HTTP/1.1 200 OK Content-Type: application/json; charset=UTF-8 Content-Length: 458 { "cluster_name" : "myes", "status" : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 5, "active_shards" : 10, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 }
https://www.elastic.co/guide/en/logstash/2.3/index.html
手動的輸入內容,而且打印出來。
/opt/logstash/bin/logstash -e \ 'input { stdin {} } output { stdout{} }'
手動的輸入內容,而且打印出來,格式更加的好看。
/opt/logstash/bin/logstash -e \ 'input { stdin {} } output { stdout{codec => rubydebug} }'
將輸出的內容打印出來同時存入到es裏。
input插件
官方參考連接 https://www.elastic.co/guide/en/logstash/current/input-plugins.html
logstash
官方參考連接 https://www.elastic.co/guide/en/logstash/current/index.htm
output插件
官方參考連接 https://www.elastic.co/guide/en/logstash/current/output-plugins.html
/opt/logstash/bin/logstash -e \ 'input { stdin {} } output { stdout { codec => rubydebug } elasticsearch { hosts => ["http://10.0.0.204:9200"] index => "test-log-%{+YYYY.MM.dd}" } }'
https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html
寫一個輸出到前臺,而且將日誌存儲到es裏面的配置文件,啓動在前臺。
[root@elk01-node2 ~]# cd /etc/logstash/conf.d/ [root@elk01-node2 conf.d]# cat demo.conf input{ stdin {} } filter{ } output{ stdout { codec => rubydebug } elasticsearch { hosts => ["10.0.0.204:9200"] index => "demo-log-%{+YYYY.MM.dd}" } } [root@elk01-node2 conf.d]# /opt/logstash/bin/logstash -f demo.conf
寫一個收集系統日誌的配置文件
[root@elk01-node2 conf.d]# cat file.conf input{ file { path => ["/var/log/messages", "/var/log/secure"] # path 指定日誌收集的位置 start_position => "beginning" # 指定從日誌文件什麼地方開始讀。 type => "system-log" # 指定日誌的類型,這個是自定義的。用來作if判斷。 } } filter { } output{ if [type] == "system-log" { # 這裏的等於不是=> 而是== , 須要注意一下。 elasticsearch { hosts => ["10.0.0.204:9200"] index => "system-log-%{+YYYY.MM}" } } } [root@elk01-node2 conf.d]# /opt/logstash/bin/logstash -f file.conf
這裏須要注意下:若是使用了type來作if判斷,那麼在日誌裏就不能出現type字段了。
input{ file { path => ["/var/log/messages", "/var/log/secure"] start_position => "beginning" type => "system-log" } file { path => ["/var/log/elasticsearch/myes.log"] start_position => "beginning" type => "myes-log" } } filter { } output{ if [type] == "system-log" { elasticsearch { hosts => ["10.0.0.204:9200"] index => "system-log-%{+YYYY.MM}" } } if [type] == "myes-log" { elasticsearch { hosts => ["10.0.0.204:9200"] index => "myes-log-%{+YYYY.MM.dd}" } } } [root@elk01-node2 conf.d]# /opt/logstash/bin/logstash -f file.conf
在上面配置文件基礎上,添加對java日誌文件的收集。須要注意java日誌的日誌格式。java日誌內容 一行會有不少的內容。須要用到multiline模塊
https://www.elastic.co/guide/en/logstash/2.3/plugins-codecs-multiline.html
[root@elk01-node2 conf.d]# cat codec.conf input{ file { path => ["/var/log/elasticsearch/myes.log"] start_position => "beginning" type => "myes" codec => multiline { # 下面這三行的意思是,匹配到以[開頭的文件以前的內容,就合併到上一行。 pattern => "^\[" # 匹配的表達式 negate => true # 匹配的結果,這是一個bool類型。 what => "previous" # 動做,合併到上一行。previous合併到上一行,next是合併到下一行。 } } } filter { } output{ if [type] == 'myes' { elasticsearch { hosts => ["10.0.0.204:9200"] index => "myes_log-%{+YYYY.MM.dd}" } } }
能夠將其添加到kibana裏面去,能清楚看到效果。
注:
https://www.elastic.co/guide/en/logstash/2.3/plugins-codecs-json.html
爲何要用json來收集日誌?
獲取日誌裏面的參數信息,es不能直接作到的(若是你會ruby就能夠本身寫了),用json就能夠輕鬆的獲取到。(用app客戶端的訪問日誌來舉例,客戶端,uid等等。)
獲取方式
ps: nginx能夠將日誌寫成json格式的。
修改nginx配置文件
[root@elk01-node2 nginx]# vim nginx.conf log_format access_log_json'{"user_ip":"$http_x_real_ip","lan_ip":"$remote_addr","log_time":"$time_iso8601","user_req":"$request","http_code":"$status","body_bytes_sent":"$body_bytes_sent","req_time":"$request_time","user_ua":"$http_user_agent"}'; access_log /var/log/nginx/access_json.log access_log_json; [root@elk01-node2 nginx]# systemctl restart nginx
編寫收集nginx日誌的配置文件,啓動在前臺進行測試。
[root@elk01-node2 conf.d]# cat nginx.conf input{ file { path => ["/var/log/nginx/access_json.log"] type => "nginx-log" codec => "json" } } filter { } output{ if [type] == 'nginx-log' { stdout { codec => rubydebug} } } [root@elk01-node2 conf.d]# /opt/logstash/bin/logstash -f nginx.conf # 模擬數據 [root@web01-node1 ~]# ab -n 100 -c 2 http://10.0.0.204/aaa/ [root@web01-node1 ~]# ab -n 100 -c 2 http://10.0.0.204/
效果
確認沒問題後寫入到es裏
[root@elk01-node2 conf.d]# cat nginx.conf input{ file { path => "/var/log/nginx/access_json.log" codec => "json" type => "nginx-log" } } filter{ } output{ if [type] == 'nginx-log' { elasticsearch { hosts => ["10.0.0.204:9200"] index => "nginxlog-%{+YYYY.MM.dd}" } } } [root@elk01-node2 conf.d]# /opt/logstash/bin/logstash -f nginx.conf # 模擬數據 [root@web01-node1 ~]# ab -n 100 -c 2 http://10.0.0.204/aaa/ [root@web01-node1 ~]# ab -n 100 -c 2 http://10.0.0.204/
遇到的問題: