Linux ELK 安裝（服務器架設篇）

時間 2019-11-08

原文原文鏈接

ELK簡介

ELK是三個開源軟件的縮寫，分別表示：Elasticsearch , Logstash, Kibana , 它們都是開源軟件。新增了一個FileBeat，它是一個輕量級的日誌收集處理工具(Agent)，Filebeat佔用資源少，適合於在各個服務器上搜集日誌後傳輸給Logstash，官方也推薦此工具。java

Elasticsearch是個開源分佈式搜索引擎，提供蒐集、分析、存儲數據三大功能。它的特色有：分佈式，零配置，自動發現，索引自動分片，索引副本機制，restful風格接口，多數據源，自動搜索負載等。node

Logstash 主要是用來日誌的蒐集、分析、過濾日誌的工具，支持大量的數據獲取方式。通常工做方式爲c/s架構，client端安裝在須要收集日誌的主機上，server端負責將收到的各節點日誌進行過濾、修改等操做在一併發往elasticsearch上去。linux

Kibana 也是一個開源和免費的工具，Kibana能夠爲 Logstash 和 ElasticSearch 提供的日誌分析友好的 Web 界面，能夠幫助彙總、分析和搜索重要數據日誌。git

Filebeat隸屬於Beats。目前Beats包含四種工具：json

Packetbeat（蒐集網絡流量數據）
Topbeat（蒐集系統、進程和文件系統級別的 CPU 和內存使用狀況等數據）
Filebeat（蒐集文件數據）
Winlogbeat（蒐集 Windows 事件日誌數據）

通常而言，ELK主要用在海量零散數據的彙總和信息提取分析上。在分佈式系統的日誌統計，大數據的數據分析，業務數據的快速檢索，服務器集羣上每臺服務器的運行狀況查詢監控等方面有很強大的功能。bootstrap

拿ELK在分佈式系統上的日誌收集舉例。隨着微服務的流行，分佈式的使用，以往日誌文件寫在具體的服務器上的某一位置作法變得不符合需求，首先是服務器愈來愈多並且後端服務集羣橫跨多個服務器致使日誌愈來愈散，不管是開發，測試仍是線上的日誌定位愈來愈難，準確的找到有用的信息須要運維/開發不段的排查，這時ELK就派上用場了，它將服務集羣裏面的日誌收集彙總並創建索引，當出現問題是定位問題就像Google這類搜素引擎同樣高效簡單。vim

安裝

通常單臺機器就能夠安裝了，我這裏爲了貼近實際使用，分爲3個機器來部署一個入門的ELK。後端

具體的結構以下api

主機	IP	系統	部署服務
thinkvmc01	192.168.50.207	CentOS7	ElasticSearch
thinkvmc02	192.168.50.19	CentOS7	Logstash
thinkvmc03	192.168.50.54	CentOS7	Kibana

安裝準備

ELK是須要Java的，建議安裝Java8。這裏我就囉嗦了瀏覽器

# 先檢查JDK環境
[thinktik@thinkvmc01 ~]$ java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

開始安裝

安裝ELK不難，按照官方的文檔便可，官網連接下

開源搜索與分析 · Elasticsearch

咱們先安裝 ElasticSearch。先下載 Linux 通用安裝包 elasticsearch-6.7.1.tar.gz
。固然爲了簡單，你也能夠下載具體Linux發行版的預編譯包，這樣安裝更加簡單，不過缺乏靈活性。

thinkvmc01 先安裝ES

# 下載
[thinktik@thinkvmc01 thinktik]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.tar.gz
--2019-04-08 22:51:05--  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.230.222, 2a04:4e42:1a::734
Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.230.222|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 148542786 (142M) [application/x-gzip]
Saving to: ‘elasticsearch-6.7.1.tar.gz.1’

 2% [>                                                                ] 3,699,945   42.4KB/s  eta 25m 36s

....

# 下載完畢，解壓
[thinktik@thinkvmc01 thinktik]# ls
elasticsearch-6.7.1.tar.gz  java8  jdk-8u201-linux-x64.tar.gz
[thinktik@thinkvmc01 thinktik]# tar -zxvf elasticsearch-6.7.1.tar.gz 
elasticsearch-6.7.1/
elasticsearch-6.7.1/lib/

....

elasticsearch-6.7.1/logs/
elasticsearch-6.7.1/plugins/

# 進入安裝文件目錄
[thinktik@thinkvmc01 thinktik]# cd elasticsearch-6.7.1
[thinktik@thinkvmc01 elasticsearch-6.7.1]# ls
bin  config  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.textile
[thinktikt@thinkvmc01 elasticsearch-6.7.1]# cd config/
[thinktik@thinkvmc01 config]# ls
elasticsearch.yml  jvm.options  log4j2.properties  role_mapping.yml  roles.yml  users  users_roles
# 修改配置，綁定咱們的網卡。不修改默認爲127.0.0.1，那樣其他的機器上的Logstash，Kibana就無法訪問這臺機的ES了
[thinktik@thinkvmc01 config]# vim elasticsearch.yml 

#修改以下
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# IP地址
network.host: 192.168.50.207
#
# Set a custom port for HTTP:
# 端口，默認9200
http.port: 9200

# 啓動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
warning: Falling back to java on path. This behavior is deprecated. Specify JAVA_HOME
[2019-04-08T23:11:44,120][INFO ][o.e.e.NodeEnvironment    ] [ZVfIMzv] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [26.7gb], net total_space [28.9gb], types [rootfs]
[2019-04-08T23:11:44,126][INFO ][o.e.e.NodeEnvironment    ] [ZVfIMzv] heap size [1015.6mb], compressed 

....

# 這裏報了錯，很明顯了
ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-04-08T23:12:06,558][INFO ][o.e.n.Node               ] [ZVfIMzv] stopping ...
[2019-04-08T23:12:06,636][INFO ][o.e.n.Node               ] [ZVfIMzv] stopped
[2019-04-08T23:12:06,637][INFO ][o.e.n.Node               ] [ZVfIMzv] closing ...
[2019-04-08T23:12:06,673][INFO ][o.e.n.Node               ] [ZVfIMzv] closed

# 咱們按它的提示該系統配置
[thinktik@thinkvmc01 bin]$ vim /etc/security/limits.conf
[thinktik@thinkvmc01 bin]$ su
Password: 

#添加以下配置
    * soft nofile 65536
    * hard nofile 131072
    * soft nproc 2048
    * hard nproc 4096

# 繼續啓動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
# 報錯，那麼繼續修改
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-04-08T23:19:38,825][INFO ][o.e.n.Node               ] [ZVfIMzv] stopping ...
[2019-04-08T23:19:38,844][INFO ][o.e.n.Node               ] [ZVfIMzv] stopped
[2019-04-08T23:19:38,845][INFO ][o.e.n.Node               ] [ZVfIMzv] closing ...
[2019-04-08T23:19:38,887][INFO ][o.e.n.Node               ] [ZVfIMzv] closed
[2019-04-08T23:19:38,889][INFO ][o.e.x.m.p.NativeController] [ZVfIMzv] Native controller process has stopped - no new native processes can be started

# 繼續修改
[thinktik@thinkvmc01 bin]$ su
Password: 
[root@thinkvmc01 bin]# sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
# 繼續啓動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
[2019-04-08T23:22:37,612][INFO ][o.e.c.s.ClusterApplierService] [ZVfIMzv] new_master {ZVfIMzv}{ZVfIMzviR5ie4WVCaO9CZA}{B3vTE3wKSriPc-LwHC8J-A}{192.168.50.207}{192.168.50.207:9300}{ml.machine_memory=1927471104, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {ZVfIMzv}{ZVfIMzviR5ie4WVCaO9CZA}{B3vTE3wKSriPc-LwHC8J-A}{192.168.50.207}{192.168.50.207:9300}{ml.machine_memory=1927471104, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-04-08T23:22:37,792][INFO ][o.e.h.n.Netty4HttpServerTransport] [ZVfIMzv] publish_address {192.168.50.207:9200}, bound_addresses {192.168.50.207:9200}
[2019-04-08T23:22:37,792][INFO ][o.e.n.Node               ] [ZVfIMzv] started
[2019-04-08T23:22:38,740][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [ZVfIMzv] Failed to clear cache for realms [[]]
[2019-04-08T23:22:38,850][INFO ][o.e.l.LicenseService     ] [ZVfIMzv] license [41e1ad3d-893b-48c6-98b1-71e02ab1a367] mode [basic] - valid
[2019-04-08T23:22:38,873][INFO ][o.e.g.GatewayService     ] [ZVfIMzv] recovered [0] indices into cluster_state

# 成功

驗證

[thinktik@thinkvmc01 ~]$ netstat -nlp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      -                   
# 9200,9300 被ES監聽
tcp6       0      0 192.168.50.207:9200     :::*                    LISTEN      12829/java          
tcp6       0      0 192.168.50.207:9300     :::*                    LISTEN      12829/java          
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 ::1:25                  :::*                    LISTEN      -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
raw6       0      0 :::58                   :::*                    7           -                   
raw6       0      0 :::58                   :::*                    7           -    

# 防火牆開放端口
[root@thinkvmc01 thinktik]# firewall-cmd --zone=public --add-port=9200/tcp --permanent
success
[root@thinkvmc01 thinktik]# firewall-cmd --zone=public --add-port=9300/tcp --permanent
success
[root@thinkvmc01 thinktik]# firewall-cmd --reload
success


# thinkvmc02 主機驗證 thinkvmc01 的ES效果。你用瀏覽器驗證下面的地址也能夠 
[thinktik@thinkvmc02 ~]$ curl -i http://192.168.50.207:9200/
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 493

{
  "name" : "ZVfIMzv",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "mhuFY2EcRl6Bt9xqKiyY7Q",
  "version" : {
    "number" : "6.7.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "2f32220",
    "build_date" : "2019-04-02T15:59:27.961366Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

你用瀏覽器驗證下面的地址也能夠

到這裏ES就安裝好了

thinkvmc02 先安裝Logstash

# 驗證java
[thinktik@thinkvmc02 java8]$ java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

# 下載
[thinktik@thinkvmc02 java8]$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.7.1.tar.gz
--2019-04-08 23:32:55--  https://artifacts.elastic.co/downloads/logstash/logstash-6.7.1.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.110.222, 2a04:4e42:36::734
Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.110.222|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 175824421 (168M) [application/x-gzip]
Saving to: ‘logstash-6.7.1.tar.gz’

 6% [==>                                                              ] 10,605,295   113KB/s  eta 9m 57s
 
...

[thinktik@thinkvmc02 ~]$ ls
java8  jdk-8u201-linux-x64.tar.gz  logstash-6.7.1.tar.gz
[thinktik@thinkvmc02 ~]$ tar -zxvf logstash-6.7.1.tar.gz 
...
logstash-6.7.1/x-pack/src/test/java/org
logstash-6.7.1/x-pack/src/test/java/org/logstash
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test/RSpecIntegrationTests.java
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test/RSpecTests.java
logstash-6.7.1/LICENSE.txt
logstash-6.7.1/logstash-core/lib/logstash/build.rb


[thinktik@thinkvmc02 ~]$ cd logstash-6.7.1
[thinktik@thinkvmc02 logstash-6.7.1]$ ls
bin     CONTRIBUTORS  Gemfile       lib          logstash-core             modules     tools   x-pack
config  data          Gemfile.lock  LICENSE.txt  logstash-core-plugin-api  NOTICE.TXT  vendor
[thinktik@thinkvmc02 logstash-6.7.1]$ cd config/
[thinktik@thinkvmc02 config]$ ls
jvm.options  log4j2.properties  logstash-sample.conf  logstash.yml  pipelines.yml  startup.options
[thinktik@thinkvmc02 config]$ cp logstash-sample.conf logstash.conf 
[thinktik@thinkvmc02 config]$ vim logstash.conf 
# 這裏將ES地址寫對就能夠了
    input {
      beats {
        port => 5044
      }
    }
    
    output {
      elasticsearch {
        hosts => ["http://192.168.50.207:9200"]
        index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
        #user => "elastic"
        #password => "changeme"
      }
    }

[thinktik@thinkvmc02 config]$ vim logstash.yml 
# 這裏寫正確本身的IP   
    # ------------ Metrics Settings --------------
    #
    # Bind address for the metrics REST endpoint
    #
    http.host: "192.168.50.19"


# 啓動
[thinktik@thinkvmc02 bin]$ ./logstash -f ../config/logstash.conf 
Sending Logstash logs to /home/thinktik/logstash-6.7.1/logs which is now configured via log4j2.properties
[2019-04-08T23:47:53,295][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-04-08T23:47:53,324][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.7.1"}
[2019-04-08T23:48:08,245][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-04-08T23:48:09,323][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.50.207:9200/]}}
# 日誌顯示ES地址對了
[2019-04-08T23:48:09,919][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://192.168.50.207:9200/"}
[2019-04-08T23:48:10,080][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-04-08T23:48:10,096][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-04-08T23:48:10,174][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://192.168.50.207:9200"]}
[2019-04-08T23:48:10,250][INFO ][logstash.outputs.elasticsearch] Using default mapping template
[2019-04-08T23:48:10,318][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2019-04-08T23:48:11,308][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2019-04-08T23:48:11,360][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x583666d7 run>"}
[2019-04-08T23:48:11,499][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
# 日誌顯示5044，9600被監聽
[2019-04-08T23:48:11,589][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2019-04-08T23:48:12,194][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

# 檢查下端口監聽
[thinktik@thinkvmc02 ~]$ netstat -nlp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      -                   
tcp6       0      0 :::5044                 :::*                    LISTEN      27467/java          
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 ::1:25                  :::*                    LISTEN      -                   
tcp6       0      0 192.168.50.19:9600      :::*                    LISTEN      27467/java          
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
raw6       0      0 :::58                   :::*                    7           -                   
raw6       0      0 :::58                   :::*                    7           -       
# 防火牆打開
[root@thinkvmc02 thinktik]# firewall-cmd --zone=public --add-port=9600/tcp --permanent
success
[root@thinkvmc02 thinktik]# firewall-cmd --zone=public --add-port=5044/tcp --permanent
success
[root@thinkvmc02 thinktik]# firewall-cmd --reload
success

到這裏logstash安裝完畢

thinkvmc03 先安裝Kibana

[thinktik@thinkvmc03 ~]$ java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

# 修改配置
[thinktik@thinkvmc03 config]$ pwd
/home/thinktik/kibana-6.7.1-linux-x86_64/config
[thinktik@thinkvmc03 config]$ vim kibana.yml
# 這裏修改成本身的IP，端口默認5601
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "192.168.50.54"
# 這裏修改ES服務的地址
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://192.168.50.54:9200"]

# 啓動
[thinktik@thinkvmc03 bin]$ ./kibana

  log   [16:04:24.455] [info][status][plugin:kibana@6.7.1] Status changed from uninitialized to green - Ready
  log   [16:04:24.507] [info][status][plugin:elasticsearch@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [16:04:24.510] [info][status][plugin:xpack_main@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [16:04:24.523] [info][status][plugin:graph@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch


# 檢查
[thinktik@thinkvmc03 config]$ netstat -nlp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      -                   
# 監聽正常
tcp        0      0 192.168.50.54:5601      0.0.0.0:*               LISTEN      27474/./../node/bin 
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 ::1:25                  :::*                    LISTEN      -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
raw6       0      0 :::58                   :::*                    7           -                   
raw6       0      0 :::58                   :::*                    7           -      
# 防火牆開放端口
[root@thinkvmc03 config]# firewall-cmd --zone=public --add-port=5601/tcp --permanent
success
[root@thinkvmc03 config]# firewall-cmd --reload
success

kibana效果

到這裏就完成了ELK基礎搭建

thinkvmc02 先安裝Filebeat

接下來咱們安裝 Filebeat，使用ELKF架構來實現log4j的日誌收集。

爲了方便Filebeat安裝在thinkvmc03上與thinkvmc02的Logstash造成分佈式結構來模擬日誌數據的收集與傳輸

官方的安裝教程也很簡單，屬於基本操做了

[thinktik@thinkvmc03 ~]$ tar -zxvf filebeat-6.7.1-linux-x86_64.tar.gz 
filebeat-6.7.1-linux-x86_64/.build_hash.txt
filebeat-6.7.1-linux-x86_64/fields.yml
filebeat-6.7.1-linux-x86_64/LICENSE.txt
filebeat-6.7.1-linux-x86_64/NOTICE.txt
filebeat-6.7.1-linux-x86_64/kibana/
filebeat-6.7.1-linux-x86_64/kibana/5/
filebeat-6.7.1-linux-x86_64/

...
filebeat-6.7.1-linux-x86_64/module/traefik/access/machine_learning/visitor_rate.json
filebeat-6.7.1-linux-x86_64/module/traefik/access/manifest.yml
filebeat-6.7.1-linux-x86_64/module/traefik/module.yml
filebeat-6.7.1-linux-x86_64/filebeat.reference.yml
filebeat-6.7.1-linux-x86_64/filebeat

# 修改配置使filebeat對準咱們的輸出
[thinktik@thinkvmc03 filebeat-6.7.1-linux-x86_64]$ vim filebeat.yml 



    #=========================== Filebeat inputs =============================
    # 設置Filebeat讀取/home/thinktik/ELKF_TEST.log日誌
    filebeat.inputs:
    
    # Each - is an input. Most options can be set at the input level, so
    # you can use different inputs for various configurations.
    # Below are the input specific configurations.
    
    - type: log
    
      # Change to true to enable this input configuration.
      # 這裏設爲True開啓日誌讀入
      enabled: true
    
      # Paths that should be crawled and fetched. Glob based paths.
      paths:
        - /home/thinktik/ELKF_TEST.log
        #- /var/log/*.log
        #- c:\programdata\elasticsearch\logs\*

    #-------------------------- Elasticsearch output ------------------------------
    # 直接輸出到Elasticsearch 這裏咱們不建議直接輸出
    #output.elasticsearch:
      # Array of hosts to connect to.
      # hosts: ["192.168.50.207:9200"]
    
      # Enabled ilm (beta) to use index lifecycle management instead daily indices.
      #ilm.enabled: false
    
      # Optional protocol and basic auth credentials.
      #protocol: "https"
      #username: "elastic"
      #password: "changeme"
    
    #----------------------------- Logstash output --------------------------------
    # 這裏纔是Logstash，直接輸出到logstash 這裏咱們建議直接輸出，地址配對就能夠
    output.logstash:
      # The Logstash hosts
      hosts: ["192.168.50.19:5044"]
    
      # Optional SSL. By default is off.
      # List of root certificates for HTTPS server verifications
      #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
    
      # Certificate for SSL client authentication
      #ssl.certificate: "/etc/pki/client/cert.pem"
    
      # Client Certificate Key

# 保存後啓動
[thinktik@thinkvmc03 filebeat-6.7.1-linux-x86_64]$ ./filebeat 

# 後續的你直接修改/home/thinktik/ELKF_TEST.log，寫入一些數據到這個文件裏面等待Kibana顯示出來效果
# 咱們先理下思路，流程是：filebeat -> Logstash -> ES -> Kibana
# 若是沒有問題，那麼咱們再Kibana檢查效果

檢查ELKF效果