Elasticsearch + Logstash + Kibana搭建

 本次部署是以單機部署，服務器IP爲：192.168.1.101，ELK版本爲7.1.1

1、環境準備

　　1) ELK須要JDK 8.*支持

~]$ vim /etc/profile
　　　export JAVA_HOME=/usr/local/jdk1.8.0_201
　　　export JAVA_BIN=/usr/local/jdk1.8.0_201/bin
　　　export PATH=$PATH:$JAVA_BIN
　　　export CLASSPATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
　　　export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
　　　export JAVA_HOME JAVA_BIN PATH CLASSPATH
~]$ source /etc/profile   #刷新環境變量

~]$ java -version   #查看java版本
　　java version "1.8.0_201"
　　Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
　　Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

　　2) 系統設置

#修改系統鏈接數
~]$ vim /etc/sysctl.conf
　vm.max_map_count = 655360
　fs.file = 1000000
~]$ sysctl -p  #刷新配置

#修改系統打開文件數爲65535
~]$ vim /etc/security/limits.conf
　　* soft nofile 65536
　　* hard nofile 65536
　　* soft nproc 65536
　　* hard nproc 65536
~]$ ulimit -n #查看系統文件數
　　ulimit -n 65535  #若是系統

~]$ vim /etc/security/limits.d/20-nproc.conf
*  soft  nproc 1024 修改成  * soft nproc 2048

　　3）將ELK組件包上傳到服務器，本次部署版本爲7.1.1

　　kibana-7.1.1-linux-x86_64.tar

　　elasticsearch-7.1.1-linux-x86_64.tar

　　logstash-7.1.1.tar

　   elasticsearch-head-master.zip

2、Elasticsearch部署

1）解壓ES安裝包
tar -xf elasticsearch-7.1.1-linux-x86_64.tar
mv elasticsearch-7.1.1-linux-x86_64.tar /apps/elasticsearch
2）建立啓動用戶
useradd  elastic 
3）建立啓動日誌目錄,數據目錄，PID目錄
mkdir -p /apps/log/elasticsearch
mkdir -p /apps/lib/elasticsearch
mkdir -p /apps/run/elasticsearch
4）賦予對應權限
chown -R elastic:elastic /apps/elasticsearch
chown -R elastic:elastic /apps/log/elasticsearch
chown -R elastic:elastic /apps/lib/elasticsearch
chown -R elastic:elastic /apps/run/elasticsearch

5）修改ES配置文件
~]$ vim /etc/elasticsearch/config/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#判斷節點是否屬於統一集羣，多臺ES集羣名稱要一致
cluster.name: daoran
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#節點名稱，寫主機名就行
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#ES數據目錄
path.data: /apps/lib/elasticsearch
#
# Path to log files:
#ES日誌目錄
path.logs: /apps/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#解決centos6可能會報錯，因此添加這行
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#network.host: 172.17.0.2只能本地訪問，若是想瀏覽器訪問改成0.0.0.0
network.host: 192.168.1.101
#
# Set a custom port for HTTP:
#
#默認ES端口
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#開啓這行註釋(集羣初始主節點)
cluster.initial_master_nodes: ["node-1"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#添加新參數，這樣head插件能夠訪問ES
#是否支持跨域，默認爲false
http.cors.enabled: true 
#當設置容許跨域，默認爲*,表示支持全部域名，若是咱們只是容許某些網站能訪問，那麼可使用正則表達式。好比只容許本地地址。 /https?:\/\/localhost(:[0-9]+)?/
http.cors.allow-origin: "*"
#解決若是安裝了x-pack插件會致使head訪問不了問題,未安裝x-pack不須要添加
http.cors.allow-headers: "Authorization,X-Requested-With,Content-Length,Content-Type"
#是否開啓x-pack驗證，默認是false（此項須要理解ES運行後再開啓，不然坑比較多，前期建議先關閉），若是開啓還須要增長ssl方式，否定則會報錯
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false

6）設置jvm內存，通常設置爲物理內存一半
~]$vim /apps/elasticsearch/config/jvm.options
　　-Xms1g
　　-Xmx1g
7）以守護進程啓動Elasticsearch
~]$ su elastic #不能用root啓動，不然會報錯
elastic]$ cd /apps/elasticsearch
elastic]$ ./bin/elasticsearch -d

     8）打開網頁輸入 192.168.1.101:9200，若是啓動正常就會看到以下頁面

2、部署Elasticsearch-head

~]$ yum install httpd -y
~]$ unzip -o elasticsearch-head-master.tar
~]$ cp -r elasticsearch-head-master/* /var/www/html/
~]$ vim /var/www/html/_site/app.js
　　將配置文件中的
　　this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
　　修改成：
　　this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.1.101:9200";
#啓動httpd服務
~]$ service httpd start

 服務器啓動正常狀況下， 在瀏覽器輸入 192.168.1.101，就會看到以下頁面：

 

 

 3、部署Kibana

~]$ tar -xf kibana-7.1.1-linux-x86_64.tar
~]$ mv kibana-7.1.1-linux-x86_64 /apps/kibana
~]$ vim /apps/kibana/config/kibana.yml

# Kibana is served by a back end server. This setting specifies the port to use.
#Kibana默認服務端口
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#服務訪問地址
server.host: "192.168.1.101"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name.  This is used for display purposes.
#server.name: "your-hostname"

# The URLs of the Elasticsearch instances to use for all your queries.
#鏈接Elasticsearch
elasticsearch.hosts: ["http://192.168.1.101:9200"]

# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#Kibana日誌
kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

················

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
#i18n.locale: "en"
#添加中文支持
i18n.locale: "zh-CN"

 

因爲Kibana是須要在前臺運行，因此使用screen
~]$screen   #這樣就另開啓一個終端窗口了
~]$cd /apps/kibana/
~]$./bin/kibana

　　啓動後按ctrl+a+d組合鍵，這樣在上面另啓的screen屏裏啓動的kibana服務就一直運行在前臺了....

正常啓動後，訪問 192.168.1.101:5601就能夠看到以下頁面：

 

 4、部署Logstash

　Logstash只需解壓就行，沒有特別須要更改的

~]$tar -xf logstash-7.1.1.tar.gz
~]$mv logstash-7.1.1 /apps/logstas

 

5、x-pack登陸認證（有這方面需求的能夠配置一下）

因爲ELK如今版本已經自帶了x-pack因此不須要再去安裝x-pack只需開啓就行
##############################################Elasticsearch 
修改Elasticsearch配置文件，開啓x-pack驗證
~]$vim /apps/elasticsearch/config/elasticsearch.yml
········
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

從新啓動Elasticsearch
~]$su elastic
elastic]$cd /apps/elasticsearch/
elastic]$./bin/elasticsearch

##############################################Logstach
修改Logstach配置文件，取消以下幾行的註釋並修改內容
~]$vim /apps/logstash/config/logstash.yml
·········
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: "elastic"
xpack.monitoring.elasticsearch.password: "changeme"
xpack.monitoring.elasticsearch.hosts: ["https://192.168.1.101:9200"]

再次訪問 192.168.1.101:9200頁面就會須要帳戶/密碼驗證，默認：elastic / changeme

 

############################################## Elasticsearch-head
重啓head插件
~]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
再次訪問head時，URL須要加上帳戶和密碼，不然訪問不到頁面：
192.168.1.101/?auth_user=elastic&auth_password=changeme

##############################################Kibana~]
取消鏈接Elasticsearch帳戶的註釋
~]$vim /apps/kibana/config/kibana.yml
········
elasticsearch.username: "elastic"
elasticsearch.password: "changeme"

重啓Kibana，就會看到以下登陸頁面