elasticsearch集羣安裝+安全驗證+kibana安裝

準備環境

啓動4個centos容器, 並暴露相對應端口（個人本機ip爲172.16.1.236，如下涉及到的地方須要修改成本身的ip）

node_name	ip	http port	transport port
es01	docker宿主機ip	9205:9200	9305: 9300
es02	docker宿主機ip	9206:9200	9306: 9300
es03	docker宿主機ip	9207:9200	9307: 9300

kibana : 5601:5601node

執行如下命令啓動centos容器並暴露相應端口（使用-p 來對外映射docker容器端口）

docker run -tid --name centos1 -p 9205:9200 -p9305:9300 --privileged=true centos:latest /sbin/init
docker run -tid --name centos2 -p 9206:9200 -p9306:9300 --privileged=true centos:latest /sbin/init
docker run -tid --name centos3 -p 9207:9200 -p9307:9300 --privileged=true centos:latest /sbin/init
#centos4用於kibana安裝
docker run -tid --name centos4 -p 5601:5601 --privileged=true centos:latest /sbin/init

注意：若是使用shell工具鏈接，增長 -p22:22 參數
參考es快速啓動準備es安裝包等數據

配置修改

修改system.conf配置 /etc/systemd/system.conf

sed -i "s/#DefaultLimitMEMLOCK=/DefaultLimitMEMLOCK=infinity/g" /etc/systemd/system.conf

修改limits.cong配置 /etc/security/limits.conf

修改以下

# nofile 最大打開文件描述符數
# nproc 最大進程數
# memlock 最大內存鎖定

echo "* soft nofile 65536
* hard nofile 131072
* soft nproc 4096
* hard nproc 4096
* soft memlock unlimited
* hard memlock unlimited" >> /etc/security/limits.conf

修改sysctl配置 /etc/sysctl.conf

修改以下

echo "vm.max_map_count = 262145" >> /etc/sysctl.conf

#生效
sysctl -p

修改es佔用內存 /opt/es/config/jvm.options

sed -i "s/-Xms1g/-Xms"800m"/g" /opt/es/config/jvm.options
sed -i "s/-Xmx1g/-Xmx"800m"/g" /opt/es/config/jvm.options

開啓ES內存鎖定 /opt/es/config/elasticsearch.yml

增長以下配置

bootstrap.memory_lock: true

ES配置文件詳解

節點類型

主節點
候選主節點的設置方法是設置node.mater爲true，默認狀況下，node.mater和node.data的值都爲true，即該節點既能夠作候選主節點也能夠作數據節點。因爲數據節點承載了數據的操做，負載一般都很高，因此隨着集羣的擴大，建議將兩者分離，設置專用的候選主節點。當咱們設置node.data爲false，就將節點設置爲專用的候選主節點了。

node.master: true
node.data: false

數據節點
數據節點負責數據的存儲和相關具體操做，好比CRUD、搜索、聚合。因此，數據節點對機器配置要求比較高，首先須要有足夠的磁盤空間來存儲數據，其次數據操做對系統CPU、Memory和IO的性能消耗都很大。一般隨着集羣的擴大，須要增長更多的數據節點來提升可用性。

node.master: false
node.data: true

client節點
即不會被選做主節點，也不會存儲任何索引數據。該節點只能處理路由請求，處理搜索，分發索引操做等，從本質上來講該客戶節點表現爲智能負載平衡器。

node.master = false
 node.data = false

配置集羣名稱

不一樣的節點配置同一個cluster.name可組成同一個集羣，確保不一樣的集羣使用不一樣的cluster.name
配置以下 :

cluster.name: es-cluster-test

配置ES節點名稱

node.name表示集羣節點的名稱，集羣中節點進行區分，若是不配置則默認爲主機名
配置以下

node.name: es01

配置ES節點監聽地址

若是不配置，默認是監聽在127.0.0.1 和 [::1]，同時以development的方式啓動。

#監聽在指定ip上
network.host : 172.17.0.1

#監聽在全部ip上
network.host : 0.0.0.0

日誌數據路徑配置

配置方式

path.data: /opt/data/es
path.logs: /opt/log/es

path:
    data: /opt/data/es
    logs: /opt/log/es

path:  
    logs: /opt/log/es
    data:    
        - /opt/data/es-A    
        - /opt/data/es-B   
        - /opt/data/es-C

集羣發現配置

discovery.seed_hosts 配置方式以下
用於多個集羣節點進行發現，組成集羣

discovery.seed_hosts: ["192.168.1.10:9300", "192.168.1.11", "seeds.mydomain.com"]

discovery.seed_hosts:
   - 192.168.1.10:9300
   - 192.168.1.11
   - seeds.mydomain.com

cluster.initial_master_nodes 配置方式以下
用於集羣在第一次啓動時，指定能夠參與選舉的主節點列表（node.master: true）

cluster.initial_master_nodes: ["es01", "es02", "es03"]

cluster.initial_master_nodes:
    -es01
    -es02
    -es03

discovery.seed_hosts 若是不配置，會自動監聽本地迴環地址將本地多個elasticsearch實例加入到集羣中。

jvm配置

/opt/es/config/jvm.options (通常配置爲機器內存大小的一半)

sed -i "s/-Xms1g/-Xms"800m"/g" /opt/es/config/jvm.options
sed -i "s/-Xmx1g/-Xmx"800m"/g" /opt/es/config/jvm.options

配置Elasticsearch（本次測試配置）

es01

cluster.name: es-cluster-test
node.name: es01
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9306", "172.16.1.236:9307"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

es02

cluster.name: es-cluster-test
node.name: es02
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9305", "172.16.1.236:9307"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

es03

cluster.name: es-cluster-test
node.name: es03
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9305", "172.16.1.236:9306"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

啓動Elasticsearch

啓動

分別進入不一樣的容器啓動elasticsearch
命令以下

/opt/es/bin/elasticsearch -d

查看容器情況（容器內）

查看集羣是否啓動成功

[elasticsearch@813bf8515935 /]$ curl localhost:9200/_cat/nodes
172.17.0.7 28 91  9 0.58 0.66 0.90 dilm - es02
172.17.0.6 15 91  5 0.58 0.66 0.90 dilm - es01
172.17.0.8 35 91 13 0.58 0.66 0.90 dilm * es03

查看當前節點

[elasticsearch@813bf8515935 /]$ curl localhost:9200
{
  "name" : "es03",
  "cluster_name" : "es-cluster-test",
  "cluster_uuid" : "Syj18FUrR1GdnGzghBIacQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

查看節點健康情況

[elasticsearch@813bf8515935 /]$ curl localhost:9200/_cat/health?v
epoch      timestamp cluster         status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1605859559 08:05:59  es-cluster-test green           3         3      0   0    0    0        0             0
      -                100.0%

查看容器情況（宿主機）

因爲使用docker容器安裝，所以也能夠經過宿主機映射端口訪問，本次測試宿主機爲windows

開啓集羣安全驗證

修改配置

分別在不一樣的Es節點修改配置開啓安全驗證

echo "xpack.security.enabled: true" >> /opt/es/config/elasticsearch.yml
echo "xpack.security.transport.ssl.enabled: true" >> /opt/es/config/elasticsearch.yml

生成證書

選擇其中一個節點生成CA證書，默認狀況下生成證書放在 /opt/es

#建立ca證書 /opt/es/bin/elasticsearch-certutil ca -v
[elasticsearch@c824e845075b es]$ /opt/es/bin/elasticsearch-certutil ca -v
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]:    #輸入回車，使用默認
Enter password for elastic-stack-ca.p12 :      #回車，暫不輸入密碼

與上一步驟同一個節點建立節點見認證用的證書

#建立節點間證書 /opt/es/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
[elasticsearch@c824e845075b es]$ /opt/es/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.

    * All certificates generated by this tool will be signed by a certificate authority (CA).
    * The tool can automatically generate a new CA for you, or you can provide your own with the
         -ca or -ca-cert command line options.

By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -keep-ca-key (retain generated CA key)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Enter password for CA (elastic-stack-ca.p12) :      #暫留空
Please enter the desired output file [elastic-certificates.p12]: #默認
Enter password for elastic-certificates.p12 :        #ca證書密碼，這次爲空

Certificates written to /opt/es/elastic-certificates.p12

This file should be properly secured as it contains the private key for
your instance.

This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

配置ES節點使用這個證書

# 將生成證書拷貝到 /opt/es/config/certs目錄下
[elasticsearch@c824e845075b es]$ mkdir -p  /opt/es/config/certs
[elasticsearch@c824e845075b config]$ mv /opt/es/elastic-* /opt/es/config/certs/
# 將certs目錄拷貝到其它Es節點 （在宿主機進行該操做）
#拷貝certs目錄到本地
PS C:\Users\Administrator> docker cp centos2:/opt/es/config/certs C:\Users\Administrator\Desktop
#拷貝certs目錄到其它兩個節點
PS C:\Users\Administrator> docker cp C:\Users\Administrator\Desktop\certs centos1:/opt/es/config
PS C:\Users\Administrator> docker cp C:\Users\Administrator\Desktop\certs centos3:/opt/es/config
#在每一個節點修改 /opt/es/config/elasticsearch.yml 配置
[elasticsearch@813bf8515935 /]$ echo "xpack.security.transport.ssl.verification_mode: certificate 
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12" >> /opt/es/config/elasticsearch.yml

重啓

#在每一個容器殺死elastic進程並重啓
[elasticsearch@c824e845075b config]$ kill -9 $(ps -ef | grep 'elasticsearch' | grep '/bin/java' | grep -v grep | awk '{print $2}')
#啓動
[elasticsearch@c824e845075b config]$ /opt/es/bin/elasticsearch -d

任意一個節點生成密碼

[elasticsearch@6ebd0bc8cc5d certs]$ /opt/es/bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

測試

#此時直接訪問會報無權限異常，須要增長 -u elastic 參數進行訪問
[elasticsearch@6ebd0bc8cc5d certs]$ curl localhost:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

#增長 -u elastic
[elasticsearch@6ebd0bc8cc5d certs]$ curl -u elastic localhost:9200
Enter host password for user 'elastic':
{
  "name" : "es01",
  "cluster_name" : "es-cluster-test",
  "cluster_uuid" : "Syj18FUrR1GdnGzghBIacQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

kibana的安裝配置

準備

kibana版本號與elasticsearch相同，本文使用7.6.2
將主備好的kibana壓縮包移動到容器內，並進入到容器

PS C:\Users\Administrator> docker cp C:\Users\Administrator\Downloads\kibana-7.6.2-linux-x86_64.tar.gz centos4:/opt

PS C:\Users\Administrator> docker exec -it centos4 /bin/bash

解壓安裝

[root@db0759d8c6c8 /]# useradd kibana
[root@db0759d8c6c8 /]# chown -R kibana /opt
[root@db0759d8c6c8 /]# su kibana
[kibana@db0759d8c6c8 /]$ cd /opt/
[kibana@db0759d8c6c8 opt]$ tar -zxvf /opt/kibana-7.6.2-linux-x86_64.tar.gz -C /opt && mv kibana-7.6.2-linux-x86_64 kibana

配置文件修改並啓動

kibana配置文件修改

[kibana@db0759d8c6c8 opt]$ vi kibana/config/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
server.name: "mykibana"
elasticsearch.hosts: ["http://172.16.1.236:9205"]
kibana.index: ".kibana"
elasticsearch.username: "kibana"
elasticsearch.password: "123123"
i18n.locale: "zh-CN"

啓動

[kibana@db0759d8c6c8 opt]$ /opt/kibana/bin/kibana

在宿主訪問 127.0.0.1:5601 進行測試

能夠查看集羣狀態信息linux

本文同步分享在博客「Java入門到入墳」（JianShu）。
若有侵權，請聯繫 support@oschina.cn 刪除。
本文參與「OSC源創計劃」，歡迎正在閱讀的你也加入，一塊兒分享。面試

elasticsearch集羣安裝+安全驗證+kibana安裝

推薦閱讀：

準備環境

配置修改

修改system.conf配置 /etc/systemd/system.conf

修改limits.cong配置 /etc/security/limits.conf

修改sysctl配置 /etc/sysctl.conf

修改es佔用內存 /opt/es/config/jvm.options

開啓ES內存鎖定 /opt/es/config/elasticsearch.yml

ES配置文件詳解

節點類型

配置集羣名稱

配置ES節點名稱

配置ES節點監聽地址

日誌數據路徑配置

集羣發現配置

jvm配置

配置Elasticsearch（本次測試配置）

es01

es02

es03

啓動Elasticsearch

啓動

查看容器情況（容器內）

查看容器情況（宿主機）

開啓集羣安全驗證

修改配置

生成證書

kibana的安裝配置

準備

配置文件修改並啓動