ElasticSearch 7.8.1集羣搭建

通往集羣的大門

集羣由什麼用?

高可用

  高可用(High Availability)是分佈式系統架構設計中必須考慮的因素之一,它一般是指,經過設計減小系統不能提供服務的時間。若是系統每運行100個時間單位,會有1個時間單位沒法提供服務,咱們說系統的可用性是99%。html

負載均衡

  將流量均衡的分佈在不一樣的節點上,每一個節點均可以處理一部分負載,而且能夠在節點之間動態分配負載,以實現平衡。node

高性能

  將流量分發到不一樣機器,充分利用多機器多CPU,從串行計算到並行計算提供系統性能。linux

ES集羣的基本核心概念

Cluster集羣

  一個ElasticSearch集羣由一個或多個節點(Node)組成,每一個集羣都有一個共同的集羣名稱做爲標識。bootstrap

Node節點

  一個ElasticSearch實例即一個Node,一臺機器能夠有多個實例,正常使用下每一個實例應該會部署在不一樣機器上。ElasticSearch的配置文件中能夠經過node.master、node.data來設置節點類型。瀏覽器

  node.master:表示節點是否具備稱爲主節點的資格服務器

    true表明的是有資格競選主節點架構

    false表明的是沒有資格競選主節點app

  node.data:表示節點是否存儲數據cors

Node節點組合

主節點+數據節點(master+data)

    節點即有稱爲主節點的資格,又存儲數據負載均衡

node.master: true
node.data: true

數據節點(data)

  節點沒有成爲主節點的資格,不參與選舉,只會存儲數據

node.master: false
node.data: true

客戶端節點(client)

  不會成爲主節點,也不會存儲數據,主要是針對海量請求的時候,能夠進行負載均衡

node.master: false
node.data: false

分片

  每一個索引有一個或多個分片,每一個分片存儲不一樣的數據。分片可分爲主分片(primary shard)和複製分片(replica shard),複製分片是主分片的拷貝。默認每一個主分片有一個複製分片,一個索引的複製分片的數量能夠動態地調整,複製分片匆匆不與它的主分片在同一個節點上。

搭建ES集羣

搭建步驟

  • 拷貝ES7.8.1安裝包3份,分別命名es-a,es-b,es-c
  • 分別修改elasticsearch.yml文件
  • 分別啓動a、b、c三個節點
  • 打開瀏覽器輸入:ip:port/_cat/health?v,若是返回的nodt.total是3,表明集羣搭建成功

配置文件

#集羣名稱,三臺集羣,要配置相同的集羣名稱!!!
cluster.name: my-application
#節點名稱
node.name: node-1 #是否是有資格主節點
node.master: true
#是否存儲數據
node.data: true
#最⼤集羣節點數
node.max_local_storage_nodes: 3 #⽹關地址
network.host: 0.0.0.0
#端⼝
http.port: 9200
#內部節點之間溝通端⼝
transport.tcp.port: 9300
#es7.x 以後新增的配置,寫⼊候選主節點的設備地址,在開啓服務後能夠被選爲主節點
discovery.seed_hosts: ["localhost:9300","localhost:9400","localhost:9500"]
#es7.x 以後新增的配置,初始化⼀個新的集羣時須要此配置來選舉master
cluster.initial_master_nodes: ["node-1", "node-2","node-3"] #數據和存儲路徑
path.data: /Users/louis.chen/Documents/study/search/storage/a/data
path.logs: /Users/louis.chen/Documents/study/search/storage/a/logs

注意

  由於我是一臺服務器上,搞了3搞ES啓動端口號不一樣,拷貝了3份es,具體還要跟實際狀況相應調整

真實配置-a

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集羣名稱
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集羣名稱
node.name: node-1
# 是否是有資格主節點
node.master: true
# 是否存儲數據
node.data: true
# 最大集羣節點數,由於3個集羣,因此配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 數據存儲路徑
path.data: /var/soft/es7.8.1/elasticsearch-7.8.1/data
# 日誌存儲路徑
# Path to log files:
#
path.logs: /var/soft/es7.8.1/elasticsearch-7.8.1/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 網關地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9200
# 內部節點之間溝通端口
transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x以後新增的配置,寫入候選主節點的設備地址,在開啓服務後能夠被選爲主節點
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x以後新增的配置,初始化一個新的集羣時須要此配置來選舉master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

真實配置-b

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集羣名稱
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集羣名稱
node.name: node-2
# 是否是有資格主節點
node.master: true
# 是否存儲數據
node.data: true
# 最大集羣節點數,由於3個集羣,因此配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 數據存儲路徑
path.data: /var/soft/es-b/data
# 日誌存儲路徑
# Path to log files:
#
path.logs: /var/soft/es-b/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 網關地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9201
# 內部節點之間溝通端口
transport.tcp.port: 9400
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x以後新增的配置,寫入候選主節點的設備地址,在開啓服務後能夠被選爲主節點
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x以後新增的配置,初始化一個新的集羣時須要此配置來選舉master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

只修改了,2個端口、數據和日誌存儲路徑

真實配置-c

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集羣名稱
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集羣名稱
node.name: node-3
# 是否是有資格主節點
node.master: true
# 是否存儲數據
node.data: true
# 最大集羣節點數,由於3個集羣,因此配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 數據存儲路徑
path.data: /var/soft/es-c/data
# 日誌存儲路徑
# Path to log files:
#
path.logs: /var/soft/es-c/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 網關地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9202
# 內部節點之間溝通端口
transport.tcp.port: 9500
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x以後新增的配置,寫入候選主節點的設備地址,在開啓服務後能夠被選爲主節點
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x以後新增的配置,初始化一個新的集羣時須要此配置來選舉master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

只修改了,2個端口、數據和日誌存儲路徑

踩坑錄(必看

坑一

 

坑二

  由於我是在一臺linux上搭建的集羣,而後用端口號9200、920一、9202區分開來的,es-b、es-c是拷貝es-a的,可是es-a的data目錄有數據,須要將es-b和es-c的data目錄清空,而後重啓es便可!!!

啓動

 

集羣搭建成功

配置kibana

打開配置kibana.yml,添加elasticsearch.hosts: ["http://192.168.199.170:9200", "http://192.168.199.170:9201", "http://192.168.199.170:9202"]

 

而後重啓便可

相關文章
相關標籤/搜索