elasticsearch 安裝教程(centos)

elasticserach 安裝
@(ELK 日誌系統)

1、首先安裝 supervisor

supervisor 是 linux 用於管理後臺程序的守護進程、方便管理集羣
supervisor 官網http://supervisord.org/introd...

2、安裝 es

下載最新版的 es 官網地址
準備好兩臺服務器 (可以互相 ping 通)
10.66.30.221
10.66.2.90
es 配置文件示例

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descript ive name for your cluster:
#
cluster.name: es-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-2
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /home/es/path/to/data
#
# Path to log files:
#
path.logs: /home/es/path/to/logs
bootstrap.memory_lock: true
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
network.host: 10.66.2.90
#
# Set a custom port for HTTP:
#
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.zen.ping.unicast.hosts:
        - 10.6.2.90
        - 10.66.30.221
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

es 啓動常見的錯誤

• 問題 1：

$ ./elasticsearch
...
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
[2016-10-31T04:55:45,240][INFO ][o.e.n.Node               ] [vJDcSkt] stopping ...
[2016-10-31T04:55:45,249][INFO ][o.e.n.Node               ] [vJDcSkt] stopped
[2016-10-31T04:55:45,249][INFO ][o.e.n.Node               ] [vJDcSkt] closing ...
[2016-10-31T04:55:45,257][INFO ][o.e.n.Node               ] [vJDcSkt] closed

如圖所示有兩個錯誤

錯誤 1 的解決辦法
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
修改 /etc/security/limits.conf 文件，添加或修改以下行：

退出當前帳號，而後從新登陸

• hard nofile 65536
• soft nofile 65536

錯誤 2 的解決辦法

max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

切換到root用戶修改配置sysctl.conf
vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=262144
並執行命令：
sysctl -p
而後，從新啓動elasticsearch，便可啓動成功。

• 問題 2：

max number of threads [1024] for user [root] is too low, increase to at least [2048]
解決辦法

切換到 root 用戶，進入 limits.d 目錄下修改配置文件。 vim /etc/security/limits.d/90-nproc.conf

修改以下內容：
* soft nproc 1024
修改成

* soft nproc 2048

• 問題 3：

ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
緣由： 這是在由於 Centos6 不支持 SecComp，而 ES5.2.0 默認 bootstrap.system_call_filter 爲 true 進行檢測，因此致使檢測失敗，失敗後直接致使 ES 不能啓動。 解決： 在 elasticsearch.yml 中配置 bootstrap.system_call_filter 爲 false，注意要在 Memory 下面:

bootstrap.memory_lock: false
bootstrap.system_call_filter: false

• 問題 4：elasticsearch 索引問題

logstash 傳入 es 的序列必須全爲小寫，不然會報 400 的錯誤

解決：

直接在 filebeat.yml 中設置 index

output.logstash:
# The Logstash hosts
hosts: ["10.66.140.37:5044"]
worker: 1
loadbalance: true
index: "opennbk-filebeat"

• 問題 5：elasticsearch 使用 supervisord 啓動時須要切換用戶到非 root 啓動問題

ES 不能用 root 帳戶啓動, 不然會拋出異常 可是使用 supervisord -c …supervisor.conf 啓動的時候會出現一下錯誤

解決:

這個時候咱們能夠吧 supervisor.conf 文件中的 program 選項中的 user (誰來啓動) 註釋, 而後修改 program 選項中的 command(啓動命令) 爲:su -c 「ES 啓動腳本的目錄」 非 root 用戶.

• 問題 6

ERROR: [1] bootstrap checks failed

修改 /etc/security/limits.conf 文件，添加或修改以下行：

*   soft    memlock unlimited
*   soft    memlock unlimited

退出當前帳號，而後從新登陸

3、supervisor 配置 es

[program: es_node2]
command=/home/es/elasticsearch-6.0.0/bin/elasticsearch
numprocs=1
user=es
autostart=true
autorestart=true

es 插件 head 安裝

1. 首先安裝 nodejs
2. 全局安裝 grunt

npm install -g grunt-cli

1. 克隆 head 工程

git clone https://github.com/mobz/elasticsearch-head.git

1. 修改 head 目錄下的 Gruntfile.js 文件

添加 hostname

connect: {
                          server: {
                                  options: {
                                          port: 9100,
                                          hostname: '*',
                                          base: '.',
                                          keepalive: true
                                  }
                          }
                  }

修改 head 目錄下的 app.js 文件。 vim/_site/app.js

app.App = ui.AbstractWidget.extend({

defaults: {
                     base_uri: null
             },
             init: function(parent) {
                     this._super();
                     this.prefs = services.Preferences.instance();
                     this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://10.66.2.90:9200" || "http://10.66.30.221:9200";
                     if( this.base_uri.charAt( this.base_uri.length - 1 ) !== "/" ) {
                             // XHR request fails if the URL is not ending with a "/"
                             this.base_uri += "/";
                     }
                     if( this.config.auth_user ) {

修改 es 目錄下的 elasticsearch.yml 文件

http.cors.enabled: true
http.cors.allow-origin: "*"

grunt server 啓動