ElasticSearch經常使用配置（內置帳號密碼修改、自定義角色自定義帳號，日誌按期刪除等)

時間 2019-11-07

標籤 elasticsearch 經常使用配置內置帳號密碼修改自定義角色日誌按期刪除欄目日誌分析简体版

原文原文鏈接

自定義內置帳號

帳戶elastic爲elasticsearch超級管理員，擁有全部權限
帳戶kibana用於kibana組件獲取相關信息用於web展現
帳戶logstash_system用於logstash服務獲取elasticsearch的監控數據
注意：此步驟需先啓動elasticsearch服務

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/x-pack/setup-passwords interactive
Initiating the setup of reserved user elastic,kibana,logstash_system passwords.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

驗證內置帳戶訪問

若不提供用戶名密碼則返回401java

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}

提供相應用戶信息後可訪問，若用戶權限不足則返回403node

使用logstash_system用戶訪問linux

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u logstash_system:logstash_system
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用kibana用戶訪問web

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定義角色

添加角色接口爲 POST /_xpack/security/role/json

下述示例爲添加超級管理員角色的方法bootstrap

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty' -d '{
>   "run_as": [ "elastic" ],
>   "cluster": [ "all" ],
>   "indices": [
>     {
>       "names": [ "*" ],
>       "privileges": [ "all" ]
>     }
>   ]
> }'
{
  "role" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty'
{
  "admin" : {
    "cluster" : [
      "all"
    ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "all"
        ]
      }
    ],
    "run_as" : [
      "elastic"
    ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定義帳戶

添加用戶接口爲 POST /_xpack/security/user/vim

下述爲添加martin帳戶並添加至admin角色操做方法segmentfault

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/martin?pretty' -d '{
>   "password" : "123456",
>   "full_name" : "Martin Lei",
>   "roles" : ["admin"],
>   "email" : "martin@martin.com"
> }'
{
  "user" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/martin?pretty'
{
  "rocshen" : {
    "username" : "martin",
    "roles" : [
      "admin"
    ],
    "full_name" : "Martin Lei",
    "email" : "martin@martin.com",
    "metadata" : { },
    "enabled" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u martin:123456 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 4883 88   2.5mb   2.5mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0  24.2kb  24.2kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                        
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  630  0 703.3kb 703.3kb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0  33.3kb  33.3kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0   6.5kb   6.5kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

修改帳戶密碼

修改密碼需使用超級管理員權限即elastic帳戶，接口爲POST _xpack/security/user//_password
curl參數含義以下bash

-XPOST 使用post方法傳遞參數
-H 指定http協議的header信息
-u 指定用於認證的用戶信息用戶名與密碼使用冒號分隔
-d 指定具體要傳遞的參數信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/kibana/_password?pretty' -d '{"password": "123456"}'
{ }

密碼修改後使用老密碼訪問則返回401，使用更新後的密碼則正常

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
{
"error" : {
"root_cause" : [
  {
    "type" : "security_exception",
    "reason" : "failed to authenticate user [kibana]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  }
],
"type" : "security_exception",
"reason" : "failed to authenticate user [kibana]",
"header" : {
  "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
}
},
"status" : 401
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:123456
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

配置ldap賬號認證

ldap服務安裝可參考：https://segmentfault.com/a/11...app

添加下述ldap相關述配置 bind_dn爲ldap的管理DN

bind_password爲管理dn的密碼
user_search.base_dn爲linux系統帳戶信息導入ldap的信息
user_search.attribute爲帳戶在ldap中的標識信息
group_search.base_dn爲linux系統組信息導入ldap的信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ vim config/elasticsearch.yml 

......

network.host: 10.59.30.96
bootstrap.system_call_filter: false

xpack.ssl.key: elasticsearch/elasticsearch.key
xpack.ssl.certificate: elasticsearch/elasticsearch.crt
xpack.ssl.certificate_authorities: ca/ca.crt
xpack.security.transport.ssl.enabled: true

xpack:
  security:
    authc:
      realms:
        ldap1:
          type: ldap
          order: 0
          url: "ldap://10.59.30.95"
          bind_dn: "cn=Manager, dc=martin, dc=com"
          bind_password: 123456
          user_search:
            base_dn: "ou=People,dc=martin,dc=com"
            attribute: uid
          group_search:
            base_dn: "ou=Group,dc=martin,dc=com"
          unmapped_groups_as_roles: false

配置AD域賬號認證
添加下ldap相關述配置至elasticsearch.yml，此處爲接着上述LDAP配置添加，若是隻需配置AD認證請將ldap相關配置刪除便可；

domain_name爲AD域的域名
url爲AD域的地址
bind_dnw爲隨意的域帳戶名稱（格式爲user@domain）

bind_password爲上述帳戶的密碼

xpack:
security:
authc:
  realms:
    ldap1:
      type: ldap
      order: 0
      url: "ldap://10.59.30.94"
      bind_dn: "cn=Manager, dc=martin, dc=com"
      bind_password: 123456
      user_search:
        base_dn: "ou=People,dc=martin,dc=com"
        attribute: uid
      group_search:
        base_dn: "ou=Group,dc=martin,dc=com"
      unmapped_groups_as_roles: false
    active_directory:
      type: active_directory
      order: 1
      domain_name: martin.com
      url: ldap://ad.martin.com
      bind_dn: martin@martin.com
      bind_password: AD.123456

重啓elasticsearch服務並使用ldap域帳戶user01登陸

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ killall java
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/elasticsearch -d
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat?pretty'
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用AD域帳戶martin登陸

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl http://10.59.30.96:9200/_cat?pretty -u martin:AD.123456
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

爲域帳戶信息映射角色

接口爲：POST /_xpack/security/role_mapping/

下述爲映射user1*帳戶爲管理員角色的操做步驟

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty' -d '{
>   "roles": [ "admin" ],
>   "enabled": true,
>   "rules": {
>     "any": [
>       {
>         "field": {
>           "username": "/user1*/"
>         }
>       }
>     ]
>   }
> }'
{
  "role_mapping" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty'
{
  "ldap_user_admin" : {
    "enabled" : true,
    "roles" : [
      "admin"
    ],
    "rules" : {
      "any" : [
        {
          "field" : {
            "username" : "/user1*/"
          }
        }
      ]
    },
    "metadata" : { }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

驗證域帳戶權限，使用user01無權訪問indices接口，使用user11能夠訪問；

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user11:user11 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 6178 44  5.9mb  5.9mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0 11.7kb 11.7kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                      
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  777  0  1.1mb  1.1mb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0 40.2kb 40.2kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0 12.8kb 12.8kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

ES數據按期刪除

若是不刪除ES數據，將會致使ES存儲的數據愈來愈多，磁盤滿了以後將沒法寫入新的數據。這時可使用腳本定時刪除過時數據。

#/bin/bash
#es-index-clear
#只保留15天內的日誌索引
LAST_DATA=`date -d "-15 days" "+%Y.%m.%d"`
#刪除上個月份全部的索引
curl -XDELETE 'http://ip:port/*-'${LAST_DATA}'*'

能夠視我的狀況調整保留的天數，這裏的ip和port一樣設置爲不存儲數據的那臺機器。該腳本只須要在ES中一臺機器定時運行便可。

crontab -e添加定時任務：
0 1 * * * /search/odin/elasticsearch/scripts/es-index-clear.sh
天天的凌晨一點清除索引。

常見報錯

No subject alternative names matching IP address

[2018-01-10T19:19:35,483][WARN ][o.e.x.s.t.n.SecurityNetty4Transport] [fzP4t-4] exception caught on transport layer [[id: 0x5d97fe48, L:/0:0:0:0:0:0:0:1:49121 ! R:/0:0:0:0:0:0:0:1:9300]], closing connection
    io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
......
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found

解決方案爲一種是關閉IPv6地址，另外一種是修改ES_HOME/config/elasticsearch.yml中的network.host值爲本機eth0的IP

原文：https://www.leiyawu.com/2018/05/07/es/

參考文檔
官方安裝步驟：https://www.elastic.co/guide/...
配置內置帳戶密碼：
https://www.elastic.co/guide/...
修改帳戶密碼：
https://www.elastic.co/guide/...
用戶相關操做：
https://www.elastic.co/guide/...
使用LDAP認證： https://www.elastic.co/guide/...
用戶角色映射： https://www.elastic.co/guide/...