Elasticsearch定時刪除索引

在Elasticsearch中，咱們經常須要按時間來創建索引，以便咱們從此的使用與管理，同時咱們也常常按時間去刪除一些老的數據。好比只保留最近3天的數據，只需將超多3天的索引數據刪除就行了。刪除索引有不少種方法，你能夠本身編寫腳本刪除索引，也能夠手動刪除索引curl -XDELETE http://127.0.0.1:9200/index-name，這裏使用Curator工具刪除索引。

Curator

Curator 是elasticsearch 官方的一個索引管理工具，能夠經過配置文件的方式幫助咱們對指定的一批索引進行建立/刪除、打開/關閉、快照/恢復等管理操做。

安裝參考文檔： Curator Reference [5.6] » Installation

用法

Usage: curator [OPTIONS] ACTION_FILE

  Curator for Elasticsearch indices.

  See http://elastic.co/guide/en/elasticsearch/client/curator/current

Options:
  --config PATH  Path to configuration file. Default: ~/.curator/curator.yml
  --dry-run      Do not perform any changes.
  --version      Show the version and exit.
  --help         Show this message and exit.
複製代碼

配置

github上給出了配置示例：github.com/elastic/cur…

這裏，列出最經常使用的2個配置文件

curator.yml配置

# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
client:
 hosts:
 - 127.0.0.1
 port: 9200
 url_prefix:
 use_ssl: False
 certificate:
 client_cert:
 client_key:
 aws_key:
 aws_secret_key:
 aws_region:
 ssl_no_validate: False
 http_auth:
 timeout: 30
 master_only: False

logging:
 loglevel: INFO
 logfile:
 logformat: default
 blacklist: ['elasticsearch', 'urllib3']
複製代碼

delete_indices.yml配置(刪除3天以上的索引配置)

# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True. If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
  1:
 action: delete_indices
 description: >-
      Delete indices older than 3 days (based on index name), for test-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
 options:
 ignore_empty_list: True
 timeout_override:
 continue_if_exception: False
 disable_action: False
 filters:
 - filtertype: pattern
 kind: prefix
 value: test-
 exclude:
 - filtertype: age
 source: name
 direction: older
 timestring: '%Y.%m.%d'
 unit: days
 unit_count: 3
 exclude:
複製代碼

使用示例

執行`

curator --config /etc/elasticsearch_curator/curator.yml /etc/elasticsearch_curator/delete_indices.yml
複製代碼

運行結果：

ubuntu@localhost:/etc/elasticsearch_curator$ curator --config /etc/elasticsearch_curator/curator.yml /etc/elasticsearch_curator/delete_indices.yml 
2018-12-20 14:22:06,854 INFO      Preparing Action ID: 1, "delete_indices"
2018-12-20 14:22:06,863 INFO      Trying Action ID: 1, "delete_indices": Delete indices older than 3 days (based on index name)
2018-12-20 14:22:06,965 INFO      Deleting selected indices: ['test-2018.12.10']
2018-12-20 14:22:06,965 INFO      ---deleting index test-2018.12.10
2018-12-20 14:22:07,262 INFO      Action ID: 1, "delete_indices" completed.
2018-12-20 14:22:07,262 INFO      Job completed.
複製代碼

能夠看到刪除了超過3天的索引。

定時執行任務

配置好Curator後，爲了完成定時刪除索引的功能還要配置定時任務。 執行crontab -e，添加如下一行：

0 23 * * * curator --config /etc/elasticsearch_curator/curator.yml /etc/elasticsearch_curator/delete_indices.yml >> /home/ubuntu/elk/data/curator/curator.log 2>&1
複製代碼

天天23:00定時執行刪除索引的任務。

參考文檔：
Curator Reference
Curator examples
Curator從入門到實戰

---

關注微信公衆號，按期推送文章！