ElasticSearch 命令行管理工具Curator

1、背景


elastic官網如今已經大面積升級到了5.x版本,然而針對elasticsearch的命令行管理工具curator如今仍然是4.0版本。java

剛開始找到此工具,深深的怕由於版本更迭沒法使用,還好在試用以後,顧慮煙消雲散。node

  • 應用目的

我主要用到了它刪除數據的功能。由於直接經過curl命令刪除數據須要輸入密碼,然而在利用<< EOF的方式也沒法繞過密碼校驗。python

懶得本身寫python腳本,既然已經有了對應工具,何不立刻用起來。json

 

2、安裝


一、Signing Keyedit

Download and install the public signing key:vim

rpm -- import  https: //packages.elastic.co/GPG-KEY-elasticsearch

二、Repository Configurationedit

Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example curator.repocentos

Warning
The repositories are different for CentOS/RHEL 6 and 7 due to library and path differences. Be sure to use the correct version for your system!bash

RHEL/CentOS 6
[curator- 4 ]
name=CentOS/RHEL  6  repository  for  Elasticsearch Curator  4 .x packages
baseurl=http: //packages.elastic.co/curator/4/centos/6
gpgcheck= 1
gpgkey=http: //packages.elastic.co/GPG-KEY-elasticsearch
enabled= 1
RHEL/CentOS 7
[curator- 4 ]
name=CentOS/RHEL  7  repository  for  Elasticsearch Curator  4 .x packages
baseurl=http: //packages.elastic.co/curator/4/centos/7
gpgcheck= 1
gpgkey=http: //packages.elastic.co/GPG-KEY-elasticsearch
enabled= 1

三、Binary Package Installationedit

Starting with version 4.1.2, Curator comes with a binary version. What this really means is that the source is compiled, and all required libraries are bundled with the curator binary, so there are no conflicts.curl

There are separate binary packages for RedHat variants. The binary packages resulting from the creation process have been tested on CentOS 6 & 7, with a different binary for each. They may work on similar variants and/or derivatives, but they have not been tested.elasticsearch

yum install elasticsearch-curator

This will install the necessary files into /opt/elasticsearch-curator and make a symlink at /usr/bin/curator that points to the curator binary in the aforementioned directory.工具

3、配置


運行報錯
curator_cli     
Click will abort further execution because Python  3  was configured to use ASCII as encoding  for  the environment.  Either run  this  under Python  2  or consult http: //click.pocoo.org/python3/ for mitigation steps.
This system lists a couple of UTF- 8  supporting locales that
you can pick from.  The following suitable locales where
discovered: aa_DJ.utf8, aa_ER.utf8, aa_ET.utf8, af_ZA.utf8, am_ET.utf8, an_ES.utf8, ar_AE.utf8, ar_BH.utf8, ar_DZ.utf8, ar_EG.utf8, ar_IN.utf8, ar_IQ.utf8, ar_JO.utf8, ar_KW.utf8, ar_LB.utf8, ar_LY.utf8, ar_MA.utf8, ar_OM.utf8, ar_QA.utf8, ar_SA.utf8, ar_SD.utf8, ar_SY.utf8, ar_TN.utf8, ar_YE.utf8, as_IN.utf8, ast_ES.utf8, ayc_PE.utf8, az_AZ.utf8, be_BY.utf8, bem_ZM.utf8, ber_DZ.utf8, ber_MA.utf8, bg_BG.utf8, bho_IN.utf8, bn_BD.utf8, bn_IN.utf8, bo_CN.utf8, bo_IN.utf8, br_FR.utf8, brx_IN.utf8, bs_BA.utf8, byn_ER.utf8, ca_AD.utf8, ca_ES.utf8, ca_FR.utf8, ca_IT.utf8, crh_UA.utf8, cs_CZ.utf8, csb_PL.utf8, cv_RU.utf8, cy_GB.utf8, da_DK.utf8, de_AT.utf8, de_BE.utf8, de_CH.utf8, de_DE.utf8, de_LU.utf8, doi_IN.utf8, dv_MV.utf8, dz_BT.utf8, el_CY.utf8, el_GR.utf8, en_AG.utf8, en_AU.utf8, en_BW.utf8, en_CA.utf8, en_DK.utf8, en_GB.utf8, en_HK.utf8, en_IE.utf8, en_IN.utf8, en_NG.utf8, en_NZ.utf8, en_PH.utf8, en_SG.utf8, en_US.utf8, en_ZA.utf8, en_ZM.utf8, en_ZW.utf8, es_AR.utf8, es_BO.utf8, es_CL.utf8, es_CO.utf8, es_CR.utf8, es_CU.utf8, es_DO.utf8, es_EC.utf8, es_ES.utf8, es_GT.utf8, es_HN.utf8, es_MX.utf8, es_NI.utf8, es_PA.utf8, es_PE.utf8, es_PR.utf8, es_PY.utf8, es_SV.utf8, es_US.utf8, es_UY.utf8, es_VE.utf8, et_EE.utf8, eu_ES.utf8, fa_IR.utf8, ff_SN.utf8, fi_FI.utf8, fil_PH.utf8, fo_FO.utf8, fr_BE.utf8, fr_CA.utf8, fr_CH.utf8, fr_FR.utf8, fr_LU.utf8, fur_IT.utf8, fy_DE.utf8, fy_NL.utf8, ga_IE.utf8, gd_GB.utf8, gez_ER.utf8, gez_ET.utf8, gl_ES.utf8, gu_IN.utf8, gv_GB.utf8, ha_NG.utf8, he_IL.utf8, hi_IN.utf8, hne_IN.utf8, hr_HR.utf8, hsb_DE.utf8, ht_HT.utf8, hu_HU.utf8, hy_AM.utf8, ia_FR.utf8, id_ID.utf8, ig_NG.utf8, ik_CA.utf8, is_IS.utf8, it_CH.utf8, it_IT.utf8, iu_CA.utf8, iw_IL.utf8, ja_JP.utf8, ka_GE.utf8, kk_KZ.utf8, kl_GL.utf8, km_KH.utf8, kn_IN.utf8, ko_KR.utf8, kok_IN.utf8, ks_IN.utf8, ku_TR.utf8, kw_GB.utf8, ky_KG.utf8, lb_LU.utf8, lg_UG.utf8, li_BE.utf8, li_NL.utf8, lij_IT.utf8, lo_LA.utf8, lt_LT.utf8, lv_LV.utf8, mag_IN.utf8, mai_IN.utf8, mg_MG.utf8, mhr_RU.utf8, mi_NZ.utf8, mk_MK.utf8, ml_IN.utf8, mn_MN.utf8, mni_IN.utf8, mr_IN.utf8, ms_MY.utf8, mt_MT.utf8, my_MM.utf8, nb_NO.utf8, nds_DE.utf8, nds_NL.utf8, ne_NP.utf8, nhn_MX.utf8, niu_NU.utf8, niu_NZ.utf8, nl_AW.utf8, nl_BE.utf8, nl_NL.utf8, nn_NO.utf8, nr_ZA.utf8, nso_ZA.utf8, oc_FR.utf8, om_ET.utf8, om_KE.utf8, or_IN.utf8, os_RU.utf8, pa_IN.utf8, pa_PK.utf8, pap_AN.utf8, pl_PL.utf8, ps_AF.utf8, pt_BR.utf8, pt_PT.utf8, ro_RO.utf8, ru_RU.utf8, ru_UA.utf8, rw_RW.utf8, sa_IN.utf8, sat_IN.utf8, sc_IT.utf8, sd_IN.utf8, se_NO.utf8, shs_CA.utf8, si_LK.utf8, sid_ET.utf8, sk_SK.utf8, sl_SI.utf8, so_DJ.utf8, so_ET.utf8, so_KE.utf8, so_SO.utf8, sq_AL.utf8, sq_MK.utf8, sr_ME.utf8, sr_RS.utf8, ss_ZA.utf8, st_ZA.utf8, sv_FI.utf8, sv_SE.utf8, sw_KE.utf8, sw_TZ.utf8, szl_PL.utf8, ta_IN.utf8, ta_LK.utf8, te_IN.utf8, tg_TJ.utf8, th_TH.utf8, ti_ER.utf8, ti_ET.utf8, tig_ER.utf8, tk_TM.utf8, tl_PH.utf8, tn_ZA.utf8, tr_CY.utf8, tr_TR.utf8, ts_ZA.utf8, tt_RU.utf8, ug_CN.utf8, uk_UA.utf8, unm_US.utf8, ur_IN.utf8, ur_PK.utf8, ve_ZA.utf8, vi_VN.utf8, wa_BE.utf8, wae_CH.utf8, wal_ET.utf8, wo_SN.utf8, xh_ZA.utf8, yi_US.utf8, yo_NG.utf8, yue_HK.utf8, zh_CN.utf8, zh_HK.utf8, zh_SG.utf8, zh_TW.utf8, zu_ZA.utf8
 
When used with Python  3  (and the DEB and RPM packages of Curator are compiled
and bundled with Python  3 ), Curator requires the locale to be unicode. Any of
the above unicode definitions are acceptable.
To set the locale to be unicode,  try :
$ export LC_ALL=en_US.utf8
$ curator_cli [ARGS]
Alternately, you should be able to specify the locale on the command-line:
$ LC_ALL=en_US.utf8 curator_cli [ARGS]
Be sure to substitute your unicode variant  for  en_US.utf8
配置修正
vim /etc/profile
#添加如下內容
export LC_ALL=en_US.utf- 8
export LANG=en_US.utf- 8
 
source /etc/profile
 

4、應用


主要利用其刪除功能,先展現下其命令行功能表:

$ curator_cli --help
Usage: curator_cli [OPTIONS] COMMAND [ARGS]...
 
Options:
   --config PATH       Path to configuration file. Default:
                       ~/.curator/curator.yml
   --host TEXT         Elasticsearch host.
   --url_prefix TEXT   Elasticsearch http url prefix.
   --port TEXT         Elasticsearch port.
   --use_ssl           Connect to Elasticsearch through SSL.
   --certificate TEXT  Path to certificate to use  for  SSL validation.
   --client-cert TEXT  Path to file containing SSL certificate  for  client auth.
   --client-key TEXT   Path to file containing SSL key  for  client auth.
   --ssl-no-validate   Do not validate SSL certificate
   --http_auth TEXT    Use Basic Authentication ex: user:pass
   --timeout INTEGER   Connection timeout in seconds.
   --master-only       Only operate on elected master node.
   --dry-run           Do not perform any changes.
   --loglevel TEXT     Log level
   --logfile TEXT      log file
   --logformat TEXT    Log output format [ default |logstash|json].
   --version           Show the version and exit.
   --help              Show  this  message and exit.
 
Commands:
   allocation        Shard Routing Allocation
   close             Close indices
   delete_indices    Delete indices
   delete_snapshots  Delete snapshots
   forcemerge        forceMerge index/shard segments
   open              Open indices
   replicas          Change replica count
   show_indices      Show indices
   show_snapshots    Show snapshots
   snapshot          Snapshot indices

我用到的刪除命令示例:

#!/bin/bash

dy=`date -d"1 weeks ago" +%Y.%m.%d`
echo "start deleting elasticsearch index by $dy"
curator_cli --host "127.0.0.1" --port "9200" --http_auth elastic:gl2016 delete_indices --filter_list "{\"filtertype\":\"pattern\",\"kind\":\"regex\",\"value\":\"logstash-(.+)-$dy\"}" >> /tmp/curator.log



相關文章
相關標籤/搜索