elasticsearch 使用指南

文檔： www.elastic.co/guide/en/el… blog.csdn.net/napoay/arti…

安裝中文分析器

// 下載elasticsearch
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.zip

./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.2.4/elasticsearch-analysis-ik-6.2.4.zip

// 訪問不了github 能夠先下載到本地，在安裝
./bin/elasticsearch-plugin install file:///mnt/elasticsearch-6.2.4/elasticsearch-analysis-ik-6.2.4.zip
複製代碼

配置

ulimit 設置爲大於65536

設置文件 /etc/sysctr.conf 參數
vm.max_map_count=655360
遠程訪問
修改文件 ./elasticsearch-6.2.4/config/elasticsearch.yml
network.host: 0.0.0.0   // 設置過此項，會認爲切換到生產環境，會檢測系統文件最大打開數量是否達標，不然啓動失敗

啓動參數
設置ES_HEAP_SIZE參數
export ES_HEAP_SIZE=10g
此外，你也能夠經過命令行參數的形式，在程序啓動的時候把內存大小傳遞給它，若是你以爲這樣更簡單的話：

./bin/elasticsearch -Xmx10g -Xms10g 


確保堆內存最小值（ Xms ）與最大值（ Xmx ）的大小是相同的，防止程序在運行時改變堆內存大小， 這是一個很耗系統資源的過程。

一般來講，設置 ES_HEAP_SIZE 環境變量，比直接寫 -Xmx -Xms 更好一點。

複製代碼

守護進程啓動

./bin/elasticsearch -d  // -d參數守護進程啓動
複製代碼

中止

$ ./bin/elasticsearch -p /tmp/elasticsearch-pid -d
$ cat /tmp/elasticsearch-pid && echo
15516
$ kill -SIGTERM 15516
複製代碼

索引(庫)

// 建立索引(庫)
curl -XPUT 'localhost:9200/lovewith_v1?pretty'

// 查看索引
curl -X GET "localhost:9200/lovewith"

// 刪除索引
curl -X DELETE "localhost:9200/lovewith_v2"
複製代碼

建立索引別名

// 設置單個別名
curl -X PUT "localhost:9200/lovewith_v1/_alias/lovewith"

// https://www.elastic.co/guide/en/elasticsearch/reference/6.2/indices-aliases.html
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d' { "actions" : [ { "add" : { "index" : "lovewith_v2", "alias" : "lovewith" } }, { "remove" : { "index" : "lovewith_v1", "alias" : "lovewith" } } ] } '

// 查看索引別名

curl -X GET "localhost:9200/_alias/lovewith*"

// 刪除別名
curl -X DELETE "localhost:9200/lovewith_v1/_alias/lovewith"

複製代碼

建立 type（表）

curl -X PUT "localhost:9200/lovewith/_mapping/${type名}" -H 'Content-Type: application/json' -d' { "fulltext": { "_all": { "analyzer": "ik" }, "properties": { "id": { "type": "integer" }, "tag": { "type": "text" //text取代了string，當一個字段是要被全文搜索的，好比Email內容、產品描述，應該使用text類型。設置text類型之後，字段內容會被分析，在生成倒排索引之前，字符串會被分析器分紅一個一個詞項。text類型的字段不用於排序，不多用於聚合（termsAggregation除外） "analyzer": "ik_max_word", // not_analyzed 不分詞 "search_analyzer": "ik_max_word" }, "color": { "type":"keyword", //keyword類型的數據只能徹底匹配，適合那些不須要分詞的數據，好比email地址、主機名、狀態碼和標籤。若是字段須要進行過濾(好比查找已發佈博客中status屬性爲published的文章)、排序、聚合。keyword類型的字段只能經過精確值搜索到。 } "created_at": { "type": "date", "format": "EEE MMM dd HH:mm:ss Z YYYY" } } } '

// 查看 type
curl -X GET "localhost:9200/lovewith_v1/_mapping/${type名}"

// 檢測 type 是否存在
curl -X HEAD "localhost:9200/lovewith_v1/_mapping/${type名}"
複製代碼

mapping

// 查看索引mapping
curl -XGET "http://localhost:9200/lovewith/_mapping?pretty"
curl -X GET "localhost:9200/_all/_mapping"
curl -X GET "localhost:9200/_mapping"

// 查看具體字段mapping
curl -X GET "localhost:9200/lovewith/_mapping/${type名}/field/title"

// 修改mapping
curl -X PUT "localhost:9200/lovewith/_mapping/${type名}" -H 'Content-Type: application/json' -d' { "properties": { "color": { "type": "keyword", "index": true } } } '

複製代碼

分詞測試

// 標準模式
curl -X POST "localhost:9200/_analyze" -H 'Content-Type: application/json' -d' { "analyzer": "standard", // ik_max_word 中文分詞模式 "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog\u0027s bone." } '

複製代碼

bulk 批處理

// 方式一 http
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' -d' { "index" : { "_index" : "lovewith_v1", "_type" : "${type名}", "_id" : "1" } } { "field1" : "value1" } { "delete" : { "_index" : "lovewith_v1", "_type" : "${type名}", "_id" : "2" } } { "create" : { "_index" : "lovewith_v1", "_type" : "${type名}", "_id" : "3" } } { "field1" : "value3" } { "update" : {"_id" : "1", "_type" : "${type名}", "_index" : "lovewith_v1"} } { "doc" : {"field2" : "value2"} } '

// 方式二 讀文件，大小控制在５～15M
新建一個requests 文件，文件內容以下，最後一行以換行符結束，"_id" 可不填，es 會自動生成

cat requests
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
{ "field1" : "value1" }

// 執行以下命令
curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
複製代碼

統計

// 磁盤佔比
curl -X GET "localhost:9200/_cat/allocation?v"

// 查看別名
http://127.0.0.1:9200/_cat/aliases?v

// 文檔總數
http://127.0.0.1:9200/_cat/count?v

// 統計指定索引文檔總數
http://127.0.0.1:9200/_cat/count/lovewith?v


複製代碼

搜索

// 指定字段搜索
http://127.0.0.1:9200/lovewith/album/_search?q=tag:花瓶

聚合查詢
參考：http://www.cnblogs.com/huanxiyun/articles/5888874.html
文檔：https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html

// 按數組字段精確過濾
curl -X GET "localhost:9200/lovewith/album/_search?pretty" -H 'Content-Type: application/json' -d' { "query": { "match_phrase": { "color1": 7 } } } '

// 過濾搜索 針對ketword類型字段有效
curl -X GET "localhost:9200/lovewith/_search" -H 'Content-Type: application/json' -d' { "query": { "bool": { "filter": [ { "term": { "color": "red" }}, { "term": { "brand": "gucci" }} ] } } } // 文檔分頁獲取 curl -XGET "http://localhost:9200/fqhelp_v1/_search?pretty&from=20&size=10" '

## match 與term 區別
https://www.cnblogs.com/yjf512/p/4897294.html

ANALYZED字段沒法使用term，只能使用match_phrase

// 搜索過濾
curl -X GET "localhost:9200/lovewith/superlib/_search?pretty" -H 'Content-Type: application/json' -d' { "query": { "match":{ "tags":"婚紗" } }, "post_filter":{ "term": { "status": 1} } } '

複製代碼

排序

// 默認只對數字，日期字段類型有效
// https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
curl -X GET "localhost:9200/my_index/_search" -H 'Content-Type: application/json' -d' { "sort" : [ {"date": "asc"}, {"id": "desc"} ], "query" : { "term" : { "user" : "kimchy" } } } '
複製代碼

刪除文檔

curl -X DELETE "localhost:9200/lovewith/${type名}/${文檔id}"
複製代碼

更單個文檔

curl -X PUT "localhost:9200/lovewith/${type名}/1" -H 'Content-Type: application/json' -d' { "counter" : 1, "tags" : ["red"] } '
複製代碼

添加數字段

curl -X POST "localhost:9200/lovewith/album/WGj9s2MBsz6suBcoSKEi/_update?pretty" -H 'Content-Type: application/json' -d' { "script" : "ctx._source.color1 = [7]" } '
複製代碼

更指定字段

// doc_as_upsert 爲true 文檔不存在時建立
curl -X POST "localhost:9200/lovewith_v2/superlib/1998/_update?pretty" -H 'Content-Type: application/json' -d' { "doc" : { "id": 1998, "path": "share/2012/12/15/14ad5c4a5156d21af088676dd2373b978rrJPB_650x999.jpg", "tag": "婚紗照2" }, "doc_as_upsert" : true } '
複製代碼

刪除文檔中的一個字段

// 刪除 color字段
curl -X POST "localhost:9200/lovewith/album/Vmjhs2MBsz6suBco4KFk/_update?pretty" -H 'Content-Type: application/json' -d' { "script" : "ctx._source.remove(\u0027color\u0027)" } '
複製代碼

數組字段追加元素

curl -X POST "localhost:9200/lovewith/album/Vmjhs2MBsz6suBco4KFk/_update?pretty" -H 'Content-Type: application/json' -d' { "script" : { "source": "ctx._source.color1.add(params.color1)", "lang": "painless", "params" : { "color1" : 7 } } } '
複製代碼

刪除type全部數

curl -X POST "localhost:9200/lovewith/album/_delete_by_query?conflicts=proceed" -H 'Content-Type: application/json' -d' { "query": { "match_all": {} } } '
複製代碼

備份與恢復

若是是集羣須要建立以下共享目錄

安裝 sshfs

sshfs 使用參考：使用sshfs掛載遠程服務器目錄

apt-get install fuse sshfs
mkdir /mnt/backup
chmod 755 /mnt/backup
mkdir /data/es_backup
# 每一個節點掛載一樣操做掛載/mnt/backup
sshfs 192.168.1.10:/data/es_backup /mnt/backup -o allow_other
其中的參數-o allow_other容許了其餘用戶訪問這個目錄。

測試運行ES的用戶對共享目錄是否有寫權限
sudo -u elasticsearch touch /mnt/backup/test
複製代碼

在elasticsearch.yml中加入一行:

// 設置備份目錄
path.repo: ["/mnt/backup"]
複製代碼

// 建立倉庫
curl -XPUT 'http://127.0.0.1:9200/_snapshot/my_backup?pretty' -H 'Content-Type: application/json' -d '{ "type": "fs", "settings": { "location":"/mnt/backup", "max_snapshot_bytes_per_sec" : "50mb", "max_restore_bytes_per_sec" :"50mb" } }'

// 刷新數據到磁盤
curl -X POST "localhost:9200/lovewith/_flush"

// 備份指定索引
curl -XPUT 'http://127.0.0.1:9200/_snapshot/my_backup/snapshot_1' -H 'Content-Type: application/json' -d '{ "indices": "lovewith_v1" }?wait_for_completion=true'


// 刪除索引
curl -XDELETE "http://192.168.1.10:9200/lovewith_v1"

// 恢復指定索引
將 /mnt/backup 下的全部文件傳到目標節點倉庫目錄裏
curl -XPOST 'http://127.0.0.1:9200/_snapshot/my_backup/snapshot_1/_restore' -H 'Content-Type: application/json' -d '{ "indices": "lovewith_v2" }?wait_for_completion=true'

// 查看恢復狀
curl -XGET 'http://127.0.0.1:9200/_snapshot/my_backup/snapshot_1/_status?pretty'

// 刪除快照
curl -XDELETE 'http://127.0.0.1:9200/_snapshot/my_backup/snapshot_1'
複製代碼

translog

執行時es服務須要停
bin/elasticsearch-translog truncate -d /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/
複製代碼