es操做手冊

時間 2019-11-12

標籤手冊简体版

原文原文鏈接

0 _search查詢數據時能夠指定多個index和type程序員

GET /index1,index2/type1,type2/_search

GET /_all/type1/_search  至關於查詢所有index下的type1的document

GET /_all/type1/_search?from=0&size=5 from和size爲分頁參數

1 增長一條數據，手動指定document的IDjson

PUT /index1/type1/1
{
"content1":"abcnt地方士大夫",
"age":"abc你的"
}

2 增長一條數據，自動指定document的ID數組

POST /index1/type1
{
"content1":"abcnt地方士大夫",
"age":"abc你的"
}

3 獲取一條數據的方式，並指定查詢返回字段app

GET /index1/type1/1?_source=age,content1

4 es更新數據時使用自定義版本號，只有版本號大於當前版本號才容許更新操做ide

PUT /index1/type1/1?version=5&version_type=external  （以前的_version屬性必須小於5）
{
"description":"程序員一枚~~~"
}

5 partial update對document中的部分field進行更新性能

POST /index1/type1/1/_update?version=13 （必須version等於當前版本號時才能夠修改數據，並且內容和原來相同則認爲未更改版本號不變；該操做在更新期間不會被打斷）
{
"doc":{
"description":"程序員一枚~~~7778899056"
}
}

6 經過GET /_mget 批量查找數據，須要提供index，type，id（能夠經過url參數增長，根據搜索範圍不一樣使用不一樣的查詢參數）ui

GET /_mget
{
"docs":[
{
"_index":"index1",
"_type":"type1",
"_id":"1",

  "_version":16
},
{
"_index":"index1",
"_type":"type1",
"_id":"2"
}
]
}

GET /index1/_mget
{
"docs":[
{
"_type":"type1",
"_id":"1",
"_version":16
},
{
"_type":"type1",
"_id":"2"
}
]
}

GET /index1/type1/_mget
{
"ids":[1,2]
}

7 _search搜索默認查詢前10條（timeout=1ms能夠指定超時時間）url

GET /index1/type1/_search?timeout=1ms

8 使用_search?q=xxx，爲全字段查詢，若是使用_search?q=field:xxx爲按照具體字段進行查詢；idea

PUT /index3/type3/1
{
  "date":"2019-01-02",
  "name":"the little",
  "content":"Half the ideas in his talk were plagiarized from an article I wrote last month."
}

PUT /index3/type3/2
{
  "date":"2019-01-01",
  "name":"a dog",
  "content":"is the girl, women's attention and love day. July 7th Qiqiao customs, originated in the Han "
}

PUT /index3/type3/3
{
  "date":"2019-07-01",
  "name":"very tag",
  "content":"Some of our comrades love to write long articles with no substance, very much like the foot bindings of a slattern, long as well as smelly"
}

//可是按照具體字段查詢時若是字段類型爲date或者long等時間和數值類型則使用exact value去匹配
GET /index3/type3/_search?q=date:2019-01 //只能查詢出1條數據，查詢方式爲exact value
GET /index3/type3/_search?q=2019-01 //則能查詢出3條，由於會使用full text全字匹配，會將每一可能的部分都進行分詞，只要包含則能夠查詢出來

9 使用mapping指定索引字段的類型以及是否要進行分詞，可是手動建立索引的mapping，只能給新字段指定，或者還沒建立的索引指定，mapping不能修改spa

DELETE /index3

//建立索引並指定字段的屬性
PUT /index3
{ 
  "mappings": {
    "type3": {
      "properties": {
        "date":{
          "type": "date"//日期類型的exact value匹配粗略除外，es會按照搜索的部分日期匹配出一個返回.如：GET /index3/type3/_search?q=date:2019 
        },
        "name":{
          "type": "keyword"
        },
        "no":{
           "type": "long"
        },
        "content":{
           "analyzer": "standard",
           "type": "string"
        }
      }
    }
  }
}

//添加數據
PUT /index3/type3/1
{
  "date":"2019-01-02",
  "name":"the little",
  "content":"Half the ideas in his talk were plagiarized from an article I wrote last month.",
  "no":"123"
}

PUT /index3/type3/2
{
  "date":"2019-01-01",
  "name":"a dog",
  "content":"is the girl, women's attention and love day. July 7th Qiqiao customs, originated in the Han ",
  "no":"6867858"
}

PUT /index3/type3/3
{
  "date":"2019-07-01",
  "name":"very tag",
  "content":"Some of our comrades love to write long articles with no substance, very much like the foot bindings of a slattern, long as well as smelly",
  "no":"123"
}

GET /index3/type3/_search?q=name:very tag  使用字段名稱只能exact value策略匹配能夠查詢的到，由於type指定爲keyword

View Code

10 _search的精準匹配和分詞後的全文檢索

GET /index2/type2/_search
{
  "query": {
    //bool中出現的must和should等取交集
    "bool": {
      //must要求match裏面的字段name全字匹配
      "must": [
        {
          "match": {
            "name": "ui the mark"
          }
        }
      ]
      , 
      //should要求match裏面的字段content能夠進行分詞後的查詢 
      "should": [
        {
          "match": {
            "content": "bought"
          }
        }
      ]
    }
  }
}

11 使用滾動分頁數據查詢方式，代替es的分頁功能，由於es分頁功能在深度分頁時會向coordinate節點發送大量數據，排序後在取出指定位置的數據，性能很低下

//scroll=100ms滾動查詢方式，超時時長100ms
GET /index3/type3/_search?scroll=100ms
{
  "query": {
    //查詢全部數據
    "match_all": {}
  },
  "sort": [
    {
      //排序方式按時間升序
      "date": {
        "order": "asc"
      }
    }
  ],
  //每次向後查詢3條
  "size": 3
}

12 es的DSL方式filter指定字段範圍過濾（filter不參與TF&IDF評分，只進行條件過濾）

PUT /index2/type2/1
{
  "num":10,
  "name":"ui the mark",
  "content":"Mr. Johnson had never been up in an aerophane before and he had read a lot about air accidents, so one day when a"
}

PUT /index2/type2/2
{
  "num":100,
  "title":"他的名字",
  "name":"my tag",
  "content":"He bought a gallon of gas. He put the gas into a gas can. He waited until "
}

PUT /index2/type2/3
{
  "num":1000,
  "title":"這是誰的名字",
  "name":"very lit",
  "content":"happening in the world.But radio isn't lost. It is still with us. That's because a radio is very small，and it's easy to carry. You can put one in your pocket and "
}

GET /index2/type2/_search
{
  "query": {
    //bool中出現的must和should等取交集
    "bool": {
      //should要求match裏面的字段content能夠進行分詞後的查詢 
      "should": [
        {
          "match": {
            "content": "bought"
          }
        }
      ]
      ,
      "filter": {
        "range": {
          "num": {
            "gte": 10,
            "lte": 1010
          }
        }
      }
    }
  }
}

View Code

13 使用mapping的動態屬性限定索引的document中的json內容

PUT /index3
{ 
  "mappings": {
    "type3": {
      "dynamic":"true",
      "properties": {
        "date":{
          "type": "date"
        },
        "name":{
          "type": "keyword"
        },
        "no":{
           "type": "long"
        },
        "content":{
           "type": "keyword"
        },
        "address":{
          //dynamic（默認爲true）一旦聲明爲strict，則不容許type下添加額外未指定的字段，並且dynamic可在json屬性內部嵌套
          "dynamic":"strict",
          "properties": {
            "city":{
              "type":"keyword"
            },
            "description":{
              "type":"text"
            }
          }
        }
      }
    }
  }
}

14 爲索引index添加一個別名，若是須要index從新建立，能夠經過添加刪除別名指向的索引，從而不用修改程序無縫切換

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "index4",
        "alias": "index3_alias"
      }
    }
  ]
}

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "index3",
        "alias": "index3_alias"
      }
    }
  ]
}

PUT /index4/type3/1
{
  "a":"a"
}

GET /index3_alias/type3/1

15 若是索引mapping中的字段類型已經指定，則沒法添加其餘類型的值（形如13中的索引建立方式）

//這種操做則會報錯，由於定義date字段已經指定爲日期類型
PUT /index3/type3/7
{
  "date":"asdsad",
  "name":"http litty",
  "content":"The happiest of people don’t necessarily have the best of everything;they just make the most of everything that comes along their",
  "no":"9786"
}

16 調整一個document提交到index能查詢到的時間閾值，也就是buffer的refresh時間間隔（buffer默認每秒refresh到磁盤一次，translog若是沒達到閾值大小則30分鐘持久化到磁盤1次，並清空buffer）

PUT /index5
{
  "settings": {
    //每次從buffer執行refresh到磁盤的時間爲30s    
    "refresh_interval": "30s"
  }
}

17 當咱們不關心檢索詞頻率TF（Term Frequency）對搜索結果排序的影響時，可使用constant_score將查詢語句query或者過濾語句filter包裝起來。並且term對搜索部分詞，全字匹配輸入；（filter不參與TF&IDF評分，只進行條件過濾，使用constant_score能夠取代只有filter的bool查詢，filter可以保證不參與相關度計算，只是數據過濾，因此效率要高出不少）

GET index2/type2/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "term": {
          "name": "my tag"
        }
      }
    }
  }
}

18 若是name指定了type:keyword，那麼只能使用"_all":"xxx"去匹配，由於keyword支持按字段extract value匹配和_all的full text全文檢索匹配

GET index2/type2/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match":{
            //"name":"very"不會搜索出任何內容   
            "_all":"very"//走全文檢索才能匹配出結果
          }
        }
      ]
    }
  }
}

19 filter能夠嵌套多層bool查詢

PUT /index2/type2/1
{
  "num": 1,
  "title":"你的名字",
  "name":"ui the mark",
  "content":"Mr. Johnson had never been up in an aerophane before and he had read a lot about air accidents, so one day when a"
}

PUT /index2/type2/2
{
  "num": 10,
  "title":"他的名字",
  "name":"my tag",
  "content":"He bought a gallon of gas. He put the gas into a gas can. He waited until "
}

PUT /index2/type2/3
{
  "num": 105,
  "title":"這是誰的名字",
  "name":"very lit",
  "content":"happening in the world.But radio isn't lost. It is still with us. That's because a radio is very small，and it's easy to carry. You can put one in your pocket and "
}


POST /index2/_mapping/type2
{
  "properties": {
    "name":{
      "type": "keyword"
    },
    "content":{
      "type": "text",
      "analyzer": "english"
    }
  }
}

GET /index2/type2/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "bool": {
          "must":[
            {
              "term":{
                "name":"very lit"
              }
            },
            {
              "bool":{
                "should":[
                  {
                    "match":{
                      "title": "我"
                    }
                  }
                ]
              }
            }
          ]
        }
      }
    }
  }
}

View Code

20 使用bool查詢時，若是沒有must而有should則should中必須匹配一條，若是有must，則should中的條件能夠不作任何匹配（"minimum_number_should_match": 3, should數組至少匹配3個條件）

GET /index2/type2/_search
{
  "query": {
    "bool": {
      "minimum_number_should_match": 3, 
      "should": [
        {
          "match": {
            "title": "尼瑪"
          }
        },
        {
          "match": {
            "name": "very lit"
          }
        },
        {
          "match": {
            "content":"happening"
          }
        }
      ]
    }
  }
}

21 單個field查詢時的詞量匹配，可手動控制精準程度，minimum_should_match指定在」你名字 d「 4個詞至少得匹配3個詞即爲75%

GET /index2/type2/_search
{
  "query": {
    "match": {
      "title": {
        "query": "你 名 字 d",
        "minimum_should_match": "75%"
      }
    }
  }
}

22 查詢後的結果若是想要提高某一搜索關鍵詞的評分使用boost屬性指定score

GET /index2/type2/_search
{
  "query": {
    "bool": {
      "minimum_number_should_match": 2, 
      "should": [
        {
          "match": {
            "title": {
              "query": "誰",
              "boost":5
            }
          }
        },
        {
          "match": {
            "name": "very lit"
          }
        },
        {
          "match": {
            "content":"happening"
          }
        }
      ]
    }
  }
}

23 multi_match方式的多字段，多查詢模式

GET /index2/type2/_search
{
  "query": {
    "multi_match": {
      "query": "happening like",
      //query中的搜索詞條去content和name兩個字段中來匹配，不過會因爲兩個字段mapping定義不一樣致使得分不一樣，排序結果可能有差別
      "fields": ["name","content"],
      //best_fields策略是每一個document的得分等於得分最高的match field的值；而匹配出最佳之後，其它document得分未必準確；most_fields根據每一個field的評分計算出ducoment的綜合評分
      "type":"best_fields"
    }
  }
}

結果
{
  "took": 71,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": 0.5063205,
    "hits": [
      {
        "_index": "index2",
        "_type": "type2",
        "_id": "3",
        "_score": 0.5063205,
        "_source": {
          "num": 105,
          "title": "這是誰的名字",
          "name": "happening like write",
          "content": ""
        }
      },
      {
        "_index": "index2",
        "_type": "type2",
        "_id": "2",
        "_score": 0.41043553,
        "_source": {
          "num": 10,
          "title": "他的名字",
          "name": "yes happening like write",
          "content": "happening i like"
        }
      },
      {
        "_index": "index2",
        "_type": "type2",
        "_id": "4",
        "_score": 0.34450945,
        "_source": {
          "num": 1000,
          "title": "個人名字",
          "name": "happening like write",
          "content": "happening like yeas and he had read a lot about"
        }
      }
    ]
  }
}

View Code

24 使用match_phrase對field值進行完整query詞組匹配，該詞組不作分詞直接完整匹配

GET /index2/type2/_search
{
  "query": {
    "bool": {
      "minimum_number_should_match": 1, 
      "should": [
        {
          //match_phrase短語匹配要求content字段必須包含treasure because值才能匹配得上
            "match_phrase": {
            "content": "treasure because"
          }
        }
      ]
    }
  }
}

View Code

25 使用match_phrase與slop，在使用詞組徹底匹配時，能夠在整個field值中，移動詞組內的單個詞位置，移動範圍由slop參數指定，若是經過移動後能組成要搜索的詞條，也認爲匹配成功

GET /index2/type2/_search
{
  "query": {
    "bool": {
      "minimum_number_should_match": 1, 
      "should": [
        {
          //match_phrase短語匹配要求content字段必須包含treasure because值才能匹配得上
          "match_phrase": {
            "content": {
              "query": "treasure because",
              "slop":2//treasure與because每一個詞，左右移動2個position後若是可以組合成treasure because詞組則匹配成功
            }
          }
        }
      ]
    }
  }
}

結果：
{
  "took": 15,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.53484553,
    "hits": [
      {
        "_index": "index2",
        "_type": "type2",
        "_id": "3",
        "_score": 0.53484553,
        "_source": {
          "num": 105,
          "title": "這是誰的名字",
          "name": "happening like write",
          "content": " national  treasure because  of its rare number and cute appearance. Many foreign people are so crazy about  pandas and they can’t watching these  lovely creatures all the time. Though some action"
        }
      },
      {
        "_index": "index2",
        "_type": "type2",
        "_id": "4",
        "_score": 0.45520112,
        "_source": {
          "num": 1000,
          "title": "個人名字",
          "name": "happening like write",
          "content": "happening treasure hello like because yeas and he happening like had read a lot about happening hello like"
        }
      }
    ]
  }
}

View Code

26 使用rescoring機制增長匹配的精準度，並提升搜索效率，由於match要比match_phrase的性能好10倍左右，match_phrase性能要比match_phrase+slop性能好20幾倍；rescoring可在搜索以後取出前300條數據（通常用戶分頁查詢後瀏覽不會超過10頁）進行match_phrase和slop的設置，來從新進行打分排序

GET index3/type3/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "content":"hello book"//先按照hello book分詞後匹配出結果
          }
        }
      ]
    }
  },
  "rescore":{
    "window_size":300,//從must的結果中取出300條重排序
    "query":{
      "rescore_query":{
        "match_phrase":{
          "content":{
            "query":"hello book",
            "slop":88//排序規則是按照hello和book兩個詞的position關係來決定，距離越近得分越高
          }
        }
      }
    }
  }
}
結果：
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 6,
    "max_score": 1.0520453,
    "hits": [
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "1",
        "_score": 1.0520453,
        "_source": {
          "date": "2019-01-02",
          "name": "the little",
          "content": "Half the hello book ideas in his talk were plagiarized from an article I wrote last month.",
          "no": "123"
        }
      },
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "4",
        "_score": 1.0472052,
        "_source": {
          "date": "2019-03-01",
          "name": "http litty",
          "content": "http://localhost:5601/app/kibana#/dev_tools/console?_g=() hello the book you ",
          "no": "123"
        }
      },
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "3",
        "_score": 0.8442862,
        "_source": {
          "date": "2019-07-01",
          "name": "very tag",
          "content": "Some of our hello  comrades love book to write long articles with no substance, very much like the foot bindings of a slattern, long as well as smelly",
          "no": "123"
        }
      },
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "5",
        "_score": 0.6407875,
        "_source": {
          "date": "2019-05-01",
          "name": "http litty",
          "content": "There are hello moments in life when you miss book someone so much that you just want to pick them from your dreams",
          "no": "564",
          "description": "描述"
        }
      },
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "6",
        "_score": 0.52347976,
        "_source": {
          "date": "2019-06-01",
          "name": "http litty",
          "content": "The happiest of hello people don’t necessarily have the best of everything;they just make the you most of everything that comes along their book",
          "no": "9786"
        }
      },
      {
        "_index": "index3",
        "_type": "type3",
        "_id": "2",
        "_score": 0.1252801,
        "_source": {
          "date": "2019-02-01",
          "name": "a dog",
          "content": "is the girl,hello women's attention and love day. July 7th Qiqiao customs, originated in the Han ",
          "no": "6867858"
        }
      }
    ]
  }
}

View Code

27 詞組+左匹配搜索實現自動完成

GET index3/type3/_search
{
  "query": {
    "match_phrase_prefix": {//詞組搜索+左前綴匹配（可用於自動完成功能）
      "title": {
        "query": "the yellow",
        "slop":10,//兩單詞左右移動位置，能匹配doc就返回
        "max_expansions": 50//詞組左匹配的時候，最多匹配50條（該參數頗有必要，若是不限制匹配條數則可能出現性能急劇降低，由於要針對全部索引進行左前綴過濾，這種狀況是災難的）
      }
      
    }
  }
}

View Code

28 在bool組合查詢下，1 先進行條件過濾篩選，2 在進行字段分詞檢索

GET /index3/type3/_search
{
  "query": {
    "bool": {
      "filter": [//name和日期交際
        {
          "term":{
            "name":"http litty"
          }
        },
        {
          "terms":{
              "date":["2019-06-01","2019-03-01"]//日期條件並集
          }
        }
      ],
      "must": [
        {
          "match":{//模糊檢索content字段
            "content":"The happiest of"
          }
        }
      ]
    }
  }
}

View Code

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。