Elasticsearch

Elasticsearch

ElasticSearch是一個基於Lucene的搜索服務器。它提供了一個分佈式多用戶能力的全文搜索引擎，基於RESTful web接口。Elasticsearch是用Java開發的，並做爲Apache許可條款下的開放源碼發佈，是當前流行的企業級搜索引擎。設計用於雲計算中，可以達到實時搜索，穩定，可靠，快速，安裝使用方便。

 

Elasticsearch Reference

參考文檔：

https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html

 

ES安裝

下載elasticsearch-2.2.0.tar.gz，解壓，執行/bin/elasticsearch;

訪問地址：http://node1:9200

 

安裝插件 head

執行bin/plugin install mobz/elasticsearch-head

訪問地址：http://node1:9200/_plugin/head

若是用root用戶執行啓動腳本會提示：

Exception in thread "main" java.lang.RuntimeException: don't run elasticsearch as root.

建立elasticsearch用戶執行

adduser –U elasticsearch

而後以 elasticsearch用戶登陸，啓動程序。

 

集羣部署

部署三臺機器，部署程序，而後修改配置文件 config/elasticsearch.yml

配置：

cluster.name: my-application
node.name: node1
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s
discovery.zen.ping.unicast.hosts: ["node1","node2", "node3"]

集羣名字cluster.name相同才能組成一個集羣，修改node.name對應的主機名。

 

防止腦裂配置

discovery.zen.minimum_master_nodes=2

 

Curl

經過curl操做Elasticsearch數據：

curl -XPUT ‘http://node1:9200/index1/’ -d ‘傳輸的數據’

-X指定http請求方法：HEAD, PUT, GET, PST, DELETE

-d 指定傳輸的數據

 

–GET：獲取對象的當前狀態；

–PUT：改變對象的狀態；      冪等

–POST：建立對象；

–DELETE：刪除對象；          冪等

–HEAD：獲取頭信息。

 

Elasticsearch與關係型數據庫的對比

Database	Index
Table	Type
Row	Document
Column	Field

 

 

 

 

 

建立Index

curl -XPUT 'http://node1:9200/test/'

 

建立Type

curl -XPOST http://node1:9200/test/emp/ -d '{"name" : "John"}' 自動建立id,也能夠本身指定id
curl -XPUT http://node1:9200/test/emp/1 -d '{"name" : "Tom"}'  須要指定id

 

查詢

curl -XGET http://node1:9200/test/emp/1

{"_index":"test","_type":"emp","_id":"1","_version":2,"found":true,"_source":{"name" : "Jim"}}

curl -XGET http://node1:9200/test/emp/1?pretty

{
  "_index" : "test",
  "_type" : "emp",
  "_id" : "1",
  "_version" : 2,
  "found" : true,
  "_source" : {
    "name" : "Jim"
  }
}

pretty 格式化結果，更好看

 

查詢帶報文頭

curl -i http://node1:9200/test/emp/1?pretty

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 134
{
  "_index" : "test",
  "_type" : "emp",
  "_id" : "1",
  "_version" : 2,
  "found" : true,
  "_source" : {
    "name" : "Jim"
  }
}

 

查詢_source

curl -XGET http://node1:9200/test/emp/1/_source

{"name" : "Jim"}

 

查詢指定field的信息

curl -XGET http://node1:9200/test/emp/1?_source=name

 

查詢type信息

curl -XGET http://node1:9200/test/emp/_search?pretty

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "AVYw5zrHBTthl-73fXCK",
      "_score" : 1.0,
      "_source" : {
        "name" : "John"
      }
    }, {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "2",
      "_score" : 1.0,
      "_source" : {
        "name" : "John"
      }
    }, {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "1",
      "_score" : 1.0,
      "_source" : {
        "name" : "Jim"
      }
    } ]
  }
}

 

查詢指定條件數據

curl -XGET http://node1:9200/test/emp/_search?q=name:Jim

 

 

Domain Specific Language DSL 查詢

curl -XGET http://node1:9200/test/emp/_search?pretty -d  \
'{"query":
    {"match":
        {"name":"Jim"}
    }
}'

 

獲取多個文檔

curl -XGET http://node1:9200/_mget?pretty -d \
'{"docs":[{"_index":"test","_type":"emp","_id":1},{"_index":"test","_type":"emp","_id":2}]
}'

curl -XGET http://node1:9200/test/emp/_mget?pretty -d \
'{"ids":["1","2"]
}'

 

檢查文檔是否存在，只返回報文頭

curl -i -XHEAD http://node1:9200/test/emp/1?pretty

 

更新字段信息

curl -XPUT http://node1:9200/test/emp/1?pretty -d \
'{"name":"Tom",}'

 

局部更新(增長first_name字段) 只能用POST

curl -XPOST http://node1:9200/test/emp/1/_update -d '{"doc":{"first_name":"Jim"}}'

 

刪除文檔

curl -XDELETE http://node1:9200/test/emp/1

指定id

 

查詢設置 副本數量，分片數量

curl -XGET http://node1:9200/test/_settings?pretty

{
  "test" : {
    "settings" : {
      "index" : {
        "creation_date" : "1469698597831",
        "uuid" : "VFew5vmlQUC3Uieu62w23Q",
        "number_of_replicas" : "1",
        "number_of_shards" : "5",
        "version" : {
          "created" : "2020099"
        }
      }
    }
  }
}

 

設置分片及副本數

curl -XPUT http://node1:9200/test/_settings -d'{"settings":{"number_of_shards":3,"number_of_replicas":2}}'

 

查詢mapping信息

curl -XGET http://node1:9200/test/_mapping?pretty

用於修改字段相關屬性，如字段類型，是否使用分詞器，使用哪一種分詞器

curl -XPOST http://node1:9200/test/emp/_mapping -d \
'{
    "emp": {
        "properties": {
              "name" : {
                "type" : "string",
                "analyzer" : "standard"
              }
        }
    }
}'

 

 

解析器

curl -XGET 'http://node1:9200/_analyze?analyzer=standard&pretty=true' -d '很高興認識你'

使用解析器查看結果

具體說明參考官方文檔：https://www.elastic.co/guide/en/elasticsearch/reference/2.2/analyzer.html

 

Java API

官方文檔：

https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.2/index.html

 

測試demo：

注意：這裏使用term時注意使用小寫，因爲使用了分詞器

  1 import org.elasticsearch.action.get.GetResponse;
  2 import org.elasticsearch.action.index.IndexResponse;
  3 import org.elasticsearch.action.search.SearchResponse;
  4 import org.elasticsearch.action.search.SearchType;
  5 import org.elasticsearch.client.Client;
  6 import org.elasticsearch.client.transport.TransportClient;
  7 import org.elasticsearch.common.settings.Settings;
  8 import org.elasticsearch.common.transport.InetSocketTransportAddress;
  9 import org.elasticsearch.common.unit.TimeValue;
 10 import org.elasticsearch.common.xcontent.XContentFactory;
 11 import org.elasticsearch.index.query.QueryBuilder;
 12 import org.elasticsearch.index.query.QueryBuilders;
 13 import org.elasticsearch.search.SearchHit;
 14 import org.junit.After;
 15 import org.junit.Before;
 16 import org.junit.Test;
 17 
 18 import java.io.IOException;
 19 import java.net.InetAddress;
 20 import java.net.UnknownHostException;
 21 
 22 /**
 23  * Created by Edward on 2016/7/29.
 24  */
 25 public class MyTest {
 26 
 27     public static Client client;
 28 
 29     @Before
 30     public void init() throws UnknownHostException {
 31         Settings settings = Settings.settingsBuilder()
 32                 .put("cluster.name", "my-application").build();
 33         client = TransportClient.builder().settings(settings).build()
 34                 .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node1"), 9300))
 35                 .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node2"), 9300))
 36                 .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node3"), 9300));
 37     }
 38     @After
 39     public void closeClient()
 40     {
 41         client.close();
 42     }
 43 
 44     @Test
 45     public void createIndex() throws IOException {
 46         //建立索引
 47         IndexResponse response = client.prepareIndex("test", "emp", "1")
 48                 .setSource(XContentFactory.jsonBuilder()
 49                         .startObject()
 50                         .field("name", "Tom")
 51                         .field("age", "21")
 52                         .field("message", "Elasticsearch")
 53                         .endObject())
 54                 .get();
 55         System.out.println(response);
 56     }
 57 
 58     @Test
 59     public void queryIndex(){
 60         //獲取索引信息
 61         GetResponse res = client.prepareGet("test", "emp", "1").get();
 62         System.out.println(res.getSource());
 63     }
 64 
 65     @Test
 66     public void search(){
 67 
 68         SearchResponse response = client.prepareSearch("test")
 69                 .setTypes("emp")
 70                 .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
 71                 //注意term使用了分詞，這裏tom使用小寫
 72                 .setQuery(QueryBuilders.termQuery("name", "tom"))                 // Query
 73                 //.setPostFilter(QueryBuilders.rangeQuery("age").from(12).to(18))     // Filter
 74                 //.setFrom(0).setSize(60).setExplain(true)
 75                 .execute()
 76                 .actionGet();
 77         System.out.println(response.getHits().getHits().length);
 78         //SearchResponse response = client.prepareSearch().execute().actionGet();
 79         System.out.println(response.toString());
 80     }
 81 
 82 
 83     @Test
 84     public void scrollQuery(){
 85 
 86         //注意因爲使用了分詞，這裏tom使用小寫
 87         QueryBuilder qb = QueryBuilders.termQuery("name", "tom");
 88 
 89         SearchResponse scrollResp = client.prepareSearch("test")
 90                 .setTypes("emp")
 91                 .setSearchType(SearchType.SCAN)
 92                 .setScroll(new TimeValue(60000))
 93                 .setQuery(qb)
 94                 .setSize(100).execute().actionGet(); //100 hits per shard will be returned for each scroll
 95 //Scroll until no hits are returned
 96         while (true) {
 97             scrollResp = client.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet();
 98             for (SearchHit hit : scrollResp.getHits().getHits()) {
 99                 //Handle the hit...
100                 System.out.println(hit.getSource());
101             }
102 
103             //Break condition: No hits are returned
104             if (scrollResp.getHits().getHits().length == 0) {
105                 break;
106             }
107         }
108     }
109 }