elasticsearch的restful API和Java API

時間 2019-11-09

標籤 elasticsearch restful api java 欄目日誌分析简体版

原文原文鏈接

本人如今使用的是elasticsearch 5.2.1的,服務器IP爲192.168.5.182.因此在Java API和jar包中會有所不一樣.json

經常使用的restful API以下:服務器

http://192.168.5.182:9200/_cat/health?v 健康檢查
http://192.168.5.182:9200/_cat/indices?v 查看索引
PUT http://192.168.5.182:9200/test_index?pretty 添加索引
DELETE http://192.168.5.182:9200/test_index 刪除索引
PUT http://192.168.5.182:9200/ecommerce/product/1
BODY {
   "name":"zhonghua yagao",
   "desc":"caoben zhiwu",
   "price":40,
   "producer":"zhonghua producer",
   "tags":["qingxin"]
} 爲索引添加數據,ecommerce索引,product type,1 ID
GET http://192.168.5.182:9200/ecommerce/product/1 查詢數據
PUT http://192.168.5.182:9200/ecommerce/product/1
BODY {
   "name":"jiaqiangban zhonghua yagao",
   "desc":"caoben zhiwu",
   "price":40,
   "producer":"zhonghua producer",
   "tags":["qingxin"]
} 更新索引數據,方式一,必須帶上全部數據
POST http://192.168.5.182:9200/ecommerce/product/1/_update
BODY {
   "doc": {
       "name":"gaolujie yagao"
   }
} 更新索引數據,方式二
DELETE http://192.168.5.182:9200/ecommerce/product/1 刪除索引數據
GET http://192.168.5.182:9200/ecommerce/product/_search 搜索全部
GET http://192.168.5.182:9200/ecommerce/product/_search?q=name:yagao&sort=price:desc <query string search>
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "query":{
> "match_all":{}
> }
> }' <query DSL查詢>
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "query":{
> "match":{
> "name":"yagao"
> }
> },
> "sort":[
> {"price":"desc"}
> ]
> }' 排序查詢
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "query":{
> "match_all":{}
> },
> "from":1,
> "size":1
> }' 分頁查詢
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
{
"query":{
"match_all":{}
},
"_source":["name","price"]
}' 只查詢指定的字段
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
{
> "query":{
> "bool":{
> "must":{
> "match":{
> "name":"yagao"
> }
> },
> "filter":{
> "range":{
> "price":{
> "gt":25
> }
> }
> }
> }
> }
> }' 查詢yagao的price範圍,大於25 <query filter>
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "query":{
> "match":{
> "producer":"yagao producer"
> }
> }
> }' 全文檢索<full-text search>
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
{
"query":{
"match_phrase":{
"producer":"yagao producer"
}
}
}' 短語搜索<phrase search>
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "query":{
> "match":{
> "producer":"producer"
> }
> },
> "highlight":{
> "fields":{
> "producer":{}
> }
> }
> }' 高亮顯示<highlight search>
PUT http://192.168.5.182:9200/ecommerce/_mapping/product
BODY {
   "properties":{
       "tags":{
           "type":"text",
           "fielddata":true
       }
   }
} 將文本field的fielddata屬性設置爲true
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "aggs":{
> "group_by_tags":{
> "terms":{
> "field":"tags"
> }
> }
> }
> }' 對tags聚合,會顯示明細
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
{ "size":0,
"aggs":{
"group_by_tags":{
"terms":{
"field":"tags"
}
}
}
}' 對tags聚合,不顯示明細,只顯示聚合
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "size":0,
> "query":{
> "match":{
> "name":"yagao"
> }
> },
> "aggs":{
> "group_by_tags":{
> "terms":{
> "field":"tags"
> }
> }
> }
> }' 搜索包含條件的聚合
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "size":0,
> "aggs":{
> "group_by_tags":{
> "terms":{
> "field":"tags"
> },
> "aggs":{
> "avg_price":{
> "avg":{
> "field":"price"
> }
> }
> }
> }
> }
> }' 聚合計算平均值
curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
> {
> "size":0,
> "aggs":{
> "group_by_tags":{
> "terms":{
> "field":"tags",
> "order":{
> "avg_price":"desc"
> }
> },
> "aggs":{
> "avg_price":{
> "avg":{
> "field":"price"
> }
> }
> }
> }
> }
> }' 聚合後降序排序restful

curl -XGET 'http://192.168.5.182:9200/ecommerce/product/_search' -d'
{
"size":0,
"aggs":{
"group_by_price":{
"range":{
"field":"price",
"ranges":[
{
"from":0,
"to":20
},
{
"from":20,
"to":40
},
{
"from":40,
"to":60
}
]
},
"aggs":{
"group_by_tags":{
"terms":{
"field":"tags"
},
"aggs":{
"average_price":{
"avg":{
"field":"price"
}
}
}
}
}
}
}
}' 按照價格區間分組後再聚合tags平均價格
PUT http://192.168.5.182:9200/company
BODY {
"mappings": {
"employee": {
"properties": {
"age": {
"type": "long"
},
"country": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
},
"fielddata":true
},
"join_date": {
"type": "date"
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"position": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"salary": {
"type": "long"
}
}
}
}
} 給country創建正排索引app

在Java API中,咱們須要先找到相應的jar包,maven中的配置以下(開始以前請先執行上面的給country創建正排索引的restful API)curl

<dependency>
   <groupId>org.elasticsearch.client</groupId>
   <artifactId>transport</artifactId>
   <version>5.2.1</version>
</dependency>

5.2.1中只須要配這一個就能夠了,固然不一樣的版本配置的都不一樣,高版本的須要配elasticsearch

<dependency>
   <groupId>org.elasticsearch</groupId>
   <artifactId>elasticsearch</artifactId>
</dependency>

咱們依然在resources文件中作以下配置(注意restful API中使用的是9200端口,而Java API使用的是9300端口)maven

elasticsearch:
  clusterName: aubin-cluster
  clusterNodes: 192.168.5.182:9300

配置類以下ui

@Getter
@Setter
@Configuration
@ConfigurationProperties(prefix = "elasticsearch")
public class ElasticSearchConfig {

   private String clusterName;

   private String clusterNodes;

    /**
     * 使用elasticsearch實現類時才觸發
     *
     * @return
     */
   @Bean
   public TransportClient transportClient() {
      // 設置集羣名字
      Settings settings = Settings.builder().put("cluster.name", this.clusterName).build();
      TransportClient client = new PreBuiltTransportClient(settings);
      try {
         // 讀取的ip列表是以逗號分隔的
         for (String clusterNode : this.clusterNodes.split(",")) {
            String ip = clusterNode.split(":")[0];
            String port = clusterNode.split(":")[1];
            client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(ip), Integer.parseInt(port)));
         }
      } catch (UnknownHostException e) {
         e.printStackTrace();
      }

      return client;
   }
}

在5.2.1中使用的是InetSocketTransportAddress,這是一個具體的類,而在高版本中此處爲TransportAddress,這是一個接口.this

咱們作一個數據類url

@Component
public class DataEs {
    @Autowired
    private TransportClient transportClient;

    /**
     * 添加原始數據
     * @throws IOException
     */
    @PostConstruct
    private void init() throws IOException {
        transportClient.prepareIndex("company","employee","1").setSource(XContentFactory.jsonBuilder().startObject()
                .field("name","jack")
                .field("age",27)
                .field("position","technique software")
                .field("country","China")
                .field("join_date","2018-01-01")
                .field("salary",10000)
                .endObject()).get();
        transportClient.prepareIndex("company","employee","2").setSource(XContentFactory.jsonBuilder().startObject()
                .field("name","marry")
                .field("age",35)
                .field("position","technique manager")
                .field("country","China")
                .field("join_date","2018-01-01")
                .field("salary",12000)
                .endObject()).get();
        transportClient.prepareIndex("company","employee","3").setSource(XContentFactory.jsonBuilder().startObject()
                .field("name","tom")
                .field("age",32)
                .field("position","senior technique software")
                .field("country","China")
                .field("join_date","2017-01-01")
                .field("salary",11000)
                .endObject()).get();
        transportClient.prepareIndex("company","employee","4").setSource(XContentFactory.jsonBuilder().startObject()
                .field("name","jen")
                .field("age",25)
                .field("position","junior finance")
                .field("country","USA")
                .field("join_date","2017-01-01")
                .field("salary",7000)
                .endObject()).get();
        transportClient.prepareIndex("company","employee","5").setSource(XContentFactory.jsonBuilder().startObject()
                .field("name","mike")
                .field("age",37)
                .field("position","finance manager")
                .field("country","USA")
                .field("join_date","2016-01-01")
                .field("salary",15000)
                .endObject()).get();
    }

    /**
     * 員工搜索應用程序
     * 搜索職位中包含technique的員工
     * 同時要求age在30到40歲之間
     * 分頁查詢,查找第一頁
     */
    public void executeSearch() {
        SearchResponse searchResponse = transportClient.prepareSearch("company")
                .setTypes("employee")
                .setQuery(QueryBuilders.matchQuery("position", "technique"))
                .setPostFilter(QueryBuilders.rangeQuery("age").from(30).to(40))
                .setFrom(0).setSize(1)
                .get();
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (int i = 0;i < hits.length;i++) {
            System.out.println(hits[i].getSourceAsString());
        }
    }

    /**
     * 員工聚合分析應用程序
     * 首先按照country國家來進行分組
     * 而後在每一個country分組內,再按照入職年限進行分組
     * 最後計算每一個分組內的平均薪資
     */
    public void executeAggregation() {
        SearchResponse searchResponse = transportClient.prepareSearch("company")
                .addAggregation(AggregationBuilders.terms("group_by_country").field("country")
                .subAggregation(AggregationBuilders.dateHistogram("group_by_join_date")
                .field("join_date").dateHistogramInterval(DateHistogramInterval.YEAR)
                .subAggregation(AggregationBuilders.avg("avg_salary").field("salary"))))
                .execute().actionGet();
        Map<String,Aggregation> aggrMap = searchResponse.getAggregations().asMap();
        StringTerms groupByCountry = (StringTerms) aggrMap.get("group_by_country");
        Iterator<StringTerms.Bucket> groupByCountryBucketIterator = groupByCountry.getBuckets().iterator();
        while (groupByCountryBucketIterator.hasNext()) {
            StringTerms.Bucket groupByCountryBucket = groupByCountryBucketIterator.next();
            System.out.println(groupByCountryBucket.getKey() + ":" + groupByCountryBucket.getDocCount());
            Histogram groupByJoinDate = (Histogram) groupByCountryBucket.getAggregations().asMap().get("group_by_join_date");
            Iterator<? extends Histogram.Bucket> groupByJoinDateIterator = groupByJoinDate.getBuckets().iterator();
            while (groupByJoinDateIterator.hasNext()) {
                Histogram.Bucket groupByJoinDateBucket = groupByJoinDateIterator.next();
                System.out.println(groupByJoinDateBucket.getKey() + ":" + groupByJoinDateBucket.getDocCount());
                Avg avg = (Avg) groupByJoinDateBucket.getAggregations().asMap().get("avg_salary");
                System.out.println(avg.getValue());
            }
        }
    }
    public void close() {
        transportClient.close();
    }
}

在主程序中調用以下(通常咱們能夠先不執行搜索操做,先注入數據,由於elasticsearch自己有一個秒級寫讀的問題,若是數據寫入,得須要1秒的時間才能讀取出來)

@SpringBootApplication
public class EsApplication {
   public static void main(String[] args) {
      ApplicationContext applicationContext = SpringApplication.run(EsApplication.class, args);
      DataEs dataEs = (DataEs) applicationContext.getBean(DataEs.class);
      dataEs.executeSearch();
      dataEs.executeAggregation();
      dataEs.close();
   }
}