elasticsearch 常見查詢及聚合的JAVA API

時間  2019-12-07
標籤 elasticsearch 常見 查詢 聚合 java api 欄目 日誌分析 简体版
原文   原文鏈接

ES 常見查詢sql

（1）根據ID 進行單個查詢數組

GetResponse response = client.prepareGet("accounts", "person", "1").setOperationThreaded(false).get();

相對於sql 的 select * from accounts.person  where id=1 ;elasticsearch

 

（2）分頁查詢全部記錄ui

QueryBuilder qb=new MatchAllQueryBuilder();
        SearchResponse response= client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)
                .setSize(100).get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

response.getHits()是全部命中記錄 相較於sql select * from accounts.person limit 100;spa


（3）根據多條件組合與查詢code

QueryBuilder qb=QueryBuilders.boolQuery().must(QueryBuilders.termQuery("title","JAVA開發工程師")).must(QueryBuilders.termQuery("age",30)) ;

        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

must 就像sql裏的and   相較於sql  select * from accounts.person where title='JAVA開發工程師' and age=30orm

（4）多條件或查詢blog

 QueryBuilder qb=QueryBuilders.termQuery("user","kimchy14");
        QueryBuilder qb1=QueryBuilders.termQuery("user","kimchy15");

        SortBuilder sortBuilder=SortBuilders.fieldSort("age");
        sortBuilder.order(SortOrder.DESC);
        QueryBuilder s=QueryBuilders.boolQuery().should(qb).should(qb1);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).addSort(sortBuilder).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

should 就像sql裏的or  SortBuilder 的做用不言而喻就是用來排序 以上代碼相較於sql  select * from   accounts.person where user='kimchy14' or  user='kimchy15'   ;   排序

   

（5）範圍查詢 索引

// RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
       // RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gt(30 );
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gte(30 );
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

範圍查詢rangeQuery.from（30，true）方法是大於30  後面的參數是是否包含 爲true的話就是大於等於30 to就至關於小於 若是也有包含參數爲true的話就是小於等於  gt 是大於 gte是大於等於   lt是小於 lte是小於等於  第一句的builder就至關於 select * from accounts.person where age >=30 and age<=30;   

          

（6）包含查詢

List<String> strs=new ArrayList<>();
        strs.add("kimchy14");
        strs.add("kimchy15");
        strs.add("kimchy16");
        QueryBuilder qb=QueryBuilders.termsQuery("user",strs);

        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

包含查詢使用termsQuery 能夠傳列表 也能夠傳多個參數 或者數組 setFetchSource有兩個參數 第一個參數是包含哪些參數 第二個參數是排除哪些參數   以上這段代碼就至關於sql  select age from accounts.person where user in ('kimchy14','kimchy15','kimchy16');

 

（7）專門按id進行的包含查詢

QueryBuilder qb=QueryBuilders.idsQuery(0+"");

        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits(); 
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

 

（8）按通配符查詢

QueryBuilder qb = QueryBuilders.wildcardQuery("user", "k*hy17*");
         //Fuzziness fuzziness=Fuzziness.fromEdits(2);

      // QueryBuilder qb = QueryBuilders.fuzzyQuery("user","mchy2").fuzziness(fuzziness);
        //QueryBuilder qb = QueryBuilders.prefixQuery("user", "kimchy2");
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("user",null).setFrom(0)
                .setSize(100);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }

通配符查詢像咱們sql裏的like 可是還不同 like的百分號能夠加到先後    elasticsearch技術解析與實戰中有一句話 是這麼說的 爲了不極端緩慢的通配符查詢 通配符索引詞不該該以一個通配符開頭 通配符查詢應該避免以通配符開頭  

 

  常見統計  統計分爲指標 和 桶 桶就是咱們統計的樣本  指標就是咱們平時所查的count  sum  與sql不同的是 咱們還能夠將統計的樣本拿到 就是response.getHits

（9）統計count

AggregationBuilder  termsBuilder = AggregationBuilders.count("ageCount").field("age");

        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }
        ValueCount valueCount= response.getAggregations().get("ageCount");
        long value=valueCount.getValue();

這段代碼就至關於 sql select count(age) ageCount form accounts.person  where age >=30 and age<=30   

（10）查詢最大值

 AggregationBuilder  termsBuilder = AggregationBuilders.max("max").field("age");

        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }
        Max valueCount= response.getAggregations().get("max");
        double value=valueCount.getValue();

 

（11）統計總和

AggregationBuilder  termsBuilder = AggregationBuilders.sum("sum").field("age");

        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }
        Sum valueCount= response.getAggregations().get("sum");
        double value=valueCount.getValue();

 

（12）平均數

AggregationBuilder  termsBuilder = AggregationBuilders.avg("avg").field("age");

        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }
        Avg valueCount= response.getAggregations().get("avg");
        double value=valueCount.getValue();

 

（13）統計樣本基本指標

AggregationBuilder  termsBuilder = AggregationBuilders.stats("stats").field("age");

        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();
        SearchHits searchHits =  response.getHits();
        for(SearchHit hit:searchHits.getHits()){
            logger.log(Level.INFO , hit.getSourceAsString());
        }
        Stats valueCount= response.getAggregations().get("stats");
        logger.log(Level.INFO,"max"+valueCount.getMaxAsString());
        logger.log(Level.INFO,"avg"+valueCount.getAvgAsString());
        logger.log(Level.INFO,"sum"+valueCount.getSumAsString());
        logger.log(Level.INFO,"min"+valueCount.getMinAsString());
        logger.log(Level.INFO,"count"+valueCount.getCount());

 


分組統計 至關於group by 後拿各組指標進行統計 

 （14）分組求各組數據

 AggregationBuilder  termsBuilder = AggregationBuilders.terms("by_age").field("age");
        AggregationBuilder  sumBuilder=AggregationBuilders.sum("ageSum").field("age");
        AggregationBuilder  avgBuilder=AggregationBuilders.avg("ageAvg").field("age");
        AggregationBuilder  countBuilder=AggregationBuilders.count("ageCount").field("age");

        termsBuilder.subAggregation(sumBuilder).subAggregation(avgBuilder).subAggregation(countBuilder);
        //TermsAggregationBuilder all = AggregationBuilders.terms("age").field("age");
        //all.subAggregation(termsBuilder);
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(36,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFetchSource(null,"gender").setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();

        Aggregations terms= response.getAggregations();
        for (Aggregation a:terms){
            LongTerms teamSum= (LongTerms)a;
            for(LongTerms.Bucket bucket:teamSum.getBuckets()){
                logger.info(bucket.getKeyAsString()+"   "+bucket.getDocCount()+"    "+((Sum)bucket.getAggregations().asMap().get("ageSum")).getValue()+"    "+((Avg)bucket.getAggregations().asMap().get("ageAvg")).getValue()+"    "+((ValueCount)bucket.getAggregations().asMap().get("ageCount")).getValue());

            }
        }

第一行 termsBuilder 就至關於根據年齡對數據進行分組 group by   後面對sumBuilder avgBuilder countBuilder等就是在組內 求和 求平均數 求數量

 （15）多分組求各組數據

TermsAggregationBuilder all = AggregationBuilders.terms("by_gender").field("gender");
         AggregationBuilder age = AggregationBuilders.terms("by_age").field("age");
        AggregationBuilder  sumBuilder=AggregationBuilders.sum("ageSum").field("age");
        //AggregationBuilder  avgBuilder=AggregationBuilders.avg("ageAvg").field("age");
       // AggregationBuilder  countBuilder=AggregationBuilders.count("ageCount").field("age");
         all.subAggregation(age.subAggregation(sumBuilder));
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(32,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(rangeQueryBuilder).addAggregation(all);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response=  sv.get();

        Aggregations terms= response.getAggregations();
        for (Aggregation a:terms){
            StringTerms stringTerms= (StringTerms)a;
             for(StringTerms.Bucket bucket:stringTerms.getBuckets()){
              //  logger.info(bucket.getKeyAsString());
                Aggregation aggs=bucket.getAggregations().getAsMap().get("by_age");
                 LongTerms terms1= (LongTerms)aggs;
                for (LongTerms.Bucket bu:terms1.getBuckets()){
                     logger.info(bucket.getKeyAsString()+"  "+bu.getKeyAsString()+" "+bu.getDocCount()+"    "+((Sum)bu.getAggregations().asMap().get("ageSum")).getValue());
                }

            }
        }

每增長一個分組指標就須要多加一個termsBuilder  其餘等一切跟普通分組同樣 每次拿到

      

          以上就是我總結的基本的查詢 聚合 等常見功能 其餘等諸如 求各組前多少數據是用topHits 這些基本夠咱們平常操做了 。

          最後咱們總結下    精確查詢用term 組合查詢用bool 範圍用range    and查詢用must    or查詢用should  not查詢用must not  常見的接收聚合返回結果的類型 ValueCount   AVG  SUM  MAX  MIN  按照英文意義就能夠理解  分組聚合查詢時候還須要根據實際狀況看是返回那種terms 

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程