關於elasticsearch的先聚合和過濾、先過濾再聚合的詳解

時間  2019-11-12
標籤 關於 elasticsearch 聚合 過濾 詳解 欄目 日誌分析 简体版
原文   原文鏈接

對於elasticsearch的聚合和過濾，他的結果並不會受到你寫的順序而影響。換句話說就是你不管是在聚合語句的前面寫過濾條件，仍是在過濾語句後面寫過濾條件都不會影響他的結果。他都會先過濾再聚合和關係數據庫同樣先where後group by。 可是若是你想過濾條件不影響聚合(agg)結果，而只是改變hits結果；能夠使用setPostFilter() 這個方法node

eg：所有數據 代碼：數據庫

SearchResponse response = null;  
SearchRequestBuilder responsebuilder = client.prepareSearch("company")  
               .setTypes("employee").setFrom(0).setSize(250);  
AggregationBuilder aggregation = AggregationBuilders  
               .terms("agg")  
               .field("age")  ;  
response = responsebuilder
                      .addAggregation(aggregation)
                      .setExplain(true).execute().actionGet();
SearchHits hits = response.getHits(); 
Terms agg = response.getAggregations().get("agg");

結果： 僅聚合結果不過濾（注意看hits和agg裏的結果）elasticsearch

{
    "took":100,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":7,
        "max_score":1,
        "hits":[
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"5",
                "_score":1,
                "_source":{
                    "name":"Fresh",
                    "age":22
                },
                "_explanation":Object{...}
            },
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"10",
                "_score":1,
                "_source":{
                    "name":"Henrry",
                    "age":30
                },
                "_explanation":Object{...}
            },
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"9",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"jiangsu",
                        "city":"nanjing",
                        "area":{
                            "pos":"10001"
                        }
                    }
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"2",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"jiangsu",
                        "city":"nanjing"
                    },
                    "name":"jack_1",
                    "age":19,
                    "join_date":"2016-01-01"
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"4",
                "_score":1,
                "_source":{
                    "name":"willam",
                    "age":18
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"6",
                "_score":1,
                "_source":{
                    "name":"Avivi",
                    "age":30
                },
                "_explanation":Object{...}
            },
            {
                "_shard":4,
                "_node":"K7qK1ncMQUuIe0K6VSVMJA",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

一、setQuery() 寫在前面 代碼：ui

SearchResponse response = null;  
 SearchRequestBuilder responsebuilder = client.prepareSearch("company")  
                .setTypes("employee").setFrom(0).setSize(250);  
 AggregationBuilder aggregation = AggregationBuilders  
                .terms("agg")  
                .field("age")  ;  
response = responsebuilder
        	       .setQuery(QueryBuilders.rangeQuery("age").gt(30).lt(40))
                       .addAggregation(aggregation)
                       .setExplain(true).execute().actionGet();
 SearchHits hits = response.getHits(); 
 Terms agg = response.getAggregations().get("agg");

結果：code

{
    "took":538,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"anlkGjjuQ0G6DODpZgiWrQ",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

二、setQuery() 寫在後面 代碼：ci

SearchResponse response = null;  
SearchRequestBuilder responsebuilder = client.prepareSearch("company")  
               .setTypes("employee").setFrom(0).setSize(250);  
AggregationBuilder aggregation = AggregationBuilders  
               .terms("agg")  
               .field("age")  ;  
response = responsebuilder
                      .addAggregation(aggregation)
                      .setQuery(QueryBuilders.rangeQuery("age").gt(30).lt(40)
                      .setExplain(true).execute().actionGet();
SearchHits hits = response.getHits(); 
Terms agg = response.getAggregations().get("agg");

結果：get

"took":538,
  "timed_out":false,
  "_shards":{
      "total":5,
      "successful":5,
      "failed":0
  },
  "hits":{
      "total":1,
      "max_score":1,
      "hits":[
          {
              "_shard":4,
              "_node":"anlkGjjuQ0G6DODpZgiWrQ",
              "_index":"company",
              "_type":"employee",
              "_id":"3",
              "_score":1,
              "_source":{
                  "address":{
                      "country":"china",
                      "province":"shanxi",
                      "city":"xian"
                  },
                  "name":"marry",
                  "age":35,
                  "join_date":"2015-01-01"
              },
              "_explanation":Object{...}
          }
      ]
  },
  "aggregations":{
      "agg":{
          "doc_count_error_upper_bound":0,
          "sum_other_doc_count":0,
          "buckets":[
              {
                  "key":35,
                  "doc_count":1
              }
          ]
      }
  }
}

三、setPostFilter() 在聚合.aggAggregation()方法後 代碼：it

SearchResponse response = null;  
    SearchRequestBuilder responsebuilder = client.prepareSearch("company")  
               .setTypes("employee").setFrom(0).setSize(250);  
   AggregationBuilder aggregation = AggregationBuilders  
               .terms("agg")  
               .field("age")  ;  
  response = responsebuilder
                      .addAggregation(aggregation)
                      .setPostFilter(QueryBuilders.rangeQuery("age").gt(30).lt(40))  
                      .setExplain(true).execute().actionGet();
  SearchHits hits = response.getHits(); 
  Terms agg = response.getAggregations().get("agg");
結果：
{
    "took":7,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

四、setPostFilter() 在聚合.aggAggregation()方法前 代碼：io

SearchResponse response = null;  
   SearchRequestBuilder responsebuilder = client.prepareSearch("company")  
              .setTypes("employee").setFrom(0).setSize(250);  
  AggregationBuilder aggregation = AggregationBuilders  
              .terms("agg")  
              .field("age")  ;  
 response = responsebuilder
                     .setPostFilter(QueryBuilders.rangeQuery("age").gt(30).lt(40))  
                     .addAggregation(aggregation)
                     .setExplain(true).execute().actionGet();
 SearchHits hits = response.getHits(); 
 Terms agg = response.getAggregations().get("agg");

結果：ast

{
    "took":5115,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"b8cNIO5cQr2MmsnsuluoNQ",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

總結： 能夠從運行的結果很好的看出不管是setPostFilter()仍是setQuery()，它放在那的順序並不會影響他的結果。更能夠看出setQuery()這個方法的過濾條件不只會影響它的hits的結果還會影響他的聚合（agg）結果。然而對於setPostFilter()這個方法，它只會影響hits的結果，並不會影響它的聚合（agg）結果。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程