ElasticSearch 學習筆記 - 11.桶聚合

時間  2019-12-10
標籤 elasticsearch 學習 筆記 聚合 欄目 日誌分析 简体版
原文   原文鏈接

一、源數據

DELETE my-index

PUT my-index

PUT my-index/person/1
{
  "name":"張三",
  "age":27,
  "gender":"男",
  "salary":15000,
  "dep":"bigdata"
}

PUT my-index/person/2
{
  "name":"李四",
  "age":26,
  "gender":"女",
  "salary":15000,
  "dep":"bigdata"
}

PUT my-index/person/3
{
  "name":"王五",
  "age":26,
  "gender":"男",
  "salary":17000,
  "dep":"AI"
}
PUT my-index/person/4
{
  "name":"劉六",
  "age":27,
  "gender":"女",
  "salary":18000,
  "dep":"AI"
}

PUT my-index/person/5
{
  "name":"程裕強",
  "age":31,
  "gender":"男",
  "salary":20000,
  "dep":"bigdata"
}
PUT my-index/person/6
{
  "name":"hadron",
  "age":30,
  "gender":"男",
  "salary":20000,
  "dep":"AI"
}

二、Terms Aggregation

根據薪資水平進行分組，統計每一個薪資水平的人數

GET /my-index/person/_search
{
  "size": 0,
  "aggs": {
    "group_count": {
      "terms": {
        "field": "salary"
      }
    }
  }
}

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 6,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_count": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": 15000,
          "doc_count": 2
        },
        {
          "key": 20000,
          "doc_count": 2
        },
        {
          "key": 17000,
          "doc_count": 1
        },
        {
          "key": 18000,
          "doc_count": 1
        }
      ]
    }
  }
}

統計上面每一個分組的平均年齡

GET /my-index/person/_search
{
  "size": 0,
  "aggs": {
    "group_count": {
      "terms": {
        "field": "salary"
      }
      , "aggs": {
        "avg_age": {
          "avg": {
            "field": "age"
          }
        }
      }
    }
  }
}

{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 6,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_count": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": 15000,
          "doc_count": 2,
          "avg_age": {
            "value": 26.5
          }
        },
        {
          "key": 20000,
          "doc_count": 2,
          "avg_age": {
            "value": 30.5
          }
        },
        {
          "key": 17000,
          "doc_count": 1,
          "avg_age": {
            "value": 26
          }
        },
        {
          "key": 18000,
          "doc_count": 1,
          "avg_age": {
            "value": 27
          }
        }
      ]
    }
  }
}

統計每一個部門的人數

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "terms": {"field": "dep"}
    }
  }
}

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [dep] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "my-index",
        "node": "fQDwpdT2RfSfPr8ttHQCkA",
        "reason": {
          "type": "illegal_argument_exception",
          "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [dep] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
        }
      }
    ],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [dep] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [dep] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
      }
    }
  },
  "status": 400
}

根據錯誤提示」Fielddata is disabled on text fields by default.
Set fielddata=true on [dep] in order to load fielddata in memory by uninverting the inverted index.
Note that this can however use significant memory. Alternatively use a keyword field instead.」可知，須要開啓fielddata參數。只須要設置某個字段"fielddata": true便可。
此外，根據官方文檔提示se the my_field.keyword field for aggregations, sorting, or in scripts，能夠嘗試my_field.keyword格式用於聚合操做。node

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "terms": {"field": "dep.keyword"}
    }
  }
}

二、Filter Aggregation

計算男人的平均年齡

也就是統計gender字段包含關鍵字「男」的文檔的age平均值。web

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "filter": {
        "term":{"gender": "男"}
      },
      "aggs":{
        "avg_age":{
          "avg":{"field": "age"}
        }
      }
    }
  }
}

三、Filters Aggregation

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "filters":{
        "filters": [
          {"match":{"gender": "男"}},
          {"match":{"gender": "女"}}
        ]
      },
      "aggs":{
        "avg_age":{
            "avg":{"field": "age"}
        }
      }
    }
  }
}

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 6,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_count": {
      "buckets": [
        {
          "doc_count": 4,
          "avg_age": {
            "value": 28.5
          }
        },
        {
          "doc_count": 2,
          "avg_age": {
            "value": 26.5
          }
        }
      ]
    }
  }
}

四、Range Aggregation

from..to區間範圍是[from,to),也就是說包含from點，不包含to點
【例子】查詢薪資在[0,10000),[10000,20000),[2000,+無窮大)三個範圍的員工數post

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "range": {
        "field": "salary",
        "ranges": [
            {"to": 10000},
            {"from": 10000,"to":20000},  
            {"from": 20000}
        ]
      }
    }
  }
}

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 6,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_count": {
      "buckets": [
        {
          "key": "*-10000.0",
          "to": 10000,
          "doc_count": 0
        },
        {
          "key": "10000.0-20000.0",
          "from": 10000,
          "to": 20000,
          "doc_count": 4
        },
        {
          "key": "20000.0-*",
          "from": 20000,
          "doc_count": 2
        }
      ]
    }
  }
}

五、Date Range聚合

專用於日期值的範圍聚合。
這種聚合和正常範圍聚合的主要區別在於，起始和結束值能夠在日期數學表達式中表示，而且還能夠指定返回起始和結束響應字段的日期格式。
請注意，此聚合包含from值並排除每一個範圍的值。this

【例子】計算一年前以前發表的博文數和從一年前以來發表的博文總數code

GET website/_search
{
  "size": 0, 
  "aggs": {
    "group_count": {
      "range": {
        "field": "postdate",
        "format":"yyyy-MM-dd",
        "ranges": [
            {"to": "now-12M/M"},
            {"from": "now-12M/M"}
        ]
      }
    }
  }
}



{
  "took": 29,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 9,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_count": {
      "buckets": [
        {
          "key": "*-2017-10-01",
          "to": 1506816000000,
          "to_as_string": "2017-10-01",
          "doc_count": 8
        },
        {
          "key": "2017-10-01-*",
          "from": 1506816000000,
          "from_as_string": "2017-10-01",
          "doc_count": 1
        }
      ]
    }
  }
}

六、Missing聚合

基於字段數據的單桶集合，建立當前文檔集上下文中缺乏字段值（實際上缺乏字段或設置了配置的NULL值）的全部文檔的桶。
此聚合器一般會與其餘字段數據存儲桶聚合器（如範圍）一塊兒使用，以返回因爲缺乏字段數據值而沒法放置在其餘存儲桶中的全部文檔的信息。orm

GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "noDep_count": {
      "missing": {"field": "salary"}
    }
  }
}

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 9,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "noDep_count": {
      "doc_count": 3
    }
  }
}
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程