Elasticsearch 7.x 去重查詢並返回去重後的總數

Elasticsearch version: 7.8

需求是分頁去重獲取索引中的數據, 相似 MySQL 的 distinct. Elasticsearch 中的 collapse 能夠實現該需求.

collapse 官網文檔

You can use the collapse parameter to collapse search results based on field values. The
collapsing is done by selecting only the top sorted document per collapse key.

你可使用 collapse 參數根據字段值摺疊搜索結果, 摺疊是經過每一個摺疊鍵僅選擇排序最靠前的文檔來完成的.

注意:

The total number of hits in the response indicates the number of matching documents without collapsing. The total number of distinct group is unknown.

響應中的總數表示沒有摺疊的匹配文檔數, 去重後的總數是不知道的.

那麼怎麼獲取去重後的總數呢? 可使用 Aggregation 中的 cardinality 來實現.

cardinality 官方文檔

DSL example:

{
  "from": 0,
  "size": 5,
  "sort": [
    {
      "createTime": {
        "order": "desc"
      }
    }
  ],
  "collapse": {
    "field": "app_id"
  },
  "aggs": {
    "total_size": {
      "cardinality": {
        "field": "app_id"
      }
    }
  }
}

Java API example:

SortBuilder sortBuilder = SortBuilders.fieldSort(CREATE_TIME).order(SortOrder.DESC);
CollapseBuilder collapseBuilder = new CollapseBuilder(APP_ID);
AggregationBuilder aggregation = AggregationBuilders.cardinality(TOTAL_COUNT_KEY).field(APP_ID);