基於百度地圖SDK和Elasticsearch GEO查詢的地理圍欄分析系統（2）-查詢實現

 

在上一篇博客中，咱們準備好了數據。如今數據已經以咱們須要的格式，存放在Elasticsearch中了。

本文講述如何在Elasticsearch中進行空間GEO查詢和聚合查詢，以及如何準備ajax接口。

平臺的服務端部分使用的springboot+mybatis的基本開發模式。工程結構以下。

能夠看到本工程有三個module：

1）moonlight-web是controller和service層的實現；

2）moonlight-dsl封裝了ES空間索引查詢和聚合查詢的方法；

3）moonlight-dao封裝了持久化地理圍欄的方法。

咱們以客戶端請求的處理順序爲例進行講解。

 

一、controller

在controller層中，咱們實現了4個接口，分別是circle、box、polygon、heatmap，也就是圓形圈選，矩形圈選，多邊形圈選和熱力圖。

先看一下代碼的具體實現。

@RestController
@RequestMapping("/moonlight")
public class MoonlightController {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private MoonlightService moonlightService;

    @RequestMapping(value = "/circle", method = RequestMethod.GET)
    public ResponseEntity<Response> circle(HttpServletRequest request, HttpServletResponse response) {
        String point = request.getParameter("point");
        String radius = request.getParameter("radius");
        try {
            Map<String, Object> result = moonlightService.circle(point, radius);
            logger.info("circle圈選成功, points={}, radius={}, result={}", point, radius, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "circle圈選成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("circle圈選失敗, points={}, radius={}, result={}", point, radius, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "circle圈選失敗", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/box", method = RequestMethod.GET)
    public ResponseEntity<Response> box(HttpServletRequest request, HttpServletResponse response) {
        String point1 = request.getParameter("point1");
        String point2 = request.getParameter("point2");
        String point3 = request.getParameter("point3");
        String point4 = request.getParameter("point4");
        try {
            Map<String, Object> result = moonlightService.boundingBox(point1, point2, point3, point4);
            logger.info("box圈選成功, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "box圈選成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("box圈選失敗, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "box圈選失敗", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/polygon", method = RequestMethod.GET)
    public ResponseEntity<Response> polygon(HttpServletRequest request, HttpServletResponse response) {
        List<String> points = new ArrayList<>();
        Enumeration<String> paramNames = request.getParameterNames();
        while (paramNames.hasMoreElements()) {
            String paramName = paramNames.nextElement();
            if (paramName.startsWith("point")) {
                points.add(request.getParameter(paramName));
            }
        }
        try {
            Map<String, Object> result = moonlightService.polygon(points);
            logger.info("polygon圈選成功, points={}, result={}", points, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "polygon圈選成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("polygon圈選失敗, points={}, result={}", points, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "polygon圈選失敗", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/heatMap", method = RequestMethod.GET)
    public ResponseEntity<Response> heatMap(HttpServletRequest request, HttpServletResponse response) {
        try {
            List<Map<String, Object>> result = moonlightService.heatMap();
            logger.info("heatMap請求成功, result={}", result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "heatMap請求成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("heatMap請求失敗, result={}", null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "heatMap請求失敗", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
}

咱們以圓形圈選（circle接口）爲例，circle接口傳入兩個參數，一個是point，也就是中心點座標，一個是radius，也就是半徑，它乾的事情就是圈選出，point點周圍radius長度內的全部訂單數據，具體實現是調用了service層的方法，controller獲得圈選的數據後就返回了。

下面咱們來看一下service層。

 

二、service

service層是具體業務的實現。咱們這裏的service仍然比較簡單，能夠看到只是初始化了esDao的句柄，而後進行es的geo查詢。

先看一下具體代碼。

@Service
public class MoonlightService {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESDao esDao;

    public Map<String, Object> circle(String point, String radius) {
        POI center = new POI(point);
        return esDao.circle(center, Double.parseDouble(radius));
    }

    public Map<String, Object> boundingBox(String point1, String point2, String point3, String point4) {
        POI poi1 = new POI(point1);
        POI poi2 = new POI(point2);
        POI poi3 = new POI(point3);
        POI poi4 = new POI(point4);
        POI topLeft = getTopLeft(poi1, poi2, poi3, poi4);
        POI bottomRight = getBottomRight(poi1, poi2, poi3, poi4);
        logger.info("topLeft - lat={}, lng={}, bottomRight - lat={}, lng={}",
                topLeft.getLat(), topLeft.getLng(), bottomRight.getLat(), bottomRight.getLng());
        return esDao.boundingBox(topLeft, bottomRight);
    }

    public Map<String, Object> polygon(List<String> points) {
        List<POI> poiList = new ArrayList<>();
        for (String point : points) {
            POI poi = new POI(point);
            poiList.add(poi);
        }
        return esDao.polygon(poiList);
    }

    public List<Map<String, Object>> heatMap() {
        return esDao.heatMap();
    }

    private POI getTopLeft(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI topLeft = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        topLeft.setLat(maxLat);
        topLeft.setLng(minLng);
        return topLeft;
    }

    private POI getBottomRight(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI bottomRight = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        bottomRight.setLat(minLat);
        bottomRight.setLng(maxLng);
        return bottomRight;
    }
}

咱們仍然是以圓形圈選爲例，能夠看到，service代碼的邏輯就是，建立出圈選須要的數據接口，而後調用Dao層進行查詢就是了。

circle圈選須要的是一箇中心點POI類型，和一個Double半徑。

box矩形查詢須要的是左上座標點和右下座標點，裏面有兩個函數getTopLeft、getBottomRight分別能夠求出矩形的左上點和右下點。

polygon多邊形查詢須要的是一系列點，這些點順序的鏈接所繪製出來的圖形就是目標多邊形。

heatmap熱力圖什麼參數也不要，將返回必定精度的經緯度計數值，後面咱們會詳述。

以後全部的service都調用了Dao層的es查詢邏輯。因此最重要的一部分是esDao的實現，下面咱們就來看一看。

 

三、Dao

Dao層代碼是整個項目的核心，包括對Elasticsearch數據進行圈選和聚合兩部分，此外就是熱力圖數據的準備。

先看一下代碼。

@Component
public class ESDao {
    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESClient esClient;

    public Map<String, Object> circle(POI center, Double radius) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders.geoDistanceRangeQuery("location")
                .point(center.getLat(),  center.getLng())
                .from("0m")
                .to(String.format("%fm", radius))
                .includeLower(true)
                .includeUpper(true)
                .optimizeBbox("memory")
                .geoDistance(GeoDistance.SLOPPY_ARC);

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> boundingBox(POI topLeft,  POI bottomRight) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoBoundingBoxQueryBuilder geoBoundingBoxQuery = QueryBuilders.geoBoundingBoxQuery("location")
                .topLeft(topLeft.getLat(), topLeft.getLng())
                .bottomRight(bottomRight.getLat(), bottomRight.getLng());

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoBoundingBoxQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> polygon(List<POI> poiList) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoPolygonQueryBuilder geoPolygonQuery = QueryBuilders.geoPolygonQuery("location");
        for (POI poi : poiList) {
            geoPolygonQuery.addPoint(poi.getLat(), poi.getLng());
        }

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoPolygonQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public List<Map<String, Object>> heatMap() {

        TermQueryBuilder queryBuilder = termQuery("date", "2017-11-24");
        SearchRequestBuilder searchRequestBuilder = esClient.getClient()
                .prepareSearch("moon").setTypes("bj");
        SearchResponse response = searchRequestBuilder
                .setQuery(queryBuilder)
                .setFrom(0).setSize(10000)
                .setExplain(true).execute().actionGet();

        SearchHits hits = response.getHits();
        Map<String, Integer> countMap = new HashMap<>();
        for (SearchHit hit : hits) {
            Map<String, Object> source = hit.getSource();
            Map<String, Double> locationMap = (Map<String, Double>) source.get("location");
            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;
            if (countMap.containsKey(key)) {
                countMap.put(key, countMap.get(key) + 1);
            } else {
                countMap.put(key, 1);
            }
        }
        List<Map<String, Object>> result = new ArrayList<>();
        for (Map.Entry<String, Integer> entry : countMap.entrySet()) {
            String lat = entry.getKey().split("-")[0];
            String lon = entry.getKey().split("-")[1];
            Integer count = entry.getValue();
            Map<String, Object> map = new HashMap<>();
            map.put("lat", Double.parseDouble(lat));
            map.put("lng", Double.parseDouble(lon));
            map.put("count", count);
            result.add(map);
        }
        return result;
    }

    private Map<String, Object> agg(SearchRequestBuilder search) {

        Map<String, Object> resultMap = new HashMap<>();

        GroupBy groupBy = new GroupBy(search, "date_group", "date", true);
        groupBy.addSumAgg("pre_total_fee_sum", "pre_total_fee");
        groupBy.addCountAgg("order_id_count", "order_id");
        groupBy.addSumAgg("cancel_count", "type");

        List<String> xAxis = new ArrayList<>();
        List<String> profits = new ArrayList<>();
        List<String> totals = new ArrayList<>();
        List<String> cancelRatios = new ArrayList<>();
        List<Map<String, Object>> details = new ArrayList<>();

        Map<String, Object> groupbyResponse = groupBy.getGroupbyResponse();
        for (Map.Entry<String, Object> entry : groupbyResponse.entrySet()) {
            String date = entry.getKey();
            xAxis.add(date);
            Map<String, String> subAggMap = (Map<String, String>) entry.getValue();
            String profit = subAggMap.get("pre_total_fee_sum");
            profits.add(profit);
            String total = subAggMap.get("order_id_count");
            totals.add(total);
            String cancelRatioDouble = new DecimalFormat("#.0000").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            String cancelRatio = new DecimalFormat("0.00%").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            cancelRatios.add(cancelRatioDouble);

            Map<String, Object> tempMap = new HashMap<>();
            tempMap.put("profit", profit);
            tempMap.put("total", total);
            tempMap.put("cancelRatio", cancelRatio);
            tempMap.put("date", date);
            details.add(tempMap);
        }

        resultMap.put("xAxis", xAxis);
        resultMap.put("profit", profits);
        resultMap.put("total", totals);
        resultMap.put("cancelRatio", cancelRatios);
        resultMap.put("detail", details);

        return resultMap;
    }

}

es圈選部分

circle爲例，咱們構造了一個geoDistanceRangeQuery查詢，這個查詢到上一篇博客準備好的moon索引，bj type中去將數據圈選出來。

相似的咱們有矩形geoBoundingBoxQuery查詢，多邊形geoPolygonQuery查詢，具體構造查詢的方式能夠參照代碼，這個代碼仍是很簡單的，熟悉es的同窗很快能夠上手而且實現這樣的查詢，不熟悉的話能夠自行百度一下。若是還有其餘的查詢條件，能夠經過QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery)加入，例如我這裏在圈選以外加入了一個terms查詢，這個查詢至關於sql中的where product_id in (3,4) and ...。

 

es聚合部分

es聚合部分作的事情是，對查詢出的訂單進行了聚合運算，例如求和和計數，是兩個最多見的運算，這部分在這裏不詳細敘述了，請參見這篇博客。

 

熱力圖

這裏要額外說明的是，熱力圖heatmap，和圈選不同，他是查詢了最近一天type=bj分區裏的全部數據，按照座標進行了計數，能夠看到的是，計數的時候，咱們指定了精度，這裏是小數點後三位有效數字

            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;

而後將計數結果返回。百度地圖SDK會將計數結果繪製成熱力圖，這個不用咱們管，我會在另外一篇博客中講述這個過程。

 

到這裏，整個工程的基本功能就介紹完了。