saiku執行過程代碼跟蹤

時間 2019-11-06

標籤 saiku 執行過程代碼跟蹤简体版

原文原文鏈接

使用了好久的saiku，決定跟蹤一下代碼，看看它的執行核心過程：前端

1、入口controller代碼

1.一、頁面打開以後，會發送一個ajax請求

Request URL:

http://l-tdata2.tkt.cn6.qunar.com:8080/saiku/rest/saiku/api/query/execute

Request Method:

POST

1.二、controller，java文件org.saiku.web.rest.resources.Query2Resource

若是有緩存，直接輸出數據

沒有緩存，計算在輸出數據

2、service代碼

2.一、service，執行核心代碼 org.saiku.service.olap.ThinQueryService的private CellDataSet execute(ThinQuery tq, ICellSetFormatter formatter)方法

執行mdx語句

            Long start = (new Date()).getTime();
            log.debug("Query Start");
            CellSet cellSet =  executeInternalQuery(tq); //這是執行mdx語句的地方，須要較長時間
            log.debug("Query End");
            String runId = "RUN#:" + ID_GENERATOR.get();
            Long exec = (new Date()).getTime();

3、核心代碼

執行mdx語句 org.saiku.service.olap.ThinQueryService的CellSet executeInternalQuery(ThinQuery query) throws Exception 方法

    CellSet executeInternalQuery(ThinQuery query) throws Exception {
        String runId = "RUN#:" + ID_GENERATOR.getAndIncrement();
        QueryContext queryContext = context.get(query.getName());

        if (queryContext == null) {
            queryContext = new QueryContext(Type.OLAP, query);
            this.context.put(query.getName(), queryContext);
        }

        // 根據數據立方體創建olap的jdbc連接
        OlapConnection con = olapDiscoverService.getNativeConnection(query.getCube().getConnection());
        if (StringUtils.isNotBlank(query.getCube().getCatalog())) {
            con.setCatalog(query.getCube().getCatalog());
        }

        if (queryContext.contains(ObjectKey.STATEMENT)) {
            Statement s = queryContext.getStatement();
            s.cancel();
            s.close();
            s = null;
            queryContext.remove(ObjectKey.STATEMENT);
        }

        OlapStatement stmt = con.createStatement(); // 實例化Statement對象
        queryContext.store(ObjectKey.STATEMENT, stmt);

        query = updateQuery(query);

        try {
            String mdx = query.getParameterResolvedMdx();
            log.info(runId + "\tType:" + query.getType() + ":\n" + mdx);

            CellSet cs = stmt.executeOlapQuery(mdx); //這裏是執行mdx語句的過程，耗時最久
            queryContext.store(ObjectKey.RESULT, cs);
            //追蹤代碼cs使用
            log.info("cs:" + cs.toString());
            if (query != null) {
                queryContext.store(ObjectKey.QUERY, query);
            }
            //追蹤代碼query使用
            log.info("query:" + query.toString());
            return cs;
        } finally {
            stmt.close();
            queryContext.remove(ObjectKey.STATEMENT);
        }
    }

4、執行日誌：

上面的註釋，是經過日誌來做證的，日誌以下：java

2016-06-12 14:46:21,571 DEBUG [org.saiku.web.rest.resources.Query2Resource] TRACK        /query/F7CE71C7-3E29-0A6A-9BC9-FDDA1A129BB7    POST     tq:false file:/homes/saiku_search.saiku
2016-06-12 14:46:21,686 DEBUG [org.saiku.service.olap.ThinQueryService] Query Start
2016-06-12 14:46:21,814 INFO  [org.saiku.service.olap.ThinQueryService] RUN#:1    Type:QUERYMODEL:
WITH
SET [~COLUMNS] AS
    {[category_name_id].[category_name_id].[category_name].Members}
SET [~ROWS_rpt_date_rpt_date] AS
    {[rpt_date].[rpt_date].[2016-06-07]}
SET [~ROWS_partner_partner] AS
    Hierarchize({{[partner].[partner].[All partners]}, {[partner].[partner].[name].Members}})
SET [~ROWS_from_area_id_from_area_id] AS
    Hierarchize({{[from_area_id].[from_area_id].[All from_area_ids]}, {[from_area_id].[from_area_id].[name].Members}})
SET [~ROWS_utmr_page_id_utmr_page_id] AS
    {[utmr_page_id].[utmr_page_id].[All utmr_page_ids]}
SET [~ROWS_in_track_in_track] AS
    {[in_track].[in_track].[All in_tracks]}
SET [~ROWS_dist_city_dist_city] AS
    {[dist_city].[dist_city].[All dist_citys]}
SET [~ROWS_current_city_current_city] AS
    {[current_city].[current_city].[All current_citys]}
SET [~ROWS_from_value_id_from_value_id] AS
    {[from_value_id].[from_value_id].[All from_value_ids]}
SET [~ROWS_page_id_page_id] AS
    {[page_id].[page_id].[All page_ids]}
SELECT
NON EMPTY CrossJoin([~COLUMNS], {[Measures].[num], [Measures].[gid]}) ON COLUMNS,
NON EMPTY NonEmptyCrossJoin([~ROWS_rpt_date_rpt_date], NonEmptyCrossJoin([~ROWS_partner_partner], NonEmptyCrossJoin([~ROWS_from_area_id_from_area_id], NonEmptyCrossJoin([~ROWS_utmr_page_id_utmr_page_id], NonEmptyCrossJoin([~ROWS_in_track_in_track], NonEmptyCrossJoin([~ROWS_dist_city_dist_city], NonEmptyCrossJoin([~ROWS_current_city_current_city], NonEmptyCrossJoin([~ROWS_from_value_id_from_value_id], [~ROWS_page_id_page_id])))))))) ON ROWS
FROM [saiku_search_detail_cube]
2016-06-12 14:50:58,344 INFO  [org.saiku.service.olap.ThinQueryService] cs:mondrian.olap4j.FactoryJdbc41Impl$MondrianOlap4jCellSetJdbc41@2c72fc4f
2016-06-12 14:50:58,344 INFO  [org.saiku.service.olap.ThinQueryService] query:org.saiku.olap.query2.ThinQuery@3112bd55
2016-06-12 14:50:58,344 DEBUG [org.saiku.service.olap.ThinQueryService] Query End
2016-06-12 14:50:58,442 DEBUG [org.saiku.service.olap.ThinQueryService] cellSet2Matrix End
2016-06-12 14:50:58,443 DEBUG [org.saiku.service.olap.ThinQueryService] calculateTotals End
2016-06-12 14:50:58,443 INFO  [org.saiku.service.olap.ThinQueryService] RUN#:2    Size: 23/76    Execute:    276658ms    Format:    98ms    Totals:    1ms     Total: 276757ms

上面是執行了一個超級數據的日誌，紅色部分標誌出了執行時間最久的部分，日誌是我從新編譯代碼得出的，可見執行的核心代碼就是第三部分標出的紅色部分代碼web

5、olap4j引擎

第四部分的代碼，核心是創建olap的jdbc連接。下面是原文：http://www.olap4j.org/ajax

olap4j is an open Java API for OLAP.
Think of it like JDBC, but for accessing multi-dimensional data.

olap4j is a common API for any OLAP server, so you can write an analytic application on one server and easily switch it to another. Built on that API, there is a growing collection of tools and components.

這個略微有點抽象，走到這一步，說明你們已經明白了數據立方體的定義，以及上傳的xml文件就定義了一個多維數據庫（不明白的同窗翻看之前的博客：http://www.cnblogs.com/liqiu）。那麼定義好了多維數據庫，就須要獲取裏面的數據，olap4j就是這樣的一個實現了jdbc規範的多爲數據庫查詢引擎！數據庫