【問題】The coprocessor thread stopped itself due to scan timeout or scan threshold

Kylin執行查詢語句的時候報錯以下：

Error while executing SQL "select t.hotel_id_m,t.live_dt, d.day_of_week,sum(rns) from tableT t join TableD d on t.live_dt = d.daY_no group by t.hotel_id_m,t.live_dt, d.day_of_week LIMIT 50000": <sub-thread for Query ac580b70-96f2-403a-a64a-0557e599d35f GTScanRequest 143ae1ba>The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log), failing current query...

查看regionserver日誌

2017-03-20 11:10:05,436 INFO  [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: Scanned 9999001 rows from HBase.

2017-03-20 11:10:05,454 INFO  [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: The cube visit did not finish normally because scan num exceeds threshold

org.apache.kylin.gridtable.GTScanExceedThresholdException: Exceed scan threshold at 10000001

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService$1.hasNext(CubeVisitService.java:258)

at org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1$1.hasNext(HBaseReadonlyStore.java:111)

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:290)

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4117)

at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)

at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)

at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)

at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)

at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)

at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)

2017-03-20 11:10:05,459 WARN  [RpcServer.FifoWFPBQ.default.handler=59,queue=5,port=60020] ipc.RpcServer: (responseTooSlow): {"call":"ExecService(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)","starttimems":1489979376495,"responsesize":359,"method":"ExecService","processingtimems":28964,"client":"10.10.16.102:58720","queuetimems":1,"class":"HRegionServer"}

分析解決：

在${KYLIN_HOME}/conf/kylin.property配置文件中有一個配置項

kylin.query.scan.threshold=10000000

這個配置項限制了Hbase scan的行數，當scan超過kylin.query.scan.threshold行的時候仍是不能知足查詢需求，則kylin取消hbase端的查詢。

目前暫時有兩種方式解決

一、增長kylin.query.scan.threshold值，這相應也會增長hbase壓力，在Hbase可以頂住查詢壓力的狀況下，能夠增大該值

二、這次查詢語句中有LIMIT 50000，Hbase在scan了kylin.query.scan.threshold行以後，讓不能掃描出50000行。能夠減少LIMIT的值，在Hbase在scan到達kylin.query.scan.threshold以前，就能知足查詢要求。則不會出現這個問題