hive - FAILED: Error in acquiring locks

錯誤日誌

FAILED: Error in acquiring locks: Lock acquisition for LockRequest(component:[LockComponent(type:EXCLUSIVE, level:TABLE, dbname:dw, tablename:dim1, operationType:INSERT, isAcid:false), LockComponent(type:SHARED_READ, level:TABLE, dbname:ods, tablename:ds_data2, operationType:SELECT), LockComponent(type:SHARED_READ, level:TABLE, dbname:ods, tablename:dim3, operationType:SELECT)], txnid:0, user:airflow, hostname:hubserver1, agentInfo:airflow_20190731065910_5be6e4a9-8e33-4b8d-aa0b-aa778cca3105) timed out after 5503478ms. LockResponse(lockid:69648, state:WAITING)
仔細分析得知實由於lockid:69648的鎖致使運行失敗。

hive鎖

hive存在兩種鎖，共享鎖Shared (S)和互斥鎖Exclusive (X)。以上面的日誌爲例，執行的原sql大概是

insert overwrite table dim1 (bus_date='2019-07-29') select * from data2 t1 left join dim3 t2 on t1.id=t2.id;

讀的兩張表都是S鎖，被insert的dim1是X鎖。其中只觸發s鎖的操做能夠併發的執行，只要有一個操做對錶或者分區出發了x鎖，則該表或者分區不能併發的執行做業。

SHOW LOCKS <TABLE_NAME>; --查看鎖

問題解決

報錯的Sql中加入下列語句
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager;