由ReentrantLock和JPA(spring.jpa.open-in-view)致使的死鎖問題緣由分析。java
問題
在壓測過程當中,發現服務通過一段時間壓測以後出現無響應,且沒法自動恢復。spring
分析
從上述問題表象中,猜想服務出現死鎖,致使全部線程都在等待獲取鎖,從而沒法響應後續全部請求。數據庫
接下來經過jstack輸出線程堆棧信息查看,發現大量容器線程在等待數據庫鏈接jsp
"XNIO-1 task-251" #375 prio=5 os_prio=0 tid=0x00007fec640cf800 nid=0x53ea waiting on condition [0x00007febf64c5000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000081565b80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at com.alibaba.druid.pool.DruidDataSource.takeLast(DruidDataSource.java:1899) at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1460) at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1255) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1235) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1225) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:90) at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35) at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:106) at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:136) at org.hibernate.internal.SessionImpl.connection(SessionImpl.java:542)
查看DruidDataSource源碼,能夠看出當前已經沒有可用的數據庫鏈接,因此線程等待。async
DruidConnectionHolder takeLast() throws InterruptedException, SQLException { try { while (poolingCount == 0) { emptySignal(); // send signal to CreateThread create connection if (failFast && failContinuous.get()) { throw new DataSourceNotAvailableException(createError); } notEmptyWaitThreadCount++; if (notEmptyWaitThreadCount > notEmptyWaitThreadPeak) { notEmptyWaitThreadPeak = notEmptyWaitThreadCount; } try { // 當有新的鏈接被建立或者其餘線程釋放鏈接,就會被喚醒 notEmpty.await(); // signal by recycle or creator } finally { notEmptyWaitThreadCount--; } notEmptyWaitCount++; if (!enable) { connectErrorCountUpdater.incrementAndGet(this); throw new DataSourceDisableException(); } } } catch (InterruptedException ie) { notEmpty.signal(); // propagate to non-interrupted thread notEmptySignalCount++; throw ie; } decrementPoolingCount(); DruidConnectionHolder last = connections[poolingCount]; connections[poolingCount] = null; return last; }
再查看其餘容器線程狀態,發現有8個線程在等待 0x000000008437e2c8 鎖,此鎖是ReentrantLock,說明ReentrantLock已經被其餘線程持有。ide
分析多是由於某種狀況這8個線程沒有釋放數據庫鏈接,致使其餘線程沒法獲取數據庫鏈接(爲何是8個呢,由於數據庫鏈接池採用默認配置,默認最大鏈接數爲8)。性能
接下來繼續查看ReentrantLock爲何沒有正常的釋放,查看當前持有該鎖的線程信息,發現該線程持有了ReentrantLock鎖,可是又再等待數據庫鏈接。因爲異常致使上一次獲取到鎖以後沒有釋放(沒有在finally代碼塊中釋放鎖),若是數此線程能夠獲取到數據庫鏈接,下次可能就會釋放鎖,應該不會致使死鎖,因此問題的根本緣由不是ReentrantLock沒有釋放鎖。ui
經過上面的分析得知,有一個線程持有了ReentrantLock鎖,可是在等待數據庫鏈接,而另外8個線程持有了數據庫鏈接,卻在等待ReentrantLock鎖,產生死鎖。this
可是正常狀況下,當數據庫操做執行完成以後,線程應該會釋放數據庫鏈接,這裏顯然沒有釋放。因爲咱們這邊使用的JPA,因此猜想多是JPA的問題。spa
聯想到在SpringBoot啓動日誌中發現JPA的警告日誌,具體以下
spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
猜測多是因爲這個配置問題,因而開始找Spring Data JPA相關文檔。發現這個配置會致使MVC的Controller執行完數據庫操做後,仍然持有數據庫鏈接。由於對於JPA(默認是Hibernate實現)來講,ToMany關係默認是懶加載,ToOne關係默認是當即加載。當咱們經過JPA查詢到一個對象以後,可能會去調用ToMany關係對應實體的get方法,獲取對應實體集合,若是此時沒有Hibernate Session會報LazyInitializationException異常,因此默認狀況下MVC的Controller方法執行完成以後纔會釋放數據庫鏈接。
查看spring.jpa.open-in-view
對應的攔截器源碼
public class OpenEntityManagerInViewInterceptor extends EntityManagerFactoryAccessor implements AsyncWebRequestInterceptor { @Override public void preHandle(WebRequest request) throws DataAccessException { EntityManagerFactory emf = obtainEntityManagerFactory(); if (TransactionSynchronizationManager.hasResource(emf)) { // ... } else { logger.debug("Opening JPA EntityManager in OpenEntityManagerInViewInterceptor"); try { // 建立EntityManager並綁定到當前線程 EntityManager em = createEntityManager(); EntityManagerHolder emHolder = new EntityManagerHolder(em); TransactionSynchronizationManager.bindResource(emf, emHolder); AsyncRequestInterceptor interceptor = new AsyncRequestInterceptor(emf, emHolder); asyncManager.registerCallableInterceptor(key, interceptor); asyncManager.registerDeferredResultInterceptor(key, interceptor); } catch (PersistenceException ex) { throw new DataAccessResourceFailureException("Could not create JPA EntityManager", ex); } } } @Override public void afterCompletion(WebRequest request, @Nullable Exception ex) throws DataAccessException { // 關閉EntityManager if (!decrementParticipateCount(request)) { EntityManagerHolder emHolder = (EntityManagerHolder) TransactionSynchronizationManager.unbindResource(obtainEntityManagerFactory()); logger.debug("Closing JPA EntityManager in OpenEntityManagerInViewInterceptor"); EntityManagerFactoryUtils.closeEntityManager(emHolder.getEntityManager()); } } @Override public void afterConcurrentHandlingStarted(WebRequest request) { // 解除綁定 if (!decrementParticipateCount(request)) { TransactionSynchronizationManager.unbindResource(obtainEntityManagerFactory()); } } }
結論
因爲沒有配置spring.jpa.open-in-view
(默認爲true),JPA方法執行完成以後,並無釋放數據庫鏈接(須要等到Controller方法執行完成纔會釋放),而剛好因爲異常致使ReentrantLock鎖沒有正確釋放,進而致使其餘已經獲取到數據庫鏈接的線程沒法獲取ReentrantLock鎖,其餘線程也沒法獲取到數據庫鏈接(其中就包含持有ReentrantLock鎖的線程),最終致使死鎖。修復的方法很是簡單,finally代碼塊中釋放鎖,而且關閉spring.jpa.open-in-view
配置(可選)。
對於spring.jpa.open-in-view
這個配置大體存在兩種觀點,一種認爲須要這個配置,它有利於提高開發效率,另外一個部分人認爲這個配置會影響到性能(Controller方法執行完成以後才釋放鏈接),形成資源的浪費。可是若是執行完數據庫操做就釋放鏈接的話,就沒法經過get方法獲取ToMany關係對應的實體集合(或者獲取手動獲取,但顯然不合適)。
其實這兩種觀點沒有對錯,只不過須要根據業務實際狀況做出選擇。我猜測可能出於這種考慮,官方纔在用戶沒有主動配置spring.jpa.open-in-view
的時候,在啓動的過程當中打印出一條警告日誌,通知用戶關注此項配置,而後做出選擇。
轉載:諾頓教育