1、異常出現的場景java
一次線上訂單歷史數據字段刷新操做,3張表100多萬數據。因爲同步更新太慢大概20分鐘以上,因此採用異不的方式。代碼以下:spring
private void batchUpdate(List<SaasOrderRecordDataForUpdate> saasOrderRecordDataForUpdateList, List<SaasServiceOrderInfoDataForUpdate> saasServiceOrderInfoDataForUpdateList, List<OrderGoodsDataForUpdate> orderGoodsDataForUpdateList, List<OrderAdditionCostInfoDataForUpdate> orderAdditionCostInfoDataForUpdateList) { List<Future> asyncResultList = new ArrayList<>(); if (CollectionUtils.isNotEmpty(saasOrderRecordDataForUpdateList)) { int size = saasOrderRecordDataForUpdateList.size(); int count = size / 5000; for (int i = 0; i <= count; i++) { List<SaasOrderRecordDataForUpdate> subList = null; if (i == count) { if (5000 * i < size) { subList = saasOrderRecordDataForUpdateList.subList(5000 * i, size); } } else { subList = saasOrderRecordDataForUpdateList.subList(5000 * i, 5000 * (i + 1)); } if (CollectionUtils.isNotEmpty(subList)) { Future future = dataRefreshJobServiceForAsync.batchUpdateSaasOrderRecordDataForUpdate(subList); asyncResultList.add(future); } } } XxlJobLogger.log("批量更新訂單數:{}", saasOrderRecordDataForUpdateList.size()); if (CollectionUtils.isNotEmpty(saasServiceOrderInfoDataForUpdateList)) { int size = saasServiceOrderInfoDataForUpdateList.size(); int count = size / 5000; for (int i = 0; i <= count; i++) { List<SaasServiceOrderInfoDataForUpdate> subList = null; if (i == count) { if (5000 * i < size) { subList = saasServiceOrderInfoDataForUpdateList.subList(5000 * i, size); } } else { subList = saasServiceOrderInfoDataForUpdateList.subList(5000 * i, 5000 * (i + 1)); } if (CollectionUtils.isNotEmpty(subList)) { Future future = dataRefreshJobServiceForAsync.batchUpdateSaasServiceOrderInfoDataForUpdate(subList); asyncResultList.add(future); } } } XxlJobLogger.log("批量更新訂單服務數:{}", saasServiceOrderInfoDataForUpdateList.size()); if (CollectionUtils.isNotEmpty(orderGoodsDataForUpdateList)) { int size = orderGoodsDataForUpdateList.size(); int count = size / 5000; for (int i = 0; i <= count; i++) { List<OrderGoodsDataForUpdate> subList = null; if (i == count) { if (5000 * i < size) { subList = orderGoodsDataForUpdateList.subList(5000 * i, size); } } else { subList = orderGoodsDataForUpdateList.subList(5000 * i, 5000 * (i + 1)); } if (CollectionUtils.isNotEmpty(subList)) { Future future = dataRefreshJobServiceForAsync.batchUpdateOrderGoodsDataForUpdate(subList); asyncResultList.add(future); } } } XxlJobLogger.log("批量更新訂單商品數:{}", orderGoodsDataForUpdateList.size()); if (CollectionUtils.isNotEmpty(orderAdditionCostInfoDataForUpdateList)) { int size = orderAdditionCostInfoDataForUpdateList.size(); int count = size / 5000; for (int i = 0; i <= count; i++) { List<OrderAdditionCostInfoDataForUpdate> subList = null; if (i == count) { if (5000 * i < size) { subList = orderAdditionCostInfoDataForUpdateList.subList(5000 * i, size); } } else { subList = orderAdditionCostInfoDataForUpdateList.subList(5000 * i, 5000 * (i + 1)); } if (CollectionUtils.isNotEmpty(subList)) { Future future = dataRefreshJobServiceForAsync.batchUpdateOrderAdditionCostInfoDataForUpdate(subList); asyncResultList.add(future); } } } XxlJobLogger.log("批量更新訂單附加費數:{}", orderAdditionCostInfoDataForUpdateList.size()); if (CollectionUtils.isNotEmpty(asyncResultList)) { for (Future asyncResult : asyncResultList) { try { asyncResult.get(); } catch (Exception e) { e.printStackTrace(); } } } }
本地庫刷新沒問題,可是到了線上庫就出現以下異常:sql
### Error updating database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30005ms. ### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30005ms. at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:200) at sun.reflect.GeneratedMethodAccessor380.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433) ... 24 more Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30005ms. at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:81) at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:82) at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:68) at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:338) at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:84) at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:49) at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117) at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76) at sun.reflect.GeneratedMethodAccessor381.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63) at com.sun.proxy.$Proxy521.update(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198) ... 28 more Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30005ms. at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:669) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:183) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:148) at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128) at com.zaxxer.hikari.HikariDataSource$$FastClassBySpringCGLIB$$eeb1ae86.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.zaxxer.hikari.HikariDataSource$$EnhancerBySpringCGLIB$$f68c05a.getConnection(<generated>) at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:151) at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:115) at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:78)
2、解決辦法數據庫
經過異常能夠發現是因爲獲取不到數據庫鏈接致使,猜想是鏈接數不夠的問題,因此修改HikariPool鏈接池配置,就解決了apache
修改前:session
spring: datasource: hikari: connection-test-query: SELECT 1 FROM DUAL connection-timeout: 30000 maximum-pool-size: 20 max-lifetime: 1800000 minimum-idle: 5 connection-init-sql: SET NAMES utf8mb4
修改後:mybatis
spring: datasource: hikari: connection-test-query: SELECT 1 FROM DUAL connection-timeout: 600000 maximum-pool-size: 500 max-lifetime: 1800000 minimum-idle: 20 validation-timeout: 3000 idle-timeout: 60000 connection-init-sql: SET NAMES utf8mb4