場景:一個上傳接口,須要上傳幾十M的文件,文件中包含10幾W的數據,而後對10+W的數據進行同步批量插入,每次批量插入1W。最後返回結果。java
項目上線一段時間後,上傳接口出現問題,數據庫用的MySQL5.7.21,報了以下錯誤:mysql
2018-07-16 01:30:03.497 ERROR com.alibaba.druid.pool.DruidDataSource Line:1594 - discard connection com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,480 milliseconds ago. The last packet sent successfully to the server was 1,480 milliseconds ago. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155) at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379) at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498) at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63) at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79) at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109) at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:137) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61) at com.sun.proxy.$Proxy63.query(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141) at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434) at com.sun.proxy.$Proxy25.selectList(Unknown Source) at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231) at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137) at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75) at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53) at com.sun.proxy.$Proxy51.selectReportListByReportDate(Unknown Source) at com.rxwx.service.report.impl.PrcmtReportServiceImpl.generateProcurementReport(PrcmtReportServiceImpl.java:97) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$8c585594.generateProcurementReport(<generated>) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$9a830f32.generateProcurementReport(<generated>) at com.rxwx.task.sup.GenarateReportJob.selfGenReportTask1(GenarateReportJob.java:37) at com.rxwx.task.sup.GenarateReportJob$$FastClassBySpringCGLIB$$64b31449.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503) ... 58 common frames omitted
首先百度了一發,按照網上的解決方案,修改了my.cnf中的超時時間:git
wait_timeout=31536000 interactive_timeout=31536000
將原來默認的8小時改成1年。這裏單位是秒。github
修改了德魯伊的:testOnBorrow = true。spring
跑了一天,MMP,凌晨又出現這個錯誤了。sql
一個電話把睡夢中的我吵醒,擾我週末擾我夢,沒得辦法,重啓下服務,重啓下MySQL,先上傳吧,睡覺。數據庫
週一到公司谷歌了一發,修改my.cnf的net_read_timeout和net_write_timeout 參數,將這兩個參數調大,改成6000。因爲項目中的數據存在無效數據,因此刪除了10天以前的無效數據,減小了表中的數量。明天看效果。apache
show global variables like "%timeout%";服務器
connect_timeout網絡
鏈接超時 mysql鏈接共有6次握手,3次TCP協議這個跟connect_timeout參數沒有關係,另外3次跟connect_timeout參數有關係,該參數主要是爲了防止網絡不佳時應用重連致使鏈接數漲太快,通常默認便可。
delayed_insert_timeout
這是爲MyISAM INSERT DELAY設計的超時參數,在INSERT DELAY停止前等待INSERT語句的時間
interactive_timeout
服務器關閉交互式鏈接前等待活動的秒數。交互式客戶端定義爲在mysql_real_connect()中使用CLIENT_INTERACTIVE選項的客戶端。參數默認值:28800秒(8小時)
lock_wait_timeout
鎖等待超時時間
net_read_timeout / net_write_timeout
這個參數只對TCP/IP連接有效,分別是數據庫等待接收客戶端發送網絡包和發送網絡包給客戶端的超時時間,這是在Activity狀態下的線程纔有效的參數
slave_net_timeout
這是Slave判斷主機是否掛掉的超時設置,在設定時間內依然沒有獲取到Master的迴應就人爲Master掛掉了
wait_timeout
服務器關閉非交互鏈接以前等待活動的秒數。
效果很明顯。今天沒有出現問題。