线上应用,配置的数据库连接数为50,正常情况是已经够用了,但是有天发现50个连接全部被占满,并且长时间无法恢复,重启服务后会好一段时间。

<>1、问题现象

Druid获取MySQL数据库连接超时,超时时间设置的为60s
2019-09-23 15:38:24.627-|XNIO-3 task-80-|ERROR-|7f
ba6a1c917f42878b43f937df37e575-|10.123.18.241-|-|/api/v1/qimao/admin/distributor
/add-|-|Java/1.8.0_181-|c.q.k.q.web.controller.admin.DistributorController-|
method:addDistributor org.springframework.transaction.
CannotCreateTransactionException: Could not open JDBC Connection for transaction
; nested exception is com.alibaba.druid.pool.GetConnectionTimeoutException:
wait millis60000, active 10, maxActive 10, creating 0 at org.springframework.
jdbc.datasource.DataSourceTransactionManager.doBegin(
DataSourceTransactionManager.java:289) at org.springframework.transaction.
support.AbstractPlatformTransactionManager.getTransaction(
AbstractPlatformTransactionManager.java:373) at org.springframework.transaction.
interceptor.TransactionAspectSupport.createTransactionIfNecessary(
TransactionAspectSupport.java:447) at org.springframework.transaction.
interceptor.TransactionAspectSupport.invokeWithinTransaction(
TransactionAspectSupport.java:277) at org.springframework.transaction.
interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org
.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.
CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com
.qiyi.kpp.qimao.web.service.impl.
DistributorServiceImpl$$EnhancerBySpringCGLIB$$e95f0879.addDistributor(<
generated>) at com.qiyi.kpp.qimao.web.controller.admin.DistributorController.
addDistributor(DistributorController.java:174) at com.qiyi.kpp.qimao.web.
controller.admin.DistributorController$$FastClassBySpringCGLIB$$9c955533.invoke(
<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.
java:204) at org.springframework.aop.framework.
CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:157) at org.springframework.aop.framework.
adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:
52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:168) at org.springframework.aop.interceptor.
ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.
springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.
CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com
.qiyi.kpp.qimao.web.controller.admin.
DistributorController$$EnhancerBySpringCGLIB$$100c45af.addDistributor(<generated
>) at sun.reflect.GeneratedMethodAccessor210.invoke(Unknown Source) ............
. at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(
ServletInitialHandler.java:272) at io.undertow.servlet.handlers.
ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.
servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.
java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:
211) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:809)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:617) at java.lang.Thread.run(Thread.java:745) Caused by: com.alibaba.druid.
pool.GetConnectionTimeoutException: wait millis 60000, active 10, maxActive 10,
creating0 at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(
DruidDataSource.java:1512) at com.alibaba.druid.pool.DruidDataSource.
getConnectionDirect(DruidDataSource.java:1255) at com.alibaba.druid.filter.
FilterChainImpl.dataSource_connect(FilterChainImpl.java:5007) at com.alibaba.
druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.java:680) at
com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java
:5003) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.
java:1233) at com.alibaba.druid.pool.DruidDataSource.getConnection(
DruidDataSource.java:1225) at com.alibaba.druid.pool.DruidDataSource.
getConnection(DruidDataSource.java:90) at org.springframework.jdbc.datasource.
DataSourceTransactionManager.doBegin(DataSourceTransactionManager.java:246) ...
100 common frames omitted
<>2、解决办法

添加配置,当连接超过30秒钟后会强制进行回收
<!-- 超过时间限制是否回收 --> druid.removeAbandoned=true <!-- 超时时间;单位为秒。 --> druid.
removeAbandonedTimeout=30 <!-- 关闭abanded连接时输出错误日志 --> druid.logAbandoned=true
最终重现了这个问题,打印日志如下
2019-09-27 16:07:21.369-|AsyncResolver-bootstrap-executor-0-|INFO-|-|-|-|-|-|-|
c.n.d.shared.resolver.aws.ConfigClusterResolver-|Resolving eureka endpoints via
configuration2019-09-27 16:07:28.954-|Druid-ConnectionPool-Destroy-786728464-|
ERROR-|-|-|-|-|-|-|com.alibaba.druid.pool.DruidDataSource-|abandon connection,
owner thread: XNIO-3 task-862, connected at : 1569571561703, open stackTrace at
java.lang.Thread.getStackTrace(Thread.java:1552) at com.alibaba.druid.pool.
DruidDataSource.getConnectionDirect(DruidDataSource.java:1313) at com.alibaba.
druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:5007) at
com.alibaba.druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.
java:680) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(
FilterChainImpl.java:5003) at com.alibaba.druid.pool.DruidDataSource.
getConnection(DruidDataSource.java:1233) at com.alibaba.druid.pool.
DruidDataSource.getConnection(DruidDataSource.java:1225) at com.alibaba.druid.
pool.DruidDataSource.getConnection(DruidDataSource.java:90) at org.
springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(
DataSourceTransactionManager.java:246) at org.springframework.transaction.
support.AbstractPlatformTransactionManager.getTransaction(
AbstractPlatformTransactionManager.java:373) at org.springframework.transaction.
interceptor.TransactionAspectSupport.createTransactionIfNecessary(
TransactionAspectSupport.java:447) at org.springframework.transaction.
interceptor.TransactionAspectSupport.invokeWithinTransaction(
TransactionAspectSupport.java:277) at org.springframework.transaction.
interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org
.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.
CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com
.qiyi.kpp.qimao.web.service.impl.
DistributorServiceImpl$$EnhancerBySpringCGLIB$$ec30275b.addDistributor(<
generated>) at com.qiyi.kpp.qimao.web.controller.admin.DistributorController.
addDistributor(DistributorController.java:174) at com.qiyi.kpp.qimao.web.
controller.admin.DistributorController$$FastClassBySpringCGLIB$$9c955533.invoke(
<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.
java:204) at org.springframework.aop.framework.
CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:157) at org.springframework.aop.framework.
adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:
52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:168) at org.springframework.aop.interceptor.
ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.
springframework.aop.framework.ReflectiveMethodInvocation.proceed(
ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.
CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com
.qiyi.kpp.qimao.web.controller.admin.
DistributorController$$EnhancerBySpringCGLIB$$34df12db.addDistributor(<generated
>)
运行程序,当连接超过30秒钟后会强制进行回收,并输出异常日志。再根据异常日志分析(看看owner thread在干啥,为什么一直占据数据库连接)即可

技术
©2019-2020 Toolsou All rights reserved,
java实现抢红包功能AndroidStudio开发笔记1--第一个appMybatis错误解决:There is no getter for property named '*' in 'class Java.lang.String单个按键控制多种流水灯状态用Python做自动化测试(pytest框架的精髓)关于keras使用fit_generator中遇到StopIterationMySQL面试必会!Python基础知识整理笔记Redis队列实现java秒杀系统,无脚本,可用于生产崮德好文连载 - 活该你是工程师(自序)