*****
Ignore this - was a dumb infinite loop problem, not hanging at all
*****
An application containing a mapped entity successfully processing many thousand of transactions per day hung - Looking at the thread dump through jconsole showed one thread in this state - unmoving for a very long time. Looking at the jdk source code, the offending line of DelegatingMethodAccessorImpl merely calls the same method in a delegate object. Many other threads were blocked on the database because this thread had a row lock, so the whole app ground to a halt, otherwise the jvm seemed fine - responsive to JMX calls, memory use was well below the maximum, GC proceeding ok.
Its a one-off incident but in a critical system so I would like to understand what happened and how to prevent it.
Anyone seen anything like this or have any ideas?
Code:
State: RUNNABLE
Total blocked: 321 Total waited: 48,201
Stack trace:
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.invoke(CGLIBLazyInitializer.java:157)
com.ubs.boxer_setts.entity.AssetFlow$$EnhancerByCGLIB$$57af84d8.getTaskFlagsVersion(<generated>)
com.ubs.boxer_setts.taskplugin.AssetFlowAutoMatchTaskHandler.getPreviousVersion(AssetFlowAutoMatchTaskHandler.java:71)
com.ubs.boxer_setts.taskplugin.AssetFlowAutoMatchTaskHandler.run(AssetFlowAutoMatchTaskHandler.java:51)
com.ubs.boxer_setts.taskplugin.AssetFlowAutoMatchTaskHandler.run(AssetFlowAutoMatchTaskHandler.java:20)
com.ubs.boxer_core.taskplugin.GenericEntityTaskHandler.run(GenericEntityTaskHandler.java:73)
com.ubs.boxer_core.pmp.PrimaryMessageProcessor.runTask(PrimaryMessageProcessor.java:682)
com.ubs.boxer_core.pmp.PrimaryMessageProcessor.processMessageTree(PrimaryMessageProcessor.java:1098)
com.ubs.boxer_core.pmp.PrimaryMessageProcessor.processMessage(PrimaryMessageProcessor.java:454)
com.ubs.boxer_core.pmp.PrimaryMessageProcessor.run(PrimaryMessageProcessor.java:285)
java.lang.Thread.run(Thread.java:595)
Thanks,
David
Login
FAQ
Search