Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: C3PO Occasional Apparent Deadlock
PostPosted: Wed Jun 27, 2007 4:52 pm 
Newbie

Joined: Mon Jan 08, 2007 6:32 pm
Posts: 1
Hello

Using Hibernate 3.2.1 / c3p0 0.9.1-pre9 we're getting occasional APPARENT DEADLOCK messages :

What does that exactly mean ?

2007-06-27 17:37:18,861 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$Deadl
ockDetector@323993b6 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
2007-06-27 17:37:18,876 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$Deadl
ockDetector@323993b6 -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 3
Active Threads: 3
Active Tasks:
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask@4227a2f6 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolTh
read-#2)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask@383be668 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolTh
read-#0)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask@6493d2af (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolTh
read-#1)
Pending Tasks:
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@24ed12bf
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@4869cc89
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@19f35b83
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@37fd02c4
Pool thread stack traces:
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1340)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:41)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask.run(GooGooStatementCache.java:404)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1340)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:41)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask.run(GooGooStatementCache.java:404)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1340)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:41)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StatementCloseTask.run(GooGooStatementCache.java:404)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)


....


2007-06-27 17:38:18,909 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - Task com.mchange.v2.c3p0.stmt.GooGooStatementCache$1Sta
tementCloseTask@4227a2f6 (in deadlocked PoolThread) failed to complete in maximum time 60000ms. Trying interrupt().
2007-06-27 17:38:18,909 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - Task com.mchange.v2.c3p0.stmt.GooGooStatementCache$1Sta
tementCloseTask@383be668 (in deadlocked PoolThread) failed to complete in maximum time 60000ms. Trying interrupt().
2007-06-27 17:38:18,909 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - Task com.mchange.v2.c3p0.stmt.GooGooStatementCache$1Sta
tementCloseTask@6493d2af (in deadlocked PoolThread) failed to complete in maximum time 60000ms. Trying interrupt().
2007-06-27 17:38:28,918 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$Deadl
ockDetector@323993b6 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
2007-06-27 17:38:28,926 [Timer-1] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$Deadl
ockDetector@323993b6 -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 3
Active Threads: 3




Thanks for your insights

Alain


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 28, 2007 11:18 am 
Pro
Pro

Joined: Mon Jan 24, 2005 5:39 am
Posts: 216
Location: Germany
You can find information about this here:

http://www.mchange.com/projects/c3p0/in ... perThreads
http://www.mchange.com/projects/c3p0/in ... veTaskTime

Usually its a slow or hanging database server.

_________________
dont forget to rate !


Top
 Profile  
 
 Post subject: I'm having the same problem(and maybe a solution)
PostPosted: Tue Jul 03, 2007 6:59 am 
Newbie

Joined: Mon Jan 17, 2005 1:09 pm
Posts: 19
Hello,

I had or I'm having the same problem. But I've just noticed that I'm using an mysql 3.x jdbc driver with an 5.x mysql database :-) . So I guess that's my problem. I'm testing this now.

I think this kind of error is mostly something like:
- DB problems
- connection problems
- bad drivers,
-.....

Not somethings to look for in your java code

best regards,
Christof

ps: don't forget to rate


Top
 Profile  
 
 Post subject: Re: C3PO Occasional Apparent Deadlock
PostPosted: Sat Jul 19, 2008 9:41 am 
Newbie

Joined: Thu Jun 14, 2007 4:19 pm
Posts: 6
lds wrote:
Hello

Using Hibernate 3.2.1 / c3p0 0.9.1-pre9 we're getting occasional APPARENT DEADLOCK messages :



Hi Ids
I have similar deadlocks :(
See my post here with detailed description:
http://forum.hibernate.org/viewtopic.php?p=2386237

btw:
Have you found a solution or a reson of this deadlocks?

Regards!
Adam


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jul 21, 2008 4:04 am 
Newbie

Joined: Mon Jul 21, 2008 3:42 am
Posts: 2
Ok I am using c3p0-0.9.1.2.jar. Previously I am also having APPARENT DEADLOCK messages every now and then.

For my case, besides this deadlock, I also encountered a memory leak in my application.

So therefore, I do a heap dump and analyze my memory usage. In the memory analysis, PreparedStatement were occupying large part of my memory usage.

After some trial and error, I had finally found out if you set maxStatementsPerConnection to be less than the number of PreparedStatement opened during a lifetime of a connection, the PreparedStatement will be cached indefinitely, causing a memory leak and then in turn the APPARENT DEADLOCK.

For example,

conn = connPool.getConnection()

p1 = conn.preparedStatement();
p2 = conn.preparedStatement();

conn.close()


if connPool maxStatementsPerConnection is 1, then the memory leak will occur. To resolve, set it to greater or equal to 2


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 23, 2008 8:18 am 
Newbie

Joined: Wed Sep 06, 2006 1:02 pm
Posts: 3
Hi notorand

Many thanks for you reply.
I will definitely try to run your scenario and I will try to recreate this issue (APPARENT DEADLOCK).

btw:
Your application which you use to recreate this problem was single-threaded or multithreaded?

btw II:
Which database have you use to recreate this error?

Kind regards,
Adam Wo┼║niak


Top
 Profile  
 
 Post subject: Re: C3PO Occasional Apparent Deadlock
PostPosted: Thu May 20, 2010 9:12 am 
Newbie

Joined: Thu May 20, 2010 8:48 am
Posts: 1
We were able to solve our application's APPARENT DEADLOCK problems by analyzing stack traces. A good idea is to set up a script to trigger stack trace dumping when APPARENT DEADLOCK appears in the log file.

There's a nasty mechanism where a single long-running Oracle query can block other c3p0-managed connections from preparing statements.

A single long-running query will hold the synchronization lock on the OracleConnection it is using.

If OracleStatements have to be evicted from the c3p0 statement cache a c3p0 background thread will try to call OracleStatement.close. OracleStatement holds a reference to the original statement it was created with and will try to obtain lock on it. This will block, if the long running query is using the same connection.

If there's a lot of churn on the statement cache this will happen more often and once in a while all the c3p0 background threads may be blocked by this mechanism.

C3P0 uses background threads to prepare statements (see http://www.java2s.com/Open-Source/Java- ... e.java.htm line 540).

When all the background threads are stuck waiting for access to the single connection no statements will be prepared and all Hibernate operatations will be blocked like so:
Code:
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00002aaab3b7c4d8> (a com.mchange.v2.c3p0.stmt.GlobalMaxOnlyStatementCache)
        at java.lang.Object.wait(Object.java:485)
        at com.mchange.v2.c3p0.stmt.GooGooStatementCache.acquireStatement(GooGooStatementCache.java:552)
        at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkoutStatement(GooGooStatementCache.java:168)
        - locked <0x00002aaab3b7c4d8> (a com.mchange.v2.c3p0.stmt.GlobalMaxOnlyStatementCache)
        at com.mchange.v2.c3p0.impl.NewPooledConnection.checkoutStatement(NewPooledConnection.java:277)
        - locked <0x00002aab175582d0> (a com.mchange.v2.c3p0.impl.NewPooledConnection)
        at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:199)
        - locked <0x00002aaab1b3a1b8> (a com.mchange.v2.c3p0.impl.NewProxyConnection)
        at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:534)
        at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:452)
        at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161)
        at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1577)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.