I got this error:
Code:
javax.persistence.PersistenceException: org.hibernate.HibernateException: Transaction was rolled back in a different thread!
randomly after 5 days, and it kills my DB connection from then on.
I'm using CMT in JBoss 6.2, so every DAO injects the PersistenceContext, and has its transactions managed for it.
This is my Spring setup:
Quote:
<jee:jndi-lookup jndi-name="java:jboss/persistence/ds"
id="entityManagerFactory" expected-type="javax.persistence.EntityManagerFactory" />
<bean id="entityManager"
class="org.springframework.orm.jpa.support.SharedEntityManagerBean">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager" >
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
And my DS is setup so:
Quote:
<datasources>
<datasource jndi-name="java:/DS" pool-name="DS" enabled="true" use-java-context="true">
<connection-url>jdbc:sqlserver://srvr:1433;databaseName=DEV</connection-url>
<driver>sqlserver2008</driver>
<pool>
<min-pool-size>5</min-pool-size>
<max-pool-size>50</max-pool-size>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<user-name>jb</user-name>
<password>jboss</password>
</security>
<validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mssql.MSSQLValidConnectionChecker"/>
</validation>
</datasource>
<drivers>
<driver name="sqlserver2008" module="com.microsoft">
<driver-class>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class>
</driver>
</drivers>
</datasources>
My hunch is that because it's CMT, there is one connection only, which is passed around via @PersistenceContext, so the same EntityManager is used for everything, so to reconnect, it would have to get a new EntityManager from the EntityManagerFactory, but it doesn't know to do that.
I've now put 'synchronized' on the code that writes to the DB (which might cause a lock on a table) and I haven't had the problem since, but it also hasn't been 5 days yet, and using synchronized is not a scalable solution.
Code:
01:27:16,084 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000001:-33c59422:53f48181:1f01a in state RUN
01:27:16,089 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff7f000001:-33c59422:53f48181:1f01a invoked while multiple threads active within it.
01:27:16,089 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012108: CheckedAction::check - atomic action 0:ffff7f000001:-33c59422:53f48181:1f01a aborting with 1 threads active!
01:27:16,588 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000001:-33c59422:53f48181:1f01a in state CANCEL
01:27:17,102 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000001:-33c59422:53f48181:1f01a in state CANCEL_INTERRUPTED
01:27:17,103 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012120: TransactionReaper::check worker Thread[Transaction Reaper Worker 0,5,main] not responding to interrupt when cancelling
TX 0:ffff7f000001:-33c59422:53f48181:1f01a -- worker marked as zombie and TX scheduled for mark-as-rollback
01:27:17,103 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012110: TransactionReaper::check successfuly marked TX 0:ffff7f000001:-33c59422:53f48181:1f01a as rollback only
01:37:45,891 WARN [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (Camel (camelContext) thread #0 - JmsConsumer[inQueue]) SQL Error: 0, SQLState: 08S01
01:37:45,891 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (Camel (camelContext) thread #0 - JmsConsumer[inQueue]) Connection timed out
01:37:45,925 WARN [org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl] (Transaction Reaper Worker 0) Transaction afterCompletion called by a background thread! Delaying action until the original thread can handle it. [status=4]
01:37:45,950 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012113: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] missed interrupt when cancelling TX 0:ffff7f000001:-33c59422:53f48181:1f01a -- exiting as zombie (zombie count decremented to 0)
01:37:45,987 INFO [org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl] (Camel (camelContext) thread #0 - JmsConsumer[inQueue]) HHH000106: Forcing container resource cleanup on transaction completion
So, I want to understand how this happens, and why it makes my connection unrecoverable from then on. Ideally I can fix the root problem, but I'd be happy with just a measure to reconnect when this happens. I thought valid-connection-checker was supposed to do that. But it didn't help. Should I maybe change the flush strategy? A different entityManager class? A different Transaction Manager?