Hello,

We have recently moved to using JBoss 3.2.4RC2 with Hibernate and using the TreeCache since we want transactional cache. However I have a test, that writes/updates, and than deletes an object using Hibernates Session API. After the transaction is completed, we get the following error each time we call evict on that class (we have an MBean that will flush cache on demand). Is this a bug or am I forgetting to do something.

Thanks,

Rich

10:22:29,802 ERROR [IdentityLock] IdentityLock.acquireWriteLock(): lock could not be acquired after 10000 ms. Lock map ownership Read lock owners: [<null>:12, <null>:15]
Write lock owner: null

10:22:29,803 ERROR [HierarchyServiceBean] net.sf.hibernate.cache.CacheException: org.jboss.cache.lock.TimeoutException: IdentityLock.acquireWriteLock(): lock could not be acquired after 10000 ms. Lock map ownership Read lock owners: [<null>:12, <null>:15]
Write lock owner: null

10:22:29,805 ERROR [LogInterceptor] EJBException, causedBy:
net.sf.hibernate.cache.CacheException: org.jboss.cache.lock.TimeoutException: IdentityLock.acquireWriteLock(): lock could not be acquired after 10000 ms. Lock map ownership Read lock owners: [<null>:12, <null>:15]
Write lock owner: null

at net.sf.hibernate.cache.TreeCache.clear(TreeCache.java:88)
at net.sf.hibernate.cache.TransactionalCache.clear(TransactionalCache.java:82)
at net.sf.hibernate.impl.SessionFactoryImpl.evict(SessionFactoryImpl.java:579)

Some of the exception was missing from my original posting

Caused by: org.jboss.cache.lock.TimeoutException: IdentityLock.acquireWriteLock(): lock could not be acquired after 10000 ms. Lock map ownership Read lock owners: [<null>:2]
Write lock owner: null

at org.jboss.cache.lock.IdentityLock.acquireWriteLock(IdentityLock.java:151)
at org.jboss.cache.Node.acquireWriteLock(Node.java:412)
at org.jboss.cache.Node.acquire(Node.java:381)
at org.jboss.cache.TreeCache.findNode(TreeCache.java:2851)
at org.jboss.cache.TreeCache._remove(TreeCache.java:1805)
at org.jboss.cache.TreeCache._remove(TreeCache.java:1782)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:395)
at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:2640)
at org.jboss.cache.TreeCache.remove(TreeCache.java:1183)
at net.sf.hibernate.cache.TreeCache.clear(TreeCache.java:85)

Do you run a clustered system (clustered TreeCache)?

Yes, with one server in the cluster. Is that the issue?

Verify that you have configured TreeCache to be run in local mode:



The default behaviour of the TreeCache is to "discover" that it's alone on the segment on startup, but the timeout value of 10000 ms in the stacktrace you gave, correlates good with the default setting of SyncReplTimeout.

Maybe the cache tries to discover cache friends every time a sync op. is executed....

Two things:

First where does this attribute get defined (which file, jboss-service.xml ?)

Also, evict of the cache works until I do either the write/update/delete (I haven't figured out which one yet does this). So why would this attribute make a difference only after a update of some type?

Thanks,

Rich

Since you use Hibernate I guess that the TreeCache configuration can bee found in treecache.xml. Look for CacheMode and SyncReplTimeout there.

We are using Hibernate 2.1.2. I can't seem to find the treecache.xml file with that release.
Is that my problem? Am I suppose to upgrade Hibernate above 2.1.2 in order to use JBoss 3.2.4 (which is the release of JBoss that we are using)?

Hibernate is hard-coded to configure the TreeCache from a file named treecache.xml found at / of your classpath (Look at the source of net.sf.hibernate.cache.TreeCache and org.jboss.cache.PropertyConfigurator).

So, somewhere on your classpath there should be such a file present -- unless there's a FileNotFoundException somewhere in your log.

The hibernate distro contains the example config file src/treecache.xml.

I can find this treecache.xml with Hibernate 2.1.4, but not Hibernate 2.1.2. Does this mean I need to upgrade Hibernate to use this file?

If you like having known bugs hanging around then -- no, you should not upgrade to 2.1.4 :-)

Since you seem to get some error messages from TreeCache classes related to Hibernate stuff, then Hibernate must have had support for TreeCache in 2.1.2.

Anyhow, try to locate the treecache.xml file on your classpath, and then find out how it got there.

Ok, adding the treecache.xml file did solve my problem (I got a copy of the file from the Hibernate 2.1.4 distribution). But now I believe that auto commit must be enabled because when the container issues the commit at the end of the transaction, I get the following exception. Any ideas?

Thanks,

Rich

16:47:56,704 ERROR [TreeCache] getCurrentTransaction(): registration with transaction failed
java.lang.IllegalStateException: Already committed.
at org.jboss.tm.TransactionImpl.registerSynchronization(TransactionImpl.java:702)
at org.jboss.cache.TreeCache.getCurrentTransaction(TreeCache.java:2557)
at org.jboss.cache.TreeCache.put(TreeCache.java:1155)
at net.sf.hibernate.cache.TreeCache.put(TreeCache.java:67)
at net.sf.hibernate.cache.UpdateTimestampsCache.invalidate(UpdateTimestampsCache.java:50)
at net.sf.hibernate.impl.SessionImpl.afterTransactionCompletion(SessionImpl.java:581)
at net.sf.hibernate.engine.CacheSynchronization.afterCompletion(CacheSynchronization.java:29)
at org.jboss.tm.TransactionImpl.doAfterCompletion(TransactionImpl.java:1398)
at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:421)
at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:457)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:325)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:149)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:118)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
at org.jboss.ejb.Container.invoke(Container.java:713)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.java:60)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:186)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:466)
at org.jboss.invocation.jrmp.server.JRMPInvoker.invoke(JRMPInvoker.java:367)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:261)
at sun.rmi.transport.Transport$1.run(Transport.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:144)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:460)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:701)
at java.lang.Thread.run(Thread.java:534)

Actually it is more like Hibernate is not participating in the container transaction ever since I moved in the treecache.xml file. Is there a setting that I over looked? My treecache.xml file looks as follows:

<?xml version="1.0" encoding="UTF-8"?>

<!-- ===================================================================== -->
<!-- -->
<!-- Sample TreeCache Service Configuration -->
<!-- -->
<!-- ===================================================================== -->

<server>

<classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>


<!-- ==================================================================== -->
<!-- Defines TreeCache configuration -->
<!-- ==================================================================== -->

<mbean code="org.jboss.cache.TreeCache"
name="jboss.cache:service=TreeCache">

<depends>jboss:service=Naming</depends>
<depends>jboss:service=TransactionManager</depends>

<!--
Configure the TransactionManager
-->
<attribute name="TransactionManagerLookupClass">net.sf.hibernate.transaction.JBossTransactionManagerLookup</attribute>


<!--
Node isolation level : SERIALIZABLE
REPEATABLE_READ (default)
READ_COMMITTED
READ_UNCOMMITTED
NONE
-->
<attribute name="IsolationLevel">NONE</attribute>

<!--
Valid modes are LOCAL
REPL_ASYNC
REPL_SYNC
-->
<attribute name="CacheMode">LOCAL</attribute>

<!-- Name of cluster. Needs to be the same for all clusters, in order
to find each other
-->
<attribute name="ClusterName">dev_cluster_hierarchy</attribute>

<!-- JGroups protocol stack properties. Can also be a URL,
e.g. file:/home/bela/default.xml
<attribute name="ClusterProperties"></attribute>
-->

<attribute name="ClusterConfig">
<config>
<!-- UDP: if you have a multihomed machine,
set the bind_addr attribute to the appropriate NIC IP address -->
<!-- UDP: On Windows machines, because of the media sense feature
being broken with multicast (even after disabling media sense)
set the loopback attribute to true -->
<UDP mcast_addr="239.192.35.35" mcast_port="45577"
ip_ttl="64" ip_mcast="true"
mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
loopback="false"/>
<PING timeout="2000" num_initial_members="3"
up_thread="false" down_thread="false"/>
<MERGE2 min_interval="10000" max_interval="20000"/>
<FD shun="true" up_thread="true" down_thread="true"/>
<VERIFY_SUSPECT timeout="1500"
up_thread="false" down_thread="false"/>
<pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
up_thread="false" down_thread="false"/>
<pbcast.STABLE desired_avg_gossip="20000"
up_thread="false" down_thread="false"/>
<UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10"
down_thread="false"/>
<FRAG frag_size="8192"
down_thread="false" up_thread="false"/>
<pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
shun="true" print_local_addr="true"/>
<pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
</config>
</attribute>

<!--
Max number of entries in the cache. If this is exceeded, the
eviction policy will kick some entries out in order to make
more room
-->
<attribute name="MaxCapacity">20000</attribute>

<!--
The max amount of time (in milliseconds) we wait until the
initial state (ie. the contents of the cache) are retrieved from
existing members in a clustered environment
-->
<attribute name="InitialStateRetrievalTimeout">20000</attribute>

<!--
Number of milliseconds to wait until all responses for a
synchronous call have been received.
-->
<attribute name="SyncReplTimeout">10000</attribute>

<!-- Max number of milliseconds to wait for a lock acquisition -->
<attribute name="LockAcquisitionTimeout">15000</attribute>

<!-- Max number of milliseconds we hold a lock (not currently
implemented) -->
<attribute name="LockLeaseTimeout">60000</attribute>

<!-- Name of the eviction policy class. Not supported now. -->
<attribute name="EvictionPolicyClass"></attribute>
</mbean>


</server>

Remove the TransactionManagerLookupClass attribute. Hibernate sets its own adaptor. Make sure you set hibernate.transaction.manager_lookup_class.

Hello Gavin,

I'm getting the same error after removing the attribute. When you say "Make sure you set hibernate.transaction.manager_lookup_class." I assume you mean in the MBean configuration which I am doing.

I'm still using Hibernate 2.1.2, is this file compatible with that release? I couldn't find an example of this file in the 2.1.2 distribution that I have so I copied it for 2.1.4 release.

Thanks,

Rich