Hi,
we have some problem when rolling back XA transactions (we use XA enabled connection pools) using Hibernate in Weblogic 8.1 server.

Hibernate version: 3.0.5
Weblogic version: 8.1 SP 4
Database: Oracle 9.2.0.6.0

We've found the problem during the performance tests. Under certain load conditions transaction rollbacks do not work correctly anymore. Weblogic behaves quite strangely. It starts to throw XA_PROTO exceptions when the application tries to start a transaction. The error comes only after some time after the tests start. And when it comes, then nothing works anymore. After analysing the logs together with BEA support we found out, that the database connections were left in some not consistent state, which does not allow their further usage.
The problem is in Hibernate. We wrote a small CMT session bean, which uses Hibernate to access the database and reproduced the same error using the load test.
So, so far so good. After reading some docus, it turned out that new transaction factory CMTTransactionFactory was added to the Hibernate 3.0 release (we developed using Beta and then switched to the release, so I do not know whether it was 3.0.1, 3.0.2 ...). So with CMTTransactionFactory everything works fine.
The problem is only that we can't replace JTATransactionFactory with CMTTransactionFactory, because we have some places where transactions are started manually (BMT). And they do not work with CMTTransactionFactory for some reasons (for example NullPointerException in CMTTransactionFactory).
My question is, what does JTATransactionFactory different from CMTTransactionFactory and why was CMTTransactionFactory created? Did anybody has the same problems? Does JTATransactionFactory work only with XA wrong or there is some more general issue?
I'm thinking about the possible solutions of our problem and one of them were to customize JTA-/CMTTransactionFactory, not sure whether it makes sense.

Sergiy

CMTTransactionFactory always assume a Tx will be started by the container,
JTATransactionFactory will start one an associate it to the JTA context if none were associated, or use the existent one if it exists already in the JTA context.

never forget to flush and close the session unless you're using hibernate.transaction.flush_before_completion and/or auto_close_session in a CMT environment.

In your case it might be possible that you don't correctly use hibernateTx.begin();
hibernateTx.commit();
in your JTA tx.

In non XA environment we did not see any difference between JTATransactionFactory and CMTTransactionFactory. So I assume they do nearly the same things, just CMTTransactionFactory does a little bit less then JTATransactionFactory and can be used only withing CMT beans. It works only if a transaction already exists. JTATransactionFactory works always if JTA Transaction Manager is present. It starts a new transaction or reuses the existing one. And I though it does not metter who has started the transaction: container or somebody using UserTransaction. So your explanation Emmanuel does not satisfy me :(.
As to commit: we do everything correct. As I already said, the problem comes only when using XA transactions and under the load test with the high rate of rollbacks. Single rollbacks work fine. Rollbacks of non-XA transactions work fine as well. But with multiple concurrent rollbacks something goes wrong. To reproduce the error, just program a CMT session bean with one method with the trx attribute "required" and use Hibernate session and transaction to access a test db table. Additionally, the connection pools must be XA-enabled. Then use JUnitPerf to program a load test with for example 15 concurrent threads.
It does not work. It fails. It fails, but it fails after several minutes of normal behavior. Weblogic starts to use database connections concurrently in different threads with different XA transactions (which is actually quite legal for XA case - one connection can take part in several XA transactions) mixing up connection states somehow and coming to XA_PROTO exception at the end.
It would be good to hear from somebody, why was CMTTransactionFactory added to the release and why JTATransactionFactory is not enough for managed environments? Why does the reference document suggest to use CMTTransactionFactory for CMT beans and JTATransactionFactory for BMT?

Sergiy

hug, I explained the exact same thing!

Great thread :)

To be honest this :
- really sound like a bug in the connection pool handling of Weblogic (a connection pool should not leave Tx in inconsistent state anyway)
- or sounds like your code share the same connectino accross 2 different threads, witch is absolutly forbidden by the JTA spec.

Look at the code, there is no fondamental difference between CMTTxFactory and JTATxFactory when the Tx boundaries are managed by the container. CMTTxFactory has been added to workaround some bogus app servers Tx Managers.

If it's a bug in the Weblogic code/our application, then why does everything work with CMTTxFactory? We just replace JTATxFactory with CMTTxFactory, comment begin/commit/rollback Hibernate transaction and use getCurrentSession and everything works!
Here is the code, which produces the error (session bean method throwing an exception) with JTATxFactory. Calling this method with 15 threads during several minutes produces the error for sure. Tell me, is there something wrong?

... and there is a fundamental difference between JTATxFactory and CMTTxFactory. The last one does not use UserTransaction. And its rollback, wasRolledBack, wasCommited method throw NullPointerException, when trying to call them (javax.transaction.Transaction transaction is not initialized in CMTTransaction class). So I assume that CMTTransaction is not supposed to be called to rollback/commit transactions. So, is there really no fundamental difference?

it seems that we found a possible problem source: session.connection(). If we call session.connection() and after that do trx rollback, it leads to our XAER_PROTO problem.
In the Hibernate documentation is stated that one has to do connection.rollback/commit, when using session.connection(). But in case of XA transactions it's not allowed resulting in: Cannot call rollback when using distributed transactions.
So it seems that we can't do anything with the connection. The only thing we do is trx.rollback() and session.close().
What is the right way to deal with plain JDBC connections obtained from the Hibernate Session?

All above works assuming that the following is configured in the hibernate config xml:

The Hibernate documentation certainly does not say you have to commit/rollback a JDBC connection if you access it through Hibernate.

Most certainly not!

really? Here is the excerpt from the Hibernate 3.0.5 API for Session.connection() method:

It says "...commit/rollback upon the connection". I understood it as if I have to call connection.rollback/commit. Am I wrong? Did I get it wrong? Frankly saying, I agree that it does not make sense to do those rollback/commit, but it's stated in the documentation. Before we got the XAER_PROTO problem we did not do any rollback/commit upon connections. But because of the problem we had to dig all that stuff. But anyway it does not metter actually what the documentatio says. I admit that Hibernate is an open source tool and I do not require (I can't actually require anything in this case :) ) commercial level documentation. It's enough that the Hibernate is already a very very good FREE tool. So instead of pointing me that "the Hibernate documentation certainly does not say you have to commit/rollback a JDBC connection" (which assumes that I have not read any documentation before posting the question) it would be enough to say that one should not call rollback/commit on the connection.
Anyway, I have now another question:
there is "hibernate.connection.release_mode" property in the hibernate configuration. The documentation :) suggests to have it set to "after_statement" in JTA environment. According to the source this setting triggers connection close after every JDBC statement execution (and not at the end of the transaction or session close). Why is this suggested? Why is it so important to release a connection after a JDBC statement execution in a JTA environment? This setting helps us to handle our problem partially. And I would like to understand the reasons for having this setting. This would help us enormously.

Hi,
just wanted to update my thread. We solved the problem recently. Setting release_mode to auto (after_statement) helped. Places where we used session.connection() to obtain and to work with the connection were corrected to close the connection using ConnectionManager.afterStatement() method, which we made accessible through SessionImpl class (Session interface) having changed the source. Unfortunatly that was the only way we knew to let the Hibernate know that we want to close the connection. Direct connection.close() is not captured by Hibernate and it tries to reuse the connection, when one tries to issue any HQL queries.
Setting release_mode to auto solved our problem for all those cases, where we used Hibernate HQL or Session methods to access the database. But the cases, where we played with the connection manually obtaining it through session.connection() still produced XAER_PROTO error. We thought that we had to do the same thing that Hiberate does with release_mode=auto - releases connection after a statement execution. So after having worked with the connection obtained from Hibernate we call session.forceReleaseConnection method, which we added to Session interface and SessionImpl class and which delegated the call to getConnectionManager().afterStatement(). And the error disappeared.
I do not know, what should we do with the extension we did to the Hibernate source. Whether it was a bug or known issue. Whether the thing we did is right or there are other better solution. Unfortunatly I did not get any reasoable answer in this thread and did not see that anybody was interested in our problem.

Best wishes,
Sergiy