We have a complex structure of Document object with collections and a lot of sub elements.
One of these elements is call Feature.
When we need to insert a Feature, we need to do a query first to verify a unicity constraint, then insert it if not already exist.

PS: We could not used a composite id for Feature because of max length of key

The performance problem is that when we do this query , the session call autoFlushIfRequired() that takes lot of time.
It takes a lot of time because it check all object in session to know if they are dirty, and there are a lot of objects in the session (more than 1000)

Time with a lot of queries is 52 seconds, and autoFlushIfRequired() takes 50 seconds

If I set flush mode to NEVER, it is much more faster, but the query did not works because object are not inserted

Any ideas ??

Thanks a lot for your help

Hibernate version:3.0.5

Yes, we have the same problem. Our session has a large number of objects that can't be evicted (one strategy for dealing with this problem) as they are all taking part in the calculation happening in this particular transaction. However due to the use of HQL finders in this calculation, many auto-flushes are occuring that are eating up 95% of the time that this calculation takes! Unfortunately we can't turn off the flushing as this calculation also modifies that data.

What is particularly frustrating is that our system keeps track of exactly what pojos/attributes are dirty in the transaction so we have the information to hand to tell Hibernate exactly what to flush but there appears to be no way to tell it this information to improve the dreadful performance of the flushes.

I spent some time implementing some custom flush listeners to replace the Hibernate implementation but their default versions (that I based my versions on) appear to do other collection/pojo related initialisation that assumes that ALL collection/pojo entries will be proccessed in each flush - which they're not in my version as I am only flushing those collections/pojos that have changed. This caused sporadic instability with hibernate collections which was unfortunate as the performance increase was significant. We eventually rolled back our changes and are stripping out HQL finder calls and replacing them with programmatic model navigation to avoid the auto flushes. This seems the only way to improve the performance in this case (if not the memory consumption due to the cache size).

Talk about doing things the hard way.

implement Interceptor.findDirty()

Or just call flush() manually when you know you need to.

I quite agree, it was the hard way - that flushing code is rather complex! :) Okay, we already have implemented the findDirty() interceptor although the profiling that we did indicated that this wasn't the hotspot (although it did make a small difference).

That findDirty however only returns the attributes on the pojo specified as a parameter that are dirty, what we really wanted was a findDirty that we could implement that would provide Hibernate with ALL the pojos / parameters that were dirty in one hit. That would remove all of the loops in the Hibernate flush code that have to hit hit every single pojo and collection. It was in this general processing that the time was taken, presumably as we have a very large & highly connected session cache at this point.

Mind you, I am assuming that its not as simple as I put it above,

a) because no such API exists in Hibernate, and if it was that straight-forward I would imagine it would be there.
&
b) because something else is happening in this flush code as when I stopped Hibernate iterating over all pojos/collections things started started to misbehave.

As for manually calling flush()... due to the nature of the calculations, stale data and the finders we wish to use, we need to flush... We want these flushes to happen its just that they take too long.