We have a system that creates and persists a reasonably large number of objects on a daily basis (up to 50,000). When the objects are initially created, they are persisted with the following code:



(All persistence happens on a separate thread to the main application processing.)

When an object is updated, it is first cloned (a requirement of other parts of the system) and the same code as above called to persist the update to the database. The system is responsible for generating it's own Ids therefore the cloned objects have the same Id as the object being managed by the EntityManager.

When commit is called, we are seeing exceptionally high CPU usage; in fact as objects are created and updated quickly, the CPU usage frequently stays at 100%.

Connecting a profiler to the system indicates that the majority of the time is being spent in the method: org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(FlushEvent) which is called automatically by Hibernate following a commit.

I assume (but have been unable to confirm) that rather than simply writing the current change to the database, it is synchronizing all 50k managed objects which would go some way to explaining the high CPU usage.

Has anyone come across any similar issues and/or have any suggestions as to how avoid this problem?

Thanks

Steve

Hi,

please see
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3967

Please try the patch I propose in org.hibernate.engine.Cascade.java
and let me know if it improves the performance.
(if it does, please give a vote to HHH-3967)

thanks
Guenther

Hi,

after reviewing your post, I recognize that you are:
1. committing each of the 50.000 separately (= totally 50.000 commits = 50.000 flushes in commit-phase)
2. I guess you continue to work with the same persistence context without ever calling clear().
That would mean, that your persistent context is gradually increasing until it reaches 50.000 objects.
In summary you perform 50.000 flushes with having the last flushes scanning about 50.000 objects each time.

Solution:

1. Do commit more rarely.
2. Call clear after commit if possible, or detach the objects which you don't need anymore.

Generally:
Always examine if it is really necessary to have all object attached until the commit.
If yes, then avoid the use of flush were possible (consider that queries can call flush implicitly).
Otherwise you could call intermediate flush() + clear() keeping the persistent context always small.