Hey all,


I have about 100.000 objects that need to be saved in a postgresdb. Each of those objects contains an array with by average 2 other classes. So in total 300.000 objects are saved to the database.

my code is simple, i have a for loop running over the 100.000 objects and for each object i do a session.save(object);
After saving those objects i do a session.flush(); and session.clear();

The problem is when doing the flush it never stops. I have ran the insert multiple times and after a while it just seems to keep on inserting :-).
The method call never ends.

My memory augments and augments.

When i do my flush and clear after 10.000 objects it does commit, but starts committing the second batch off 20.000 objects and keeps on committing there.

2 questions now, how can i debug my application and find out where it hangs.
And how do i make sure it clears completely the session so my memory won't keep on augmenting.

thank you

nobody?

i know it's possible to use hibernate with + 1.000.000 objects, so there must be a way

these are my properties:

<property name="hibernate.connection.pool_size">20</property>
<property name="hibernate.jdbc.fetch_size">500</property>
<property name="hibernate.jdbc.batch_size">20</property>
<property name="hibernate.jdbc.use_get_generated_keys">true</property>
<property name="hibernate.cache.use_query_cache">true</property>
<property name="hibernate.use_outer_join">true</property>
<property name="hibernate.max_fetch_depth">1</property>

You can try:
1) divide your 100.000 objects saves in many transactions (starttransaction; 100 objects saves,commit, session.evict)
one transaction for 100.000 objects saves is not good for real database use.

see:
http://www.hibernate.org/hib_docs/reference/en/html_single/#performance-sessioncache
2)

Add something like this in your loop:



There is a lot of information about this in the forums.

I've done what pojo said, and putted that in my loop.
Still after 10000 it really slows down.

Other idea's?

sorry yyop, not pojo :-)


Ok i've been timing my different statements.
The commit itself is very fast, it's the flush always that slows down.
I've set it up that it should flush every 100 classes and commit every 1000.
Flushmode is set to COMMIT.

Is there a way to speed up the flush?


thanks