I'm fairly new to Hibernate and I am having some performance issue on inserts into the database.

I have a couple of questions on how Hibernate does inserts into the database.

so lets say I have the following piece of code.

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<1000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
}
tx.commit();
session.close();

Will there be 1000 individual insert statements or will it make batch up the inserts into 1 call?

If I switch the code to look like this.

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

for ( int i=0; i<100000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
if ( i % 20 == 0 ) { //20, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}

How are session.flush() and session.clear() suppose to give me a performance boost? Does the session.flush() do the inserts as 1 statement or does it do all 20 inserts as 1 call to the database?

Is there a better way to do batch processing of several records?



tx.commit();
session.close();

Can you describe the perfomance issue? Is the database slowing down over time? The first insert speedy and the next n are progressively slower.

Hi,


That depends....
If you are using a databas-located identifier-generator (like identity) there would be no way for hibernate to do batch-inserts, so there will be 1000 single inserts. If your identity-generation happens without use of the database and batch-inserting is supported hibernate will do batch-inserts according to the value of hibernate.jdbc.batch_size. E.g. batch_size=200 means 5 inserts with 200 customers each.


As for the batch-inserts: it's just the same as without the flush()/clear().
Flushing/clearing does not necessarily improve performance - espacially not when batch-inserting is used. What it does is that it keeps the size of the hibernate-session within reasonable bounds. If you do save 100000 Customer-instances without flushing you're sure to experience a dramatic slowdown because of increasing garbage-collection-activity in the JVM before your application dies with an OutOfMemoryError: Without the clear() the saved instances can't be garbage-collected because the hibernate persistency-context is still holding on to them.

To find the best interval-size in which to do the flush/clear-thing does heavily depend on your application, so the only way to find out the best values is to try.

Regards

piet

Ok so I just ran a simple test with a simple DAO where I inserted 10,000 records.
First test took 37 seconds.

I then modified the hibernate configuration file by adding the following:
<property name="hibernate.jdbc.batch_size">30</property>
<property name="hibernate.order_inserts">true</property>
<property name="hibernate.order_updates">true</property>

and add the following code to my DAO

if(i %20 == 0)
{
session.flush();
session.clear();
}
and the ran inserted another 10,000 records and that test took 51 seconds.

I do see things in the logs like

- Executing batch size: 20
- success of batch update unknown: 0
- success of batch update unknown: 1
- success of batch update unknown: 2
- success of batch update unknown: 3
- success of batch update unknown: 4
- success of batch update unknown: 5
- success of batch update unknown: 6
- success of batch update unknown: 7
- success of batch update unknown: 8
- success of batch update unknown: 9
- success of batch update unknown: 10
- success of batch update unknown: 11
- success of batch update unknown: 12
- success of batch update unknown: 13
- success of batch update unknown: 14
- success of batch update unknown: 15
- success of batch update unknown: 16
- success of batch update unknown: 17
- success of batch update unknown: 18
- success of batch update unknown: 19

Also what exactly do the following entries in the logs mean?

- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020635
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020636
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020637
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020638
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020639
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020640
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020641
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020642
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020643
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020644
- Invalidating space [NMRW.FOO_TXN], timestamp: 5098157369020645

Oh yeah and piet.t I believe the identifier-generator is not database-located.