-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: Bulk insert very slow
PostPosted: Thu Apr 01, 2010 6:57 am 
Newbie

Joined: Thu Apr 01, 2010 6:47 am
Posts: 7
I'm using Postgre with Hibernate but when i try to insert many records in database its very slow.
10.000 inserts takes more than half a minute.
In hibernate.cfg i set up <property name="hibernate.jdbc.batch_size">100</property>
I'm using the following code from the book java persistance with hibernate:

Code:
public static void DDos() {
      Session session = HibernateSessionFactory.getSession();
      Transaction tx = session.beginTransaction();
         for ( int i=0; i<10000; i++ ) {
            User user = new User("bivanise", "bivanise");
            session.save(user);
            if ( i % 100 == 0 ) {
               session.flush();
               session.clear();
         }
      }
      tx.commit();
      session.close();
   }


Top
 Profile  
 
 Post subject: Re: Bulk insert very slow
PostPosted: Thu Apr 01, 2010 7:49 am 
Expert
Expert

Joined: Tue Jun 16, 2009 3:36 am
Posts: 990
JDBC-batch insert is ineffective when using native id generator.
Do you use native id generator on your User entity?


Top
 Profile  
 
 Post subject: Re: Bulk insert very slow
PostPosted: Tue Apr 06, 2010 6:37 am 
Newbie

Joined: Thu Apr 01, 2010 6:47 am
Posts: 7
This is the code from .hbm file

<id name="userId" column="user_id">
<generator class="sequence">
<param name="sequence">user_id_sequence</param>
</generator>
</id>


Top
 Profile  
 
 Post subject: Re: Bulk insert very slow
PostPosted: Wed Apr 07, 2010 5:14 am 
Expert
Expert

Joined: Tue Jun 16, 2009 3:36 am
Posts: 990
In first line I would check with following code if your jdbc-driver is supporting BatchUpdates at all.
(not all drivers do effectively implements this feature).

Code:
Class.forName("org.postgresql.Driver");
java.sql.Connection con = java.sql.DriverManager.getConnection("your_connectionUrl",userName,password);
boolean supportsbatchinserts_updates = con.getMetaData().supportsBatchUpdates());
con.close();


Anyway I guess that the bottleneck is the database itself.
Create some stack with jstack during the half minute the application is running.
Then look at the stacktraces you got.
If the stacks are almost always executing in socketRead or socketWrite,
then it is effectively the database (or a extremely slow network) which is the bottleneck.


Top
 Profile  
 
 Post subject: Re: Bulk insert very slow
PostPosted: Sun Jan 11, 2015 4:31 pm 
Newbie

Joined: Sun Jan 11, 2015 4:28 pm
Posts: 5
Did you try looking at the SQL your code generates? I suspect Hibernate is allocating a sequence id from database before every insert.
Check this link for an explanation: http://korhner.github.io/hibernate/hibernate-performance-traps-part-3/


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.