-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: Slow batch insertion on InnoDB
PostPosted: Wed Sep 19, 2012 8:42 pm 
Newbie

Joined: Wed Sep 19, 2012 8:33 pm
Posts: 4
Hi.
I am new in Hibernate. First of all, thank you for shareing such a great tool.
Here is my problem.
I am using hibernate with MySql 5.5 with InnoDB engine.

I made a test for batch insert like this:
Code:
Session session = HibernateUtil.getSessionFactory().openSession();
      Transaction tx = session.beginTransaction();
      
      long now = System.currentTimeMillis();
      for(int i=0 ; i < 100000 ; i++)
      {
         LandTrack lt = new LandTrack();
         Point p = new Point();
         p.setId((i % 6000)+1);
         lt.setPoint(p);
         LandTrackId ltid = new LandTrackId();
         ltid.setId(i);
         ltid.setOrder(0);
         lt.setId(ltid);
         
         session.save(lt);
         
         
         if ( i % 20 == 0 ) { //20, same as the JDBC batch size
              //flush a batch of inserts and release memory:
              session.flush();
              session.clear();
          }
      }
      tx.commit();
      session.close();
      
      long stop = System.currentTimeMillis();
      double t = (double)((double)(stop-now)/(double)1000);
      System.out.print("\n time: " + t);


The time result result was astounding - 35 seconds.
For 100 000 rows, 35 seconds is too much. How can I improve this time ?
I didn't configured MySql, it is just a fresh installation.
My hibernate.cfg.xml file :

Code:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
    <session-factory name="SessionFactory">
        <property name="hibernate.bytecode.use_reflection_optimizer">false</property>
        <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
        <property name="hibernate.connection.password">root</property>
        <property name="hibernate.connection.url">jdbc:mysql://localhost/mydb</property>
        <property name="hibernate.connection.username">root</property>
        <property name="hibernate.current_session_context_class">thread</property>
        <property name="hbm2ddl.auto">create</property>
       
        <property name="hibernate.dialect">org.hibernate.dialect.MySQL5Dialect</property>

        <property name="hibernate.jndi.class">com.sun.jndi.fscontext.RefFSContextFactory</property>
        <property name="hibernate.jndi.url">file:/</property>
        <property name="hibernate.search.autoregister_listeners">false</property>
        <property name="hibernate.jdbc.batch_size">20</property>
        <property name="hibernate.cache.use_second_level_cache">false</property>

        ...
<!-- mapping resources -->
...
    </session-factory>
</hibernate-configuration>



Top
 Profile  
 
 Post subject: Re: Slow batch insertion on InnoDB
PostPosted: Thu Sep 20, 2012 7:15 am 
Newbie

Joined: Wed Sep 19, 2012 8:33 pm
Posts: 4
I tried inserting without batch:
Code:
      Session session = HibernateUtil.getSessionFactory().openSession();
      Transaction tx = session.beginTransaction();
      
      long now = System.currentTimeMillis();
      for(int i=0 ; i < 100000 ; i++)
      {
         LandTrack lt = new LandTrack();
         Point p = new Point();
         p.setId((i % 6000)+1);
         lt.setPoint(p);
         LandTrackId ltid = new LandTrackId();
         ltid.setId(i);
         ltid.setOrder(0);
         lt.setId(ltid);
         
         session.save(p);


      }
      tx.commit();
      session.close();
      
      long stop = System.currentTimeMillis();
      double t = (double)((double)(stop-now)/(double)1000);
      System.out.print("\n" + t);


The result is same : 35 second. What I'm doing wrong ? I am using hibernate 3.2.


Top
 Profile  
 
 Post subject: Re: Slow batch insertion on InnoDB
PostPosted: Thu Sep 20, 2012 7:42 am 
Newbie

Joined: Wed Sep 19, 2012 8:33 pm
Posts: 4
I have downloaded hibernate 4.1.7 release and from now batch insert for loop in the very first post at top, lasts 20 sek.
But i think it is still to slow. On another page i reed, that someone achived 400K/sec. insertion on InnoDB.
My computer specification : 2 GB ram , core 2 duo 2300 mhz.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.