-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 6 posts ] 
Author Message
 Post subject: Bulk Save in Nhibernate
PostPosted: Thu Aug 16, 2007 11:57 am 
Newbie

Joined: Thu Aug 16, 2007 11:45 am
Posts: 3
Hi All,

I have a requirement to persist an entire ILIST to database. The IList which gets populated can have about 10000-20000 objects at a time. Iterating through the Ilist and peristing 1 object can bog down the performance.

Is there any way I can make a bulk save(save an entire ILIST). Does Nhibernate supports this. I did search on google but didnt help..

Let me know if this is feasible. If there is any other alternative do let me know.

Its urgent.. Awaiting replies..

Thanks in advance.


Regards,
Shirish..


Top
 Profile  
 
 Post subject: Re: Bulk Save in Nhibernate
PostPosted: Mon Aug 20, 2007 1:52 am 
Newbie

Joined: Thu Aug 16, 2007 11:45 am
Posts: 3
Hi friends,

I googled the query and find a work around, which is saving objects in a batch. But the limit of the batch is 50-100. I have around 10K-20K records. Is it possible to have a larger batch size, if yes what can be the max batch size I can use without causing out of memory exception.

Also I read somewhere the to get an efficient performance, I need to set the batch size of the driver in Hibernate config file which should match the batch size I am actually using. Where in config file can I specify the batch size and how this relates to performance.

The above is just a workaround, but I still feel there should be a way for persisting entire collection in single Save.

Kindly reply to this post.. Its urgent.. Thnks in advance..

Regards,
Shirish.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Aug 20, 2007 5:30 am 
Regular
Regular

Joined: Tue Aug 08, 2006 4:28 am
Posts: 96
Location: Hong Kong
large volume bulk insert is not what NH strong at. You may try SqlBulkCopy.

Or if you still want NH to do the work, check this out http://www.ayende.com/Blog/archive/2006/09/16/7269.aspx

Well I have not tried any of the two approaches before. Hope they help.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Aug 21, 2007 1:28 am 
Newbie

Joined: Thu Aug 16, 2007 11:45 am
Posts: 3
Thnks Canton...

As the article mentions NH is not suitable for bulk batch inserts. So I need to selectively set the batch size.

Any idea what should be the ideal size of the batch? What parameters should be considered while setting the batch size.

Thanks in advance.


Regards,
Shirish.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Aug 21, 2007 3:57 pm 
Beginner
Beginner

Joined: Fri Jan 12, 2007 1:08 am
Posts: 41
Remember that an ORM is rarely going to be the right answer to every problem that you encounter. IMHO you want to avoid premature optimization and look elsewhere when it becomes clear that a scenario does not fit in with the chosen paradigm. I do not believe that NHibernate is the right answer for this particular requirement.

I'm not sure if you want to perform a bulk INSERT or a bulk UPDATE. Assuming that the answer is INSERT and that the underlying database in SQL Server 2005 the most performant approach that I have found is (and it's very efficient):

Code:
CREATE TABLE t1
(
  c1 int NOT NULL,
  c2 int NOT NULL,
  c3 int NOT NULL,
  c4 varchar (50) NOT NULL,
  c5 varchar (50) NOT NULL,
  c6 varchar(50) NOT NULL
)

public void BulkInsert()
{
  using (SqlCommand cmd = new SqlCommand(@"BULK INSERT t1 FROM 'c:\test_insert.txt'", CONNECTION))
  {
    using (StreamWriter sw = new StreamWriter(@"c:\test_insert.txt"))
    {
      for (int i = 1; i <= LOOP_COUNT; i++)
      {
        sw.WriteLine(i + "\t" + i + "\t" + i + "\t" + i + "\t" + i + "\t" + i);
      }
    }
    cmd.ExecuteNonQuery();
  }
}


Top
 Profile  
 
 Post subject: Re: Bulk Save in Nhibernate
PostPosted: Wed Apr 13, 2011 8:02 am 
Newbie

Joined: Wed Apr 13, 2011 6:59 am
Posts: 2
Location: USA
Using Java, use can get the Connection from Hibernate using the doWork method from Session.
Code:
final String[] insertStms = new String[2000];
int count = 0;
for (Widget w : widgets) {
  insertStms[count] = w.getInsertStmt();
  count++;
  if (count == insertStms.length) {
  session.doWork(new Work() {
   @Override
   public void execute(Connection connection) throws SQLException
   {
   Statement stmt = connection.createStatement();
   for (String sql : insertStms) {
    stmt.addBatch(sql);
   }
   stmt.executeBatch();
   stmt.close();
   }
  });
  count = 0;
   }
  }
}



For more details, go to http://i4hq.com/116-gbulk-inserts-with-hibernate-and-sqlserver


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 6 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.