These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 
Author Message
 Post subject: Performance problem on mass delete
PostPosted: Tue Jul 19, 2011 3:59 am 
Beginner
Beginner

Joined: Fri Nov 14, 2008 7:11 am
Posts: 31
We have a table with about 10 millions of rows and we need to delete a large number of rows (about 1 million) with a delete statement like

Code:
delete from table where someid = ?


We are doing this (using Spring) via

Code:
final String queryString = "delete from MyEntity e where e.anotherEntity.id = ?";
getHibernateTemplate().bulkUpdate(queryString, anId);


but the delete still takes much too long (a few minutes).

Doing some research in DB forums the reason seems to be the DB transaction log which would become very big with such a mass delete statement. Their advice is to split the mass delete into many smaller transactions. Is there any way to let Hibernate do so automatically or do I have to write my own programmatic transactions? Till now I use transactions by annotation.

Thanks in advance,
Ole


Top
 Profile  
 
 Post subject: Re: Performance problem on mass delete
PostPosted: Wed Dec 21, 2011 4:33 am 
Beginner
Beginner

Joined: Fri Nov 14, 2008 7:11 am
Posts: 31
As this problem has popped up again I'd like to kindly ask for any help again.


Top
 Profile  
 
 Post subject: Re: Performance problem on mass delete
PostPosted: Wed Dec 21, 2011 5:38 am 
Senior
Senior

Joined: Tue Oct 28, 2008 10:39 am
Posts: 196
Did you trace the statement that is being executed? Is there an index on the column "someid"? Is your DB perfoming a full table scan or does it use some other index that's not suitable you that statement? What does EXPLAIN PLAN (or whatever you DB uses) tell you about the statement?
Is there only one statement or does the spring template execute more statements?


Top
 Profile  
 
 Post subject: Re: Performance problem on mass delete
PostPosted: Thu Dec 22, 2011 9:52 am 
Beginner
Beginner

Joined: Fri Nov 14, 2008 7:11 am
Posts: 31
CDillinger wrote:
Did you trace the statement that is being executed?

Yes, the statement itself is fast.

CDillinger wrote:
Is there an index on the column "someid"?

There is an index for that column.

CDillinger wrote:
Is your DB perfoming a full table scan or does it use some other index that's not suitable you that statement?

As there is an index for the column that one is used.

CDillinger wrote:
What does EXPLAIN PLAN (or whatever you DB uses) tell you about the statement?

That the statement is fast.

CDillinger wrote:
Is there only one statement or does the spring template execute more statements?

It is only one statement.

The problem is not the statement itself. It is very slow because of the transaction log that has to be written for 1 million rows.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.