-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 10 posts ] 
Author Message
 Post subject: High-performance deleting of many entities
PostPosted: Fri Apr 03, 2009 8:31 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
Hi,
I am searching for a high-performance deleting of many (up to 5.000-10.000) entities. I have tries the following:

1. Usage EntityManager.merge(Entity) & EntityManager.remove(Entity)

Problem:
- The performance is very slow.
- High performance problems with EntityListener

------------------------------------------------------------------------

2. Usage of "WHERE id IN (:ID_LIST)" clause

Example: DELETE FROM a WHERE a.b_id in ([static_list_of_ids])

Problem:
- Problems (Exception) if to many ids

------------------------------------------------------------------------

3. Usage of native DELETE Query (one query for each entity, and each related entity)

Example: Contact with two Address
DELETE from Contact c WHERE c.id=:ID
DELETE from Address a WHERE a.id=:ID
DELETE from Address a WHERE a.id=:ID

Problem:
- 3 queries for each contact
- EntityListener will not informed (low prio)

Someone know a high-performance way to delete many entities?
My vision is to collect all ids to be delete and delete them in one query. Is this possible without the "WHERE IN" clause?

Thanks and best regards
QStorm

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 8:41 am 
Expert
Expert

Joined: Fri Aug 19, 2005 2:11 pm
Posts: 628
Location: Cincinnati
Don't use hibernate or your other entity manager. Use a native JDBC access method. Evict/null out the objects from your entity manager.

for the where in(xx,xx) clause, divide it up into batches of a size where it throws no exception.

_________________
Chris

If you were at work doing this voluntarily, imagine what you'd want to see to answer a question.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 10:19 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
How to evict an entity from a EntityManager without using clear() or close() which have effect to all contained entities?

Thanks
QStorm

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 06, 2009 2:19 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
I have tried EntityManager.remove() to remove an already deleted entity from the manager. But is that the way?

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 07, 2009 10:43 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
Have nobody a solution for my last issue?

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 07, 2009 11:04 am 
Expert
Expert

Joined: Fri Aug 19, 2005 2:11 pm
Posts: 628
Location: Cincinnati
Sorry. I'm not all too familiar with the EntityManager behavior.

_________________
Chris

If you were at work doing this voluntarily, imagine what you'd want to see to answer a question.


Top
 Profile  
 
 Post subject: Re: High-performance deleting of many entities
PostPosted: Tue Aug 11, 2009 10:40 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
Any news to this issue? I have a bad performance lack, after 10.000 entities has been deleted. It seems to be that the objects are still in the memory (entity manger) although there are removed via EntityManager.remove(..). Or it could be that the internal remove operation will be perform on EntityManger.close(), but some of my manager must not close.

Did someone knows an other way to remove entities, which are deleted via a native statement, from the manager?

Thanks

Best regards
QStorm

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject: Re: High-performance deleting of many entities
PostPosted: Tue Aug 11, 2009 10:53 pm 
Newbie

Joined: Tue Aug 11, 2009 10:49 pm
Posts: 2
?????????

_________________
logiciel emailing


Top
 Profile  
 
 Post subject: Re: High-performance deleting of many entities
PostPosted: Wed Aug 12, 2009 2:23 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
Julia2009 wrote:
?????????

?!?

OK I can write again about the issue:
I will remove a Set of entities (maybe 10.000) from the database. The EntityManager.remove(entity) method is to slow for this operation. So I delete the entities via native SQL from the database (DELETE from Contact c WHERE c.id=:ID). My question is now: How can I remove the native deleted entities from all EntityManager (in a high-performance way)?

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
 Post subject: Re: High-performance deleting of many entities
PostPosted: Wed Sep 09, 2009 11:13 am 
Regular
Regular

Joined: Thu Jun 08, 2006 5:32 pm
Posts: 52
Here is the answer to this issue: viewtopic.php?f=1&t=998914

_________________
http://nocxsville.myminicity.com/


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 10 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.