-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: Performance Optimization for large data set
PostPosted: Tue May 15, 2012 12:52 pm 
Newbie

Joined: Tue May 15, 2012 12:01 pm
Posts: 1
Hi,

I'm new to Hibernate so I apologize in advance for any knowledge/vocab gaps.

I'm running a bunch of reports that require a rather large data set retrieved from the database. The first report retrieves ~500,000 items from the database, the next about 300k, the last about 300k. I'm setting all collections of objects to null immediately after use (prior to the next retrieval of data).

The first time I execute my Hibernate query, it takes about 40 sec; the next about 2-3 min, the following takes maybe 5 minutes, and then the last query (for the last report) takes a very long time, about 1-2 hrs. When I run this report by itself, it runs in about 5 minutes.

Does anyone know what the issue might be and how I can fix this?

Thank you in advance.


Top
 Profile  
 
 Post subject: Re: Performance Optimization for large data set
PostPosted: Wed May 16, 2012 2:28 am 
Expert
Expert

Joined: Wed Mar 03, 2004 6:35 am
Posts: 1240
Location: Lund, Sweden
My first guess is that when you get to the third report there are lots and lots of already loaded entities in the first-level cache of the Hibernate Session object. Then, when you run the third report Hibernate will do it's automatic dirty-checking "magic" which can take a long time if the number of objects are large. There are a few possible solutions to this:

1. Use a new Session for each report
2. Call Session.clear() before each report to get rid of everything in the first-level cache
3. Call Session.setFlushMode(FlushMode.MANUAL) to avoid the automatic flushing and dirty-checking


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.