-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: Faster way to Query and divide to bulks
PostPosted: Thu Jan 03, 2008 11:15 am 
Beginner
Beginner

Joined: Tue Jul 03, 2007 11:11 am
Posts: 22
Hi all
I got Hql string which contain query that return List<String>,
Each String is the identifier row METADATA table.

Quote:
SELECT md.id FROM MetaData as md WHERE md.metadataType ="X";


METADATA table description:

Quote:
MetaData tabe :
String id, String metadataType , Date metadataChangeDate , String MetaDataText


The HQL query may result in thousands to several million records of identifiers ids.
After I get the List of metadata ids I divide them to 1000 and insert it to another table (bulk table) for later processing.

Performance and speed is the main issue here,
The question is should I get all records at once using:
(Pseudo code)
Code:
Query query = getEntityManager().createNamedQuery("SelectTypeXFromMetaData");
query.setParameter(x,'x');
List<String>  =  query.list ()

Loop on the list:
   Create 1000 bulks
   Insert into the bulk table


Or it will be faster to tell the query to divide the result 1000 each time?
Example:

Code:
Query query = getEntityManager().createNamedQuery("SelectTypeXFromMetaData");
query.setParameter(x,'x');

while (list contain items){
  query.setFirstResult(start);
  query.setMaxResults(rowNum);
  List<String> =  query.list ()
   Insert into the bulk table
}



Another alternative is JDBC or oracle stored proc,

Please advise
Thank you very much!
Sharon


Top
 Profile  
 
 Post subject: massive select/insert
PostPosted: Thu Jan 03, 2008 12:01 pm 
Expert
Expert

Joined: Mon Nov 26, 2007 2:29 pm
Posts: 443
Hibernate discourages the use of stored procedures for anything but "SELECT" operations, because whatever is inserted or deleted in the tables without Hibernate knowing, results in the objects in the session being outdated.
So you can discard the stored procedure solution.

Using the firstResult/maxResult alternative seems reasonable,that's what it is for. Remember that once you call ".list()" you are forcing your application to retrieve and store all those results in memory, so whatever you do in order to reduce the size of those resultsets is good.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.