how to handle large table in Hibernate?

yangleibj · **Posted:** Sat May 08, 2004 3:19 am

hi,
when i insert or retrieve some record which is very large ,the exception will be generated java.lang.OutOfMemoryError,why?the two table is parent/child relationship,and i set the lazy=false
the code is following:
JournalEntryDAO dao = new JournalEntryDAO(
"companyb");
String batchNo = "batchNo";
JournalBatch journalBatch = dao.retrieve(batchNo);
if (journalBatch != null) {
System.out.println(journalBatch.getBatchNo());
dao.deleteBatch(batchNo);
}
journalBatch = new JournalBatch();
journalBatch.setBatchNo(batchNo);
Collection journalEntries = new ArrayList();
for(int i=0;i<50000;i++)
{
String referenceNo1 = "referenceNo"+i;
JournalEntry journalEntry1 = new JournalEntry();
journalEntry1.setReferenceNo(referenceNo1);
journalEntry1.setBatchNo(journalBatch);
journalEntries.add(journalEntry1);
}

journalBatch.setJournalEntries(journalEntries);
dao.create(journalBatch);
JournalBatch batchLoad=dao.retrieve(batchNo);
System.out.println(((JournalEntry)batchLoad.getJournalEntries().toArray()[9998]).getReferenceNo());
the hbm file is following:the parent
<bag name="journalEntries" inverse="true" cascade="all">
<key>
<column name="batchNo"/>
</key>
<one-to-many class="JournalEntry"/>
</bag>
the child<many-to-one
name="batchNo"
column="batchNo"
class="JournalBatch"
not-null="true"
>
<column name="batchNo" />
</many-to-one>
Thanks in advance for all the help.

baliukas · **Posted:** Sat May 08, 2004 4:33 am

"System.out.println(((JournalEntry)batchLoad.getJournalEntries().toArray()[9998]).getReferenceNo());"
Are you sure this is not the cause of OutOfMemoryError ?
Why to you load database to memory if you need just "referenceNo" at 9998 ?

yangleibj · **Posted:** Sat May 08, 2004 4:41 am

Thanks for your reply.
i am sure it is not caused by the the sentence,for when i comment it ,the same to uncomment.
this is just test code,for in production,the system will need such a large data insert and retrieve.
what is the reason of this? is it caused by the lazy=false?why?can anybody help me?Thanks.

baliukas wrote:

"System.out.println(((JournalEntry)batchLoad.getJournalEntries().toArray()[9998]).getReferenceNo());"
Are you sure this is not the cause of OutOfMemoryError ?
Why to you load database to memory if you need just "referenceNo" at 9998 ?

baliukas · **Posted:** Sat May 08, 2004 6:18 am

If you do not need data in memory then load it "lazy".

yangleibj · **Posted:** Sat May 08, 2004 8:35 pm

hi,baliukas
but i need it,this will cause OutOfMemory exception?can i make it work?
can u help me?
Thanks for your kindness.

yangleibj · **Posted:** Sun May 09, 2004 9:54 pm

can anyone help me?[b][/b]

emmanuel · **Posted:** Mon May 10, 2004 12:22 pm

Check the size of every single object (and linked one wo lazy) * 50000, how much do you have ?

yangleibj · **Posted:** Wed May 12, 2004 5:08 am

hi,emmanuel
thanks for your reply.
but i don't konw how to estimate the size of the object,may u tell me?
Thanks.

yangleibj · **Posted:** Wed May 12, 2004 5:19 am

and how do u think about this situation? what caused this?
thanks.

yangleibj · **Posted:** Wed May 12, 2004 5:33 am

there is no "standard way" to handle large table with hibernate ?
such as retrieve 100k rows records from one table.
Thanks.

christian · **Posted:** Wed May 12, 2004 5:39 am

Why would you load your database into memory? Use a stored procedure.

yangleibj · **Posted:** Wed May 12, 2004 5:45 am

hi,christian
thanks for your reply.
what is the standard way of hibernate to deal with this situation?
i mean ,can i use hibernate to deal with the situation?
if not,what is the better way?
Thanks

christian · **Posted:** Wed May 12, 2004 5:46 am

You should not load your database into memory and therefore, not use Hibernate to load and save millions of objects. You implement a Stored Procedure in the database and call it, using straight SQL and JDBC.

yangleibj · **Posted:** Wed May 12, 2004 8:53 pm

hi ,christian
Thanks for your help.
YangLei

yangleibj · **Posted:** Wed May 12, 2004 10:11 pm

hi ,christian
can u tell me what is the max rows Hibernate allowed?
thanks