-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 8 posts ] 
Author Message
 Post subject: problem inserting big number off objects
PostPosted: Fri May 27, 2005 8:33 am 
Beginner
Beginner

Joined: Thu Sep 16, 2004 4:36 am
Posts: 22
When i do a commit after saving 100.000 objects that has an avarage off a list with 2 object ( meaning avarage 100.000 Parent objects, 200.000 joined-subclasses) my inserts go ok, but afterwards it updates the subclass to put the correct parent id in it. How it should be thus.
But it never stops with sending the update queries.

The application never inserts into the database (meaning the commit never finishes) because it keeps on updating the subclass.

mapping:
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 2.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">
<hibernate-mapping auto-import="false" >
<class name="A"
table="A"
dynamic-update="false"
dynamic-insert="false"
select-before-update="false"
>

<id
name="idInternal"
column="idInternal"
type="long"
unsaved-value="-1"
>
<generator class="sequence">
<param name="sequence">A_sequence</param></generator>
</id>
<some properties here>
<list
name="BInternal"
table="A_B"
lazy="false"
inverse="false"
cascade="all-delete-orphan"
where="B_index is not null"
>

<key
column="A_id"
>

</key>

<index
column="B_index"
/>

<one-to-many
class="B"
/>
</list>
</class>
</hibernate-mapping>

class B:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 2.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">

<hibernate-mapping
auto-import="false"
>
<class
name="B"
table="B"
dynamic-update="false"
dynamic-insert="false"
select-before-update="false"
>

<id
name="idInternal"
column="idInternal"
type="long"
unsaved-value="-1"
>
<generator class="sequence">
<param name="sequence">B_sequence</param> </generator>

</id>
<some properties here>
</class>
</hibernate-mapping>


code:
FlushMode.COMMIT
tx = session.beginTransAction()
for loop containing:
session.save(A);
end for loop

tx.commit(); //this keeps on going for mmore then 2 hours and a half before i kill the app.

I use EhCache, outer join set to false autocommit=false;

any input welcome.

thanks


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 27, 2005 9:46 am 
Regular
Regular

Joined: Tue May 24, 2005 10:19 am
Posts: 65
see:http://forum.hibernate.org/viewtopic.php?t=942937&highlight=


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 27, 2005 2:05 pm 
Beginner
Beginner

Joined: Thu Sep 16, 2004 4:36 am
Posts: 22
zesj i know but doing that it fails after a certain time also, and keeps slowing down...


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 27, 2005 2:37 pm 
Regular
Regular

Joined: Tue May 24, 2005 10:19 am
Posts: 65
put more info... code, database, sql...


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 30, 2005 5:48 am 
Beginner
Beginner

Joined: Thu Sep 16, 2004 4:36 am
Posts: 22
code is simple and attached to first post

SQL i can't give (employee doesn't allow)
database is pgsql


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 30, 2005 9:01 am 
Regular
Regular

Joined: Tue May 24, 2005 10:19 am
Posts: 65
repeated: You can try:
1) divide your 100.000 objects saves in many transactions (starttransaction; 100 objects saves,commit, session.evict)
one transaction for 100.000 objects saves is not good for real database use.

2)you try call gc to get memory.
3)see:
http://www.hibernate.org/hib_docs/refer ... ssioncache


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 30, 2005 10:24 am 
Beginner
Beginner

Joined: Thu Sep 16, 2004 4:36 am
Posts: 22
found the problem
if i use a uuid.hex generator (that isn't done by the database) 10000 objects inserted takes 59 seconds (cool and fast)
but that means my id's are at least 32 long and in string (what makes my search heavier and my joins to slow).

SO i should find a way to use as id still a long, but generated locally (less database requests)


thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 30, 2005 12:10 pm 
Beginner
Beginner

Joined: Sat May 07, 2005 12:01 pm
Posts: 33
I think your problem is the "sequence" generator, which hits the db for each ID. Try the "hilo" generator, which fetches a block of ID numbers, then updates them in memory until it runs out and gets the next batch from the db (32,000+ is the default I think, so it should be much faster).


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 8 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.