These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: performance issues
PostPosted: Fri Aug 31, 2007 8:56 pm 
Newbie

Joined: Thu Dec 22, 2005 10:21 pm
Posts: 4
I'm trying to persist a collection of ~20000 entities (albums), but it's taking forever (>2 hours, and within an hour generates over 1 GB of logs if I allow it to log the sql). There are a couple mappings, and I tried to set the cascade to "none" and lazy="true" to limit the work it must do, but this didn't seem help:

<property name="Title" column="title" type="string" length="8000" not-null="true" />
<many-to-one name="Artist" class="Artist" column="artist_id" cascade="none" not-null="true" />
<property name="Year" column="year" type="int" not-null="true" />
<property name="Era" column="era" type="string" length="8000" not-null="true" />
<property name="Genre" column="genre" type="string" length="8000" not-null="true" />
<property name="CreatedDate" column="created_date" type="DateTime" not-null="true" />
<property name="ModifiedDate" column="modified_date" type="DateTime" not-null="true" />
<bag name="Tracks" inverse="true" lazy="true" cascade="none">
<key column="album_id" />
<one-to-many class="Track" />
</bag>

From the logs it appears the engine flushes a single insertion, then does tons of internal processing, then one more insertion etc.

13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - Processing unreferenced collections
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - scheduling collection removes/(re)creates/updates
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - Flushed: 1 insertions, 0 updates, 0 deletions to 14486 objects
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
13:58:47.933 [4] DEBUG NHibernate.Impl.Printer - listing entities:
13:58:47.933 [4] DEBUG NHibernate.Impl.Printer - Artist{
...
inserts an album
...
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - loading [Artist#8625]
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - attempting to resolve [Artist#8625]
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - resolved object in session cache [Artist#8625]
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - find: from Artist where Id = :id
13:58:47.933 [4] DEBUG NHibernate.Engine.QueryParameters - named parameters: {id=8625}
13:58:47.933 [4] DEBUG NHibernate.Hql.Classic.QueryTranslator - compiling query
13:58:47.933 [4] DEBUG NHibernate.Impl.SessionImpl - flushing session
13:58:47.949 [4] DEBUG NHibernate.Impl.SessionImpl - Flushing entities and processing referenced collections
13:58:48.058 [4] DEBUG NHibernate.Impl.AbstractVisitor - Processing collection for role Tracks
... (repeat for hundreds of lines)

Any advice? This isn't usable in this state, and I'm not sure what else to do. In this particular case, there's no real need to process any artists other than to enforce the fk-constraints, and no need to process tracks at all.

thanks!!
--craig


Top
 Profile  
 
 Post subject:
PostPosted: Mon Sep 03, 2007 11:35 am 
Newbie

Joined: Tue Mar 21, 2006 3:11 pm
Posts: 10
Can you post the code, or a snippet of the code, you are using to do the inserts?

If you aren't using Transactions, then this might explain why you are getting some slow down. If you can hold off performing Commits, then you can do a bunch of work and then Commit periodically. Below is a quick and dirty code snippet showing what I mean.

Code:
ITransaction txn = session.BeginTransaction()

int commitCnt = 0;
while (someCondition)
{
// do a bunch of work

    // if needed commit periodically
    if (commitCnt++ > 100) // whatever # works for you
    {
        txn.Commit();
        txn = session.BeginTransaction();
        commitCnt = 0;
    }
}
txn.Commit();



Top
 Profile  
 
 Post subject:
PostPosted: Mon Sep 03, 2007 12:05 pm 
Contributor
Contributor

Joined: Sun Jun 26, 2005 5:03 am
Posts: 51
Location: London, UK
You also want to get them out of the session after committing them as NHibernate has to check the state of all of the objects on each flush.

Had a case of this sometime last year; batching and evicting the objects reduced the commit time from ~30 sec to < 1 sec from recollection

_________________
Paul Hatcher
NHibernate Team


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.