Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 3:32 am 
Newbie

Joined: Thu May 26, 2016 3:14 am
Posts: 4
Hi

When I persist entities using Hibernate

hibernate-commons-annotations-4.0.5.Final.jar
hibernate-entitymanager-4.3.11.Final.jar
hibernate-core-4.3.11.Final.jar
hibernate-jpa-2.1-api-1.0.0.Final.jar
hibernate-ehcache-4.3.11.Final.jar
hibernate-validator-5.2.4.Final.jar

then I receive after some time the following warnings

2016-05-26 09:19:56,186 | 080-exec-4 | WARN | LoadContexts | HHH000100: Fail-safe cleanup (collections) : org.hibernate.engine.loading.internal.CollectionLoadContext@81aa398<rs=org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet@34070de0>
2016-05-26 09:19:56,186 | 080-exec-4 | WARN | CollectionLoadContext | HHH000160: On CollectionLoadContext#cleanup, localLoadingCollectionKeys contained [10] entries

and persisting becomes slower and slower and stops completely at some point with

java.lang.OutOfMemoryError: Java heap space
at java.lang.reflect.Array.newInstance(Array.java:75)
at org.hibernate.type.descriptor.java.ArrayMutabilityPlan.deepCopyNotNull(ArrayMutabilityPlan.java:43)
at org.hibernate.type.descriptor.java.MutableMutabilityPlan.deepCopy(MutableMutabilityPlan.java:52)
at org.hibernate.type.AbstractStandardBasicType.deepCopy(AbstractStandardBasicType.java:321)
at org.hibernate.type.AbstractStandardBasicType.deepCopy(AbstractStandardBasicType.java:317)
at org.hibernate.type.TypeHelper.deepCopy(TypeHelper.java:67)
at org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:268)
at org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:144)
at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:1115)
at org.hibernate.loader.Loader.processResultSet(Loader.java:973)
at org.hibernate.loader.Loader.doQuery(Loader.java:921)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:355)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:325)
at org.hibernate.loader.Loader.loadCollectionBatch(Loader.java:2300)
at org.hibernate.loader.collection.plan.LegacyBatchingCollectionInitializerBuilder$LegacyBatchingCollectionInitializer.initialize(LegacyBatchingCollectionInitializerBuilder.java:100)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:693)
at org.hibernate.event.internal.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:92)
at org.hibernate.internal.SessionImpl.initializeCollection(SessionImpl.java:1933)
at org.hibernate.collection.internal.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:722)
at org.hibernate.engine.internal.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:894)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:360)
at org.hibernate.loader.Loader.doList(Loader.java:2554)
at org.hibernate.loader.Loader.doList(Loader.java:2540)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2370)
at org.hibernate.loader.Loader.list(Loader.java:2365)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:497)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:387)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:236)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1300)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:103)
at org.hibernate.jpa.internal.QueryImpl.list(QueryImpl.java:573)
at org.hibernate.jpa.internal.QueryImpl.getResultList(QueryImpl.java:449)

Any idea what might be causing this?

Thanks

Michael


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 4:53 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1383
You might be using ScrollableResultSets which you don't explicitly close after usage:

Code:
EntityManager entityManager = null;
EntityTransaction txn = null;
ScrollableResults scrollableResults = null;
try {
    entityManager = entityManagerFactory().createEntityManager();

    txn = entityManager.getTransaction();
    txn.begin();

    int batchSize = 25;

    Session session = entityManager.unwrap( Session.class );

    scrollableResults = session
        .createQuery( "select p from Person p" )
        .setCacheMode( CacheMode.IGNORE )
        .scroll( ScrollMode.FORWARD_ONLY );

    int count = 0;
    while ( scrollableResults.next() ) {
        Person Person = (Person) scrollableResults.get( 0 );
        processPerson(Person);
        if ( ++count % batchSize == 0 ) {
            //flush a batch of updates and release memory:
            entityManager.flush();
            entityManager.clear();
        }
    }

    txn.commit();
} catch (RuntimeException e) {
    if ( txn != null && txn.isActive()) txn.rollback();
        throw e;
} finally {
    if (scrollableResults != null) {
        scrollableResults.close();
    }
    if (entityManager != null) {
        entityManager.close();
    }
}


You should always close the scrollableResults in a finally block.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 5:11 am 
Newbie

Joined: Thu May 26, 2016 3:14 am
Posts: 4
Thanks very much for your quick reply, but I just checked our code and we do not use ScrollableResults anywhere.

Thanks

Michael


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 5:17 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1383
Are you fetching large amounts of data, like tens of thousands of entities with your HQL queries. This can also be the case if you have mapped a one-to-many association which has tons of child entities.

It might be an issue, but you have to provide a replicating test case for it.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 6:05 am 
Newbie

Joined: Thu May 26, 2016 3:14 am
Posts: 4
we have something like

"User" -- one2many --> "BlogEntry" -- one2many --> "Artifact" -- one2many --> "BlobVersion"
-- one2many --> "MetaVersion"

which could result in ten of thousands, but at the moment we only have 3 users with maybe 10 entries with one artifact with maybe 2 versions per entry. So not that much ;-)

The reading is no problem, but the writing is the problem. So when we add a new "BlogEntry" and containing for example an image (BlobVersion) and some text (MetaVersion).

We can reproduce the problem using JMeter with

1 Thread
50 Iterations

Thanks

Michael


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Thu May 26, 2016 8:13 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1383
Maybe it's the BlobVersion because the Blob free() method is called by the ResourceRegistryStandardImpl only after the Transaction is ended.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Sun May 29, 2016 5:15 am 
Newbie

Joined: Thu May 26, 2016 3:14 am
Posts: 4
Thanks for your hint, but I assume that it should become free eventually, right?

I have turned on Java VisualVM and observe, that the "Total unloaded" keeps increasing (until JVM crashes).

A colleague of mine pointed out that this might be cause by

https://docs.jboss.org/hibernate/stable ... hcode.html

Also I have found the following article which describes a similar behaviour

http://stackoverflow.com/questions/1071 ... ue-in-java

Any help on this very much appreciated!

Thanks

Michael


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Wed Nov 23, 2016 10:14 am 
Newbie

Joined: Fri May 15, 2015 11:29 am
Posts: 2
Have you find a solution to this problem?


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Sat Sep 02, 2017 2:19 am 
Newbie

Joined: Sat Sep 02, 2017 2:15 am
Posts: 1
vlad wrote:
Are you fetching large amounts of data, like tens of thousands of entities with your HQL queries. This can also be the case if you have mapped a one-to-many association which has tons of child entities.

It might be an issue, but you have to provide a replicating test case for it.


It is the case in my issue. How to resolve it?


Top
 Profile  
 
 Post subject: Re: Fail-safe cleanup (collections) : org.hibernate.engine.loadi
PostPosted: Sat Sep 02, 2017 12:29 pm 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1383
As I explained in this article, you should use pagination. There's no reason not to use it.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.