I'm trying to use hibernate search to index about 9000 objects with 9,000,000 nested objects.
I get this exception after 1 hour of processing:
[java] java.lang.OutOfMemoryError: Java heap space
[java] 01:09:15,128 ERROR AssertionFailure:22 - an assertion failure occured (this may indicate a bug in Hibernate)
[java] org.hibernate.annotations.common.AssertionFailure: Access a Sealed WorkQueue whcih has not been sealed
[java] at org.hibernate.search.backend.WorkQueue.getSealedQueue(WorkQueue.java:38)
[java] at org.hibernate.search.backend.impl.BatchedQueueingProcessor.performWorks(BatchedQueueingProcessor.java:120)
[java] at org.hibernate.search.backend.impl.PostTransactionWorkQueueSynchronization.afterCompletion(PostTransactionWorkQu
eueSynchronization.java:50)
[java] at org.hibernate.transaction.JDBCTransaction.notifyLocalSynchsAfterTransactionCompletion(JDBCTransaction.java:243)
[java] at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:121)
[java] at com.broadlane.portal.contracts.ContractManager.indexContracts(Unknown Source)
[java] at com.broadlane.portal.contracts.ContractManager.main(Unknown Source)
[java] 01:09:15,128 ERROR JDBCTransaction:246 - exception calling user Synchronization
[java] org.hibernate.annotations.common.AssertionFailure: Access a Sealed WorkQueue whcih has not been sealed
[java] at org.hibernate.search.backend.WorkQueue.getSealedQueue(WorkQueue.java:38)
[java] at org.hibernate.search.backend.impl.BatchedQueueingProcessor.performWorks(BatchedQueueingProcessor.java:120)
[java] at org.hibernate.search.backend.impl.PostTransactionWorkQueueSynchronization.afterCompletion(PostTransactionWorkQu
eueSynchronization.java:50)
[java] at org.hibernate.transaction.JDBCTransaction.notifyLocalSynchsAfterTransactionCompletion(JDBCTransaction.java:243)
[java] at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:121)
[java] at com.broadlane.portal.contracts.ContractManager.indexContracts(Unknown Source)
[java] at com.broadlane.portal.contracts.ContractManager.main(Unknown Source)
[java] Indexed 7034 contracts
I tried to increase the heapsize to 1MB, but the indexing fails a little bit latter.
Is there a recommended strategy to do the original indexing when dealing with millions of objects? Like break up the indexing in several pieces instead of doing everything in one query like this:
Session session = HibernateUtil.getSessionFactory().getCurrentSession();
FullTextSession fullTextSession = Search.createFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
List contracts = session.createCriteria(Contract.class).list();
int i = 0;
for (Object contract : contracts) {
fullTextSession.index(contract);
i++;
}
tx.commit();
Thanks,
-Christophe
|