Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 2 posts ] 
Author Message
 Post subject: java.lang.OutOfMemoryError During Mass Indexing
PostPosted: Fri Jul 07, 2017 10:55 am 
Newbie

Joined: Fri Jul 07, 2017 10:39 am
Posts: 1
When performing mass indexing with elasticsearch and the following parameters

fullTextSession.createIndexer().batchSizeToLoadObjects(25)
.cacheMode(CacheMode.IGNORE)
.threadsToLoadObjects(16)

I'am getting a java.lang.OutOfMemoryError: GC overhead limit exceeded exception.

The indexing runs fine for the first 100,000 records and then this exception occurs.

Can someone please provide advice on how to tweak the mass indexer to perform as quickly as possible with elasticsearch without this exception happening?

Thanks.


Top
 Profile  
 
 Post subject: Re: java.lang.OutOfMemoryError During Mass Indexing
PostPosted: Tue Jul 11, 2017 7:46 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2530
Location: Third rock from the Sun
Did you try the obvious: check how much memory is needed?

The index rebuild process is a very intensive operation and it will be significantly faster if you can allow it to scale up on the number of threads and use larger buffers. Using many buffers and threads of course consumes more memory, so make sure to allow your JVM to use a reasonable heap size.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.