-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 
Author Message
 Post subject: Hib. search - Rebuilding the whole index using flushToIndexe
PostPosted: Fri Jan 07, 2011 4:53 am 
Beginner
Beginner

Joined: Wed Nov 03, 2010 1:05 pm
Posts: 24
Hello all,

I have a simple question regarding the rebuild of the index with hibernate search.
I know that there is two way to rebuild the index, but I've choosen to use flushToIndexes() as follow:
Code:
fullTextSession.setFlushMode(FlushMode.MANUAL);
fullTextSession.setCacheMode(CacheMode.IGNORE);
transaction = fullTextSession.beginTransaction();
//Scrollable results will avoid loading too many objects in memory
ScrollableResults results = fullTextSession.createCriteria( Email.class )
    .setFetchSize(BATCH_SIZE)
    .scroll( ScrollMode.FORWARD_ONLY );
int index = 0;
while( results.next() ) {
    index++;
    fullTextSession.index( results.get(0) ); //index each element
    if (index % BATCH_SIZE == 0) {
        fullTextSession.flushToIndexes(); //apply changes to indexes
        fullTextSession.clear(); //free memory since the queue is processed
    }
}
transaction.commit();


I would like to know if I can do a kind of batch with this process and being sure that gives no problem if users are still using the application during the index rebuild.

Thank you very much and happy new year ;-)


Top
 Profile  
 
 Post subject: Re: Hib. search - Rebuilding the whole index using flushToIndexe
PostPosted: Fri Jan 07, 2011 4:59 pm 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
It should be safe. What errors are you afraid about?


Top
 Profile  
 
 Post subject: Re: Hib. search - Rebuilding the whole index using flushToIndexe
PostPosted: Mon Jan 10, 2011 5:23 am 
Beginner
Beginner

Joined: Wed Nov 03, 2010 1:05 pm
Posts: 24
I'm afraid if users enter new data during the index building process and so not taken into account in the index.


Top
 Profile  
 
 Post subject: Re: Hib. search - Rebuilding the whole index using flushToIndexe
PostPosted: Mon Jan 10, 2011 8:23 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

I probably was to quick with my answer. There could of course be problems. However, using ScrollableResults should still work. Even if your re-indexing is in progress while data changes they should be picked up. Basically there are two cases:
1. The data gets changed after the original (old) data was indexed in the indexing loop. This is fine since the event based indexing will just update the indexed data
2. The data gets changed before the main indexing loop has indexed it. In this case the event based indexing will update the index first and there would be a risk that the main indexing loop would then index stale data. However, afaiu ScrollableResults will make sure that the latest version of the object gets retrieved. I am not sure whether I would rely on this though. At least not without some testing.

If you have a last modified column or something in your data/entity you could rely on this field to keep indexing while there are no modifications anymore. The safest option of course would be to not allow changes to entities while re-indexing.

--Hardy


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.