hi,

I need to manually index my entities periodically. Please let me know if there will be any issues if the following approach is taken.

1. Batch manual indexing every 15 minutes
2. Optimizing the index occasionally without purging

What is the effect of avoiding purging since optimizing seems to clean up the index?

Thanks,
Seema

Sure you can do that. Provided 15 minutes is enough to re-index. The question is more why you want to do that and why you can not rely on the automatic index updates? Search performance will suffer while you re-indexing. Maybe you should use the master/slave setup? You could re-index on the master and regularly (eg every 15 minutes) update the slave indexes. But it all depends on your requirements.

Optimizing and purging are orthogonal. In fact there is no point to optimize the index if you just purged it. Optimizing is useful after a batch index or in regular intervals during automatic index synchronization.

Purging removed all entities of the specified class (or the whole index). In the Lucene context this means that the documents are marked for deletion. They are not searchable anymore, but the data is actually still in the index. Optimizing will then remove the data and perform an update of the index data structure. Just optimizing the index has nothing to do with purging. If you haven't deleted some documents before optimized is called nothing will be removed.

--Hardy

Purging removed all entities of the specified class (or the whole index). In the Lucene context this means that the documents are marked for deletion. They are not searchable anymore, but the data is actually still in the index. Optimizing will then remove the data and perform an update of the index data structure. Just optimizing the index has nothing to do with purging. If you haven't deleted some documents before optimized is called nothing will be removed.


ok. Hope I can assume that both the following approaches are fine and there is no need to favor one over the other.
1. Re-index (on the existing index) and then optimize
2. Purge the existing index and then re-index without optimizing


Thanks,
Seema

Hi,

I think your usecase might be a good fit for a master/slave setup. You have to keep in mind though that if your procedures manipulate the database while you are still having a 'old' copy of the index you might get synchronisation problems. It depends really on the requirements of your application.

Regarding your second question - "Re-index and then optimize" is not the same as "Purge the existing index and then re-index without optimizing". If for example your database procedure deleted a record which was previously indexed, the corresponding Lucene document would not get deleted from the index during a re-indexing. You would have to purge first. Also, it it recommended to optimize the index after batch indexing. Optimizing is not only there to remove documents marked for deletion, but the process also performs other optimizations, eg segment merging.
I recommend you read more about this in "Hibernate Search in Action" or "Lucene in Action". Either book would give you a deeper insight into Lucene.

--Hardy