hi,

Our application is deployed in a WAS cluster. The client does not want to go for a JMS solution only for indexing purposes. Using a shared folder like NFS also seems to cause problems when it comes to locking by different nodes. So we want to turn off automatic indexing and just run a Quartz job which does manual indexing once in every 15 minutes. In this job, I plan to purge all the entities in the index, do indexing and then do the index optimization. What are the pros and cons of this approach? Will the search speed be affected a lot?

Thanks,
Seema

Hi,
your approach looks fine.
You probably want to build the index on a fresh copy, so you don't empty your existing index during the indexing process, and then swith to using the new index when you're done indexing.
Give a look to the source in Hibernate Search to investigate how the "indexing switch" is done in the org.hibernate.search.store.FSSlaveDirectoryProvider
(the same stragegy is used in the JMS based solution); you could make your own DirectoryProvider implementations, and I think we could bundle it in Search if you like that.

Thanks for the tip, I will try that. But I had a question. If I search on the same index which is currently being indexed/purged/optimized, will I get incorrect results? Or will it just make the search slow?

Thanks,
Seema

optimized doesn't change the results; but purging will make all your entities unavailable until you don't index all your data again: it won't give results for the objects not indexed; so users may not find all results.
That's why I am telling you that you should rebuild your index in a separate index, and then replace them "hot" by implementing a DirectoryProvider.
(unless you are rebuilding your index at a time that the index being empty for a couple of minutes doesn't worry anybody)


yes it will be a bit slower, just because when an index segment is changed some buffers need to be re-read from disk, but this happens all the time anyway and is usually not a problem.

hi,

I looked at the FSSlaveDirectoryProvider implementation. In this case, the hot swapping takes place after copying the index between the two directories. The index is "read only", no writing happens to this index since slaves do not write to the index.

But in my case, while one directory is being updated, the reading needs to happen from the other. I need the hot swapping of directories to happen after the purging, optimizing and indexing is complete. How do I do this check? The getDirectory() method in the DirectoryProvider seems to be called for both read and write operations. So how do I distinguish?

Thanks,
Seema