I need to implement a "site search" functionality for a web site.
All content is updated on the backend server, where indexing takes place, and queries are run on two frontend servers, where the web application is hosted.
How do I make the index available to the frontend servers?

I've read about JMS, JGroups, Infinistore and all this fine stuff, but they seem overkill for my needs.
I'd just be happy with a scheduled cron copy of the index across machines via scp or similar.

Will I get a corrupted index copy if I transfer the index during an indexing operation?
In such a case, is there anything simple around this?

My servers are hosted on EC2 instances (if that makes any difference).

Thank you.

Well I think I got it: the backend is a filesystem-master and the cron task can transfer the copy over to the frontend servers whenever.

On the backend:

hibernate.search.default.directory_provider=filesystem-master
hibernate.search.default.sourceBase=/tmp/thecopy

On the frontend I just leave the default settings.


(it feels lonely to be the only one to answer your own posts :-)

Should I also use a filesystem-slave on the frontend, just to be totally safe?
I'd copy the index via scp to the "source" location of the slave instead of overwriting the main index itself.

Is locking going to mess with my setup?

What if I copy the index over while the master is writing the copy (aka source)?
Will I copy the lock file as well? Will the slave be influenced by a lock file in its own copy?

My understanding is that the creation of the index copy is not "atomic" (like a directory rename) because it is "incremental ... to reduce the copy time" (from the docs). So I'm not sure the whole thing is going to work yet...

Please advise!

Give us some time ;) it's saturday early morning, and long holidays as well !


that's basically what the master/slave do, as you have figured out. The difference over scp / rsync is that having the backend trigger the copy, it knows when it's safe to do (it's in sync with it's own timer to switch writing to the other directory).


yes


up to you, if you do that with your own script.. it's not needed but should not make trouble on the slaves;


No, since the slaves should not write to it. They will get an exception if attempting to write to it, so it might be safer to keep the lock so you will notice that some configuration is wrong ;)