So I finally go the master-slave setup working over JMS/ActiveMQ. Now my question is this. What are the recommended ways of integrating mass-indexing with the master instance? Ideally, what I'd like to have is a task that fires off on some schedule that will perform the mass-index operation on the entire database. This would happen concurrently with the search listener receiving and processing index-update requests from slaves.

Assuming I set the exclusive-lock flag off, does anyone foresee any issues? Has anyone done something similar? My main concern is what would happen if the master attempts to process a slave request while the mass-indexer is running.

On a side-note, the mass-index step seems to experience deadlocks fairly frequently. I don't know if this is an issue with the c3p0 connection pool I'm using or something inherent in the indexing code itself. Anyone else have issues with this?

Thanks,

Michael

Hi Michael,

You should run the massindexer on the master only, not on the slaves.
In any case the Massindexer was not meant to be used on a "live system", as the index is emptied at start and so clients might search on an empty index: the replacement with the new index is currently not atomical; you should use it for maintenance purposes.
Generally Hibernate Search should be able to keep the database and index always in sync, so this shouldn't be needed.. why are you needing to start it periodically?


you might deadlock if the connection pool is not big enough, as in some stages it might need more than N connections in parallel to continue; make sure to monitor your pool and either reduce the
number of thread, or enlarge the allowed maximum size of the pool.

Hi Sanne,

Yes, I am talking about running this on the master instance, on a periodic basis and concurrently with index updates the master may be processing from slaves. The database on which my Lucene indices depend is being refreshed by an external process once every hour, so I need a way to periodically sweep through all the data and update the entire index.

As far as the connection pool deadlocking, how can I know the peak number of connections the indexer may invoke so as to configure the pool appropriately?