We encounter a performance regression in massive reindex with Hibernate Search 4.0. After migration towards this new version, massive reindex starts to last more than 20 minutes with high iowait on host (with hibernate 3.6.8, it took approximately 5 minutes, ~100k entities database).

After some investigation, we found that downgrading to lucene 3.3.0 brings back original performance. I implemented a trivial ProgressMonitor and noticed the following speed :

Hibernate 4.0 - Lucene 3.3.0 : 3000entities / 15sec.
Hibernate 4.0 - Lucene 3.4.0 or Lucene 3.5.0 : 150 entities / 15sec.

As performance regression seems to be linked with lucene dependency, we think that it is related with fsync bug fix introduced in 3.4.0 (it explains the higher iowait noticed). With a breakpoint to count fsync hit in FSDirectory, it confirms that fsync is called frequently (~25 times / s.).
If we choose ram index storage, there is also no performance regression.

Does anyone else encounter this performance regression ? Do you know this problem ? Is there any configuration tweak available to disable fsync during massive reindex or any plan to enhance massive reindex performance ?


Thanks

Hi,

thanks for this thorough report. I was not aware of any performance regression. Which Lucene issue are you referring to?

--Hardy

Hi,

after some investigation I assume you mean this issue - https://issues.apache.org/jira/browse/LUCENE-3418.
Really it is a bug fix and before the fix data could be lost. The unfortunate side effect is a loss in performance. One thing we can do on the mass indexer side is to reduce the amount of index commits to alleviate the performance drop. How much this is possible and will help has to be seen. I created https://hibernate.onjira.com/browse/HSEARCH-1019 to track the issue.

--Hardy

is the combination of HS4 and Lucene 3.4 and up always that slow or it is only in your special use case?

I would rather risk some data loss (the occasions seems to occur rather seldom), which can be corrected by rebuilding the index, than having such a slow indexing performance.

kind regards

These figures are relative to a bug in the MassIndexer specifically; the overhead of using indexing during "normal" event driven operations can be significantly different and depends on your data values and general schema and indexing options; it can be much faster, simple (most trivial schema and data) reaches figures up to 12 millions entities in 3 minutes on my dual core laptop during MassIndexer operation (before this bug); Most of the work performed by the MassIndexer is usually data loading from the database, a step which is skipped when you're dealing with event-triggered indexing as the data is usually in large part already available in the persistence context; it can be slower when the indexing operation needs to load additional entities from the database.