Hi,

just upgraded to HS 3.2 and tested the massindexer. Amazing work guys, really fast, low memory usage, just what i've been looking for :)

But, some litte issues still bothering me. What about this progressMonitor of the massindexer and/or the whole slf4j thing. I'm not really into this, and a quick look into the documentation didnt help me at all.
Is there an option that turns off the progressMonitor (or just let him print a msg every 10% (oder x documents)?

thank you, it's indeed the result of a long quest:)


Sure, there are some open issues related to documenting this API which where postponed as we where need more feedback: release early, release often should become the new strategy soon.
Have a look into the current only implementation: org.hibernate.search.impl.SimpleIndexingProgressMonitor. It shouldn't have a great impact on performance, it only considers logging some minor stats every 50 entities.
Unfortunately while extending it is easy, it's not yet possible to replace the implementation (without recompiling Search): we would like to make sure the interface is stable before making it a public API, so please try implementing whatever you need and provide feedback on it.
Please see and comment on HSEARCH-370


Either you're having a very small database and can disable it completely (set the logging level for the class to silent "info"), or if you have a huge database you likely want some feedback more often, as it could take some time to go from 20% to 20%.. The safest default I could think of is to log requently, a period of 50 seemed a good start and has proven to be negligible as performance hit compared to the actual indexing job.

You're welcome to propose alternative implementations, and this is the right time to ask for changes in the API.

Turning off the logging of the progressMonitor decreases my overall indexing process duration from 140minutes to round about 50min with a db that contains 1.1 million entrys. just fyi
(which is, btw, really really nice to see. again, thx a lot guys :) )

thanks for reporting that, I've added a link from HSEARCH-370 to this thread as additional motivation to improve this.