We are using Hibernate 3.2.5 with Search 3.0, and it works fine.

Now we have some entities referring to say huge PDF contents, whose content must be extracted before indexing; and the text extraction process takes some time: now this puts the transaction in hold before text extraction is completed. What is the appropriate way to do that?

We have set org.hibernate.worker.execution to asynch. Index writing is indeed asynchronous, but text extraction of course isn't, as it happens in the getter method (annotated with "Field") call. We have searched the search documentation and the forum, and found a lot of documentation and discussion concerning scaling the index writing process, but not about queueing the text extraction process.

We could build a @ClassBridge that puts the text extraction in a separate queue, but maybe there is a simpler way.

Now this is quite problematic.
I initially did the extraction at the same time as the indexing, but it does not work well at all since the indexing is post transactional and and some lazy objects might need initialization.

What might work I think is to write a LazilyGeneratedField which would implement o.apache.lucene.document.Fieldable and lazily parse the PDF. That way the field processing will only happen during the async process.

If it works, I would appreciate if you could create a wiki page in the community area to capture the knowledge.

Thanks for your support. I am already testing your solution and preparing a wiki contribution.

Published here:

http://www.hibernate.org/432.html

Cheers

Nice, thanks!