If I use FullTextQuery.DOCUMENT_ID in a projection to get the document ID out of a search, what (if anything) guarantees that the IndexReader I subsequently get through the ReaderProvider will be the same one that was used for the search? I'm using shared readers, but index updates may be going on in the background. Does Hibernate hold onto the IndexReader until the end of the transaction? Thanks!

There are several parts to the answer of the question. First of all, there is no guarantee whether you get a same IndexReader. Each time you call ReaderProvider#openIndexReader, IndexReader#reopen() is called under the hood, depending on whether or not the index has changed. So there is no guarantee there.

Now to the transactions. Here a lot will depend on the isolation level and whether you are using transactional batching (which is the default). In this case all Lucene work is batched and executed as a transaction synchronization. So the question is whether in your setup it is possible for an update/delete transaction to complete while another read transaction is in progress. In the default isolation level of READ COMMITTED this can in fact happen, so there is no guarantee here. You could change the isolation level, but that might have other implications and should be carefully thought through.

Is there a particular use-case you have in mind?

--Hardy

It's possible that an update/delete could happen, and there's not much I can do about that, unfortunately.

The use case I have in mind is running Lucene's Highlighter module on a field in the index to show users the text that matched. Ideally, it would be great if I could get a callback invoked by Hibernate Search for each hit so that I can pass the document and IndexReader to the highlighter. But I don't think that's possible in the current release, right? So my next thought was to project the document ID and document from the Hibernate Search query result and iterate through them. But if there's a chance that the IndexReader could be different, then the document IDs are obsolete. I would have to run a separate search via the native Lucene API just to get the same documents again.

Hi,

another solution which does not require a IndexReader is to re-analyze the text. Depending on your setup this might only be minimally slower. In fact the Lucene folks even say that "Whether to index term vectors or reanalyze the text is an application-dependent decision: run your own tests to measure the difference in runtime and index size for each approach.".

You can store the original text in the index and project the value. Once you have the value you use the right analyzer to create the token stream and pass everything to the highlighter.

--Hardy

I'm concerned about how much RAM that could consume; the text field is lengthy. But I will investigate further. Thanks.