Hi,

I'm using Hibernate Search 4.1.1.Final (with Lucene 3.5) and I want to update fields in my document.
The following blog post explain how to do it with Lucene :
http://hrycan.com/2009/11/26/updating-d ... in-lucene/

I know that the IndexWriter is "managed" by Hibernate Search and I don't want to mess with it. My need is quite simple and I just want to give to the backend an extra UpdateLuceneWork.

My document is composed by multiple entities and I have heavy process to compute the whole document. When an entity changed I want to update only those fields and not the entire document.
Example :
Document#D

• Entity#A.name
• Entity#B.price

When entity#A changed I don't want to recompute the entity#B.price (I use custom ClassBridges). I just want to update the name on the document.

Any help will be greatly appreciated,
Guillaume.

PS : I saw the following post but I don't want to extend the framework (if possible) : viewtopic.php?f=9&t=1013864

Hi,

what is your main concern with using automatic index updates? Are you having performance issues? Is there a special use case you want to solve? I am asking, because what you are asking is not possible unless you start customizing the code base. If you provide more context we can find maybe another solution for your problem.

--Hardy

Hi Hardy,

We're using Hibernate Search in a simple way with annotations on entities (no problem) and in a "complex" way to build "cross-domains documents".
We have a modular application with multiple separated domains and we want to build a consolidated document for searching purpose.

Example :
Invoice (domain Financial) have a weak reference on Person (domain Reality). Each domains have their own EntityManager/PersistenceContext/Database.
I want to build a document with the full name of a Person and the account number of an Invoice.
My bridge is a Stateless bean where I can use services to read an invoice and the person associated to build my document.
Indexing is really fast but retrieving data across domains can be slow... this is why I need to index only what's really changed.

Maybe the solution is to use two separated index. One for entities with Hibernate Search on top, and an other one for cross-domains documents with Lucene ?
If you don't understand something feel free to ask... my English is not perfect and can be confusing :)

Thanks for your help,
Guillaume.

Search is not really build for this use case in mind, but what do you mean with weak reference? You just keep the invoice id and load the actual invoice when needed? Or are you talking about some lazy loading features? What's the reason behind having separate databases?


How do you know that the invoice was already indexed when you try to update the name? Since there is no actual update functionality in Lucene you would have to query the index first to get the invoice related fields to be added to the new document. In fact that's maybe what you could do. On the update operation you could use a search + projection to get the invoice related fields and use this information in the bridge to add fields (instead of the service call).

--Hardy

There are many uses cases and lots of users asked for update able fields, unfortunately the approach described in that blog post is very limited and has many flaws; I could list some but asking on the Lucene mailing list will likely be more appropriate and get you a more complete list of solid reasons to avoid that approach.

A better sound approach is what Andrzej Bialecki is proposing for Lucene 4, but it's not ready and he might needs some help if you want to contribute:
https://issues.apache.org/jira/browse/LUCENE-3837

We would of course be very glad to integrate such a capability in Hibernate Search.

If you want to apply the technique from the blog you linked, feel free to patch Hibernate Search; it should be easy to add a new UpdateLuceneWork as you mentioned, or even patch the existing one.

What other users - like myself - used to do to workaround this "limitation" of Lucene is to make extensive use of Hibernate's second level cache, so that reloading often needed references to create the updated Document is fast. By the way, that would be much faster than using any Lucene update operation, since extracting documents from the index is often slow: it's meant for fast matches on queries, not for extracting stored fields: a relational database is more suited for the task.

By weak reference I mean that Invoice only store a unique number about the Person and not a Person entity.


Yes.
When the Person's name changed, we fire an event to notify our bridges. The bridge load the corresponding entity(ies) and call index().


I would prefer to implement a "good" approach that can be merged to Hibernate Search (and benefit from others).


Thanks, I will take a look :)


Yes you're right... we already use Hibernate's second level cache but with some anti-pattern on top :'(


Thanks for your replies they helped me to rethink my code.
Guillaume.

By all means, if you find a way that would be an extremely cool contribution and I would be glad to help.

I don't want to discourage you, just pointing out that it's not as simple as it looks like in that blog, as the approach would only work when all fields are stored in the index and no custom analyzers are applied; that are quite some strong limitations.
Also it doesn't address any thought about concurrent updates.

If you are interested in helping on LUCENE-3837, I would say it's a pretty though one but I'm sure Andrej would appreciate any kind of help so even if you're not hard-core Lucene but can help testing and reviewing that would be great.
He gave a good presentation about the design of it in Berlin, which was filmed:

http://vimeo.com/album/1968418/video/44300229