Just read the wiki page about Lucene usage with Hibernate. Oddly enough, I just got a requirement to implement some type of search engine and the current lean is toard Lucene.

I've never used Lucene before and need to figure out a way to make operations against Lucene be transactional (my understanding is that it does not support transactional operations.

Not knowing much about Lucene, it seems to me I have two options:
1) update Lucene through JMS, where the JMS feed from the Hibernate interceptor is XA transaction aware;
2) Buffer the updates to Lucene in a class that registers a synch with the transaction manager managing the connection used by hibernate and then apply updates oly when the transaction commits.

#1 risks slowing the operation down, while #2 risks mixed hueristics issues.

Anyone used Lucene in this manner and have a better approach?

bump

Are you going to store index in RDBMS ?

If you're in read mostly case, use the first one, it'll be the easiest way. The $100 question is Are you sure to need guaranteed transaction concistency ?
In the #1 case, you're sure to have your message in the failed queue is lucene goes wrong. You're not sure of concistency between DB and Lucene. The concistency is limited to 1 change in DB = 1 message sent and received properly.
Actually it's the same for #2 case, no way to have concistency between Lucene and DB : lucene will always be late.

Can you do that with Lucene? I thought Lucene could only utilize the underlying file system for index storage.

You can implement RDBMS "file system" yourself and use db server transactions see org.apache.lucene.store.* package, but I do not think it will perform (lusene locks index), most of databases support text search internaly and it must be better to use SQL extensions and procedures.
I use lucene, but my use case is very trivial script. Lucene is good for this use case:

1. export db to text file (if you do not want SQL overhead on transactional system)
2. parse files (or execute SQL on DB directly) to build index
3. copy index to web server (index can contain data too)
4. rename directory with new index

It performs very well, if your change data peridicaly and it must be trivial to implement this kind of script.

We're using the Interceptor example provided on this site to integrate Lucene with Hibernate in our project. We ran into a situation where we think the Lucene index is getting updated (via onFlushDirty) before Hibernate commits changes to the database. If for some reason Hibernate encounters an error and needs to rollback, the lucene index does not get rolled back. So we have an issue there with keeping the lucene index in synch with the database.
We're looking into it - I'm wondering if anybody else has run into this same type of problem and how/if they resolved it?

Hi!

There is a framework called Compass that adresses some of your issues. I have tried it, and it seems quite usable. At least, the support is very good.

http://www.compassframework.org/

Christian

Hibernate Annotations has a integration with lucene, but has the same problem as you right now. I guess I should do the work on a postTransactionCompletionEventListener

But it that case, the changes won't be seen until the tx commit.
To work around that, you need a tx management on top of lucene.

This is against the forum rules. If you continue in this behavior, I will have to ask Christian to ban you.

;-)

Oh, and please consider commerical support. ;-)

You can update index using commands to undo changes:

class Update extends Command{

Document oldDoc, newDoc;

void execute(){
deleteOldDoc();
insertNewDoc();
}

void undo(){
deleteNewDoc();
insertOldDoc();

}


}

This stuff will not be transactional, but it must solve 99% of problems for text search (store all executed commands in list and call undo for all commands if trasaction fails). I asume you can tolerate some "dirty" data for text search in practice.