Lucene Searches and Hibernate - Rehydration

botah · **Joined:** Mon Feb 16, 2004 2:36 pm **Posts:** 12

All,

I don't know if this is the 'best' place to ask this but here goes.

I have my full domain model happily implemented in hibernate. I have a requirement to implement some advanced text AND attribute searches. So I implemented my search in Lucene. Searching my index was easy and clean. HOWEVER, I implemented the results of my lucene query by rehydrating my POJOs (the same that hibernate has mapped) attribute by attribute.

Code:

            
searcher = new IndexSearcher(Case.LUCENE_SEARCH_INDEX);
            Hits hits = searcher.search(query);
            logger.info(hits.length() + " total matching documents");
            for (int i = 0; i < hits.length(); i++) {
                Document doc = hits.doc(i);
                String id = doc.get("id");
                Case aCase = new UserStory();
                aCase.setId(new Long(id));
                aCase.setDescription(doc.get("description"));
                aCase.setTitle(doc.get("title"));
                aCase.getCaseType().setDescription(doc.get("caseType"));
                results.add(aCase);
            }

When creating the index from the POJOs I added all attributes that I would want on my search results as keywords in the index. (I think this may be the source of my problem). Then to show the results, I just rehydrate the POJO from the keywords in the lucene results doc. Easy enough.

However, it turns out that my solution is not extendable. If I have different result views that have additional attributes, I am out of luck.

So, my question is this. Which is the best approach:
A) Use lucene to index and search and store all attributes that I might want in all views then rehydrate my POJOs using setters.
B) Use lucene to index and search, but only fetch the PK from lucene and do a hibernate session.load(pk) to get the REAL object? I am worried about the double search hit. But this approach is extendable nicely. This might look something like this:

Code:

searcher = new IndexSearcher(Case.LUCENE_SEARCH_INDEX);
            Hits hits = searcher.search(query);
            logger.info(hits.length() + " total matching documents");
            for (int i = 0; i < hits.length(); i++) {
                Document doc = hits.doc(i);
                String id = doc.get("id");
                Case aCase = hibSession.load(id);
                results.add(aCase);

              }

C) Bag Lucene all together and do straight HQL queries and loose some of the rich text searches the Lucene gives me?
D) Other

Thanks in advance

botah · **Joined:** Mon Feb 16, 2004 2:36 pm **Posts:** 12

Just to clarify option two....
I might make a list of keys and do a hibernate find where the ids in the list so I dont run 100s of Hib. load queries. Even so, not sure I like this option.

Thoughts?

imario · **Posted:** Thu Feb 19, 2004 2:55 am

botah wrote:

I might make a list of keys and do a hibernate find where the ids in the list so I dont run 100s of Hib. load queries. Even so, not sure I like this option.
Thoughts?

I have done exactly this too (with our own persistence layer, not hibernate, .... for now).
For sure, you have to read the database too, which might cost some time, but I have found using the above solution (where id in (..., .., ...)) is fast enough.

Beside this, in an dynamic system i have found it is not always easy to keep the lucene-index in sync with the database in realtime.

So, if you rehydrate it is possible that the hibernate-object is not really existent in the database, on the other hand if you reread them from the database, it could look like the search result do not really match.

But for sure, this depends on your application.

ko5tik · **Posted:** Thu Feb 19, 2004 7:36 am

botah wrote:

All,

So, my question is this. Which is the best approach:
A) Use lucene to index and search and store all attributes that I might want in all views then rehydrate my POJOs using setters.

Do you find that your pojo attributes lost something inhiberante index? Kind of ucool.

botah wrote:

B) Use lucene to index and search, but only fetch the PK from lucene and do a hibernate session.load(pk) to get the REAL object? I am worried about the double search hit. But this approach is extendable nicely. This might look something like this:

Code:

searcher = new IndexSearcher(Case.LUCENE_SEARCH_INDEX);
            Hits hits = searcher.search(query);
            logger.info(hits.length() + " total matching documents");
            for (int i = 0; i < hits.length(); i++) {
                Document doc = hits.doc(i);
                String id = doc.get("id");
                Case aCase = hibSession.load(id);
                results.add(aCase);

              }

I prefer this approach. I have to indexcontents of my infomation management system - so complicated hibernate structure is first rendered to text, using velocity.
( like subchapter of book, news entry or whatever )
Then I store keywords ( explicitely given ) and fulltext
( maybe title or whatever ) in search document fields.

As you may expect, keywords are indexed / stored ,
full text only indexed and title also indexed / stored

Then on hit I retrieve ID of piece of information ( be it
news, chapter or even pictire ) - and render it.

A must also say, that big parts of my information are pretty static,
so I do batch reindexing from time to time.

botah wrote:

C) Bag Lucene all together and do straight HQL queries and loose some of the rich text searches the Lucene gives me?

Worst scenario.... Guy which was before me on this task
created a really lame database, with search that dod not found anything :) ( though he did not used hibernate for this,
it was plain jdbc... )

botah wrote:

D) Other

Thanks in advance