Null-values in search result

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

Hi,

I am using Hibernate Search, and I am still very much only trying to get it to work. My current problem that I have struggled with for a while is that I get null values in alla fields of the returned persistent object.

The scenario: I search with a query that has an exact match. I get one hit, but it contains nulls only.

This is how the call-method looks like (very much inspired by the docs on Hibernate Search):

Code:

       
Session session = HibernateUtil.getSessionFactory().openSession();
        FullTextSession fullTextSession = Search.createFullTextSession(session);
        QueryParser parser = new QueryParser("question", new StopAnalyzer()); 
        org.apache.lucene.search.Query luceneQuery;
        try {
            luceneQuery = 
                parser.parse(question); 
        } catch (ParseException e) {
            throw new FilterExecuteException(getClass(), e);
        }
        org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery(luceneQuery);
        fullTextQuery.setFirstResult(offset);
        List<QAItem> results = fullTextQuery.list(); //return a list of managed objects

I get nulls on all field of the QAItems in the results list (field are id, question, captionAnswer, answerDetails).

The QAItem class is annotated as follows.

Code:

  
@Entity
@Indexed  
public class QAItem {

 @Id 
    @GeneratedValue
    @DocumentId 
    private Long id;

    @Field(index=Index.TOKENIZED, store=Store.YES)  
    @Boost(3)                                        
    private String question;

    @Field(index=Index.TOKENIZED, store=Store.YES)    
    @Boost(3)                          
    private String answerCaption;

    @Field(index=Index.TOKENIZED, store=Store.YES)    
    private String answerDetails;

The lucene index has been created, and I get no complains when I start Hibernate and load things.

Any help greatly appreciated...since I seem to be really stuck this time ;-)

/Lars

emmanuel · **Posted:** Mon Mar 26, 2007 3:46 pm

You mean you have a list of one result
this result is not null, it's a QAItem but all those fields are null
assertNull( ie qaItem.getId() );

I can't see how this happen. Hibernate Search retrieve the object from hibernate (session.get()) underneath.

Can you play with your debugger to see what is going on (BTW don't check the field nullability through your debugger, some don't play nice with Hibernate proxy generation

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

Hi Emmanuel,

thanks for your response!

emmanuel wrote:

You mean you have a list of one result
this result is not null, it's a QAItem but all those fields are null
assertNull( ie qaItem.getId() );

Yes, exactly. To very explicit, this is what I get from toString on the results-list:

Code:

[[Id: null Question:null AnswerCaption:null AnswerDetails:null]]

More details:

1) I have also tried to call with a regular Hibernate Session/Query and the object is there in the database. If I change the question, the list length changes. So it seems that this is somehow the right object, but it has no values for some reason.

2) Perhaps some versions don't play with each other? I am using lucene-core-2.0.0.jar (tried 2.1.0 but it complained that a method was missing); hibernate-annotations-3.2.1.jar; hibernate-3.2.2.jar; lucene-snowball-2.1.0.jar; lucene-wordnet-2.1.0.jar; hibernate-3.2.2.jar; hibernate-commons-annotations.jar

Quote:

I can't see how this happen. Hibernate Search retrieve the object from hibernate (session.get()) underneath.

Can you play with your debugger to see what is going on (BTW don't check the field nullability through your debugger, some don't play nice with Hibernate proxy generation

I am not really using a debugger but I can give you Log4J debug print outs. Sorry for long printout. The printout starts from when the method given above is called (its in the LuceneRetrieveFiler class). I added flush and close to the code as a test but that did not change anything...

Its seems that the answer "is there" for a while but then I get nulls in the printout further on. The returns from NullableType looks ok: first comes the answerCaption then answerDetails. These are the correct strings seen in the debug.

An interesting details (perhaps) is that Printer debug shows the result correctly *after* the log only shows nulls...which looks strange. Not sure why Printer is called though perhaps due to flush.

Suggestions??

Code:

     [java] DEBUG [main] (LuceneRetrieveFilter.java:77) - Executing FAQ question: "What is this newsgroup"
     [java] DEBUG [main] (DefaultLoadEventListener.java:171) - loading entity: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (DefaultLoadEventListener.java:255) - creating new proxy for entity
     [java] DEBUG [main] (JDBCContext.java:233) - after autocommit
     [java] DEBUG [main] (ConnectionManager.java:402) - aggressively releasing JDBC connection
     [java] DEBUG [main] (SessionImpl.java:422) - after transaction completion
     [java] DEBUG [main] (SessionImpl.java:832) - initializing proxy: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (DefaultLoadEventListener.java:332) - attempting to resolve: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (DefaultLoadEventListener.java:369) - object not resolved in any cache: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (AbstractEntityPersister.java:3031) - Fetching entity: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (Loader.java:1843) - loading entity: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (AbstractBatcher.java:358) - about to open PreparedStatement (open PreparedStatements: 0, globally: 0)
     [java] DEBUG [main] (ConnectionManager.java:419) - opening JDBC connection
     [java] DEBUG [main] (AbstractBatcher.java:393) - select qaitem0_.id as id3_0_, qaitem0_.question as question3_0_, qaitem0_.answerCaption as answerCa3_3_0_, qaitem0_.answerDetails as answerDe4_3_0_, qaitem0_.qaItemStatus as qaItemSt5_3_0_, qaitem0_.dialogId as dialogId3_0_ from QAItem qaitem0_ where qaitem0_.id=?
     [java] DEBUG [main] (AbstractBatcher.java:476) - preparing statement
     [java] DEBUG [main] (NullableType.java:133) - binding '2' to parameter: 1
     [java] DEBUG [main] (AbstractBatcher.java:374) - about to open ResultSet (open ResultSets: 0, globally: 0)
     [java] DEBUG [main] (Loader.java:682) - processing result set
     [java] DEBUG [main] (Loader.java:687) - result set row: 0
     [java] DEBUG [main] (Loader.java:1164) - result row: EntityKey[se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (Loader.java:1346) - Initializing object from ResultSet: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (AbstractEntityPersister.java:2027) - Hydrating entity: [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (NullableType.java:172) - returning 'What is this newsgroup' as column: question3_0_
     [java] DEBUG [main] (NullableType.java:172) - returning 'comp.dcom.sys.cisco, which is ' as column: answerCa3_3_0_
     [java] DEBUG [main] (NullableType.java:172) - returning 'comp.dcom.sys.cisco, which is gatewayed to the mailing list<a href="mailto:cisco@spot.colorado.edu">cisco@spot.colorado.edu</a>, is a newsgroup for discussion of ciscohardware, software, and related issues. Remember that you can alsoconsult with cisco tec' as column: answerDe4_3_0_
     [java] DEBUG [main] (EnumType.java:79) - Returning null as column qaItemSt5_3_0_
     [java] DEBUG [main] (NullableType.java:166) - returning null as column: dialogId3_0_
     [java] DEBUG [main] (Loader.java:709) - done processing result set (1 rows)
     [java] DEBUG [main] (AbstractBatcher.java:381) - about to close ResultSet (open ResultSets: 1, globally: 1)
     [java] DEBUG [main] (AbstractBatcher.java:366) - about to close PreparedStatement (open PreparedStatements: 1, globally: 1)
     [java] DEBUG [main] (AbstractBatcher.java:525) - closing statement
     [java] DEBUG [main] (Loader.java:839) - total objects hydrated: 1
     [java] DEBUG [main] (TwoPhaseLoad.java:107) - resolving associations for [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (TwoPhaseLoad.java:206) - done materializing entity [se.icepeak.acc.datamodel.QAItem#2]
     [java] DEBUG [main] (StatefulPersistenceContext.java:777) - initializing non-lazy collections
     [java] DEBUG [main] (Loader.java:1874) - done entity load
     [java] DEBUG [main] (LuceneRetrieveFilter.java:89) - Got results: [[Id: null Question:null AnswerCaption:null AnswerDetails:null]]
     [java] DEBUG [main] (AbstractFlushingEventListener.java:58) - flushing session
     [java] DEBUG [main] (AbstractFlushingEventListener.java:111) - processing flush-time cascades
     [java] DEBUG [main] (AbstractFlushingEventListener.java:154) - dirty checking collections
     [java] DEBUG [main] (AbstractFlushingEventListener.java:171) - Flushing entities and processing referenced collections
     [java] DEBUG [main] (AbstractFlushingEventListener.java:210) - Processing unreferenced collections
     [java] DEBUG [main] (AbstractFlushingEventListener.java:224) - Scheduling collection removes/(re)creates/updates
     [java] DEBUG [main] (AbstractFlushingEventListener.java:85) - Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
     [java] DEBUG [main] (AbstractFlushingEventListener.java:91) - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
     [java] DEBUG [main] (Printer.java:83) - listing entities:
     [java] DEBUG [main] (Printer.java:90) - se.icepeak.acc.datamodel.QAItem{id=2, qaItemStatus=null, answerDetails=comp.dcom.sys.cisco, which is gatewayed to the mailing list<a href="mailto:cisco@spot.colorado.edu">cisco@spot.colorado.edu</a>, is a newsgroup for discussion of ciscohardware, software, and related issues. Remember that you can alsoconsult with cisco tec, answerCaption=comp.dcom.sys.cisco, which is , question=What is this newsgroup, dialogId=null}
     [java] DEBUG [main] (AbstractFlushingEventListener.java:290) - executing flush
     [java] DEBUG [main] (ConnectionManager.java:467) - registering flush begin
     [java] DEBUG [main] (ConnectionManager.java:476) - registering flush end
     [java] DEBUG [main] (AbstractFlushingEventListener.java:321) - post flush
     [java] DEBUG [main] (SessionImpl.java:273) - closing session
     [java] DEBUG [main] (ConnectionManager.java:378) - performing cleanup
     [java] DEBUG [main] (ConnectionManager.java:439) - releasing JDBC connection [ (open PreparedStatements: 0, globally: 0) (open ResultSets: 0, globally: 0)]
     [java] DEBUG [main] (JDBCContext.java:215) - after transaction completion
     [java] DEBUG [main] (ConnectionManager.java:402) - aggressively releasing JDBC connection
     [java] DEBUG [main] (SessionImpl.java:422) - after transaction completion
     [java] DEBUG [main] (Explore.java:224) - No reset of FAQ

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

Not sure if this is related. But I am using lucene-core-2.0.0. This is since 2.1.0 seems not to work well with Hibernate Search. However, I just saw that 2.1.0 is included in Hibernate Search. So this is a bit odd. This is what I get if I use lucene-core-2.1.0:

Code:

     [java] Caused by: java.lang.reflect.InvocationTargetException
     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     [java]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [java]     at java.lang.reflect.Method.invoke(Method.java:597)
     [java]     at org.hibernate.context.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:301)
     [java]     ... 8 more
     [java] Caused by: java.lang.NoSuchMethodError: org.apache.lucene.document.Document.add(Lorg/apache/lucene/document/Field;)V
     [java]     at org.hibernate.search.engine.DocumentBuilder.getDocument(DocumentBuilder.java:234)     [java]     at org.hibernate.search.event.FullTextIndexEventListener.onPostInsert(FullTextIndexEventListener.java:154)
     [java]     at org.hibernate.action.EntityIdentityInsertAction.postInsert(EntityIdentityInsertAction.java:90)
     [java]     at org.hibernate.action.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:66)
     [java]     at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:248)
     [java]     at org.hibernate.event.def.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:298)
     [java]     at org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:181)
     [java]     at org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:107)
     [java]     at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:187)
     [java]     at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:172)
     [java]     at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:94)
     [java]     at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:70)
     [java]     at org.hibernate.impl.SessionImpl.fireSaveOrUpdate(SessionImpl.java:507)
     [java]     at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:499)
     [java]     at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:495)
     [java]     ... 13 more
     [java] Java Result: 1

[/code]

emmanuel · **Posted:** Tue Mar 27, 2007 9:42 am

To use Lucene 2.1.0, you need to use Hibernate Annotations 3.3.0.GA, the associated hibernate-commons-annotations.jar, which you already seem to use??), anf Hibernate Search 3.0.0.Beta1
Can you give it a try?

emmanuel · **Posted:** Tue Mar 27, 2007 9:49 am

Well apparently you have the correct object loaded

Code:

se.icepeak.acc.datamodel.QAItem{id=2, qaItemStatus=null, answerDetails=comp.dcom.sys.cisco, which is gatewayed to the mailing list<a href="mailto:cisco@spot.colorado.edu">cisco@spot.colorado.edu</a>, is a newsgroup for discussion of ciscohardware, software, and related issues. Remember that you can alsoconsult with cisco tec, answerCaption=comp.dcom.sys.cisco, which is , question=What is this newsgroup, dialogId=null} 

So I don't really understand. Can you show the precise loading code including how LuceneRetrieveFilter.java:89 print the object

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

emmanuel wrote:

To use Lucene 2.1.0, you need to use Hibernate Annotations 3.3.0.GA, the associated hibernate-commons-annotations.jar, which you already seem to use??), anf Hibernate Search 3.0.0.Beta1
Can you give it a try?

Ok good, now I can use Lucene 2.1.0. Unfortunately, the problem did not go away with that change. The problem is still there.

/Lars

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

emmanuel wrote:

Well apparently you have the correct object loaded

Code:

se.icepeak.acc.datamodel.QAItem{id=2, qaItemStatus=null, answerDetails=comp.dcom.sys.cisco, which is gatewayed to the mailing list<a href="mailto:cisco@spot.colorado.edu">cisco@spot.colorado.edu</a>, is a newsgroup for discussion of ciscohardware, software, and related issues. Remember that you can alsoconsult with cisco tec, answerCaption=comp.dcom.sys.cisco, which is , question=What is this newsgroup, dialogId=null} 

So I don't really understand. Can you show the precise loading code including how LuceneRetrieveFilter.java:89 print the object

Ok.

This is how it looks in the LuceneRetrieveFilter (it only has one method execute which looks as follows):

Code:

    public List<QAItem> execute(String question, List<QAItem> qaItems, 
                                int offset, 
                                int maxNoOfHits) 
        throws FilterExecuteException {
        Session session2 = HibernateUtil.getSessionFactory().openSession();
        FullTextSession fullTextSession2 = Search.createFullTextSession(session2);
//          QueryParser parser = new QueryParser("answerCaption", new LuceneAnalyzer()); // TODO
        QueryParser parser = new QueryParser("question", new StopAnalyzer()); // TODO
        org.apache.lucene.search.Query luceneQuery;
        try {
            log.debug("Executing FAQ question: " + question);
            luceneQuery = 
                parser.parse(question); // TODO
        } catch (ParseException e) {
            throw new FilterExecuteException(getClass(), e);
        }
        org.hibernate.Query fullTextQuery = fullTextSession2.createFullTextQuery(luceneQuery);
        fullTextQuery.setFirstResult(offset);
        if (maxNoOfHits > 0) {
            fullTextQuery.setMaxResults(offset + maxNoOfHits);
        }
        List<QAItem> results = fullTextQuery.list(); //return a list of managed objects
        log.debug("Got results: " + results);
//         session.close();
        return results;
    }
    

This is how it looks in QAItem:

Code:

@Entity
@Indexed  // for lucene indexing
    // TODO: could not get default property to work
    // the index should not be hard-coded like this!
public class QAItem {

    @Id 
    @GeneratedValue
    @DocumentId // for lucene indexing (required)
    private Long id;

    @Field(index=Index.TOKENIZED, store=Store.YES)   
    @Boost(3)                                        
    private String question;

    @Field(index=Index.TOKENIZED, store=Store.YES)   
    @Boost(3)                          
    private String answerCaption;

    @Field(index=Index.TOKENIZED, store=Store.YES)    
    private String answerDetails;

    private QAItemStatus qaItemStatus;

    private Long dialogId;

    public QAItem (){
    }

    /**
     * Gets the value of id
     *
     * @return the value of id
     */
    public final Long getId() {
        return this.id;
    }

    /**
     * Sets the value of id
     *
     * @param argId Value to assign to this.id
     */
    public final void setId(final Long argId) {
        this.id = argId;
    }

    /**
     * Gets the value of question
     *
     * @return the value of question
     */
    public final String getQuestion() {
        return this.question;
    }

    /**
     * Sets the value of question
     *
     * @param argQuestion Value to assign to this.question
     */
    public final void setQuestion(final String argQuestion) {
        this.question = argQuestion;
    }

    /**
     * Gets the value of answerCaption
     *
     * @return the value of answerCaption
     */
    public final String getAnswerCaption() {
        return this.answerCaption;
    }

    /**
     * Sets the value of answerCaption
     *
     * @param argAnswerCaption Value to assign to this.answerCaption
     */
    public final void setAnswerCaption(final String argAnswerCaption) {
        this.answerCaption = argAnswerCaption;
    }

    /**
     * Gets the value of answerDetails
     *
     * @return the value of answerDetails
     */
    public final String getAnswerDetails() {
        return this.answerDetails;
    }

    /**
     * Sets the value of answerDetails
     *
     * @param argAnswerDetails Value to assign to this.answerDetails
     */
    public final void setAnswerDetails(final String argAnswerDetails) {
        this.answerDetails = argAnswerDetails;
    }


    /**
     * Gets the value of qaItemStatus
     *
     * @return the value of qaItemStatus
     */
    public final QAItemStatus getQaItemStatus() {
        return this.qaItemStatus;
    }

    /**
     * Sets the value of qaItemStatus
     *
     * @param argQaItemStatus Value to assign to this.qaItemStatus
     */
    public final void setQaItemStatus(final QAItemStatus argQaItemStatus) {
        this.qaItemStatus = argQaItemStatus;
    }


    /**
     * Gets the value of dialogId
     *
     * @return the value of dialogId
     */
    public final Long getDialogId() {
        return this.dialogId;
    }

    /**
     * Sets the value of dialogId
     *
     * @param argDialogId Value to assign to this.dialogId
     */
    public final void setDialogId(final Long argDialogId) {
        this.dialogId = argDialogId;
    }

    public final String toString() {
        return "[Id: " + id +
            " Question:" + question + 
            " AnswerCaption:" + answerCaption +
            " AnswerDetails:" + answerDetails + "]";
    }

emmanuel · **Posted:** Tue Mar 27, 2007 10:24 am

can you try and use getId(), getQuestion() in your toString() method?

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

Ok, I just did that and it does not change the result. (Originally I also saved results on file and got nulls there also, and then I did not have the toString-method which is only for debug)

/Lars

emmanuel · **Posted:** Tue Mar 27, 2007 10:39 am

Ah I found the issue. toString must not be final. your class must not have final methods actually

larde · **Joined:** Mon Mar 26, 2007 7:27 am **Posts:** 9

Great! That did the trick! Thanks a lot, I was not even close to that one...! ;-)

The final stuff is generated by Emacs JDEE so this may be a problem for mor e than just me...if one does not think about it.

Is this just for Hibernate Search or should I change all my persistent classes...it seems that they have worked so far besides this problem.

Lars

emmanuel · **Posted:** Tue Mar 27, 2007 2:54 pm

All persistent classes that can be lazy loaded.
Nah, I'm afraid to tell you emacs is no longer the IDE market leader.