-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 
Author Message
 Post subject: MultiFieldParser not working very well
PostPosted: Sun Jan 13, 2008 3:45 pm 
Newbie

Joined: Sun Dec 23, 2007 2:30 pm
Posts: 6
Tutorial.Java
Code:
package roseindia.net.dao.hibernate;

import java.io.Serializable;

import javax.persistence.Id;

import org.hibernate.annotations.Entity;
import org.hibernate.search.annotations.Analyzer;
import org.hibernate.search.annotations.DocumentId;
import org.hibernate.search.annotations.Field;
import org.hibernate.search.annotations.Index;
import org.hibernate.search.annotations.Indexed;
import org.hibernate.search.annotations.Store;


@Entity
@Indexed
@Analyzer (impl = roseindia.net.dao.hibernate.English.class)
public class Tutorial implements Serializable {

    /** identifier field */
   @Id
   @DocumentId
    private Integer id;

    /** persistent field */
    @Field(index=Index.TOKENIZED, store=Store.YES)
    private String shortdesc;

    /** persistent field */
    @Field(index=Index.TOKENIZED, store=Store.YES)
    private String longdesc;

    /** persistent field */
    //@Field(index=Index.TOKENIZED, store=Store.YES)
    private String pageurl;

    /** full constructor */
    public Tutorial(Integer id, String shortdesc, String longdesc, String pageurl) {
        this.id = id;
        this.shortdesc = shortdesc;
        this.longdesc = longdesc;
        this.pageurl = pageurl;
    }

    /** default constructor */
    public Tutorial() {
    }

    public Integer getId() {
        return this.id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getShortdesc() {
        return this.shortdesc;
    }

    public void setShortdesc(String shortdesc) {
        this.shortdesc = shortdesc;
    }

    public String getLongdesc() {
        return this.longdesc;
    }

    public void setLongdesc(String longdesc) {
        this.longdesc = longdesc;
    }

    public String getPageurl() {
        return this.pageurl;
    }

    public void setPageurl(String pageurl) {
        this.pageurl = pageurl;
    }

}


English.java
Code:
public class English extends Analyzer
{
   /**
        This method useful in following situation<br>
      Assume that one of our sentContent entities contains the display name "WWE rocks" <br>
      and you want to get hits for all of the following queries: "rock", "rocks", "rocked" <br>
      and "rocking".In this case this method applies word stemming during the indexing process.
     */
    @Override
    public TokenStream tokenStream(String fieldName, Reader reader) {
        TokenStream result = new StandardTokenizer(reader);
        result = new StandardFilter(result);
        //result = new LowerCaseFilter(result);
         String fi = null;
         try{fi = result.next().toString();}catch(Exception e){}
         System.out.println("result  "+result+"   "+fi );
         result = new SnowballFilter(result, "English");
        return result;
    }
}


Main Logic for Index and Search
Code:
Session  session = _factory.openSession();


       FullTextSession fullTextSession = Search.createFullTextSession(session);


       Transaction tx=fullTextSession.beginTransaction();
       List<Tutorial> tutorials = session.createQuery("from Tutorial as tutorials").list();
       for (Tutorial tutorial : tutorials) {
          System.out.println("Indexing");
       fullTextSession.index(tutorial);
       }
       tx.commit();

-------------------
session = _factory.openSession();


          fullTextSession = Search.createFullTextSession(session);


           tx=fullTextSession.beginTransaction();
           MultiFieldQueryParser multiFieldParser = new MultiFieldQueryParser(
                  new String[] {"shortdesc","longdesc","pageurl"},
                  new StandardAnalyzer()
                  );
            Query luceneQuery = null;

            luceneQuery = multiFieldParser.parse("gun");

            org.hibernate.Query hQuery = fullTextSession.createFullTextQuery(
                                 luceneQuery,
                                 Tutorial.class
                                 );
            //hQuery.setFetchSize(2);


             List all = hQuery.list();
             System.out.println("Test Size="+all.size());
             tx.commit();



In the above case search is working only on longdesc field.
Search is not working on shordesc field,even though perfect match avilable on database.

Please guide me, If i missing any thing to get full search!!
Thanks
Ismail


Top
 Profile  
 
 Post subject: Re: MultiFieldParser not working very well
PostPosted: Mon Jan 14, 2008 2:39 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
[quote="ismail_mca"]Tutorial.Java
Code:
           MultiFieldQueryParser multiFieldParser = new MultiFieldQueryParser(
                  new String[] {"shortdesc","longdesc","pageurl"},
                  new StandardAnalyzer()
                  );


One important rule when working with Lucene and QueryParsers is to use the same analyzer at index and search time. If you use the English snowball stemmer all words (tokens) get indexed in their stemmed form (eg cars becomes car ...) For this reason it is important that you at search time apply the same analyzer and hence stemming algorithm to the search criteria. In your case:

Code:
           MultiFieldQueryParser multiFieldParser = new MultiFieldQueryParser(
                  new String[] {"shortdesc","longdesc","pageurl"},
                  new English()
                  );


I recommend using Luke (http://www.getopt.org/luke/) to actually inspect the generated Lucene index to get an understanding on what is going on.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 14, 2008 10:03 am 
Newbie

Joined: Sun Dec 23, 2007 2:30 pm
Posts: 6
Thanks for the response.

I tried the following but not working

Code:
MultiFieldQueryParser multiFieldParser = new MultiFieldQueryParser(
                  new String[] {"shortdesc","longdesc","pageurl"},
                  new English()
                  );


I have removed following from tutorial class
Code:
@Analyzer (impl = roseindia.net.dao.hibernate.English.class)


and used the following for search and it works as expected
Code:
MultiFieldQueryParser multiFieldParser = new MultiFieldQueryParser(
                  new String[] {"shortdesc","longdesc","pageurl"},
                  new StandardAnalyzer()
                  );


Any way thanks for the analyzer tip.

Pls, send me any code example for luke lucene index.

Thanks
Ismail


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 14, 2008 12:27 pm 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Interesting, the first version should work as well. The second version works, because now you are using in both cases the StandardAnalyzer. If this fulfills your usecase then great, but it is of course not the same as using a language aware analyzer.

Regarding Luke - there is really no code to send. Download Luke from the website and start it, then open the Lucene index created by Hibernate Search during indexing (only works of course if you use FSDirectoryProvider). You should be able to see the indexed Lucene documents.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.