-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: StandardAnalyzer not applied?
PostPosted: Wed Jul 20, 2011 5:53 pm 
Newbie

Joined: Fri Oct 10, 2008 9:17 am
Posts: 13
Greetings all,

I need to be able to search an index for Chinese characters and this should be possible using the StandarardAnalyzer, which I believe is the default?

However, I'm trying to execute the following code on get no hits:

Code:

String searchTerm = "中國";
BooleanQuery searchQuery = new BooleanQuery();
searchQuery.add(new TermQuery(new Term("address.country", searchTerm )), BooleanClause.Occur.SHOULD);
      
FullTextSession fullTextSession = Search.getFullTextSession( getSessionFactory().getCurrentSession() );
FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(searchQuery, Address.class );
   
logger.error("Query: " + fullTextQuery.getQueryString());



execute fullTextQuery.list() .. etc



The printout is "Query: +address.country:中國"

If I run that same query via Luke on the same index using the StandardAnalyzer I get the expected results. I'm at a bit of a loss so I'm wondering if anyone more experienced here can tell what I've missed?

I've also explicitly set the analyzer in the Hibernate config:

Code:

<property name="hibernateProperties">
          <props>
             <prop key="dialect">org.hibernate.dialect.MySQLDialect</prop>
      <prop key="hibernate.search.default.indexBase">/home/somefilesystem/indexes</prop>
              <prop key="hibernate.search.default.directory_provider">org.hibernate.search.store.FSDirectoryProvider</prop>
              <prop key="hibernate.search.analyzer">org.apache.lucene.analysis.standard.StandardAnalyzer</prop>
              <prop key="hibernate.connection.useUnicode">true</prop>
              <prop key="hibernate.connection.charSet">utf8</prop>
              <prop key="hibernate.connection.characterEncoding">utf8</prop>
           </props>
</property>



I'm using Hibernate Search 3.4.0


Top
 Profile  
 
 Post subject: Re: StandardAnalyzer not applied?
PostPosted: Thu Jul 21, 2011 3:49 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

if you are using a term query the analyzer does not get applied on the input (search) string. If you want to use a term query and have the same behavior as with a query parser you need to generate the search tokens first as the analyzer would generate them. Have a look at this related thread - viewtopic.php?f=9&t=1009934

Also, how exactly is address.country indexed?

--Hardy


Top
 Profile  
 
 Post subject: Re: StandardAnalyzer not applied?
PostPosted: Thu Jul 21, 2011 12:07 pm 
Newbie

Joined: Fri Oct 10, 2008 9:17 am
Posts: 13
The field is indexed simply like this:

Code:
@Indexed
public class Address
{
   @DocumentId
   private int id;
   @Field
   private String country

        ....
}


I have to say I'm confused.

Say the value of the field is "China 中國", using the above method, if the searchTerm is "china" I get a hit, but if it is "中國", I do not, and I don't understand why.

I tried splitting the Chinese characters up and adding them to a PhraseQuery, with the resulting query string:

+(address.country:"中 國")

This gets a hit, but is this the right way to go about it?


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.