Weird Behaviour of Hibernate search :(

sachin.mobite · **Joined:** Thu Sep 11, 2008 5:33 am **Posts:** 28

Hi All,

I am getting a Strange behaviour with it
i have stored data in multiple rows each row contains 1 English alphabet.

that is
row 1-------------------a
row2--------------------b
-
-
-
-
row----------------------z

just for testing purpose :)

it gives perfect result for some query or some specific characters only

like if i search letter a then it returns nothing and if i search letter c,d, or s it gives wrong result :(, and for rest of the letters it works fine.

here is the code for indexer......

Code:

FullTextSession fullTextSession = getFullTextSession();
      List results = fullTextSession.createCriteria(InstantMessage.class).list();
      for (Object im : results) {
         fullTextSession.index(im);
      } 

for search query

Code:

String temp[] = { "message" };
      QueryParser parser = new MultiFieldQueryParser(temp,
            new StandardAnalyzer());
      Query query = null;
      org.apache.lucene.search.Query luceneQuery = null;
      try {
         luceneQuery = parser.parse(getSearchString());
      } catch (ParseException parEx) {
         parEx.printStackTrace();
      }
      return ((FullTextEntityManager) em).createFullTextQuery(luceneQuery,
            InstantMessage.class);

Any Clue Guys

Regards,
Sachin Parnami

sanne.grinovero · **Posted:** Mon Oct 27, 2008 5:36 am

Hello,
you are getting in trouble because of the Analyzer being used.
The Analyzer is configurable, and you could plug your own implementation.
It's purpose is to split the document's content in word tokens, and is applied both during indexing of the content and during the processing of the Query.

The StandardAnalyzer, which is default and thus you are using, is being "clever" and it is targeted to analyze english language: in particular one of the things it does (besides splitting) is removing "noise": the word "a" in english is too common to be of any use, so it gets removed.

Keep in mind that FullText search is not like a SQLcolumn, it's not about exact values but about word relevance. You will probably get better results using "human english text" than the letters of the alphabet.

If you need different tokenization behaviour (or other languages) you should configure a different Analyzer.

sachin.mobite · **Joined:** Thu Sep 11, 2008 5:33 am **Posts:** 28

Hi Sanne,

Thanks for your detailed description :)

But still i have a doubt, how do i avoid the search for common words of English grammar. Because it gives incorrect result while searching those content :(

sachin.mobite · **Joined:** Thu Sep 11, 2008 5:33 am **Posts:** 28

Now the case with StandardAnalyzer() it gives same result multiple time with few words like
i tried with Hi

any clue why is it so?

sachin.mobite · **Joined:** Thu Sep 11, 2008 5:33 am **Posts:** 28

I account for your help Guys :) no clue about this problem

sanne.grinovero · **Posted:** Fri Oct 31, 2008 8:17 am

try using Luke and experiment with other Analyzers, like SimpleAnalyzer.
You really need to tell more about your problem, but please first read something about how the Analyzers work.
The "Hibernate Search in Action" may be helpful.

sachin.mobite · **Joined:** Thu Sep 11, 2008 5:33 am **Posts:** 28

Hi Sanne,

Thanks for your reply sure i am doing that too :)