Fuzzy logic

sol · **Joined:** Fri Sep 18, 2009 7:04 pm **Posts:** 12

So, I'm trying to do some very fuzzy text matching using hibernate. What my goal is is to have a keyword help system very similar to the one in Microsoft Word: The user types in a question, and a list of frequently asked questions that are most relevant are returned. In my case, the frequently asked questions will be in a database, of course.

I've tried using the Example and Criteria classes with the enableLike(MatchMode.Anywhere) method, but this doesn't seem quite fuzzy or "smart" enough; it also doesn't return a ranking of "strength of match" or "relevance" of each returned result.

Does anyone have any ideas on how I might implement something like this? Thanks.

Sol

Serethos · **Joined:** Thu Oct 04, 2007 12:22 pm **Posts:** 48

As there is afaik no fuzzy matching logic in criterias you need to implement the match mode yourself. Define a match mode
and extend the Criteria class. But implementing a fuzzy search is no trivial task. I never experimented with "Hibernate Search"
but perhaps this project will help you.

rhasselbaum · **Joined:** Thu Oct 07, 2004 4:45 pm **Posts:** 92

Yes, take a look at Hibernate Search.

sol · **Joined:** Fri Sep 18, 2009 7:04 pm **Posts:** 12

OK, so I've been trying to implement a hibernate search, and am having a lot of trouble. Here's some code snippets:

From the persistence.xml:

Code:

<property name="hibernate.search.default.directory_provider" value="org.hibernate.search.store.FSDirectoryProvider"/> 

        <property name="hibernate.search.default.indexBase" value="/var/lucene/indexes"/> 

From my services class, for indexing in Lucene:

Code:

public void indexQuestions()
   {
      EntityManager em = entityManagerFactory.createEntityManager();
      FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(em);
      List<Questions> questions = em.createQuery("SELECT question FROM Questions AS question").getResultList();
      for (Questions question : questions)
      {
         fullTextEntityManager.index(question);
      }
      em.getTransaction().commit();
      em.close();
   }

For the actual search:

Code:

public List<Questions> getQuestions(Questions toBeMatched)
   {
      EntityManager em = entityManagerFactory.createEntityManager();
      FullTextEntityManager fullTextEntityManager = org.hibernate.search.jpa.Search.getFullTextEntityManager(em);
      em.getTransaction().begin();
      
      String[] fields = new String[]{"questionText"};
      MultiFieldQueryParser parser = new MultiFieldQueryParser(fields, new StandardAnalyzer());
      try{
         org.apache.lucene.search.Query query = parser.parse(toBeMatched.getQuestiontext());
         javax.persistence.Query persistenceQuery = fullTextEntityManager.createFullTextQuery(query, Questions.class);
         List<Questions> result = persistenceQuery.getResultList();
         em.getTransaction().commit();
         em.close();
         return result;
      }
      catch(ParseException p){}
      return null;      
   }

When I try to start up my application, I get a ParseException from Lucene. What's really bizarre is that it happens before I call any methods that have anything to do with Lucene. I'm thinking maybe my annotations in the entity POJO aren't right, but I'm not sure...Thoughts?? Thanks.

Sol

sanne.grinovero · **Posted:** Tue May 18, 2010 5:01 pm

I'm sorry for the late answer, we from the Search team don't notice all posts on the wrong forum. Hibernate Search related questions should be posted on the Search forum.

If you still need help, please post to the correct forum.