How to search fields with wildcard and spaces

jeremycod · **Joined:** Mon Jul 16, 2012 1:11 pm **Posts:** 12

Hi,
I have a search box that performs a search on title field based on the given input, so the user has recommended all available titles starting with the text inserted. It works fine until space is entered. Then the result disapear. For example, I want "Learning H" to give me "Learning Hibernate" as the result. However, this doesn't happen. could you please advice me what should I use here instead.

Code:

QueryBuilder qBuilder = fullTextSession.getSearchFactory()
            .buildQueryBuilder().forEntity(LearningGoal.class).get();
      Query query = qBuilder.keyword().wildcard().onField("title")
            .matching(searchString + "*").createQuery();
        
      BooleanQuery bQuery = new BooleanQuery();
      bQuery.add(query, BooleanClause.Occur.MUST);
      for (LearningGoal exGoal : existingGoals) {
         Term omittedTerm = new Term("id", String.valueOf(exGoal.getId()));
         bQuery.add(new TermQuery(omittedTerm), BooleanClause.Occur.MUST_NOT);
      }
      @SuppressWarnings("unused")
      org.hibernate.Query hibQuery = fullTextSession.createFullTextQuery(
            query, LearningGoal.class);
      

Hibernate class is annotated as follows:

Code:

@AnalyzerDef(name = "searchtokenanalyzer", 
   tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), 
   filters = {
      @TokenFilterDef(factory = StandardFilterFactory.class),
      @TokenFilterDef(factory = LowerCaseFilterFactory.class),
      @TokenFilterDef(factory = StopFilterFactory.class, 
   params = { @Parameter(name = "ignoreCase", value = "true") }) })
@Analyzer(definition = "searchtokenanalyzer")
public class LearningGoal extends Node {

jeremycod · **Joined:** Mon Jul 16, 2012 1:11 pm **Posts:** 12

Does anybody have some idea about this?

sanne.grinovero · **Posted:** Sat Mar 09, 2013 8:06 pm

Hi, using TermQuery is suited only for keywords which have a value exactly matching the output of the Analyzer you are using: you should pre-process terms with your analyzer when you want to use the Lucene API to create Queries.

An alternative is to use the Search DSL : http://docs.jboss.org/hibernate/search/4.2/reference/en-US/html_single/#search-query-querydsl
This alternative automatically applies the correct analyzer on each field.

In the specific case, the StandardTokenizerFactory - which you're using - is splitting the terms on whitespace, so any keyword will be stored separately! you would need to query for "learning" and "hibernate" as two different terms if you want to use the TermQuery.

jeremycod · **Joined:** Mon Jul 16, 2012 1:11 pm **Posts:** 12

sanne.grinovero wrote:

In the specific case, the StandardTokenizerFactory - which you're using - is splitting the terms on whitespace, so any keyword will be stored separately! you would need to query for "learning" and "hibernate" as two different terms if you want to use the TermQuery.

Thank you for your answer, but I'm not sure if my post was not clear enough or I didn't understand your idea. The query I'm using for searching titles doesn't use TermQuery. TermQuery is used in BooleanQuery that should exclude specific LearningGoal based on id.

Code:

for (LearningGoal exGoal : existingGoals) {
         Term omittedTerm = new Term("id", String.valueOf(exGoal.getId()));
         bQuery.add(new TermQuery(omittedTerm), BooleanClause.Occur.MUST_NOT);
      }

The first query that searches LearningGoal by title, doesn't have any TermQuery. I tried even without this BooleanQuery that uses the TermQuery and the result is the same. So the following code still doesn't work:

Code:

QueryBuilder qBuilder = fullTextSession.getSearchFactory()
            .buildQueryBuilder().forEntity(LearningGoal.class).get();
      Query query = qBuilder.keyword().wildcard().onField("title")
            .matching(searchString + "*").createQuery();
org.hibernate.Query hibQuery = fullTextSession.createFullTextQuery(
            bQuery, LearningGoal.class);

Isn't this the same approach as suggested in http://docs.jboss.org/hibernate/search/4.2/reference/en-US/html_single/#search-query-querydsl ?

andlio · **Joined:** Tue Jul 19, 2011 2:16 pm **Posts:** 19

Hi,

I have the same issue. I try to search over my index with (or without) withespace, but I don't know which analyzer I have to use.

My field is annoted as the following :

Code:

@Field(name = "ALL")
@Column(name = "V_NAME", length = 40)
private String name;

And here I build my query :

Code:

Query luceneQuery =
   queryBuilder
      .bool()
         .must( // Keyword search
            queryBuilder
               .keyword() 
                  .wildcard()
                  .onField("ALL")
                  .matching(term)
               .createQuery()
         )
         .must( // Ignore soft deleted entities 
            queryBuilder
               .keyword()
                  .onField("softDelete")
                  .matching("_null_")
               .createQuery()
         )
      .createQuery();

When I search a single word it works fine (with or without wildcards). But I would like to search a name with whitespace for example (eg. "Sylvester Stallone", or "Sylvester Sta*", or "Syl* sta*"...).
How could I do it ?

jeremycod · **Joined:** Mon Jul 16, 2012 1:11 pm **Posts:** 12

andlio wrote:

When I search a single word it works fine (with or without wildcards). But I would like to search a name with whitespace for example (eg. "Sylvester Stallone", or "Sylvester Sta*", or "Syl* sta*"...).
How could I do it ?

I found a workaround for this and I described my solution in this post http://stackoverflow.com/questions/15285117/how-to-search-fields-with-wildcard-and-spaces-in-hibernate-search. In this example, I'm using the title with multiple words, but I also applied the same approach to exactly the same problem as you have (Person.lastName, Person.firstName). I also wanted to search "Sylvester Sta*" and to list Sylvester Stalone. The same approach did a job, so now it works as I wanted.

Hope this will help
Zoran

andlio · **Joined:** Tue Jul 19, 2011 2:16 pm **Posts:** 19

Hi,

Thank you very much for your help.

If I understand the way you're building the query, the query for the term "Sylverster Sta*" can return the names "Sylverster Stallone", but also "Stallone Sylvester", "State sylvester"...
For my case, the property "name" must be analyze as a single word even if it contains whitespace. It is to say, when I search for "Sylverster Sta*" I only want the names beginning with "Sylvester sta" but not "sta* sylvester"...

Any idea ?

jeremycod · **Joined:** Mon Jul 16, 2012 1:11 pm **Posts:** 12

andlio wrote:

Hi,
For my case, the property "name" must be analyze as a single word even if it contains whitespace. It is to say, when I search for "Sylverster Sta*" I only want the names beginning with "Sylvester sta" but not "sta* sylvester"...

Any idea ?

My original idea was exactly the same, but I didn't found any solution. Lucene keyword wildcard search is limited and can't search over the fields indexed and analyzed as tokens. Actually it can match individual tokens, but not phrase as you are expecting using keyword search with wildcards. I haven't tried as I need analyzed fields for other kinds of search, but you can try if that works for you to store and search fields without analysing it.