-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 13 posts ] 
Author Message
 Post subject: Help with hibernate search
PostPosted: Tue Nov 13, 2012 1:33 pm 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
Hi. can anyone help me? i create the code:

Code:
@Entity
@Table(name = "tb_comunicado", catalog = "integrabrasil", schema = "db_integra_brasil")
@SequenceGenerator(sequenceName="db_integra_brasil.tb_comunicado_seq",allocationSize=1, initialValue=0, name = "db_integra_brasil.tb_comunicado_seq")
@Indexed
@AnalyzerDef(name = "customanalyzer",
charFilters = {
      @CharFilterDef(factory = MappingCharFilterFactory.class, params = {
         @Parameter(name = "mapping", value = "br/com/dnasolution/integrabrasil/entity/mapping-chars.properties")
         }),
      @CharFilterDef(factory = HTMLStripCharFilterFactory.class)
      },
tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
  @TokenFilterDef(factory = LowerCaseFilterFactory.class),
  @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = {
    @Parameter(name = "language", value = "Portuguese")
  })
})
public class TbComunicado implements Serializable {
    private static final long serialVersionUID = 1L;
    @Id
    @GeneratedValue(strategy=GenerationType.SEQUENCE, generator="db_integra_brasil.tb_comunicado_seq")
    @Basic(optional = false)
    @Column(name = "cd_comunicado")
    private Integer cdComunicado;
   
    @Column(name="cd_cep", length=9)
    @Length(max = 9)
   private String cdCep;


   @Column(name="ds_mais_informacoes", length=2147483647)
   @Field(index=Index.TOKENIZED, store=Store.NO)
   @Analyzer(definition = "customanalyzer")
   private String dsMaisInformacoes;


and the search

Code:
    @Transactional
    public void createIndex(){
       try {
          FullTextSession fullTextSession = Search.getFullTextSession(getSessionFactory().getCurrentSession());
         fullTextSession.createIndexer().startAndWait();
      } catch (InterruptedException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }
    }

    private SimpleDateFormat dsf = new SimpleDateFormat("k:m:s:S");
   
   
   @Transactional
   public void buscarcomIndice2() throws ParseException {
      System.out.println("Entrou: " + dsf.format(new Date()));
      FullTextSession fullTextSession = Search.getFullTextSession(getSessionFactory().getCurrentSession());
      Transaction tx = fullTextSession.beginTransaction();
      QueryParser parser = new QueryParser(Version.LUCENE_30, "dsMaisInformacoes", 
//            new StandardAnalyzer(Version.LUCENE_30));
      new BrazilianAnalyzer(Version.LUCENE_30));
      String valor = "informacoes";
//      String valor = "anotação";
      FullTextQuery query = (FullTextQuery) fullTextSession.createFullTextQuery(parser.parse(valor), TbComunicado.class);
      List result = query.list();
      System.out.println("manha: " + dsf.format(new Date()));
      System.out.println(result.size());
   }


in my DB i have the text: some html text and the "informações" word.. i put in my mapping-chars.properties to change ç to c and õ to o (upper and lower) but i can't find that line..

tks


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 5:19 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

two things which might help you to narrow down your problem. First, have you checked the generated Lucene index? You can do so using the tool Luke. It will first of all show you if all the text is indexed as you expect it to be indexed.

Second, during indexing you are using a different analyzer (customanalyzer) than during search (BrazilianAnalyzer). You probably should use the same. You can get hold of your custom analyzer via SearchFactory#getAnalyzer(String).

--Hardy


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 8:46 am 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
how i can use the same in both?? where i need to change?? and another thing, how i remove the html tags???



i download de luke, and in my db i have for consolidaca but if i search for consolidaca from my code i cant find :(


tks


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 9:00 am 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
the first i resolve i only put
Code:
@TokenFilterDef(factory = BrazilianStemFilterFactory.class),


but i need to remove the html tags.. can u help me ??

tks


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 11:23 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
HTMLStripCharFilter might be for you. You could add it to the pipeline of your custom analyzer.

--Hardy


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 11:25 am 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
hardy i have tried but i always got error


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 11:51 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
What type of error? Do you have a stacktrace?


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 12:07 pm 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
i try this.

Code:
@AnalyzerDefs({
@AnalyzerDef(name = "customanalyzer",
charFilters = {
      @CharFilterDef(factory = MappingCharFilterFactory.class, params = {
         @Parameter(name = "mapping", value = "br/com/dnasolution/integrabrasil/entity/mapping-chars.properties")
         })
      },
tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
  @TokenFilterDef(factory = LowerCaseFilterFactory.class),
  @TokenFilterDef(factory = BrazilianStemFilterFactory.class),
  @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = {
    @Parameter(name = "language", value = "Portuguese")
  })
}),
@AnalyzerDef(name = "removeHtml",
charFilters = {
      @CharFilterDef(factory = HTMLStripCharFilterFactory.class)
}, tokenizer = @TokenizerDef(factory = HTMLStripCharFilterFactory.class)
      )
})


And i got:

Quote:
Type mismatch: cannot convert from Class<HTMLStripCharFilterFactory> to Class<? extends TokenizerFactory>


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Wed Nov 14, 2012 12:20 pm 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
You need to add it to the pipeline of your custom analyzer. First strip the html tags, then translate/map characters and then tokenize and apply filters:

Code:
      @AnalyzerDef(name = "customanalyzer",
      charFilters = {
            @CharFilterDef(factory = HTMLStripCharFilterFactory.class),
            @CharFilterDef(factory = MappingCharFilterFactory.class, params = {
                  @Parameter(name = "mapping", value = "br/com/dnasolution/integrabrasil/entity/mapping-chars.properties")
            })
      },
      tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
      filters = {
            @TokenFilterDef(factory = LowerCaseFilterFactory.class),
            @TokenFilterDef(factory = BrazilianStemFilterFactory.class),
            @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = {
                  @Parameter(name = "language", value = "Portuguese")
            })
      })


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Mon Nov 19, 2012 7:39 am 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
tks man, worked..

now did you know how i add stop words? in my filters and search?? because when i use luke to see he create a lot of index for my stop words /

tks


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Mon Nov 19, 2012 9:15 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Just add StopFilterFactory to your filters. It takes a file with stop words as input.

--Hardy


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Thu Nov 22, 2012 12:03 pm 
Newbie

Joined: Tue Oct 31, 2006 2:44 pm
Posts: 10
I add stopWords but now i got one problem

this is my Entity
Code:
@Entity
@Table(name = "tb_comunicado", catalog = "integrabrasil", schema = "db_integra_brasil")
@SequenceGenerator(sequenceName="db_integra_brasil.tb_comunicado_seq",allocationSize=1, initialValue=0, name = "db_integra_brasil.tb_comunicado_seq")
@Indexed
@AnalyzerDef(name = "customanalyzer",
charFilters = {
      @CharFilterDef(factory = HTMLStripCharFilterFactory.class),
      @CharFilterDef(factory = MappingCharFilterFactory.class, params = {
         @Parameter(name = "mapping", value = "br/com/dnasolution/integrabrasil/entity/mapping-chars.properties")
         })
      },
tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
  @TokenFilterDef(factory = LowerCaseFilterFactory.class),
  @TokenFilterDef(factory = BrazilianStemFilterFactory.class),
  @TokenFilterDef(factory = StopFilterFactory.class, params = {
     @Parameter(name="words", value= "br/com/dnasolution/integrabrasil/entity/stoplist.properties" ),
     @Parameter(name="ignoreCase", value="true")
  }),
  @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = {
    @Parameter(name = "language", value = "Portuguese")
  })
})
public class TbComunicado implements Serializable {
    private static final long serialVersionUID = 1L;
    @Id
    @GeneratedValue(strategy=GenerationType.SEQUENCE, generator="db_integra_brasil.tb_comunicado_seq")
    @Basic(optional = false)
    @Column(name = "cd_comunicado")
    private Integer cdComunicado;
   
    @Column(name="cd_cep", length=9)
    @Length(max = 9)
   private String cdCep;


   @Column(name="ds_mais_informacoes", length=2147483647)
   @Field(index=Index.TOKENIZED, store=Store.NO)
   @Analyzer(definition = "customanalyzer")
   private String dsMaisInformacoes;
..


and this is my search
Code:
FullTextSession fullTextSession = Search.getFullTextSession(getSessionFactory().getCurrentSession());
      Transaction tx = fullTextSession.beginTransaction();
      QueryParser parser = new QueryParser(Version.LUCENE_30, coluna,  new BrazilianAnalyzer(Version.LUCENE_30));
      FullTextQuery query = (FullTextQuery) fullTextSession.createFullTextQuery(parser.parse(valor), classe);
      List<TbComunicado> result = query.list();
//      tx.
      return result;


when i create my index he got the word "primeiro" (first in english) and index by "prim", but when i search the word "primeiro" he change to "primeir" and don't find, anyone know why he got this different words??

sorry my english :D


Top
 Profile  
 
 Post subject: Re: Help with hibernate search
PostPosted: Fri Nov 23, 2012 6:38 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
You are not using your custom analyzer in the search. You should use the same analyzer (aka customanalyzer), not just a new instance of new BrazilianAnalyzer. You can retrieved a named analyzer via SearchFactory#getAnalyzer(String).

--Hardy


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 13 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.