-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: Create a custom analyzer based on AnalyzerDef
PostPosted: Sat Nov 20, 2010 3:50 pm 
Newbie

Joined: Sun Jun 27, 2010 1:45 pm
Posts: 4
Hello,

I would like to create my custom analyzer equivalent to this definition:

Code:
@AnalyzerDef(name = "mysynonyms",
      tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
      filters = {
         @TokenFilterDef(factory = SynonymFilterFactory.class,
               params = {
                  @Parameter(name = "synonyms", value = "mysynonyms.TXT"),
                  @Parameter(name = "ignoreCase", value = "true"),
                  @Parameter(name = "expand", value = "true") }),
         @TokenFilterDef(factory = LowerCaseFilterFactory.class),
         @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
         @TokenFilterDef(factory = SnowballPorterFilterFactory.class,
               params = @Parameter(name="language", value="Slovenian"))
      })


I know how to create custom analyzers in Lucene, I am just fighting how to include SynonymFilterFactory into the top of the filter chain and how to pass the 3 parameters (synonyms, ignoreCase and expand).

So far...

Code:
import java.io.Reader;

import org.apache.lucene.analysis.ASCIIFoldingFilter;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.LowerCaseFilter;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.snowball.SnowballFilter;
import org.apache.lucene.analysis.standard.StandardFilter;
import org.apache.lucene.analysis.standard.StandardTokenizer;
import org.apache.lucene.util.Version;
import org.apache.solr.analysis.SynonymFilter;
import org.apache.solr.analysis.SynonymMap;

public class PorterWithSynonymsAnalyzer extends Analyzer {

private String language;
private SynonymMap synonymMap;
   
   public PorterWithSynonymsAnalyzer(String language) {
      super();
      this.language = language;
      boolean ignoreCase = true;
      synonymMap = new SynonymMap(ignoreCase);
   }

   @Override
   public TokenStream tokenStream(String fieldName, Reader reader) {
      return new SynonymFilter(
            new StandardFilter(
               new LowerCaseFilter(
                     new ASCIIFoldingFilter(
                           new SnowballFilter(new StandardTokenizer(Version.LUCENE_29, reader), language)
                           )
                     )
               ), synonymMap
            );
   }

}



I need the custom analyzer so I can dynamicaly set, via some application configuration parameters, the parameters for the filters.

-bob


Top
 Profile  
 
 Post subject: Re: Create a custom analyzer based on AnalyzerDef
PostPosted: Mon Nov 22, 2010 5:47 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

maybe it helps if you just look at the source of SynonymFilterFactory.java.

You can browse the code on GitHut - https://github.com/hibernate/hibernate-search
or Fisheye - https://fisheye2.atlassian.com/browse/Hibernate-search

--Hardy


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.