Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: Analyzer annotation added to field together with FieldBridge
PostPosted: Thu Nov 25, 2010 6:20 am 
Newbie

Joined: Thu Nov 25, 2010 5:04 am
Posts: 1
Hello,
this is my first post on this forum, so welcome all :)

My problem is that analyzer's annotation doesn't work when added to a field together with FieldBridge (works if Analyzer is set in property file or added on a class level), eg.:

Code:
@Entity
@Indexed
public class MyEntity {

   private Long id = -1L;
   private Address address;

   @Id
   @GeneratedValue(strategy = GenerationType.AUTO)
   @Column(name = "id")
   public Long getId() {
      return id;
   }

//edit: here have @ManyToOne, @JoinColumn and @Cascade annotations
   @Fields({
         @Field(name = "address", bridge = @FieldBridge(impl = AddressBridge.class), index = Index.TOKENIZED, analyzer = @Analyzer(impl = AddressAnalyzer.class)),
         @Field(name = "addressEx", bridge = @FieldBridge(impl = AddressBridge.class), index = Index.TOKENIZED, analyzer = @Analyzer(impl = OtherAddressAnalyzer.class)) })
   public Address getAddress() {
      return address;
   }
}


It does work, when I add the same Analyzer annotation on class level instead field level.
IMHO the implementation of the analyzer doesn't matter, because it does't work even for KeywordAnalyzer.
As a bridge I use simple implementation of FieldBridge, something like:

Code:
public class AddressBridge implements FieldBridge {

   @Override
   public void set(String name, Object value, Document document, LuceneOptions luceneOptions) {
      /*
       * here null- and instanceof checking
       */
      Address address = (Address) value;
      luceneOptions.addFieldToDocument(name + ".street", address.getStreet(), document);
      luceneOptions.addFieldToDocument(name + ".zip", address.getZip(), document);
   }
}

Bridge works ok. To check results i use Luke 1.0.1.
I use hibernate core 3.6.0 Final and hibernate search 3.3.0 CR1.

Is this a bug or I'm doing something wrong?
Does anyone have any idea how to fix it?

Thank you in advance for your help.


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Fri Nov 26, 2010 6:33 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Hi,
welcome to the forums.
I would expect the code you posted to work, could you please open a JIRA ticket and attach a patch with a testcase?
There are other testcases relating to analyzers in the package org.hibernate.search.test.analyzer, it should be easy to adapt one to verify this. One to cover this seems to be missing, so this might be a bug, in that case if you help with a test it shouldn't be hard to fix.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Thu Dec 30, 2010 3:15 am 
Newbie

Joined: Thu Dec 30, 2010 3:09 am
Posts: 2
Hi everybody,

is there a solution for this problem or an according JIRA issue. I've the same problem and wondered if it is me or a bug :-)

kind regards
Michael


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Fri Dec 31, 2010 6:36 am 
Newbie

Joined: Thu Dec 30, 2010 3:09 am
Posts: 2
Hi once more,

after some more research I can finally say: it was me!

There were two aspects preventing the analyzer to be invoked:

1) I've annotated the field with
Code:
index=Index.UN_TOKENIZED

2) My field bridge did not add any content using
Code:
options.addFieldToDocument(fieldName, fieldValue, document);


hope that helps

kind regards
Michael


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Sat Jan 01, 2011 11:45 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Quote:
is there a solution for this problem or an according JIRA issue. I've the same problem and wondered if it is me or a bug :-)

I added a new unit test to make sure, wasn't happy the test was missing but indeed it looks like there's no bug.

To recap for everybody's sake:
1) always check the value is not null/empty or the field will be skipped
2) the field must be tokenized (indexed)
3) custom FieldBridges must not be broken :)

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Tue Jan 31, 2012 3:53 am 
Newbie

Joined: Fri Jun 11, 2010 3:00 am
Posts: 14
Hi Sanne,

I have similar problem. FieldBridge is not broken, but analyzer defined for the field is not applied.

Versions:
Hibernate Search - 3.4.0.Final
Hibernate Search Analyzers - 3.4.0.Final
Hibernate Core - 3.6.3.Final

Code:

Apps.java
-------------------------------------------------------------------------------
Code:
@AnalyzerDefs({
@AnalyzerDef(name="stemmedAnalyzer",
             tokenizer = @TokenizerDef(factory= StandardTokenizerFactory.class),
             filters = {
                        @TokenFilterDef(factory=LowerCaseFilterFactory.class),
                        //@TokenFilterDef(factory=StandardFilterFactory.class),
                        @TokenFilterDef(factory=SnowballPorterFilterFactory.class,
                              params = @Parameter(name="language", value="English"))
}),
@AnalyzerDef(name="normalAnalyzer",
        tokenizer = @TokenizerDef(factory= StandardTokenizerFactory.class),
        filters = {
                   @TokenFilterDef(factory=LowerCaseFilterFactory.class)
})
})


@Entity
@Table(name = "apps_readonly", uniqueConstraints = @UniqueConstraint(columnNames = {
        "name", "appid" }))
@Indexed(index="Apps")
@Analyzer(definition="stemmedAnalyzer")
public class Apps implements java.io.Serializable  {

    @OneToOne
    @JoinColumn(referencedColumnName="appid", name="appid")
    @IndexedEmbedded(depth=1, prefix="metadata_")
    public Metadata getMetadata(){
        return this.metadata;
    }
   
}


Metadata.java
--------------------------------------------------------
Code:
@Entity
@Table(name = "metadata", uniqueConstraints = @UniqueConstraint(columnNames = "appid"))
public class Metadata implements java.io.Serializable {
   @Column(name = "description", nullable = false, length = 10000)
   
   @Fields({
      @Field(name = "stemmed_desc", index = Index.TOKENIZED, store = Store.NO),
      @Field(name = "unstemmed_desc", index = Index.TOKENIZED, store = Store.NO, analyzer=@Analyzer(definition="normalAnalyzer"))             
   })
   @FieldBridge(impl=ChunkDescriptionBridge.class, params=
             {@Parameter( name="chunkSize", value="100" ), @Parameter( name="delimeter", value=" " )})
    @Boost(value=0.0f)
    public String getDescription() {
      return this.description;
   }
}


ChunkDescriptionBridge
--------------------------------------------------------------------
Code:
public class ChunkDescriptionBridge implements ParameterizedBridge, FieldBridge {

   private Integer chunkSize = null;
   private String delimeter = null;
   private Integer noOfChunks = null;
   
   @Override
   public void set(String name, Object value, Document document,
         LuceneOptions luceneOptions) {
      String description = (String) value;
      String desc1 = "";
      String desc2 = "";
      String desc3 = "";
      String desc4 = "";
      
      if(StringUtils.isNotEmpty(description)){
         String[] descSplitArray = description.split("\\s+");
         Integer descArrayLength = descSplitArray.length;
         
         desc1 = StringUtils.join(descSplitArray, delimeter, 0, descArrayLength > chunkSize ? chunkSize: descArrayLength);
         
         if(noOfChunks > 1 && descArrayLength >= chunkSize){
            desc2 = StringUtils.join(descSplitArray, delimeter, chunkSize, descArrayLength > 2 * chunkSize ?
                  2 * chunkSize: descArrayLength);
            
            if(descArrayLength >= 2 * chunkSize){
               desc3 = StringUtils.join(descSplitArray, delimeter, 2 * chunkSize,
                     descArrayLength > 3 * chunkSize ? 3 * chunkSize: descArrayLength);               
            }
            
            if(descArrayLength > 3 * chunkSize){
               desc4 = StringUtils.join(descSplitArray, delimeter, 3 * chunkSize, descArrayLength);               
            }
         }
      }
      
      Field descField = new Field(name + "1", desc1, luceneOptions.getStore(), luceneOptions.getIndex(),
            luceneOptions.getTermVector());
      descField.setBoost(0.3f);
      document.add(descField);

      
      if(noOfChunks > 1 ) {
         descField = new Field(name + "2", desc2, luceneOptions.getStore(), luceneOptions.getIndex(),
               luceneOptions.getTermVector());
         descField.setBoost(0.03f);
         document.add(descField);

         
         descField = new Field(name + "3", desc3, luceneOptions.getStore(), luceneOptions.getIndex(),
               luceneOptions.getTermVector());
         descField.setBoost(0.003f);
         document.add(descField);

         
         descField = new Field(name + "4", desc4, luceneOptions.getStore(), luceneOptions.getIndex(),
               luceneOptions.getTermVector());
         descField.setBoost(0.003f);
         document.add(descField);      

      }
      
   }

   @Override
   public void setParameterValues(Map parameters) {
      
      this.delimeter = (parameters.get("delimeter") == null ? " " : (String)parameters.get("delimeter"));
      this.chunkSize = (parameters.get("chunkSize") == null ? 100 : Integer.parseInt((String)parameters.get("chunkSize")));
      this.noOfChunks = (parameters.get("noOfChunks") == null ? 1 : Integer.parseInt((String)parameters.get("noOfChunks")));
   }

}


Though I have applied normalAnalyzer on Metadata.description field, it still creates stemmed indices. But the field bridge was worked.

NOTE: Just to make sure I have tried with the option suggested by Michael. But it also did not work.
Code:
if(isNotEmpty(desc1)){
         luceneOptions.addFieldToDocument(name + "1", desc1, document);         
      }


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Tue Jan 31, 2012 10:59 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Hi mkkumar79,
could you please test that in a recent version? 4.0.0.Final or 4.1.0.Alpha1

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Analyzer annotation added to field together with FieldBridge
PostPosted: Sun Apr 15, 2012 11:18 am 
Newbie

Joined: Sun Apr 15, 2012 11:07 am
Posts: 5
Hi,
We are having the same issue, with the 4.0.0.Final version
We have 2 Fields using the same bridge, but with differents parameter and different analyzer.
When opening with Luke, it appears that all the fields are correctly stored, but the specified analyzer (here our NGramAnalyzer.class) isn't used.
field are not empty, index == YES and the NGramAnalyzer works as expected (we use the same in our workaround).

Here is our mapping :
Code:
    @Fields({
            @Field(index = Index.YES, store = Store.YES, bridge = @FieldBridge(impl = CompanyBusinessIdBridge.class, params = @Parameter(name = "fieldSubName", value = "ngram")), analyzer = @Analyzer(impl = NGramAnalyzer.class)),
            @Field(index = Index.YES, store = Store.YES, bridge = @FieldBridge(impl = CompanyBusinessIdBridge.class)) })
    private final Map<CompanyBusinessIdType, CompanyBusinessId> businessIds = new HashMap<CompanyBusinessIdType, CompanyBusinessId>();


and our CompanyBusinessIdBridge implementation :

Code:
public class CompanyBusinessIdBridge implements FieldBridge, ParameterizedBridge {
    public static String FIELD_SUB_NAME = "fieldSubName";
    private String fieldSubName = "";

    @Override
    @SuppressWarnings("unchecked")
    public void set(String name, Object value, Document document, LuceneOptions luceneOptions) {
        Map<CompanyBusinessIdType, CompanyBusinessId> businessIds = (Map<CompanyBusinessIdType, CompanyBusinessId>) value;
        for (Entry<CompanyBusinessIdType, CompanyBusinessId> e : businessIds.entrySet()) {
            luceneOptions.addFieldToDocument(name + fieldSubName + e.getKey().name(), e.getValue().getValue(), document);
        }
    }

    @Override
    @SuppressWarnings("rawtypes")
    public void setParameterValues(Map parameters) {
        String subName = (String) parameters.get(FIELD_SUB_NAME);
        if (subName != null) {
            fieldSubName = "_" + subName;
        } else {
            fieldSubName = "";
        }
    }
}


For now we work with a workaround (a unique fieldBrige with our 2 analyzer inside, and we add directly the value to the document, without using the luceneOptions)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.