custom similarity not working as expected

mhaller · **Joined:** Fri Aug 03, 2012 11:41 am **Posts:** 5

I've provided a custom similarity class for a given class using the annotation. (I've also tried the system property hibernate.search.similarity)

Code:

@Similarity(impl = MyGeneSearchSimilarity.class)

Code:

public class MyGeneSearchSimilarity extends DefaultSimilarity {

  private static final long serialVersionUID = 2525682785649576163L;

  /** Disabled. */
  @Override
  public float tf(float freq) {
    return 1f;
  }
    
  /** Disabled */
  @Override
  public float idf(int docFreq, int numDocs) {
    return 1f;
  }
}

When I do a projection query, the FullTextQuery.SCORE doesn't match the value in the FullTextQuery.EXPLANATION. Can you think of a reason why?

Code:

      fullTextQuery.setProjection(FullTextQuery.SCORE, FullTextQuery.THIS, FullTextQuery.EXPLANATION);
      Collection<Object[]>  scores = fullTextQuery.list();

      for (Object[] score : scores)
      {
         System.out.println("score :" + score[0].toString());
         System.out.println("object: " + score[1].toString());
         System.out.println("explanation : \n" + score[2].toString());  
      }

Code:

score :0.0015264064     <----- ????  these shoudl be equal
object: GeneSymbolData{approvedSymbol='FOZZY~withdrawn', approvedName='SLS', previousSymbols='SAMESCORE', previousNames='previous_name', synonyms='synonyms'}
explanation : 
0.5483614 = (MATCH) product of:    <------ ??? these shoudl be equal
  0.7311486 = (MATCH) sum of:
    0.0061056255 = (MATCH) product of:
      0.030528126 = (MATCH) sum of:
        0.030528126 = (MATCH) ConstantScore(approvedName:*sls*), product of:
          1.0 = boost
          0.030528126 = queryNorm
      0.2 = coord(1/5)
    0.57240236 = (MATCH) weight(match_approvedSymbol:sls^30.0 in 25), product of:
      0.9158438 = queryWeight(match_approvedSymbol:sls^30.0), product of:
        30.0 = boost
        1.0 = idf(docFreq=1, maxDocs=28)
        0.030528126 = queryNorm
      0.625 = (MATCH) fieldWeight(match_approvedSymbol:sls in 25), product of:
        1.0 = tf(termFreq(match_approvedSymbol:sls)=0)
        1.0 = idf(docFreq=1, maxDocs=28)
        0.625 = fieldNorm(field=match_approvedSymbol, doc=25)
    0.15264063 = (MATCH) weight(match_previousSymbols:sls^5.0 in 25), product of:
      0.15264063 = queryWeight(match_previousSymbols:sls^5.0), product of:
        5.0 = boost
        1.0 = idf(docFreq=1, maxDocs=28)
        0.030528126 = queryNorm
      1.0 = (MATCH) fieldWeight(match_previousSymbols:sls in 25), product of:
        1.0 = tf(termFreq(match_previousSymbols:sls)=0)
        1.0 = idf(docFreq=1, maxDocs=28)
        1.0 = fieldNorm(field=match_previousSymbols, doc=25)
  0.75 = coord(3/4)

sanne.grinovero · **Posted:** Mon Apr 18, 2016 4:34 pm

There's a restriction: for a given index, only one Similarity implementation can be configured.

Are you indexing multiple types in the same index?

Could you check if there are warnings in the log? If there's an inconsistent Similarity configuration, there should be a warning being logged about it.

mhaller · **Joined:** Fri Aug 03, 2012 11:41 am **Posts:** 5

Thanks for the reply.

I noticed a note that @Similarity was deprecated (though, I don't think it was marked that way), and decided it worked fine for us to have a global similarity defined. This did not change the strange explanation behavior though.

Since, we decided to upgrade to Hibernate Search 5.5, and I'm now extending a different base class (TFIDFSimilarity). Now the value of EXPLANATION, matches SCORE. I believe that if someone sees this in Hibernate Search 4.x and Lucene 3.x they should look close at how those *Explain methods work in their similarity base class, if they have one. But, for me this problem is gone with the globally defined class below:

Code:

public class IgnoreAllSimilarity extends TFIDFSimilarity
{

   @Override
   public float tf(float freq)
   {
      return 1f;
   }
   
   @Override
   public float idf(long docFreq, long numDocs)
   {
      return 1f;
   }
   
   @Override
   public float queryNorm(float sumOfSquaredWeights)
   {
      return 1f;
   }
   
   @Override
   public float coord(int overlap, int maxOverlap)
   {
      return 1f;
   }
   
   @Override
   public float sloppyFreq(int distance)
   {
      return 1f;
   }

   @Override
   public float lengthNorm(FieldInvertState state)
   {
      return 1f;
   }

   @Override
   public final long encodeNormValue(float f)
   {
      return 1;
   }
   
   @Override
   public final float decodeNormValue(long norm)
   {
      return 1f;
   }

   @Override
   public float scorePayload(int doc, int start, int end, BytesRef payload)
   {
      return 1f;
   }