Filtering a data set - please someone look at my filter code

Mocchi · **Joined:** Fri Mar 19, 2010 7:00 pm **Posts:** 14

OK so I have a filter where I am pretty much checking that a certain value in a row field is within a given number range.

Here is my filter:

Code:

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */



import java.io.IOException;
import java.util.HashMap;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.TermDocs;
import org.apache.lucene.index.TermEnum;
import org.apache.lucene.search.DocIdSet;
import org.apache.lucene.util.OpenBitSet;
import org.hibernate.search.annotations.Key;
import org.hibernate.search.filter.FilterKey;
import org.hibernate.search.filter.StandardFilterKey;


public class NumberRangeFilterFactory extends org.apache.lucene.search.Filter {

    private String field;
    private Integer min;
    private Integer max;
    
    /**
     * injected parameter
     */
    public void setField(String field) {
        this.field = field;
    }

    public void setMin(Integer min) {
        this.min = min;
    }

    public void setMax(Integer max) {
        this.max = max;
    }

    @Key
    public FilterKey getKey() {
        StandardFilterKey key = new StandardFilterKey();
        key.addParameter(field);
        key.addParameter(min);
        key.addParameter(max);
        return key;
    }

    public DocIdSet getDocIdSet(IndexReader reader) throws IOException {
        OpenBitSet bitSet = new OpenBitSet(reader.maxDoc());

        TermDocs termDocs = reader.termDocs();
        TermEnum termEnum = reader.terms();

        while (termEnum.next()) {

            String fieldName = termEnum.term().field();
            //if (fieldName.equals(field)) {
                
                termDocs.seek(termEnum.term());

                if (termDocs.next()) {

                     
                    int doc = termDocs.doc();
                    
                    Document document = reader.document(doc);

                    Integer actualInteger = (int)(Float.valueOf(document.get(field))).floatValue();

                    if (actualInteger >= min && actualInteger <= max) {
                        bitSet.set(doc);
                        System.out.println("INTEGER IS :: " + actualInteger );
                    }

                }
            //}
        }

        return bitSet;
    }

}

I can't say I fully understand everything that is going on within the while loop but what I do see is that all rows that match the criteria should be put in the bitset correct? But for some reason I do not think this is working.

For example there is only one row where the following check should be TRUE and it it does go in the if and prints out :

Code:

if (currentBidAmount >= min && currentBidAmount <= max) {
    System.out.println("BID AMOUNT IS :: " + actualInteger);
    bitSet.set(doc);
}

But in my main program my result set is zero.

Can anyone point me in the right direction? Are there any sites where such a process is shown in detail. Does the code above look like it should work? So far all I keep finding are examples like this that involve a Term object but a single parameter is passed to the filter (I'm passing three as of now):

https://forum.hibernate.org/viewtopic.php?f=9&t=1001911

sanne.grinovero · **Posted:** Wed Mar 24, 2010 8:00 pm

Hello,
I'm sure the issue is not depending on the fact you're having three parameters.

it's working for me using the following code, and making sure the field you are referring to is stored in the index:

Code:

@Field(index= Index.UN_TOKENIZED,store=Store.YES)

Code:

public DocIdSet getDocIdSet(IndexReader reader) throws IOException {
      OpenBitSet bitSet = new OpenBitSet( reader.maxDoc() );
      TermDocs termDocs = reader.termDocs();
      TermEnum termEnum = reader.terms();
      while ( termEnum.next() ) {
         String fieldName = termEnum.term().field();
         if ( fieldName.equals( field ) ) {
            termDocs.seek( termEnum.term() );
            if ( termDocs.next() ) {
               int doc = termDocs.doc();
               Document document = reader.document( doc );
               String fieldValue = document.get( field );
               if ( fieldValue != null ) {
                  int actualInteger = (Integer.valueOf( fieldValue )).intValue();
                  if ( actualInteger >= min && actualInteger <= max ) {
                     bitSet.set( doc );
                     System.out.println( "INTEGER IS :: " + actualInteger );
                  }
               }
            }
         }
      }
      return bitSet;
   }

I'm not sure this is the best performing solution, especially as you're reading from the document - the same issue for which you need Store.
Depending on the range between min and max it might be better to wrap a RangeQuery into the filter, or to use seek() in a loop on exact terms, scanning the whole range of possibilities and setting the bits at each seek.

If you don't store the field, you're getting null from document.get( field ), that's why I added the nullcheck (as the index might contain other entities of a different type, and not have this field stored)

Mocchi · **Joined:** Fri Mar 19, 2010 7:00 pm **Posts:** 14

Thank you so much for you response, I had given up on anyone replying. I was able to solve the issue and yes I had the indexing wrong. (@Field(index= ... ).

Thanks again!