-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

View unanswered posts | View active topics


Board index » Projects » Search, Validator, Shards

All times are UTC - 5 hours [ DST ]


Forum locked This topic is locked, you cannot edit posts or make further replies.  Page 1 of 1
  [ 9 posts ] 
  Previous topic | Next topic 
Author Message
seemarich
 Post subject: Is it possible to plugin a custom query?
PostPosted: Fri Feb 20, 2009 1:49 am 
Beginner
Beginner

Joined: Tue Feb 03, 2009 12:29 pm
Posts: 49
One of the search criteria I have is a distance range. For this, the distance between the logged in user's location and the inventory entity's location need to be calculated based on latitude and longitude of their corresponding zip codes. Then the search results would be filtered based on whether the calculated distance falls between the range specified in the criteria

How do I plugin such a custom logic when generating a Lucene query using Hibernate search? Please note that this is not the only search criteria for this search, but just one of them.

Thanks,
Seema
Top
 Profile  
 
hardy.ferentschik
 Post subject:
PostPosted: Fri Feb 20, 2009 12:15 pm 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

as part of the indexing you have to index the longitude/latitude (or whatever type of coordinates your are using) values of the inventory item. Then you build your normal Lucene search query. You also need a Filter into which you pass the users location as parameter (check the online doc for that). You then just have to calculate the distance of the user to the matching documents and return only the ones which match your distance criteria.

Check the book "Lucene in Action". I think there is a similar example in there.
This blog entry - http://sujitpal.blogspot.com/2008/02/sp ... ucene.html - might help as well.

--Hardy
Top
 Profile  
 
seemarich
 Post subject: Distance Filter - Questions
PostPosted: Sun Feb 22, 2009 2:47 pm 
Beginner
Beginner

Joined: Tue Feb 03, 2009 12:29 pm
Posts: 49
Hardy,

Thanks for the tip.
I used a bridge to get the longitude and latitude values of the inventory and add these to the index. But I had some questions on the below filter that I wrote for checking the distance and return the matching ones.

1. To get to the term of the fields that I want, do I have to iterate through all the terms like I am doing?
2. I'm not getting the negative sign in the longitude value when I call the termEnum.term().text() method. How can I get that also?
3. If the distance is within the searched range, I get the TermDocs for both longitude and latitude for the right values and set to bitSet one by one. Is this the right way of doing it?

Code:
public class DistanceFilter extends org.apache.lucene.search.Filter {

   private double latitude, longitude;

   private double lowerLimit, upperLimit;

   public DocIdSet getDocIdSet(IndexReader reader) throws IOException {
      OpenBitSet bitSet = new OpenBitSet(reader.maxDoc());
      TermEnum termEnum = reader.terms();
      while (termEnum.next()) {
         String fieldName = termEnum.term().field();
         if (fieldName.equals("longitude")) {
            double longitude = Double.parseDouble(termEnum.term().text());
         }
         if (fieldName.equals("latitude")) {
            double latitude = Double.parseDouble(termEnum.term().text());
         }
         // Call a custom method to calculate the distance
         Double distance = DistanceCalculation.getInstance()
               .distanceCalculator(latitude, longitude, getLatitude(),
                     getLongitude());
         if ((distance >= lowerLimit) && (distance <= upperLimit)) {
            TermDocs termDocs1 = reader.termDocs(new Term("longitude",
                  String.valueOf(longitude)));

            while (termDocs1.next()) {
               bitSet.set(termDocs1.doc());
            }
            TermDocs termDocs2 = reader.termDocs(new Term("longitude",
                  String.valueOf(longitude)));

            while (termDocs2.next()) {
               bitSet.set(termDocs2.doc());
            }
         }
      }
      return bitSet;
   }
...getters and setters
}
Top
 Profile  
 
hardy.ferentschik
 Post subject: Re: Distance Filter - Questions
PostPosted: Mon Feb 23, 2009 10:05 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

Quote:
1. To get to the term of the fields that I want, do I have to iterate through all the terms like I am doing?

You can also work with TermDocs and seek(). Check out the TermsFilter class. Using TermDocs you also get to the Document id and from there to the Document itself. Once you have the Document you can explicitly read the longitude and latitude values and calcualtes the distance. If the distance is in range you can just use the doc id to set the appropriate bit in the result.

If you have a look at the Lucene mailing lists you will find also other techniques to execute geospatial searches, eg the use of bounding boxes, etc.

Quote:
2. I'm not getting the negative sign in the longitude value when I call the termEnum.term().text() method. How can I get that also?

That's seems odd. Have you checked the index that it is indexed properly? You could use Luke to inspect the indexed values. Are you sure that you are not removing the '-' value in the bridge or somewhere along the analyzing process?

Quote:
3. If the distance is within the searched range, I get the TermDocs for both longitude and latitude for the right values and set to bitSet one by one. Is this the right way of doing it?

Not sure. Your code looks a little complicated. Once you calcualted the distance from one Document/Iventory to your user it should be just the question of calling result.set(docId).

--Hardy
Top
 Profile  
 
seemarich
 Post subject: Filter class
PostPosted: Wed Feb 25, 2009 12:42 pm 
Beginner
Beginner

Joined: Tue Feb 03, 2009 12:29 pm
Posts: 49
Thanks Hardy. In the below code, the result is printed correctly with the negative sign when I call the get() method of the Document object like you suggested. However, the termEnum.term().text() method call still returns the result without the negative sign. Do you see any reasons?


Code:
TermDocs td = reader.termDocs();
      TermEnum termEnum = reader.terms();
      while (termEnum.next()) {
         String fieldName = termEnum.term().field();
         if (fieldName.equals("longitude")) {
            System.out.println("val is " + termEnum.term().text());
            td.seek(termEnum.term());
            if (td.next()) {
               int doc = td.doc();
               bitSet.set(doc);
               Document document = reader.document(doc);
               System.out.println("Longitude is "
                     + document.get("longitude"));
               
            }
         }
      }


Thanks,
Seema
Top
 Profile  
 
hardy.ferentschik
 Post subject:
PostPosted: Thu Feb 26, 2009 4:22 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

I am guessing you are indexing your values TOKENIZED. This means that the default analyzer might remove the '-'. Using TermEnum you are actually iterating the actual indexed values, whereas using the Document API you are retrieving the original values provided you are using Store.YES.

That's of course just a guess right now. Check your entity mapping or post it here.

--Hardy
Top
 Profile  
 
seemarich
 Post subject: Indexing in FieldBridge
PostPosted: Thu Feb 26, 2009 10:20 am 
Beginner
Beginner

Joined: Tue Feb 03, 2009 12:29 pm
Posts: 49
Hardy,

I have a field named zip in my entity class which is annotated as below.
@Field(index = Index.TOKENIZED, store = org.hibernate.search.annotations.Store.YES, bridge = @FieldBridge(impl = ZipBridge.class))
@Column(name = "ZIP")
private String zip;

In the ZipBridge, I'm mapping it as the longitude and latitude values as shown.

Code:
Field field1 = new Field("latitude", latitude.toString(), luceneOptions
            .getStore(), luceneOptions.getIndex(), luceneOptions
            .getTermVector());
field1.setBoost(luceneOptions.getBoost());
document.add(field1);
      
Field field2 = new Field("longitude", longitude.toString(), luceneOptions
            .getStore(), luceneOptions.getIndex(), luceneOptions
            .getTermVector());
field2.setBoost(luceneOptions.getBoost());
document.add(field2);


I didn't clearly understand what you meant when you talked about 'original values' and 'indexed values'. Could you please clarify? I can see the negative sign in longitude and latitude when I use Luke also.

Thanks,
Seema
Top
 Profile  
 
hardy.ferentschik
 Post subject:
PostPosted: Thu Feb 26, 2009 11:05 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

In your code example I cannot see how you derive longitude and latitude from the string "zip", but anyhow try using Index.UN_TOKENIZED.

What I mean with 'original values' vs 'indexed values' is that from a pure search engine point of view there is no need to keep the original indexed data after the inverted index (the search data structure) is build. You are only able to see the proper longitude and latitude values in Luke and the Document, because of Store.YES. Try using Store.NO. You will see that neither Luke nor the Document in your code will have the proper values anymore. Instead null will be returned.

The TermEnum on the other hand gives you access to the indexed values. Meaning the values in the inverted index. If you want to use them you have to make sure that they are getting indexed as is (without getting modified by an analyzer). To do this you have to use Index.UN_TOKENIZED.

--Hardy
Top
 Profile  
 
seemarich
 Post subject: Works after changing to Untokenized
PostPosted: Fri Feb 27, 2009 3:23 am 
Beginner
Beginner

Joined: Tue Feb 03, 2009 12:29 pm
Posts: 49
Hardy,

It did work after I changed the mapping to use 'Index.UN_TOKENIZED'.

@Field(index = Index.UN_TOKENIZED, store = org.hibernate.search.annotations.Store.YES, bridge = @FieldBridge(impl = ZipBridge.class))
@Column(name = "ZIP")
private String zip;

Thanks,
Seema
Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  Page 1 of 1
  [ 9 posts ] 

Board index » Projects » Search, Validator, Shards

All times are UTC - 5 hours [ DST ]

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.