-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 30 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Hibernate Search with Filters
PostPosted: Sat Jan 10, 2009 5:49 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi,

How to make use of Filters in Hibernate search (lucene search).
In additional to keyword search i would like to add some more constraints.

Actually i tried with the filters.

pojo
Code:
@Entity
@Indexed
@FullTextFilterDefs( {
    @FullTextFilterDef(name = "nonDeletedPov", impl = NonDeletedPovFilterFactory.class),
})
public class POV  implements java.io.Serializable {@Id
    @DocumentId
     private String povId;
   
     private long version;
     @NaturalId
     private PAEvent paEvent;
     private PAOrganization paOrganization;

     @NaturalId
     @IndexedEmbedded(depth=1, prefix="portfolio_")
     private Portfolio portfolio;
     
    @Field(index=Index.TOKENIZED, store=Store.NO)
     private String modifiedBy;
     private Character isDeleted;
}


NonDeletedPovFilterFactory.java
Code:
public class NonDeletedPovFilterFactory {
   @Factory
    public Filter getFilter() {
        //some additional steps to cache the filter results per IndexReader
        Filter nonDeletedPovFilter =  new NonDeletedPovFilter();
        return  new CachingWrapperFilter(nonDeletedPovFilter);

    }

}


NonDeletedPovFilter.java
Code:
public class NonDeletedPovFilter extends org.apache.lucene.search.Filter {


   public DocIdSet  getDocIdSet(IndexReader reader) throws IOException {
        OpenBitSet bitSet = new OpenBitSet( reader.maxDoc() );
        TermDocs termDocs = reader.termDocs( new Term( "modifiedBy", "Ambika" ) );
        while ( termDocs.next() ) {
            bitSet.set( termDocs.doc() );
        }
        return bitSet;
    }

}


Example.java
Code:
try{
       // This step will read hibernate.cfg.xml and prepare hibernate for use
        Session session = null;
        SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
        session =sessionFactory.openSession();
        FullTextSession fullTextSession = Search.createFullTextSession(session);

        Transaction tx = fullTextSession.beginTransaction();

        
        
        MultiFieldQueryParser parser = new MultiFieldQueryParser(
            new String[] { "portfolio_portfolioName" },
            new StandardAnalyzer());
        
        Query query = parser.parse("Ambika");
        FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(query, POV.class);
        fullTextQuery.enableFullTextFilter("nonDeletedPov");
        org.hibernate.Query hibQuery = (org.hibernate.Query) fullTextQuery.list();
        List<POV> result = hibQuery.list();

tx.commit();
        session.close(); 
     }catch(Exception e){
       System.out.println(e.getMessage());
     }finally{
       }


Its giving me Null pointer Exception..

I didn't understand how and when the filters will be invoked, even the filter parameter should also be indexed???

Could anybody clear my doubts. Im really confused.

Help pls,
Ambika.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 15, 2009 10:57 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

the overall approach looks fine. Where exactly is the NullpointerException? Can you provide a stack trace?

--Hardy


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 6:17 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi,

Thanks for ur reply.

My First and foremost doubt is if we filter according to a property say "modifiedBy" should that be indexed or not required.


when i use DocIdSet in my filter it is given me NullPointerException.

but if i use BitSet in my Filter i didn get any NullPointerException, but it is getting the searchresult even though result exits.
Code:
public class NonDeletedPovFilter extends org.apache.lucene.search.Filter {

public BitSet bits(IndexReader reader) throws IOException {
        BitSet bitSet = new BitSet( reader.maxDoc() );
        TermDocs termDocs = reader.termDocs( new Term("modifiedBy", "Saniel") );
        while ( termDocs.next() ) {
            bitSet.set( termDocs.doc() );
        }
        return bitSet;
    }
}


Pls clear me whether BitSet to be used or DocIdSet, What is the major difference between these two.

Thanks,
Ambika.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 6:48 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,

of course you have to index the poroperty you want to filter on.
The DocIdSet is the recommended API. The BitSet version of filters is deprecated. As said, a stacktrace would help to narrow down the problem.

--Hardy


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 7:18 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi Hardy,

It is not giving me the stack trace completely, the thing which im able to see is
Code:
msg..null
Stack Trace..[Ljava.lang.StackTraceElement;@1e3bddb


I had run this application Example.java
Code:
try{
       // This step will read hibernate.cfg.xml and prepare hibernate for use
        Session session = null;
        SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
        session =sessionFactory.openSession();
        FullTextSession fullTextSession = Search.createFullTextSession(session);

        Transaction tx = fullTextSession.beginTransaction();

        
        MultiFieldQueryParser parser = new MultiFieldQueryParser(
            new String[] { "portfolio_portfolioName","modifiedBy","statusChangeComments" },
            new StandardAnalyzer());
        
        Query query = parser.parse("changed");
        org.hibernate.search.FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(query, POV.class);
        fullTextQuery.enableFullTextFilter("nonDeletedPov");
        List list = fullTextQuery.list();
        List<POV> result = list;
        
      if(result != null){   
        System.out.println("Result Size : "+result.size());
            
        for(int i=0;i<result.size();i++)
        {   
           System.out.println("Result::::::::::"+result.get(i).getModifiedBy());
           
        }}
        
        tx.commit();
        session.close(); 
     }catch(Exception e){
       System.out.println("msg.."+e.getMessage());
       System.out.println("Stack Trace.."+e.getStackTrace());
     }finally{
       }


In the Above example if i comment this line of code "fullTextQuery.enableFullTextFilter("nonDeletedPov");"
Im able to see the below result.
Code:
Result Size : 2
Result::::::::::Saniel
Result::::::::::Vinay


I had indexed modifiedBy property. The above things happened when i use docIdset. Even if i use BitSet it is giving the proper result. It says result is zero.
Really confused whats going wrong.

Help pls me because im really in need.

Thanks,
Ambika.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 7:23 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
when i debug, exactly in this below line im getting exception
Code:
List list = fullTextQuery.list();


when i print the stackTrace.
Code:
java.lang.NullPointerException
   at org.hibernate.search.filter.ChainedFilter.bits(ChainedFilter.java:28)
   at org.apache.lucene.search.Filter.getDocIdSet(Filter.java:49)
   at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:140)
   at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:112)
   at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:113)
   at org.apache.lucene.search.Hits.<init>(Hits.java:90)
   at org.apache.lucene.search.Searcher.search(Searcher.java:72)
   at org.hibernate.search.query.FullTextQueryImpl.getHits(FullTextQueryImpl.java:270)
   at org.hibernate.search.query.FullTextQueryImpl.list(FullTextQueryImpl.java:232)
   at com.i3l.sample.test.Example.main(Example.java:48)


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 7:58 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi,
the line numbers you've shown in the stacktrace are not matching the 3.1 release of Hibernate Search, which is required to use the DocIdSet approach of Lucene 2.4

We need to know which versions of the following jars you are using:
Hibernate
Hibernate Search
Lucene

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 8:23 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
That's right, I completly forgot to ask for the version you are using. Pre 3.1.0 you have to use the BitSet :)

--Hardy


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 8:59 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi,

Ya thats right. i was using Hibernate Search
Implementation-Version: 3.0.0.GA

Now i tried using

Hibernate : hibernate-3.2.6.ga.jar
Hibernate Search : Hibernate Search
Implementation-Version: 3.1.0.GA
(which i had downloaded from http://www.hibernate.org/6.html Hibernate Search 3.1.0 GA 04.12.2008 Production Download )

Lucene : lucene-core-2.4.0.jar

but now it giving me
Code:
Exception in thread "main" java.lang.NoClassDefFoundError: org/hibernate/event/Destructible
   at java.lang.ClassLoader.defineClass1(Native Method)
   at java.lang.ClassLoader.defineClass(Unknown Source)
   at java.security.SecureClassLoader.defineClass(Unknown Source)
   at java.net.URLClassLoader.defineClass(Unknown Source)
   at java.net.URLClassLoader.access$000(Unknown Source)
   at java.net.URLClassLoader$1.run(Unknown Source)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(Unknown Source)
   at java.lang.ClassLoader.loadClass(Unknown Source)
   at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
   at java.lang.ClassLoader.loadClass(Unknown Source)
   at java.lang.ClassLoader.loadClassInternal(Unknown Source)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Unknown Source)
   at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:100)
   at org.hibernate.cfg.Configuration.setListeners(Configuration.java:1677)
   at org.hibernate.cfg.Configuration.parseEvent(Configuration.java:1649)
   at org.hibernate.cfg.Configuration.parseSessionFactory(Configuration.java:1580)
   at org.hibernate.cfg.Configuration.doConfigure(Configuration.java:1540)
   at org.hibernate.cfg.Configuration.doConfigure(Configuration.java:1514)
   at org.hibernate.cfg.Configuration.configure(Configuration.java:1434)
   at org.hibernate.cfg.Configuration.configure(Configuration.java:1420)
   at com.i3l.sample.test.Example.main(Example.java:34)



Is the jar files which im using is making problem, from where should i download the hibernate search jar.

Thanks,
Ambika.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 9:07 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
You will have to upgrade Hibernate Core as well. Check the Compatibility Matrix - http://www.hibernate.org/6.html

In case you are using Maven or Ivy for dependecy management I would recommend you to just use the POM from the JBoss repo: http://repository.jboss.com/maven2/org/ ... te-search/

Hope this helps.

--Hardy


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 9:30 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi,

Now im not getting NullPointerException, but still im not able to get the result though the data is there according to my filter specification.

And it is throwing warning msges in my console
Code:
2009-01-19 18:57:55,515 INFO [org.hibernate.search.Version] - <Hibernate Search 3.1.0.GA>
2009-01-19 18:57:55,687 WARN [org.hibernate.search.engine.DocumentBuilderContainedEntity] - <@DocumentId specified on an entity which is not indexed by itself. Annotation gets ignored. Use @Field instead.>
2009-01-19 18:57:55,875 WARN [org.hibernate.search.engine.DocumentBuilderContainedEntity] - <@DocumentId specified on an entity which is not indexed by itself. Annotation gets ignored. Use @Field instead.>
2009-01-19 18:57:55,875 WARN [org.hibernate.search.engine.DocumentBuilderContainedEntity] - <@DocumentId specified on an entity which is not indexed by itself. Annotation gets ignored. Use @Field instead.>
2009-01-19 18:57:56,515 INFO [org.hibernate.impl.SessionFactoryImpl] - <building session factory>
2009-01-19 18:57:58,953 INFO [org.hibernate.impl.SessionFactoryObjectFactory] - <Not binding factory to JNDI, no JNDI name configured>
Result Size : 0


What needs to be done to get the filtered result.

pls help me out.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 9:39 am 
Hibernate Team
Hibernate Team

Joined: Thu Apr 05, 2007 5:52 am
Posts: 1689
Location: Sweden
Hi,


You might have @DocumentId specified on a base class, right? I think you can ignore this warning. It is probably related to this Jira issue: http://opensource.atlassian.com/project ... SEARCH-333

How does your domain model look like? Have you added some trace to your filter?

--Hardy


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 9:56 am 
Beginner
Beginner

Joined: Mon Oct 01, 2007 8:21 am
Posts: 40
Hi,

My domain POV.java
Code:
@Entity
@Indexed
@FullTextFilterDef(name = "nonDeletedPov", impl = NonDeletedPovFilterFactory.class)

public class POV  implements java.io.Serializable {


    @Id
     private String povId;
   
     private long version;
     @NaturalId
     @IndexedEmbedded(depth=3, prefix="paEvent_")
     private PAEvent paEvent;
     
     private PAOrganization paOrganization;
     private ImmutableCode povStatusCd;
     private ImmutableCode engagementLevelCd;
     
     @NaturalId
     @IndexedEmbedded(depth=1, prefix="portfolio_")
     private Portfolio portfolio;
     
     private String createdBy;
     private Date dateCreated;

     @Field(index=Index.TOKENIZED, store=Store.NO)
     private String modifiedBy;


NonDeletedPovFilterFactory.java
Code:
import org.apache.lucene.search.CachingWrapperFilter;
import org.apache.lucene.search.Filter;
import org.hibernate.search.annotations.Factory;

/**
* @author Ambika.Devi
*
*/
public class NonDeletedPovFilterFactory {
   @Factory
    public Filter getFilter() {
        //some additional steps to cache the filter results per IndexReader
        Filter nonDeletedPovFilter =  new NonDeletedPovFilter();
        return  new CachingWrapperFilter(nonDeletedPovFilter);

    }

}


NonDeletedPovFilter.java
Code:
import java.io.IOException;

import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.Term;
import org.apache.lucene.index.TermDocs;
import org.apache.lucene.search.DocIdSet;
import org.apache.lucene.util.OpenBitSet;

/**
* @author Ambika.Devi
*
*/
@SuppressWarnings("serial")
public class NonDeletedPovFilter extends org.apache.lucene.search.Filter {


   public DocIdSet getDocIdSet(IndexReader reader) throws IOException {
      OpenBitSet  bitSet = new OpenBitSet ( reader.maxDoc() );
        TermDocs termDocs = reader.termDocs( new Term( "modifiedBy", "Saniel" ) );
        while ( termDocs.next() ) {
            bitSet.set( termDocs.doc() );
        }
        return bitSet;
    }

}


when i was in debug mode
reader.maxDoc() is 26
when it comes to termDocs.next() it is not entering the block.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 10:17 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi, the NonDeletedPovFilter is looking ok, but you should avoid the CachingWrapperFilter, making your factory NonDeletedPovFilterFactory useless. Lucene's CachingWrapperFilter should be avoided when using Search, you achieve the same feature and additional benefits when asking declaratively to cache it:

Code:
@Entity
@Indexed
@FullTextFilterDef(name = "nonDeletedPov", impl = NonDeletedPovFilterFactory.class,  cache = FilterCacheModeType.INSTANCE_AND_DOCIDSETRESULTS)
public class POV  implements java.io.Serializable {
...

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 19, 2009 10:26 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Quote:
when i was in debug mode
reader.maxDoc() is 26
when it comes to termDocs.next() it is not entering the block.

I think you pointed us to the solution here: keep in mind that when you build a query your text is prepocessed by an Analyzer, when you build a Term you should make sure the text is prepared as your Analyzer would do.

Try converting
Code:
new Term( "modifiedBy", "Saniel" )
to
Code:
new Term( "modifiedBy", "saniel" )
As the default Analyzer applies lowercase transformation.

If it doesn't help, try looking in the index with a tool like Luke and search for the terms, or post your Analyzer configurations.

If you can't predict the text and/or need a more general solution take a look at QueryWrapperFilter, so you can just write "Saniel" and have lucene it analyze for you.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 30 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.