-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: [Hibernate Search] Annotation problem
PostPosted: Tue Jul 20, 2010 7:53 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Hi guys,

I have a big issue with my code. I use the annotations to configure Hibernate search, according to the "Hibernate Search in Action" book, page 127. I thought everything was ok, but in facts, not.

Here is a simple example : I have the string "Terre, céramique et plâtre" to include in my index. Again, according to the book, I use a StandardTokenizerFactory, then a StandardFilter, then a LowercaseFilter, and then a StopFilter. Recently I added an AccentFilter (my own, and the way I discovered the thing).

According to those filters, I expected to have in my lucene index the following tokens : terre|ceramique|platre (Stop Filter is supposed to stop the "et"). What I get is far away : terre|céramique|et|plâtre

Ok, maybe there is a miss in my config? I tried to remove all the filters one by one and .... surprise ... nothing changed!

So, in my case, this code
Code:
@Entity
@Indexed
@AnalyzerDef(name = "categoriestokensanalyzer", tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class))
@Analyzer(definition = "categoriestokensanalyzer")
public class Commune implements Serializable {


generates the same output as this one
Code:
@Entity
@Indexed
@AnalyzerDef(name = "categoriestokensanalyzer", tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = {
   @TokenFilterDef(factory = StandardFilterFactory.class),
   @TokenFilterDef(factory = LowerCaseFilterFactory.class),
   @TokenFilterDef(factory = AccentFilterFactory.class),
   @TokenFilterDef(factory = StopFilterFactory.class, params = {
      @Parameter(name = "words", value = "be/fgov/economie/artisans/services/stopwords.txt"),
      @Parameter(name = "ignoreCase", value = "true") }) })
@Analyzer(definition = "categoriestokensanalyzer")
public class Categories implements Serializable


Some precisions on my environment : I run my code on tomcat 6, java 6. each time I redeploy, I physically destroy the old webapp, the lucene index (I use a re-index method) and tomcat's cache. An other thing to note is that in my custom filter I added a log4j trace and my filter is properly executed ... It just look like that all the filters are executed but none are really used...

I use hibernate 3.3.0.SP1, hibernate search 3.1.1.GA, lucene 3.0.2

I'm completely lost, so if anyone could help me ...


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Wed Jul 21, 2010 7:07 pm 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Quote:
I use hibernate 3.3.0.SP1, hibernate search 3.1.1.GA, lucene 3.0.2

lucene 3 is defining a totally new way to create analyzers, so that won't work with search 3.1.1.
Could you please try using hibernate core 3.3.2, search 3.2.0, lucene 2.9.3 ?

You didn't find any error in the logs? I don't expect it to be able to create the analyzer, but if it doesn't say anything that's ugly.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 3:21 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
I tryied with the version you say and it does not change anything ...

I've no error in any log file ...


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 7:31 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Here is the complete versions log of all my jars in relation with hibernate :

hibernate-core-3.3.0.SP1
hibernate-commons-annotations-3.1.0.GA
hibernate-entitymanager-3.4.0.GA
hibernate-annotations-3.4.0.GA
com.springsource.apache.solr-1.3.0
com.springsource.apache.solr.common-1.3.0
com.springsource.apache.lucene-2.4.1
com.springsource.apache.lucene.analysis-2.4.1
com.springsource.apache.lucene.analysis.snowball-2.4.1
com.springsource.apache.lucene.search.highlight-2.4.1
com.springsource.apache.lucene.search.spell-2.4.1
lucene-core-2.9.3
lucene-snowball-2.9.2
lucene-analyser-2.9.2
hibernate-search-3.2.0.CR1
solr-core-1.4.0
solr-solrj-1.4.0

do you think there can ba some incompatibilities between the two lucene despite the package names?


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 8:54 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
why don't you update Search to the 3.2 final?
And why are you having on the classpath both spring-* and non spring copies of Solr and Lucene, of different versions? looks messy.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 9:02 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Yes I admit it it's a messy project ...

The reason why I don't update hibernate-search to the final is very simple : it is not available on the Maven repository ( http://repository.jboss.com/maven2/org/ ... te-search/ )

I'm looking to simplify the maven pom file but it's not very simple as it's not a from scratch project, it's a project I continued from sb else ...

I removed all spring related lucene & solr libraries but it doesn't change anything. At leas I've a lighter WAR file ...


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 10:01 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Okay, I've some news for you :)

I've modified my filter to this :
Code:
@Entity
@Indexed
@AnalyzerDef(name = "categoriestokensanalyzer", tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = {
   @TokenFilterDef(factory = StandardFilterFactory.class),
   @TokenFilterDef(factory = LowerCaseFilterFactory.class),
   @TokenFilterDef(factory = ISOLatin1AccentFilterFactory.class),
   @TokenFilterDef(factory = StopFilterFactory.class, params = {
      @Parameter(name = "words", value = "be/fgov/economie/artisans/services/stopwords.txt"),
      @Parameter(name = "ignoreCase", value = "true") }) })
@Analyzer(definition = "categoriestokensanalyzer")
public class Categories implements Serializable {


Now, something very strange happen. At data creation time, all peristed data go threw the filters and I get exactly what I want in my lucene index! Hoeraa!

But, at update index time, it doesn't go threw the filter anymore. So If I have to (and I have to) rebuild my index, I've a big problem ...

For info, my update code is the following (I adapted it from a code found in the book) :
Code:
    @RequestMapping("updateindex")
    public ModelAndView luceneIndex(HttpServletRequest req) {
   logger.debug("Re-Building Lucene index");
   Session session = sessionFactory.openSession();
   session.beginTransaction();
   FullTextSession fts = org.hibernate.search.Search.getFullTextSession(session);
   fts.setFlushMode(FlushMode.MANUAL);
   fts.setCacheMode(CacheMode.IGNORE);
   if (logger.isDebugEnabled()) {
       logger.debug("Beginning transaction ...");
   }
   Transaction tx = fts.beginTransaction();
   // Read the data from the database - Page 154 Hibernate search
   List<Class> reindexClasses = new ArrayList<Class>();
   reindexClasses.add(EnterpriseLang.class);
   reindexClasses.add(Categories.class);
   reindexClasses.add(Province.class);
   reindexClasses.add(Commune.class);
   reindexClasses.add(BusinessUnit.class);
   try {
       int index = 0;
       for (Class c : reindexClasses) {
      if (logger.isDebugEnabled()) {
          logger.debug("Starting reindex of " + c);
      }
      ScrollableResults results = fts.createCriteria(c).scroll(ScrollMode.FORWARD_ONLY);
      while (results.next()) {
          index++;
          fts.index(results.get(0));
          if (index % 100 == 0) {
         if (logger.isDebugEnabled()) {
             logger.debug("Flushing to index");
         }
         fts.flushToIndexes();
         fts.clear();
          }
      }

       }
       tx.commit();
   } catch (Exception e) {
       tx.rollback();
       logger.warn("Unable to update indexes", e);
   }
   return new ModelAndView("indexok");
    }


Thanks a lot for your help :)


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Thu Jul 22, 2010 10:58 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Quote:
The reason why I don't update hibernate-search to the final is very simple : it is not available on the Maven repository


Please note the Maven repository layout was changed recently:
https://repository.jboss.org/nexus/cont ... 2.0.Final/

See the blogpost for more info:
http://relation.to/Bloggers/JBossMavenRepositoryChanges

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Fri Jul 23, 2010 3:19 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Ok, thanks for the info, I've modified my Hibernate-search version to 3.2.0.Final

But it doesn't change anything to my index update problem...


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Fri Jul 23, 2010 8:39 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
I'm wondering if it is possible to implement an analyzer for the data read in the database and use the output to write the index. I think it's what lucene do, but I don't find where I can modify the analyzer.

I think that because I found this : http://lucene.apache.org/java/2_9_3/api ... lyzer.html And the ouptut I get matches exactly this Analyzer, so if i could replace it by my own analyzer, it would be wonderful! I just don't know how to do that ...


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Fri Jul 23, 2010 9:58 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Quote:
but I don't find where I can modify the analyzer.

There's a property to set in the configuration file to select the default analyzer to be used for all cases not overriden by annotations.
http://docs.jboss.org/hibernate/stable/ ... le/#d0e393

More important, could you provide a test case so I could investigate this better? If it's a bug it should be fixed.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Fri Jul 23, 2010 10:32 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Thanks for that :)

What do you need as test case? (sorry if the answer is trivial for you, but I'm beginner in this domain so ... I ask beginner's questions)


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Fri Jul 23, 2010 10:45 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
to contribute a testcase we expect a patch to be applied on the trunk of Hibernate Search, so you should prepare your environment to build Hibernate Search from sources, like explained here:
http://community.jboss.org/wiki/ContributingtoHibernateSearch

Then look at other tests for ideas: inspiration, style & conventions.

In this case I'd expect a patch which fails, to proof that we would expect something different than the current output; if you can create one, attach it to JIRA opening a new issue - that will get it in the stream of things to be handled soon by some committer, possibly fixing the problem in short time and merging your unit test in trunk together with the fix.

If you happen to provide regular contributions (also documentation improvements are much appreciated!), you could get write access to fix & commit yourself.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Mon Jul 26, 2010 6:27 am 
Beginner
Beginner

Joined: Thu Feb 25, 2010 11:33 am
Posts: 36
Ok, I will try to provide you some example and fix ... But this may take some time as I'm not a java guru yet :)

By now, I'm still struck by a Spring error, related to Hibernate :
Code:
26 Jul 2010 12:14:30,231 DEBUG DispatcherServlet:588 - Could not complete request
org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.NoClassDefFoundError: org/hibernate/action/BeforeTransactionCompletionProcess
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:924)
   at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
   at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
   at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
   at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:859)
   at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579)
   at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1555)
   at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NoClassDefFoundError: org/hibernate/action/BeforeTransactionCompletionProcess
   at org.hibernate.search.impl.FullTextSessionImpl.<init>(FullTextSessionImpl.java:103)
   at org.hibernate.search.Search.getFullTextSession(Search.java:46)
   at be.fgov.economie.artisans.controller.UtilController.luceneIndex(UtilController.java:169)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:421)
   at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:136)
   at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:326)
   at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:313)
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
   ... 21 more


I found the org/hibernate/action/ package in hibernate-core-3.3.0.SP1 but there is indeed no sign of any BeforeTransactionCompletionProcess class or interface ... I'm serching on google, but nowhere I can find in wich jar I can find it ... do you have an idea?


Top
 Profile  
 
 Post subject: Re: [Hibernate Search] Annotation problem
PostPosted: Mon Jul 26, 2010 6:42 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi, Hibernate Search 3.2 requires Hibernate Core 3.5.x (latest currently is 3.5.4)
BTW Search 3.2.1 is going to be released later today with some bugfixes.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.