-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 11 posts ] 
Author Message
 Post subject: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 6:11 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
Hi,

i created an index with HS 3.2. (core 3.5.) with the massindexer. The application i use the index in, uses HS 3.0.1 (core 3.2.6), and its not possible to change that. Now the index created with 3.2. isnt working with 3.0.1, it throws the exception:

Code:
org.hibernate.search.SearchException: Unable to read current status of Lucene IndexReader
...
Caused by: org.apache.lucene.index.CorruptIndexException: Unknown format version: -9
   at org.apache.lucene.index.SegmentInfos$2.doBody(SegmentInfos.java:360)
   at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:636)
   at org.apache.lucene.index.SegmentInfos.readCurrentVersion(SegmentInfos.java:349)
   at org.apache.lucene.index.DirectoryIndexReader.isCurrent(DirectoryIndexReader.java:150)
   at org.hibernate.search.reader.SharedReaderProvider.openReader(SharedReaderProvider.java:83)
   ... 17 more


is it possible to generate an index with HS 3.5. and use it with HS 3.0.1 at all?
(i remember that i've seen some constants in a code like "Lucene.Version_XX", but i don't remember where i've seen that)


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 7:58 am 
Pro
Pro

Joined: Wed Oct 03, 2007 2:31 pm
Posts: 205
I don't believe it is possible to down grade, for example building index with a higher version and then trying to read them with a lower version. You can obviously do it the other way around.


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 8:52 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
thx, but the other way round wouldnt help me at all. the problem is, as told before, the environment i'm searching on, isnt able to upgrade to a newer version of hibernate, so it can't make use of hibernate search 3.2 with its massindexer. So i have to find a way, to use the massindexer building up an index that can be read by hibernate search 3.0.1. Using the old way, with that pagination and all that stuff, drives me nuts, and will cost way to much time.

isnt there a way to convert the (lucene)2.9.2-index to an 2.3-index? luke can handle all of these, so at least there have to be a way to read all of them. Once i read it into memory, i could use an old version of an indexwriter to produce a 2.3-index, couldn't i?


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 9:04 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Quote:
is it possible to generate an index with HS 3.5. and use it with HS 3.0.1 at all?

This incompatibility is coming from Lucene index formats. HS 3.2 is targeting Lucene 2.9 but should work also on Lucene 2.4.1, which was the version supported by HS 3.1.
The Lucene version used by HS 3.0.1 is older and it won't work using the same one, but I think that if you use HS3.2 with Lucene 2.4 this could be able to produce an index which is usable by the older Lucene, it's only in 2.9 that internal formats changed radically (it's basically a 3.0 preview, with backwards API compatibility).

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 10:19 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
that sounds nice, but if i replace the lucene core 2.9.2 with the 2.4.1 in my indexbuilding app, i get the following exception:
Code:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.lucene.store.FSDirectory.open(Ljava/io/File;Lorg/apache/lucene/store/LockFactory;)Lorg/apache/lucene/store/FSDirectory;
   at org.hibernate.search.store.DirectoryProviderHelper.createFSIndex(DirectoryProviderHelper.java:108)
...


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Mon Jul 12, 2010 11:52 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
ah sorry; so it's not compatible, but it should be simple to convert as I expect just a bunch of method signatures to be changed;

easier than to backport the massindexer anyway.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Tue Jul 13, 2010 4:54 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
ok, now i've created twice time an index of the same db, one time with HS 3.5 (lucene 2.9.2), the other one with HS 3.1. (lucene 2.4.1). When i open one of the directorys a class is stored in, the structure is completly different. One has 25 objects (the 2.4.1. one), the other one just has 3 objects (the 2.9.2 one), obv. the second one stores all of the information in one file, while the other one distributes it to different files. another problem is that the content is binary (ok, not really binary, but a lot of control characters), i don't see any chance here, how i should be able to convert that. Any ideas?


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Tue Jul 13, 2010 8:36 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Lucene creates several segments, and periodically merges them back in one to optimize search performance. Make sure to call "optimize()" to force merging to a single main file + some metadata . The Massindexer by default optimizes at the end, so that's likely already done with 3.2.

To inspect the index, use Luke. And no, I see no way to convert it manually, as you've seen the format is complex and really meant for maximum performance.

Are you able to load the index produced by HS3.1 by HS3.0.1 ?

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Tue Jul 13, 2010 9:02 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
I don't think i'll get this working the way i wanted to. Switched back to 3.0.1 and try&errror finetuning my parameters with the scrolledResultSet. With current settings i will need approx. 8h instead of 40 minutes. Sick, but obv. there is no other way?!?


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Tue Jul 13, 2010 9:08 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
really, adapting the code of HS 3.2 to use Lucene 2.4 is easy, something around 3 lines of code.

It's harder to adapt it to Lucene 2.3, so I'd check if you're able to read the index produced by HS3.1+Lucene2.4 by your older app, if that works I can help you in doing the trivial changes, so you can have a HS3.2-Custom+Lucene2.4 to produce the index for your HS3.0.1+Lucene2.3

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: generating index with HS 3.2 for HS 3.0.1
PostPosted: Tue Jul 13, 2010 10:53 am 
Regular
Regular

Joined: Thu Nov 26, 2009 8:45 am
Posts: 78
Well, would really be nice to have that, i just do not have the time this week to check it. maybe next week. thx for your replies anyway.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 11 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.