How To Get Lucene's Directory via Hibernate Search API?

xsalefter · **Joined:** Tue Jun 01, 2010 4:18 am **Posts:** 11

Hi all,

I'm using hibernate search 4.1.0.Final. I'm trying to get Lucene's Directory via Hibernate Search API, and the docs says that I need to use IndexManager. My problem is I can't found any example about how to obtain IndexManager instance.

Yes, I'd see this code that explain how to get IndexManager, but I think it is not that I really looking for. There's a way to do this easily?

FYI, my case is I want to create index/directory to store spell checker information in the same directory/index with entity index. Or, do I going to wrong direction? Any idea/suggestion?

Thanks,
xsalefter

sanne.grinovero · **Posted:** Tue Jun 05, 2012 10:16 am

Hi,
the short answer requires a little casting:

Quote:

SearchFactoryImplementor searchFactory = (SearchFactoryImplementor) fullTextSession.getSearchFactory();
IndexManager manager = searchFactory.getAllIndexesManager().getIndexManager( "targetIndexName" );

In practice we would not expect you to need accessing the IndexManager directly very often, that's why there is not a better looking way to get a reference to it.. you think we should expose that in the SearchFactory interface itself?
Maybe we could add that or an helper.

Quote:

I'm using hibernate search 4.1.0.Final.

Make sure you upgrade to 4.1.1, important reason described here: http://in.relation.to/Bloggers/Hibernat ... eakAndMore

Quote:

FYI, my case is I want to create index/directory to store spell checker information in the same directory/index with entity index. Or, do I going to wrong direction? Any idea/suggestion?

There are several other approaches.

1)It seems Lucene is now supporting a new way to do efficient spell checking without the need for this additional index. I just heard about it and don't know much about it yet, you should best ask on the Lucene mailing list.

2) You could plug in your own IndexManager: have a look at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager, an IM is very simple as it mostly manages lifecycle/initialization of other components that you can assemble as needed to provide richer functionality.
I'm not sure if this solves all your needs but would definitely be a good way to create/initialize the indexes at the proper time.

Let us know how your experiments go! There is an open issue to provide spellchecking functionality in Search, perhaps you could contribute it. https://hibernate.onjira.com/browse/HSEARCH-269

xsalefter · **Joined:** Tue Jun 01, 2010 4:18 am **Posts:** 11

Hi, sanne.. Thanks very much for complete answer..

sanne.grinovero wrote:

Hi,

In practice we would not expect you to need accessing the IndexManager directly very often, that's why there is not a better looking way to get a reference to it.. you think we should expose that in the SearchFactory interface itself?
Maybe we could add that or an helper.

Umh, personally I have no idea. After all, I'm using HSearch for less than 1 month. But as my experience, I think yes, It's nice if HSearch have integrated feature for this. For example, now i got:

Quote:

org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@G:\VarEtcTmpLog\lucene\index\notefinder\org.xsalefter.notefinder.entity.Note\write.lock

When try to write lucene index and spell checker index in one shot in servlet listener. I'm still looking solution for this (any idea? ;) ), but refer to this post this is because the IndexWriter is already opened. This is strange because I don't accessing IndexWriter directly:

Code:

    private void createLuceneIndex() {
       this.fullTextEntityManager = Search.getFullTextEntityManager(entityManager);
      try {
         this.fullTextEntityManager.createIndexer().startAndWait();
         logger.info("Index created.");
      } catch (InterruptedException e) {
         logger.error("Indexing process interrupted: {}.", e);
      }
    }
    
    private void createLuceneSpellCheckIndex() {
       SearchFactoryImplementor searchFactory = (SearchFactoryImplementor) 
             this.fullTextEntityManager.getSearchFactory();
      IndexManager indexManager = searchFactory.getAllIndexesManager().
            getIndexManager(Note.class.getName());
      IndexReader indexReader = this.fullTextEntityManager.getSearchFactory().
            getIndexReaderAccessor().open(Note.class);
      
      Directory directory = ((DirectoryBasedIndexManager) indexManager).getDirectoryProvider().getDirectory();
      SpellChecker spellChecker = null;
      try {
         spellChecker = new SpellChecker(directory);
         LuceneDictionary dictionary = new LuceneDictionary(indexReader, "spellcheck");
         
         IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_35, defaultAnalyzer);
         spellChecker.indexDictionary(dictionary, config, true);
      } catch (IOException e) {
         logger.error("IOException when create spell checker directory: {}", e);
      } finally {
         if (spellChecker != null)
            try {
               spellChecker.close();
            } catch (IOException e) {
               logger.error("IOException when close spellChecker object: {}", e);
            }
      }
    }

In future, perhaps there's a bridge so user like me doesn't need to write bunch of code to indexing spellchecking directory. Something like

Code:

this.fullTextEntityManager.createIndexer().startAndWait();

spesialized for spellchecker, perhaps?

sanne.grinovero wrote:

Let us know how your experiments go! There is an open issue to provide spellchecking functionality in Search, perhaps you could contribute it. https://hibernate.onjira.com/browse/HSEARCH-269

You expected too much from people that use HSearch in a month :). But I can't see any branch in github for this issue. Do I miss something?

sanne.grinovero · **Posted:** Wed Jun 13, 2012 10:40 am

Hi, the line

Code:

spellChecker.indexDictionary(dictionary, config, true);

is opening an IndexWriter. It also closes it immediately after, but that means that when you invoke this method, you'll need it to point to an unlocked Directory.

At runtime Hibernate Search aggressively opens an IndexWriter to reuse it for any operation; that's a mayor performance benefit as opening IndexWriters is very slow. So since we already have an IndexWriter open on your managed Directory, the index is locked already.

The IndexManager isn't exposing the IndexWriter; and even if it did, this indexDictionary method doesn't take one.

So options you have:
- configure Hibernate Search to not keep hold on the IndexWriter (releasing locks aggressively); see exclusive_index_use at http://docs.jboss.org/hibernate/search/4.1/reference/en-US/html_single/#table-performance-parameters
- do what you need to do before the IndexManager starts it's own IndexWriter.
- use a dedicated Directory instance. I think this should be preferred, by extending one of the standard IndexManagers, you can have it use the standard Directory for the usual goals, but in addition create/initialize a secondary Directory which you would use exclusively for this purpose.

Quote:

You expected too much from people that use HSearch in a month :). But I can't see any branch in github for this issue. Do I miss something?

No worries, I don't expect you to make the perfect integration! But if you happen to pull something together and are willing to share it, we can then work on it together to polish it well enough for integration. In your advantage, we would help you with it and then maintain it, plus more users will test it in the next release and future releases. But no pressure and it doesn't need to be done quickly, you can do as you prefer, hack something together and then see if you think it's worth to share or not.

There is no branch in github for this, and I don't think John started on it, nor that he will.. it's an old issue and I'm afraid he won't show up.

To get you started:

Have a look into
org.hibernate.search.indexes.impl.DirectoryBasedIndexManager and make it your base class; extending it is trivial, as you can see in
org.hibernate.search.indexes.impl.NRTIndexManager and org.hibernate.search.test.util.RamIndexManager. The last one is in the testsuite, so you have an example on how to configure it too.

After you've got it working, this might be usefull too:
https://community.jboss.org/wiki/Contri ... nateSearch

BTW, you don't need to create a branch in Hibernate Search to create an extension such as a custom IndexManager; you can start simply by having such an implementation in your application:
http://docs.jboss.org/hibernate/search/4.1/reference/en-US/html_single/#d0e871