how to shutdown hibernate-search (lucene part) correctly?

JP-Ulm · **Joined:** Fri Sep 26, 2008 2:39 am **Posts:** 20

Hi,

My question is, how do i shutdown hibernate-search completely?

What i am doing right now is:

session.close();
sessionFactory.close();

But this is not enough - because after i do that, i can't delete the lucene index directory - so i am guessing the lucene indexwriter is still active?

I know that lucene can be accessed via SearchFactory but it does not provide any shutdown options.

I have also tried to not close the session but calling close on a FullTextSession, but that doesn't make a difference.

I am using:
Implementation-Title: Hibernate Search
Implementation-Version: 3.0.1.GA
(from META-INF/manifest.mf)

Thanks in advance for any help or clues.

(Why do i need this? - It's for unit testing, i need to be able to purge everything so the next test can run on a fresh environment(index), without restarting the JavaVM.)

hardy.ferentschik · **Posted:** Wed Jan 07, 2009 8:04 am

Hi,

I don't think that physically deleting the index directories is a good idea. I would not be surprised if you would get some wired problems if you do so, since the index directories get created at initialization time.

Why not call purgeAll in the tear down method of your test. This way you will always have a empty index. I also recommend to run the tests against a RAMDirectoryProvider. This will make the tests run faster and make them independent from the file system. You could add a switch which allows you to also run against FSDirectoryProvider in the cases where you want to debug a test and inspect the index with Luke. All this can be achieved with a custom base class for the tests.

--Hardy

JP-Ulm · **Joined:** Fri Sep 26, 2008 2:39 am **Posts:** 20

Thank you very much

hi-tech · **Joined:** Tue Feb 24, 2009 8:39 pm **Posts:** 19

After index some classes my hibernate-search code doesn't terminate.

The indexing code runs ok, but it never terminates.
I'm closing my ScrollableResults, my transaction and my session, but
it never terminates.

I don't know why, all seems in order.

I'm working with with Hibernate Search book code, listing 5.22:

Code:

   public void index(Class<?> clazz) {
      int BATCH_SIZE = 5000;
      Session session = HibernateUtilSearch.getSessionFactory().openSession();
      session.setFlushMode(FlushMode.MANUAL);
      session.setCacheMode(CacheMode.IGNORE);
      FullTextSession fts = org.hibernate.search.Search
            .getFullTextSession(session);
      log.debug("FullTextSession - closing");
      Transaction tx = fts.getTransaction();
      tx.begin();
      ScrollableResults results = fts.createCriteria(clazz).scroll(
            ScrollMode.FORWARD_ONLY);
      int index = 0;
      while (results.next()) {
         index++;
         fts.index(results.get(0));
         if (index % BATCH_SIZE == 0) {
            fts.flushToIndexes();
            log.debug("flushToIndexes: " + index);
            fts.clear();
         }
      }
      log.info(index + " indexed records");
      fts.flushToIndexes();
      fts.clear();
      tx.commit();
      results.close();
      fts.close();
   }

My Hibernate Search version is 3.1.0

hardy.ferentschik · **Posted:** Fri Mar 20, 2009 6:05 am

What do you mean with "it never terminates"? The function index() never returns? Which part of the log messages appear in th log?
The code looks ok.

--Hardy

hi-tech · **Joined:** Tue Feb 24, 2009 8:39 pm **Posts:** 19

The function index always return, but my java virtual machine does not end, and the index files remains locked.

My indexing code is called inside a for (), see:

Code:

public void procesaClases() {
      Map<String, ?> metadata = HibernateUtil.getSessionFactory()
            .getAllClassMetadata();
      Set<String> clases = metadata.keySet();
      for (String key : clases) {
         if (((AbstractEntityPersister) metadata.get(key)).hasSubclasses())
            try {
               Class<?> claseIndexada = Class.forName(key);
               if (claseIndexada.getAnnotation(Indexed.class) != null) {
                  log.info("Indexando : " + key);
                  long t1 = System.currentTimeMillis();
                  index(claseIndexada);
                  long t2 = System.currentTimeMillis();
                  log.info("Tardó :" + (t2 - t1) * 0.001 + " seg");
               }
            } catch (ClassNotFoundException e) {
               log.error("Ocurrió un error", e);
            }
      }
      log.info("Cerrando hibernate");
      HibernateUtil.getSessionFactory().close();
   }

And is called inside a main()

Code:

public static void main(String[] args) {
      ECEIndexador ece = new ECEIndexador();
      ece.procesaClases();
   }

And the output is:

11:05:17,551 INFO ECEIndexador:85 - Indexando : com.datos.Catalogo
11:05:45,537 INFO ECEIndexador:67 - 46741 registros indexados
11:05:46,662 INFO ECEIndexador:89 - Tardó :29.111 seg
11:05:46,662 INFO ECEIndexador:95 - Cerrando hibernate

From eclipse debug view, I see a java virtual machine with no threads, still running.

With the help of process explorer, I see the java virtual machine alive after running the command.

-- Arturo

sowelie · **Joined:** Wed Dec 17, 2008 12:10 pm **Posts:** 47

I am having the same problem...When my objects get indexed (be it automatically or manually) the JVM that the indexing occurred in will never close, I have to kill the process.

sowelie · **Joined:** Wed Dec 17, 2008 12:10 pm **Posts:** 47

For me, the problem was not shutting down the session factory. Make sure and call SessionFactory.close(). Thanks to s.grinovero for the answer!

annorax · **Joined:** Sat Jun 13, 2009 10:06 pm **Posts:** 2

sowelie wrote:

For me, the problem was not shutting down the session factory. Make sure and call SessionFactory.close(). Thanks to s.grinovero for the answer!

Thanks, sowelie and s.grinovero, that did the trick for me to.

It is interesting to note, however, that this behavior first appeared in the 3.1.0.GA release, i.e. until 3.1.0.CR1 (including) the JVM would shutdown correctly after indexing even if the SessionFactory had not been closed explicitly.

I assume that since 3.1.0.GA, hibernate search is using another resource obtained via the SessionFactory (perhaps a temporary session) - a resource that is never released, not even when all fullTextSessions have been closed. I do not think that this is a correct behavior. Hibernate search classes should release all acquired resources and close all sessions that have not been opened explicitly by the user.

In short, this seems to be a regression bug. Even though it never hurts for the user to explicitly close the SessionFactory, it shouldn't be necessary, and indeed was not up until the introduction of this bug.

sanne.grinovero · **Posted:** Tue Jun 16, 2009 5:26 am

Hi, you're right this got introduced during development of 3.1.0-something;
what's actually happening is that changes to different indexes are done in parallel, this is a great performance improvement and removes quite some internal locks. Unfortunately do do anything in parallel an ExecutorService is added, which will keep the JVM alive until it's properly shutdown.

Quote:

Even though it never hurts for the user to explicitly close the SessionFactory, it shouldn't be necessary, and indeed was not up until the introduction of this bug.

I wouldn't rely on such an assumption, any resource having a close() on it should carefully be closed, even if you might not see anything bad happening, you really don't know what kind of resource corruption you'r casting.
SessionFactory's close() exists since a long time and there are for sure other good reasons to use it.

When I made this patch I had the same worries you have and asked on the dev-list if I should try to close automatically, detecting somehow the situation in which a user forgets to close(); the answer from more experienced hibernate developers was that people should always close the SessionFactory. This is usually managed for you by framework helpers or application servers; if you code yourself make sure you close it.
I'm sorry for the inconvenience, there's a proper warning in the FAQ.

sanne.grinovero · **Posted:** Tue Jun 16, 2009 5:34 am

And of course I'm open for suggestions!
It would be very cool to have some *clean* way to detect the case to be able to log a warning.

erik.vido · **Joined:** Fri Jun 19, 2009 5:09 am **Posts:** 1

Hi, I had the same problem and after some research found the cause to the issue. The reason why the JVM does not shut down properly is that after indexing there are still some threads alive. This happens due to a buggy implementation of org.hibernate.search.backend.impl.lucene.QueueProcessors, which is the main class responsible for doing the actual indexing work. Indexing, no matter if set to 'sync' or 'async' mode, is done in a separate thread created by executor.execute(...), c.f. runAllAsync(), resp. runAllWaiting(). This is the preferred way to do this, but unfortunately the implementor forgot to call executor.shutdown() once the work is done. Here's what Joshua Bloch says in 'Effective Java - Item 68: Prefer executors and tasks to threads':

Quote:

"... how to tell the executor to terminate gracefully (if you fail to do this, it is likely that your VM will not exit)..."

Yep, just what happens!

Why does SessionFactory.close() do the job?

The SessionFactory.close() method, apart from other stuff, iterates over all registered listener and calls their cleanup()-method. Since the FullTextIndexEventListener is part of this group the open threads will get aggressively released.

What can you do?

Well not much! You must keep on using the SessionFactory.close() hack till the issue is properly resolved (I'll put a bug in JIRA). This is, unless you want to recompile the fixed sources yourself, which is generally a bad idea. There is a slightly less aggressive solution, but that is really not so much different:

Keep a reference to the org.hibernate.cfg.Configuration-instance and then call:

Code:

for (FlushEventListener fl : configuration.getEventListeners().getFlushEventListeners()) {
   if (fl instanceof FullTextIndexEventListener) {
      ((FullTextIndexEventListener) fl).cleanup();
   }
}

(of course you only need to do the for-loop once, to get the FullTextIndexEventListener)

Remark: About from the obvious bug, the documentation of hibernate-search is not correct. It claims that running hibernate-search in the 'sync'-mode will be done in the main-Thread, which is definitely not the case.

hardy.ferentschik · **Posted:** Fri Jun 19, 2009 5:24 am

Thanks for the detailed research into this issue. Maybe you can post the link to the Jira isse later on. Feel free to attach a patch as well ;-)

If we can confirm the issue I am sure we will get it integrated shortly.

--Hardy

sanne.grinovero · **Posted:** Fri Jun 19, 2009 6:03 am

Quote:

Remark: About from the obvious bug, the documentation of hibernate-search is not correct. It claims that running hibernate-search in the 'sync'-mode will be done in the main-Thread, which is definitely not the case.

Right I overlooked that sentence in the documentation; still the main thread is waiting for all subtasks to finish, so the behaviour is the same for you. I'll fix it but it should't matter.

Quote:

This is the preferred way to do this, but unfortunately the implementor forgot to call executor.shutdown() once the work is done. Here's what Joshua Bloch says in 'Effective Java - Item 68: Prefer executors and tasks to threads':

I didn't forget to close(), it would be silly if I had to create a new Executor for each job received by the backend, that doesn't scale. After each work is done, a new index change is likely to be submitted to the queue so the threads inside the Executor are reused. That's the whole point of using an Executor, and what Bloch means to say in his book (and also confirmed by "Java concurrency in practice", were Bloch is one of the authors).
So what I really need to do is close the Executor when I'm done with workqueues, this happens to be at SessionFactory/Application shutdown, so the code is correct.

As I said in my previous post I'm not very happy with people having to remember to close their SessionFactory, but that's recommended anyway so do it.

Quote:

You must keep on using the SessionFactory.close() hack till the issue is properly resolved (I'll put a bug in JIRA). This is, unless you want to recompile the fixed sources yourself, which is generally a bad idea. There is a slightly less aggressive solution, but that is really not so much different:

Hack? you're really supposed to close the resources you use, not to keep extra references around and mess with listeners.
As I said in previous post: frameworks (like Seam,Spring,Grails or managed EJB3) will do the housekeeping for you, but if you want to do it yourself you shouldn't blame the code when you forget.

hardy.ferentschik · **Posted:** Fri Jun 19, 2009 6:13 am

Right. it might be worth to add this explaination either to the manual and/or to the Hibernate Search wiki - https://www.hibernate.org/420.html

Maybe the manual could mention the importance of callinf close on the factory whereas on the wiki we could give the technical explaination. WDYT?

--Hardy