Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Lock not released by MassIndexer HSearch 4
PostPosted: Thu Feb 02, 2012 1:11 pm 
Pro
Pro

Joined: Wed Nov 05, 2003 7:22 pm
Posts: 208
Hi,

I'm testing against Hibernate Search 4.1.0Alpha1 and I have a problem with the lock not being released.
First of all, I'm using lockstrategy is native, which doesn't seem to work. If I kill the Tomcat process, the lock file is still there (Windows 7).

Code:
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource"/>
        <property name="configLocation" value="classpath:hibernate.cfg.xml"/>
        <property name="hibernateProperties">
            <value>
               hibernate.bytecode.provider=javaassist
               hibernate.connection.driver_class=${jdbc.driverClassName}
               hibernate.connection.is-connection-validation-required=true
               hibernate.jdbc.batch_size=30
                hibernate.dialect=${hibernate.dialect}
                hibernate.use_sql_comments=false
                hibernate.query.substitutions=true
                  hibernate.generate_statistics=true
                hibernate.search.default.directory_provider=filesystem

                hibernate.search.default.locking_strategy=native
                hibernate.search.default.indexBase=${hibsearch.hibernate_index_dir}
                hibernate.search.analyzer=nl.project.dao.hibernate.search.MyStandardAnalyzer
                hibernate.search.autoregister_listeners=true
[...]


Second, MassIndexer creates a lock when it starts running but after it completes, it's still there.

The code for calling MassIndexer is
Code:
fts.createIndexer(clazz).purgeAllOnStart(true)
                  .optimizeOnFinish(true).threadsToLoadObjects(3)
                  .batchSizeToLoadObjects(30)
                  .threadsForSubsequentFetching(12).startAndWait()


Any suggestions?

Marc


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Thu Feb 02, 2012 1:14 pm 
Pro
Pro

Joined: Wed Nov 05, 2003 7:22 pm
Posts: 208
Right, eternal locks are default unless you set
hibernate.search.default.exclusive_index_user=false

Doesn't seem like an awesome default to me.


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Thu Feb 02, 2012 2:49 pm 
Pro
Pro

Joined: Wed Nov 05, 2003 7:22 pm
Posts: 208
Hmm, after MassIndexing, Hibernate keeps a write.lock file in the directory, even if
hibernate.search.default.exclusive_index_user=false


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Thu Feb 02, 2012 3:44 pm 
Pro
Pro

Joined: Wed Nov 05, 2003 7:22 pm
Posts: 208
The lock also leads to subsequent MassIndexer failures:

Code:

org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@h:\java\projects\indexDir\nl.msw.compraventa.model.Offer\write.lock
   at org.apache.lucene.store.Lock.obtain(Lock.java:84)
   at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1108)
   at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:125)
   at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:100)
   at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:114)
   at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:101)
   at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:69)
   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
   at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
   at java.util.concurrent.FutureTask.run(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Thu Feb 02, 2012 4:09 pm 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
The property name is exclusive_index_use , does not have a final r

I don't know what Windows7 should do with the "native" lock; It uses a NIO file lock, so it's up to the operating system to cleanup. If our process is killed, we can't do much anymore to clean the lock :-/

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Mon Feb 06, 2012 11:19 am 
Pro
Pro

Joined: Wed Nov 05, 2003 7:22 pm
Posts: 208
Ok, thanks. Not sure if you saw my comment on JIRA but it does seem like the MassIndexer never releases it's write lock even with setting correctly applied.


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Mon Feb 06, 2012 12:13 pm 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Quote:
Not sure if you saw my comment on JIRA but it does seem like the MassIndexer never releases it's write lock even with setting correctly applied.

Thanks! I'll add a test and fix that soon.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Mon Apr 30, 2012 7:37 am 
Regular
Regular

Joined: Thu Oct 08, 2009 10:34 am
Posts: 55
Hey Guys,

Just stepping up from 3.4 to 4.1 and i seem to be having the same issue with regard the "write.lock" file still being present after a mass index!!


Maybe i have missed some special config when stepping up !!

My config looks like so

Quote:
#Hibernate Search properties

#Master Directory config
## DirectoryProvider
# (remote) master location where information is copied to
hibernate.search.default.sourceBase=C:/lucene/shared

# local master location
hibernate.search.default.indexBase=C:/lucene/master

# refresh every half hour,defined in secs
hibernate.search.default.refresh=120

# appropriate directory provider
hibernate.search.default.directory_provider=filesystem-master



I dont think i need more config for the upgrade(I might have missed something).

Like i said, after i do a mass index on an entity the "write.lock" file remains after it!! I see this on XP!!



Cheers,
LL


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Mon Apr 30, 2012 9:11 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Hi,
if you're still having the lock, and you're using native locking, it very likely means the application which acquired the lock is still running.

Please check it's not running in background? And don't forget to shutdown Hibernate correctly! (using SessionFactory#close or javax.persistence.EntityManagerFactory#close() depending if you are using plain Hibernate or JPA)

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Tue Jul 24, 2012 6:37 am 
Regular
Regular

Joined: Thu Oct 08, 2009 10:34 am
Posts: 55
Quote:
Please check it's not running in background? And don't forget to shutdown Hibernate correctly! (using SessionFactory#close or javax.persistence.EntityManagerFactory#close() depending if you are using plain Hibernate or JPA)


Sorry for late reply......buts surely you mean the entityManager.close and not the factory!! If the factory then how do i bypass this, i dont want to tear up and tear down the factory everytime my app has indexed!!


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Tue Jul 24, 2012 8:04 am 
Regular
Regular

Joined: Thu Oct 08, 2009 10:34 am
Posts: 55
Ok i think it is just a step up issue and change in behaviour


exclusive_index_use is defaulted to true which means that the write.lock is exists till the death(ore factory is teared down and recreated), i was use to the 3.x version seeing the write.lock file being removed after it had finished indexing. And it appears in the 4.x that if this is set to true then it will exist until the factory is destroyed.

If i set exclusive_index_use to false then it is the same behaviour i was use to in the 3.x version.

I guess it make no difference to the clients in a master/slave setup that they now have a write.lock file in their index i this is set to "true"??


Top
 Profile  
 
 Post subject: Re: Lock not released by MassIndexer HSearch 4
PostPosted: Tue Jul 24, 2012 11:48 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2296
Location: Third rock from the Sun
Hi,
yes on all your findings and questions.

It's not a problem for Hibernate Search that the lock file is still there, since all its internal functions are routed to the same IndexWriter instance for best performance.

You should set exclusive_index_use=false only if you have other applications writing to the index, so that Hibernate Search acquires the lock for the shortest time possible. Generally I think that should still be avoided, because if you have two applications writing at the same time and often contending the lock you end up with timeout exceptions.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.