Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 9 posts ] 
Author Message
 Post subject: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Wed Oct 10, 2012 8:23 am 
Beginner
Beginner

Joined: Thu Sep 01, 2005 10:09 am
Posts: 39
HI, I just get started with the infinispan support on my cluster. I sync HSearch through JMS, but I want to persist the Index anyway, so that I can restart the systems without loosing it. I choose the jdbc store, but I came around an IndexOutOfSync today after restarting some of the cluster nodes.

So my questions are: Do I need the cache loader on every node or only on the master? Maybe the out of snyc comes from here?
Second question: Should I persist the locks cache or is it temporary anyway and it makes probably no sense to persist it.

Any hints will be appreciated, as always.


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Wed Oct 10, 2012 6:34 pm 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2534
Location: Third rock from the Sun
Hi,

I presume you're talking about the Infinispan DirectoryProvider?

Quote:
Do I need the cache loader on every node or only on the master? Maybe the out of snyc comes from here?

All Infinispan configurations should be the same; yes if you configure a CacheLoader on one node you should have the same configured on the others. They can have little differences like a different password / hostname, etc.. but they should point to the same CacheStore instance if it's setup as a shared CacheStore (highly recommended) or make sure they point to different instances is you have it setup as non-shared. (For example with jdbc using different table names would be good for non-shared).

Quote:
Should I persist the locks cache or is it temporary anyway and it makes probably no sense to persist it.

You're correct: they are temporary and persisting doesn't make sense. You shouldn't even be able to, as if you're using the default lock manager I wrote it's explicitly going to ignore any cachestore.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 3:17 am 
Beginner
Beginner

Joined: Thu Sep 01, 2005 10:09 am
Posts: 39
Hi Sanne thanks for clearing things a little up.

What I'm unsecure about is what you mean with "shared cache store instance". I configured a jdbc "stringbased" cacheloader on the caches. In terms of instances that would give me two instances that only share the tables. Is that what you mean or do I need to do something different here?

I am still wondering then where the sync problems I experienced came from.

So long
Thomas


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 8:46 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2534
Location: Third rock from the Sun
Quote:
What I'm unsecure about is what you mean with "shared cache store instance".

shared is a configuration option for the CacheStores :
https://docs.jboss.org/author/display/ISPN/CacheLoaders#CacheLoaders-Configuration

Basically you have to tell Infinispan if each node is going to be connected to the same store or if they are going to have independent stores. When shared, only one node will apply updates; when it's not shared each node owning a copy of the entry you wrote will write it.

In short if you configure it as shared but they are not in practice, you might lose data.

Quote:
I am still wondering then where the sync problems I experienced came from.

Sorry forgot to ask about that. I have not seen a "IndexOutOfSync" before, what do you mean exactly? Do you have a stacktrace to post?

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 8:53 am 
Beginner
Beginner

Joined: Thu Sep 01, 2005 10:09 am
Posts: 39
Good to learn that (about the shared stores).
What I see in the master log is
Code:
2012-10-11 10:34:09,591 ERROR  LogErrorHandler: HSEARCH000058: Exception occurred org.apache.lucene.store.LockObtainFailedExce
ption: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1ef3faa
Primary Failure:
        Entity de.eiswind.paris.domain.Person  Id 13  Work Type  org.hibernate.search.backend.UpdateLuceneWork

org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1ef3faa
        at org.apache.lucene.store.Lock.obtain(Lock.java:84)
        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1097)
        at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:127)
        at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:102)
        at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:119)
        at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:77)
        at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
        at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
2012-10-11 10:34:09,591 ERROR  LuceneBackendQueueTask: HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!



Hope you can make anything with that trace.


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 9:42 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2534
Location: Third rock from the Sun
ok yes that helps.

The node which is logging the exception is not able to acquire an exclusive lock on the index.

The master node is the only one which should ever try to acquire this lock: in theory there is no contention and the lock wouldn't even be needed; the lock exists to prevent mistakes to happen in wrong configurations as you would corrupt the index.

So we need to check your configuration files; the master one is the only node which should attempt to acquire the lock, so you should configure the master with the exclusive_index_use true, while all other nodes need this to be exclusive_index_use=false. This is the most likely mistake, please check that and otherwise post your configuration files.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 9:48 am 
Beginner
Beginner

Joined: Thu Sep 01, 2005 10:09 am
Posts: 39
Hi Sanne,

thanks for helping me so far,
I found an orphaned tomcat master instance, that didn't shut down properly and I guess that orphan was holding the lock. I killed it, changed the loaders to be shared and removed the persistence from the locks cache. I hope everything is fine now.

I appreciate your efforts for the community!


So long, Thomas


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Thu Oct 11, 2012 10:10 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2534
Location: Third rock from the Sun
Hi Thomas, thanks a lot it's great to get feedback.

I hope to make it easier to setup in the next release, hopefully less moving parts to configure.
Let me know how it all works, and which error messages could be reviewed to save some time to next user ;)

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Search: Lucene/Infinispan JDBC persistence questions
PostPosted: Tue Aug 01, 2017 10:11 pm 
Newbie

Joined: Wed Jun 07, 2017 5:44 pm
Posts: 6
Can you please post your configuration files for master and slave nodes. We are also seeing lock issues. We could not progress on it.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.