-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: Parallel writes to index
PostPosted: Wed May 25, 2011 7:37 am 
Newbie

Joined: Wed May 25, 2011 7:06 am
Posts: 3
Hi,

i have a question related to the index-process of hibernate search.
Is it possible to let multiple servers write updates to the index file.
The setup i mean:
4 linux vm with one tomcat server on every vm. The vms share a network drive and one database on another linux vm.
So instead of setting up a JMS-Environment, is it possible to let every server write its changes to same shared index?


Top
 Profile  
 
 Post subject: Re: Parallel writes to index
PostPosted: Wed May 25, 2011 9:30 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi,
no that's not possible, by the design of Lucene only one VM can write at the index at the same time; that's why we propose the queue based architecture.

Generally, you can't even share a Lucene index on a shared disk: performance will be worse and you'll face many issues with locking.

To cluster the index, you either have to use the master/slave directory providers which will periodically sync remote directories, or use the clustered index via Infinispan to provide quick realtime updates to each node.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Parallel writes to index
PostPosted: Wed May 25, 2011 11:09 am 
Newbie

Joined: Wed May 25, 2011 7:06 am
Posts: 3
thx for the fast reply.
JMS or Infinispan. I havent done any of those.
Which one is lighter and faster to get to run?


Top
 Profile  
 
 Post subject: Re: Parallel writes to index
PostPosted: Wed May 25, 2011 12:08 pm 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
actually it's not about either Infinispan or JMS:
- Infinispan is a place to store the index (to share it properly)
- a queue is used to make sure only one node writes to the shared index

The queue can be either JMS or JGroups; JGroups might be a bit simpler to setup but JMS provides a persistent queue (if the master server crashes unprocessed messages are not lost).
JMS is very simple to setup in an application server, Tomcat doesn't have one so you'll have to pick one and use it embedded from your application - sorry I can't help you with that.

Infinispan is an alternative to the shared filesystems used by the backends in Search which copy the files (at file system level), you might need that only if you require changes to be visible to each node very quickly (realtime).

I'd suggest you to read the book, the JMS architecture is explained in depth there (though Infinispan didn't exist at the time of writing).

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Parallel writes to index
PostPosted: Thu May 26, 2011 2:08 am 
Newbie

Joined: Wed May 25, 2011 7:06 am
Posts: 3
Thx for all the Help!
I´ll try to go the way with JMS and the shared FS.
Its actually a very good idea to get the Book :) i´ll do that.
I will report my success or fail later. (And how long it really took)


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.