-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 8 posts ] 
Author Message
 Post subject: [Hibernate - Search]
PostPosted: Wed Feb 20, 2008 7:15 pm 
Newbie

Joined: Wed Feb 20, 2008 6:42 pm
Posts: 14
Hi,
Started using Hibernate Search 2 weeks back. Really amazed to see such a tool built on top of Hibernate.

Now, I am bumped into some sort of weird problem. I am setting hibernate.search.indexing_strategy as manual. Given the requirements of my project, I have to set it like this and it means I cannot use automatic indexing.

So now, my question is : if I am not using this strategy, what are my other options ?
I did the initial indexing of my whole database, but now, how to handle the new updates and inserts ? Should I write some daemon, to get me all the records which are updated/inserted from time to time and index all of them ? If so, how to handle the deletes ?

I am sure, there will be an easy solution, but I am too young to figure that out. Any help is greatly appreciated !!!

thanks
Anil !!!


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 20, 2008 8:38 pm 
Hibernate Team
Hibernate Team

Joined: Sun Sep 14, 2003 3:54 am
Posts: 7256
Location: Paris, France
Why do you think manual is the solution for your problem? What's wrong with the event based model? Maybe we can find some solution here.

If not, there is no magic solution to know which element to reindex, especially delete unless you put some trigger in the database. One solution is to reindex everything from scratch every night for example.

_________________
Emmanuel


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 20, 2008 9:34 pm 
Newbie

Joined: Wed Feb 20, 2008 6:42 pm
Posts: 14
Hey Emmanuel, thanks for the amazing turn around !!

Well, my reason is a simple one :

The insertion/updation/deletion happens in a very highly secured environment for which, I have only read only access. The system which does this insertion/updations is completely abstract to me.

The nightly index strategy is also no good for me, it took 3 good weeks to complete the initial indexing of some 10 million records. So, going by your words, I am at a dead end !!!

One small question though : why is the option of hibernate.search.indexing_strategy then ?

thanks a ton, for your immediate response though !!!

-- Best Regards
Anil.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 20, 2008 11:49 pm 
Hibernate Team
Hibernate Team

Joined: Sun Sep 14, 2003 3:54 am
Posts: 7256
Location: Paris, France
10 millions records should certainly not take 3 weeks to index! Something like a few hours seem more reasonable

_________________
Emmanuel


Top
 Profile  
 
 Post subject: [Hibernate - Search]
PostPosted: Thu Feb 21, 2008 12:08 am 
Newbie

Joined: Wed Feb 20, 2008 6:42 pm
Posts: 14
Really ?
well, looks like I screwed up something real big !!! :)

thanks a ton, Emmanuel !!!

-- Best Regards
Anil.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Feb 21, 2008 12:15 pm 
Hibernate Team
Hibernate Team

Joined: Sun Sep 14, 2003 3:54 am
Posts: 7256
Location: Paris, France
Check http://www.hibernate.org/hib_docs/search/reference/en/html_single/#search-batchindex

for the first pieces of advise, Also check some of the Lucene merge factors etc

_________________
Emmanuel


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 27, 2008 10:42 am 
Newbie

Joined: Fri Jan 16, 2004 9:59 am
Posts: 18
Location: La Plata-Argentina / Luxembourg
anilit99, can you share some moredata.
i Use hibernate and the search component ( i start using lucene before hibernate search, and develope a custom component, no we migrate this solution to hibernate search0
Our scenario is:

- more than 500 millions indexed object (around 20 indexed classes)
- Tables with more than 30 million records
- Simple Batch indexing in the night, takes about 4 hours
- parallel indexing (i develop a very simple grid with gridgains) takes 1,5 hour in a grid of 4 nodes (with one node takes the same as the simple indexing obviously).
- Mysql database in a no very powerfull server (dual core 2, 8 gb ram, dedicated)


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 27, 2008 10:52 am 
Newbie

Joined: Fri Jan 16, 2004 9:59 am
Posts: 18
Location: La Plata-Argentina / Luxembourg
anilit99, can you share some more data.
i Use hibernate and the search component ( i start using lucene before hibernate search, and develop a custom component, no we migrate this solution to hibernate search0
Our scenario is:

- more than 500 millions indexed object (around 20 indexed classes)
- Tables with more than 30 million records
- Simple Batch indexing in the night, takes about 4 hours
- parallel indexing (i develop a very simple grid with gridgains) takes 1,5 hour in a grid of 4 nodes (with one node takes the same as the simple indexing obviously).
- Mysql database in a no very powerfull server (dual core 2, 8 gb ram, dedicated)


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 8 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.