-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: Hibernate Search for 50 to 100 million records on MySQL
PostPosted: Wed Apr 25, 2012 3:02 am 
Regular
Regular

Joined: Sun Apr 13, 2008 3:04 am
Posts: 71
Location: Bangalore
Hi,

We are looking to consider Hibernate Search for a 100 million records of structured RDBMS data stored in MySQL.
Could experts point us to any such usage of HS in any big business production environments.

I am for HS due to annotations approach and need not worry about delays due to updates to RDBMS data.
However due to leadership of Solr as Search Server (based on market awareness & opensource from Apache), we are still hesitant to go with HS.

Any leads on what are the production deployments and their performance metric can help a lot

Regards,
Nagendra

_________________
Raja Nagendra Kumar,
C.T.O
http://www.tejasoft.com
TejaSoft - Specialists in Code Audit, Unit Testing and Merciless Re-factoring - Engineering Crisis Turnaround Experts


Last edited by nagkumar on Wed Apr 25, 2012 9:13 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Hibernate Search for 50 to 100 million records on MySQL
PostPosted: Wed Apr 25, 2012 11:36 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi Nagendra,
we could argue about "Solr leadership" :) The true leader in this market is Apache Lucene, and both Solr and Hibernate Search use Apache Lucene to manage the index.

Solr is mostly a server to provide easy access to Lucene's features, it basically exposes it via HTTP requests. Hibernate Search is not a server but embeds in your application and is specialized for Hibernate users: obviously we try do to a very good job for Hibernate/JPA users, but are not suited if not using a relational database.

100 million records is a large dataset and will require some tuning and a bit of understanding of how Lucene works, but in both cases you'll have the same problems and benefits as it really is the same library they are using. I guess you will need sharding and a multi-master configuration (but not necessarily.. depending on actual index size and workload type).

We do have big customers using these sizes or larger, unfortunately all of the big customers can't be named; still Red Hat stands behind this product so in case of need you know there is professional help and support available.. I hope you can consider that as a good hint that there are big customers sponsoring this.

While Solr can be easy to setup, using any advanced feature of Lucene is "shielded" by what the HTTP server provides. As opposing to that, we believe in making it simple enough to get started with it but we always provide access to the lower level capabilities of Lucene "native" for the case you need some very advanced feature, and this will never be possible via HTTP which will obviously never be as fast as direct access by reference.
have a look into:
http://docs.jboss.org/hibernate/search/ ... ene-native

Even while facilitating most operations and managing resources properly, we allow many extensions points and direct "raw" acces so that we don't stand in the way.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Hibernate Search for 50 to 100 million records on MySQL
PostPosted: Wed Apr 25, 2012 9:19 pm 
Regular
Regular

Joined: Sun Apr 13, 2008 3:04 am
Posts: 71
Location: Bangalore
Thank You Sanne.

As our need is purely RDBMS data full text indexing and also we are using JPA 100%, architecturally we need to go with search infrastructure which is annotations based. This way ideally HS and sharding is the way to go.

>We do have big customers using these sizes or larger

Good to note this. In case there are any public info available related such deployments, pl. point us kind of performance they have achieved..

_________________
Raja Nagendra Kumar,
C.T.O
http://www.tejasoft.com
TejaSoft - Specialists in Code Audit, Unit Testing and Merciless Re-factoring - Engineering Crisis Turnaround Experts


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.