We have a multi-tenant application where all of our searches for a given entity are guaranteed to require a particular term (which tenant we're looking at) to be matched in a literal fashion.

It occurred to me that our use of lucene would probably be much more scalable and performant if we broke out the search indexes by this required search term.. Especially the performance/scalability issue we're bumping against now is the use of lucene norms which is linear in size to the number of documents in a particular index.

I notice that there is a sharding strategy based on the primary key of the document, but in our case we'd need to have access to not only the primary key but also this foreign key by which the indexes could be separated.

As a simple example, imagine we've built a multi-user file system where users can store files online; there is no sharing of files among users. A user would search for a file and we would like to have a separate search index per user even though we are using the same hibernate session factory for all users.

Any idea whether this kind of scheme is possible with hibernate search?

hi dobes,
yes that's possible using sharding, but there's a limitation: the number of shards should be statically defined, currently you can't add a new index "on the fly".
The idea of sharding is that every user will be very likely to need a custom strategy; so the approach is to implement the base interface and plug your own strategy.

Have a look into http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-251 if you happen to be able to test latest beta.

Thanks you're right. I'm not using the new betas yet, our code is in production and I don't want to take any risks quite yet.

Apparently to make it work the way I'd like it, I'd need to be able to add shards at runtime, so I filed that as a new request here:

http://opensource.atlassian.com/project ... SEARCH-472