I've been experimenting with Hibernate Shards and have a question about the design. In my schema I want to shard my customer table and related tables e.g. customer orders. However, I have a number of fairly static tables that I would like to replicate across all shards - tables such as country, product, etc.

My question is how can I make Hibernate Shards only query a single shard when I need a list of countries. I was hoping to be able to do this in my ShardAccessStrategy implementation but the API doesn't let me see which entity the operation is being performed on. My debugger shows that the information does exist in the CriteriaFactory persistentEntity field but I am unable to access this via the API. Perhaps this would be a useful addition to the product.

Is there another approach I can take when accessing non-sharded tables?

Hi Andy, thanks for writing.

We do not yet have a good solution in place for dealing with static data, but this is a common request and we definitely need to take care of soon. The easiest thing to do is replicate that data across all your shards and then just be very careful about updating it. However, if it's important that the static data only reside on a single shard then your best bet is probably to make use of ShardedSessionFactory.getSessionFactory(). This method returns you a ShardedSessionFactory that operates on a subset of your shards (identified by ShardId). As long as you know the id of the shard where your static data lives you could do something like this:



I know that managing a separate Session sort of defeats the entire purpose of Shards, but until we get some replication support added I think this is a decent option.

Please let me know how it goes.

Max

Hi Max,



Hibernate Shards allows us to insert all global data into a single shard (we can implement this in our ShardSelectionStrategy). We can then use database replication to write this data to the other shards.

What we would really like is more data made available to our ShardAccessStrategy so we can decide whether to run the operation against a single shard or not. We have prototyped this by adding a getPersistentClass() method to the ShardOperation interface and have this working for a criteria list() operation. Do you think this approach could be adopted in Hibernate Shards? I'm not yet sure how far we can go with this approach.

My main motivation is removing the need to have two SessionFactory instances as it complicates the application design and I can foresee problems if we need to perform transactions that operate on both global and sharded tables.

I think I understand, but I'm a little concerned about using the ShardAccessStrategy in the way you're describing because the intent was never really to do any sort of filtering in there. Of course if Shards doesn't do what you need then my intent isn't worth much, right? :-) I feel like we're bumping up against a concept that so far has not been modeled in the framework. The selection and resolution strategies are used for saving and fetching individual entities. The access strategy was designed to give you flexibility in applying an operation to all your shards. But what about flexibility in deciding against which shards to execute a query? Maybe that's what is missing. Are there other use cases where developers would want to restrict the shards they query against? If so, would they always want to choose based on the type of object for which they're querying or does it need to be even more flexible?

Another thought: I wonder if it might make sense to invoke the shard resolution strategy for both entity lookups AND general queries. At least then we have a single place to filter the shards we want to operate on. Let me think about this a bit more, and in the meantime if you have more thoughts about any of this I'd love to hear them.

Thanks,
Max