We currently do not take advantage of the 2nd level cache much because we run with multiple web servers in our cluster leaving pretty much only JBoss Treecache, SwarmCache, and Coherence. Primarily, we are concerned with how Treecache, Swarmcache and Coherence will scale once we have hundreds of appservers in our cluster (MySpace for example has over 2800 application servers) because each node in the cluster needs to communicate with the other to do cache invalidation/updates it would seem prohibitive to use these second level cache providers in a large environment.

An alternative approach that we are considering is not using the second level cache at all but instead using a memcached and manually caching the objects at our DAO layer. We could, for example, deploy 100 memcached servers that would communicate amongst themselves much more efficiently than if the cache was tied to each of the appservers.

Has anybody considered developing a second level cache in hibernate that is actually an external set of cache servers? Does anybody think this is a bad idea?

We are also in the process of partitioning our database across functionalities. For example, one master database for Forums, one master for Blogs etc. This approach isn't terrible to implement with Hibernate as we denormalize some of the relationships to remove dependencies in the model and then use multiple session factories (one per each master db).

What concerns me is how/whether this approach will suffice when we reach large table sizes (users in the millions). Again, returning to MySpace or even LiveJournal, the approach that these companies took was to break their database down even further so that users are spread across multiple databases and they make a determination at the application layer which database they need to read from to get that users details.

The clearest approach to implementing something like this in Hibernate would be to force our DAO to choose a different session factory based on the username/id. Anybody see any problems with this?