I've been using hibernate for a while now and while I'm extremely impressed with the documentation in most regards, what seems to be missing (to me at least) is detailed documentaiton on dealing with performance and large datasets.

I've done a lot of searching on the hibernate site, the forums and google but am yet to turn up a good set of reference information on this subject. I guess you could say what I'm looking for is the Hibernate equivelant of the Oracle Press performance tuning books, or the "Ask Tom" columns.

I've read a number of blog entries from the hibernate team with the general theme of "java (and therefore hibernate) is not the place to do batch processing". I think there is an underlying assumption in these entries that anything dealing with large data volumes is by definition batch processing - and I disagree with that.

Where I work we've constructed a fairly complex bio-informatics/genomics LIMS and analysis platform, and we use hibernate. Where things really are set-based or amenable to batching we push it off into stored procs in the database and this works quite well. But we also have a large number of processes where the business logic is complex and used with both small and large datasets. It's simply not the kind of logic that can be set based (I'd be happy to provide domain specific examples if you're interested).

So, we end up with these transactions where we're either looking at or touching thousands or tens of thousands of objects loaded through hibernate, and we've run into quite a few performance problems. I don't expect hibernate to handle this volume of data without problems without some coaxing from us for the particular use cases. But the point of this post is that it's really hard to find out what coaxing we should be doing!

On to my question and my offer. Are you guys aware of a good source of documentation on dealing with large datasets and optimizing hibernate performance in these situations? I've looked at the available books and not found anything significant - perhaps Hibernate in Action 2nd Ed (or whatever the title is now) will change this?

If this doesn't exist in a coherent form now, and folks on the hibernate team would entertain a discussion either here or in email on these topics, I'd be happy to learn what I can, assemble the documentation and contribute it back....

Please don't construe this as an attack in any form. Hibernate is complex and free - so it's not at all shocking to me that I'd want this information and that it's not all wrapped up with a bow on it waiting for me.

Thanks,