Hi,

We are currently evaluating Hibernate as our persistence layer for our next big project. This project will be built on a pre-existing enterprise database that has 600+ tables.

As part of the evaulation process we have generated mapping files for about 300+ of those tables.

This is where we run into a potential show-stopper for us:

The startup of the application is taking 40+ seconds. Most of this time is being spent building the session factory, as it appears that the factory is loading and validating all 300+ mapping files. I will have a very difficult time introducing hibernate into our environment (especially in development) when starting the application takes this long. The code/run/debug cycle will simply be unacceptable.

I have switched the mappings to use the addCacheableFile and this has not really improved the performance that much. Again, it appears that most of the time is being spent in the creating of the factory.

Can we cut down on the amount of validation the session factory build is performing on the mappings?

Is there a way to have the factory load the mappings on demand? And if this functionality does not currently exist, how hard would it be to implement this feature in the factory implementation?

Perhaps pre-load all the class/object names and as the objects/classes are encountered by the engine, load their entire definition at that point?

I would really like to leverage hibernate for our next project and projects going forward, but this one issue could put a halt on that.

Thanks for your time.

Tyler Van Gorder
tkv@landacorp.com
Landacorp.

ps. I would be willing to work with someone to help implement this feature, I just simply don't know the internals well enough.

Have you tried to run it in profiler ?

I have not yet run it through a profiler, as we are in the evaluation phase I was hoping someone might be able to point out something obvious that can improve the performance.

I will run it through a profiler and get back to you later today.

thanks

Tyler.

40 seconds sounds about right for 300 tables. Can you partition the tables into smaller independent sets? Then developers could start the application just for the set they are working on. But in general, 40 seconds doesn't sound bad for such a large application.

I would prefer not to have to partition the tables, as we have all the meta data for the tables in an excel spreadsheet. We have a java program that reads that spreadsheet and uses velocity to generate our persistent java classes, their associated mapping files and the DDL for the database. I have not provisions (currently) in the generator to partition the objects in groups.


The 40 second startup times will kill us in the development/debug cylce. Everytime we make a coding change and want to test it, we would have to wait for the session factory to be built.

40secs X20 Times a DayX8 developers == 106 Minutes a day, Lost productivity. That is pretty costly, when you figure the salaries of our developers.
======================================


Again, it just seems like on-demand loading of the mappings could be possible, I just wanted to ask the hibernate developers how difficult this would be to refactor the session factory to do this.


Tyler.

Very difficult. Besides, we have profiled this and a large majority of the time spent in SessionFactory "creation" is actually spent in the xml parsers which actually happens during Configuration processing.

I used to work on an application mapping nearly 400 tables and 40 seconds seems unreasonable for your 300 tables. It's been awhile since I worked on that, so I can't really remember the specifics, but I seem to recall being able to build the SessionFactory locally much faster than 40 secs with more mappings.

One thing I'd recommend checking is to make sure you have your DOCTYPE definitions correct within the mapping files. Based on the DOCTYPE, Hibernate registers an EntityResolver with the XML parsers to be able to resolve the DTD locally. If the DOCTYPE def is messed up, it'll make network calls to hibernate.org which would seriously slow it down.

xml parsing is not his issue since he is using cachableFile.

The time is most likely build on the necessary classloading/generation.

But yes, try and use a profiler to verify it.

Hi Max,

Ok, I have a call tree from jprofiler, exported as HTML.

How can I get this to you? zipped up it is only 40K.


Tyler.

put it on a jira issue ?

Ok, I have created a JIRA issue and attached the call tree (zipped HTML) document.

The Key is: HHH-1154

Thanks.

Tyler

Follow up on this:

The hibernate team recommended trying 3.1 and that has significantly helped in the performance department, my load times are down from 42 seconds to 25 seconds.
===========================================

From the JIRA issue:

Thanks everyone for you help, I downloaded 3.1 RC 2 and it has definately helped, here are my numbers loading 333 persistent classes now:

33.7171 Seconds (Cached resources off, JDK 1.4)
25.5620 Seconds (Cached resources Off, -Dhibernate.cglib.use_reflection_optimizer=false, JDK 1.4)
24.7500 Seconds (Cached resources Off, -Dhibernate.cglib.use_reflection_optimizer=false, JDK 1.5)
131.375 Seconds (Cached resources, 2nd run (deserialization of mapping files))


The 25 Seconds is quite an improvement from 42. I did notice in my testing that it appears the "optimizations" seem to slow the loading down rather than speeding it up. Especially, the de-serialization vs using an XML parser to load the mapping files.

There also seems to be quite a hit when the reflection optimizer is true. Is that because performance is not realized until the engine is actively issuing queries? More curious than anything.


Thanks again.

Tyler.

ps. Is there anything else I can do to further reduce the startup time? Yes I'm greedy. 8-)

i can't belive you get 131 seconds when rereading the serialized xml that does NOT make sense.

"reflection optimizer" generates byte code to avoid reflection (property get/set become direct calls) it takes time to generate code. It is better to dissable this stuff for development, you will have better error messages without it.
Sometimes it makes sence to dissable "reflection optimizer" in production, modern JVM implementations optimize reflective calls in the same way as CGLIB, you can save permanent memory.

This is very interesting. Do you have any specifics on this? For example, could you say which JVMs optimize reflective calls (Sun's JRE 1.4.2, 5.0, ... ??)? I mean, I can always try to test it out myself, but if you already know Sun's 1.4.2 does this optimization, for example, that would save me the effort. Thanks!

I also have significantly slow startup times and it is killing my development time. As far as I can tell my doctype is proper, but could you expand on what needs to be in place for your DTD to locally resolve?

Here is my DOCTYPE