-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: High memory usage on initialization
PostPosted: Wed Dec 10, 2008 1:33 pm 
Newbie

Joined: Fri Jan 23, 2004 3:37 pm
Posts: 14
Hibernate version: Hibernate Core 3.3.1 / Hibernate Search 3.1.0

Hi.
I'm facing a huge memory usage problem, so, I'm asking help on how to reduce the runtime memory taken by Hibernate.
We have a web application which, on the previous version, with 63 tables, was taking initially 62M on tomcat.
The new version has 142 tables (139 entities), and initially, it takes 145M with the same tomcat.
Those numbers are suggesting a direct relationship between the persistent classes and the memory used.
I've spent days profiling and monitoring, and I've taken heap snapshots using Java VisualVM, and it tells me, for example, that there are 7645 (!?!) live instances of org.hibernate.loader.entity.EntityLoader, which takes 1.3M.
Not much compared to the total 145M, but the most space is used by char[] (in Strings), and I've randomly inspected them, and most are entity meta-data, like column aliases, property names, select clauses and so on...

We had production servers with 25 instances of the application with 2G of memory, but, if we don't reduce it's usage, we'll have infrastructure problems...

Does anyone has a clue on how to reduce that memory usage?


Top
 Profile  
 
 Post subject:
PostPosted: Wed Dec 10, 2008 2:52 pm 
Beginner
Beginner

Joined: Thu Oct 02, 2008 10:15 am
Posts: 21
If this helps you, we have around 10 tables and our memory footprint when there is no load is 50Meg.
And I think it is usual because hibernate creates all kinds of insert, delete,update queries and stores them.[in addition to all the application classes]
Keep you posted if I find something else.

It is like shooting in the dark . But what is the value for
hibernate.default_batch_fetch_size = ?

Try to set it to minimum just to watch the heap size.
Also, try this
hibernate.cglib.use_reflection_optimizer=false
.Should reduce it further.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Dec 10, 2008 5:43 pm 
Expert
Expert

Joined: Wed Mar 03, 2004 6:35 am
Posts: 1240
Location: Lund, Sweden
It sounds like a lot of memory...

We are getting close to 150 tables. I think the average table has about 10 columns. Largest one has maybe 50, and there are lots of many-to-many join tables with just 2 columns.

Now, the interesting part is that after a fresh restart of Tomcat we use just a about 22M of memory. With two identical web applications we use 35M.

On normal use, memory usage seems to stabilize around 200-300M.

Didn't know about the VisualVM tool. Thanks for the tip, will check it out tomorrow at work.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Dec 11, 2008 7:22 am 
Newbie

Joined: Fri Jan 23, 2004 3:37 pm
Posts: 14
The hibernate.default_batch_fetch_size = 20 and hibernate.jdbc.batch_size = 20.
These numbers I've posted are with second level cache disabled.
In production, we normally enable the second level cache (with oscache), which would lead to even more memory being used...
I'll try with hibernate.cglib.use_reflection_optimizer=false.
I've also upgraded to hibernate 3.3.1 before those numbers, and it uses javassist instead of cglib. I'll also try to use cglib to check it changes something, and then, I post the results here.
But here are some other numbers, running on tomcat, with all libraries in the shared lib folder (numbers are higher if each instance holds it's own copies of each lib). Those numbers are after starting tomcat and forcing a System.gc(), not after a high load:
* 1 app instance: 145M Heap, 62M PermGen, ~8300 loaded classes
* 3 app instances: 420M Heap, 87M PermGen, ~12000 loaded classes
* 5 app instances: 525M Heap, 111M PermGen, ~15500 loaded classes


Top
 Profile  
 
 Post subject:
PostPosted: Thu Dec 11, 2008 11:57 am 
Newbie

Joined: Fri Jan 23, 2004 3:37 pm
Posts: 14
Well, after more digging, I've made a test to raise only the "model" layer (services, daos and persistence, with spring), and the used memory was about 80M.
That made me more focused on the web layer than in Hibernate, since the total initial load was about 150M.
Although every memory gain is nice, and for that I've been trying to work with every component - including Hibernate), I'm shifting my focus now to the web layer.
By the way, the hibernate.cglib.use_reflection_optimizer was replaced by hibernate.bytecode.use_reflection_optimizer, and it defaults to false anyway.
Thanks for the help attempt.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.