Hibernate version: 3.3.1 GA

In our application we have a task, which loads large amount of data from database, then calculates something. After that small resulting records are assembled and persisted to database. Their amount can be huge. Thus we use a second-level cache and short sessions with short transactions, which generate about 50K of such records.

Such task can be performed for one hour for example and can generate several millions of described small records. And the garbage collector just have no time to collect them. So application after some time runs out of memory. And we decided that in such case we just stop this task and wait for consumed memory to be freed.

So maybe there is a way to force garbage collector to free data more frequently? Maybe someone got similar case as ours?

There isn't a lot you can do without limiting the amount of or size of objects in memory. Java has a mechanism to call the garbage collector, but it's more of a suggestion then a command. It is not guaranteed to run when you call it.

Runtime r = Runtime.getRuntime();
r.gc();

There are some other things you can try which might help.
set every object to null right after you are done with it. This will de-reference it and make it ripe for garbage collection

Object o = new Object()
//do some stuff

o = null;


Instead of returning your full annotations maybe just return a collection of lose objects or a map. In your HQL you might do something like this.

--Return as a list of objects
SELECT p.pkId, p.name, a.city
FROM People p JOIN Address a

--Return as a list of maps
SELECT new map( p.pkId as People_pkId, p.name as people.name, a.city as Address_City)
FROM People p JOIN Address a

hope this get's you closer.

We have been down a similar path once and in my experience there is nothing to gain from manually trying to invoke garbage collection. In the best case everything will be as before. In the worst case you will find that the JVM spends most if it's time doing garbage collection and almost no time doing real work. Yet, it will not reclaim any memory and occasionally you will get out-of-memory exceptions even when you have several hundred megabytes to spare.

The problem is much more likely to be some kind of "memory leak". Eg. something is somewhere keeping refereces to objects that prevents them from being garbage collected.

If you want some more information Sun has written a nice document about garbage collection: http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html

Thanks for your answers. But they weren't helpful. We have tried to explicitly call Runtime.gc() and read entire http://java.sun.com/javase/technologies ... ing_6.html .

We've inspected garbage collector's behavior using visual gc tool and heap using jmap, visualvm tools. Conclusion is that most of the loaded and created objects are being moved from "eden" space to "old generation" space, which is cleared with less frequency.

But we consider that there are no memory leaks because for example before running such task application consumed ~40MB of memory, then after running for 10 minutes it grows to ~200MB of memory and then after stopping a task consumed memory falls back to ~40MB just in 5-10 seconds. Task is running in a separate thread.

Your problem is from the amount and size of the objects your keeping in memory. You need to find a better way to do the processing that your doing. If you are really loading 50,000 or more pojo's into memory all at once that should throw up a red flag to you.

Even if you were not working in hibernate you would not want to load records in this manner. You really need to limit what you are putting up in memory and then make sure that all your objects are de-referenced.

Nordberg is right that garbage collection is impossible to control in java. It's completely unreliable. So it's us to us the programs to create better algorithms for working with large swaths of data.

I would not say that garbage collection is unreliable... In my experience experience it works fine as long as you leave it alone and don't try to control it.

Since it only takes a few seconds to clean up once the task is stopped I seem like garbage collection is working properly also in this case. I suspect that it is the task itself that somewhere keeps references to objects (probably indirectly) that prevents them from being garbage collected. But I have no idea how to debug this.

Nordberg, you are right again. I didn't mean the garbage collection was unreliable, just that mucking around with it is unreliable. I probably should have said that clearer, it did come off wrong the way I said that.