Hi,

I have a program where I load a folder with around 200 files of each 3MB. The files contain units which I load as a parent-child set and attach them to the File object. There might be tens of thousands of Unts per File. In large sets this amounts to around 1GB which I load into memory.

In File.hbm:
<set name="Unit" lazy="false" inverse="true" cascade="all">
<key column="FILE_ID"/>
<one-to-many class="Unit"/>
</set>

Now, when I start the loading procedure, the first 100 or so Files get loaded within seconds, but after that it gest painfully slow. The last files take minutes to load, although they are the same size as the first ones. I tried mapping the units as a bag, but then I get ConcurrentModificationExceptions when loading.

Is there any way to speed up the usage of Collections for large sets, what is the best collection type for large sets? Or can I tell hibernate to use a sort of on-disk cache so that not all data is saved in memory?

Regards,
Leen Toelen

You may evict some instance from session cache
if you do not need them, like


Of course you are losing the benefits of the cache then.
For e.g. batch processing of large data sets the StatelessSession
might be a good solution.