Is it possible (in the most recent release of hibernate) to add() to a lazy list without loading the whole thing into memory?

I am trying to code for a situation in which the total length of the list could vastly exceed available memory. I know I can keep it out of memory for reading by using session.filter() but I need an equivalent usage for appending.

I don't know about future Hibernate plans, but unfortunately this kind of behavior is not available yet...

May be you could define the list elements as lazy. This way they won't be instanciated/loaded when the collection is loaded - only proxies would be. This could make a big difference in memory consumption if your list elements are heavy. But would still require the instanciation of as many proxies as you have elemnts in your list :(

Que? You mean if I have a Organization with a List of Employee, I cannot add a new Employee without loading the entire Employee table?

So, all the one-to-many relationships are esenstially useless unless we are talking about a very small -many instanstances?

I guess the work-around is to add the Employee separately?

That's true. Hibernate will need to load the entire collection before adding the new element. Why so? Because it needs to enforce the Java collection semantic. Think about it...

- The collection is a set: duplicates must be avoided - so when adding a new element, Hibernate needs to know all the others (at least their ID).

- The collection is a map: keys need to be known for the map to be usable...

- The collection is a list: the indices must be known...

- The only collection that *may* not require to know anything in advance is the bag (and I'm not even sure about it) - but this is not a regular java collection type...



Useless? Why so? It will perform correctly in all cases - but with bad performance with very large collections - unless you know what you are doing and use the tool accordingly.




The easiest workaround in this case is probably to define the list element as lazy. This way Hibernate won't load their actual content but just a proxy with their IDs... This is quite efficient, believe me ;)

Think about it... Loading the element IDs will be required anyway - whatever is your strategy to update the list, you will at least need this information...

A workaround I suppose is to emulate a list with a <class>

It is concievable to append to a list without loading it. Lock the list, do a "select MAX(index) from list", add the row, unlock the list.

Hmmm.. don't see the benefit (please light my bulb if I'm wrong)...

How will you map this class to the owner? Using a collection? Then you still have the same issue... (owner -* fakelist - item)

This is true - for the list only, and for one element only, and only if you add at the end of the list (append).

The approach taken by Hibernate so far works whatever you are doing with the list. Hibernate handles all collections the same way (in this regards) - that's why it is not always optimized for some scenarios - like the one you face.

Note that you can do the update as you propose yourself - but make sure the collection is not loaded in the current session or you may have some surprise at flush() time ;)

You do it the old SQL style way.

So you implement your own list management... But there is no explicit dependency between the owner and the list items anymore. So you can't rely on Hibernate cascades to remove orphans when the the owner is deleted, etc.

That's your choice. Sure it will work. But still believe the lazy list elements is a good approach - unless you have millions of items in your list and you update it very often...

BTW, be carefull about race conditions...

You could, I suppose, keep track of an "appended list": if nothing else but add() has touched the list, then you can keep an in-memory in-sequence list of items appended, and batch-append them at flush time (or batch-append at list-initialization time if something else touches the list).

Appending to huge lists is not a marginal case.

Yeah, I saw that race. Just not sure how to lock a whole table in Hibernate.

Haven't read the full thread, but often you simply don't have to load the list, e.g. if its a unidirectional association. Then, adding a child (this is the "many" side) to a parent is just a matter of:

Child c = new Child();
Parent p = // load Parent or create a "fake" Parent with the same id
c.setParent(p);
session.save(c);

The Parent doesn't have a list of children in that case, so you don't have to load it.

So, I'll probably go with the lazy list elements for my case. How do I do this?