Hibernate version: 3.0
Name and version of the database you are using: Oracle 10g

I am having trouble speeding up a simple many-to-one association, mapped with the following .hbm.xml configuration files:

Each one of these (Associates)...


Has many of these (AssociateSessions)...


The problem I am having, and I know this is a newbie question, is how to deal with a huge number of records (millions of records, to be more specific). Turns out, not surprisingly, that when Hibernate tries to load up an Associate (creating the list of AssociateSessions along the way), it takes such a long time to load up all the records that the server/database cannot handle it (memory-wise and speed-wise).

I just dont know enough about this to really suggest a course of action. Can I use HQL to load a subset of the AssociateSessions to get around this? How do people solve a problem like this?

Any suggestions are welcome. Thanks in advance.

What about setting lazy to false instead of true? Then you'd only be loading what is needed.

Well, I would like to do that, but in the Hibernate 3.0 dtd "true" is not an option. The options that are documented (and that Hibernate accepts) are "false" (default", "proxy", and "no-proxy".

Even then, this would help some areas of my project and not others. When I go to view the list of Sessions that belong to a given Associate, won't Hibernate attempt to load all the Sessions anyway to determine which Sessions belong to this Associate?

Edit: I misunderstood what you were saying initially, thought you were talking about setting lazy="true" from the many-to-one side, not one-to-many. However, the followup question remains...is my understand of how hibernate acts in this circumstance correct?

So this has helped, and I feel silly for not picking up on it before I asked the question. Instead of loading up the items in the collection when I hit any of the mapped entities, it only does it on demand, which is a big help.

I'm not sure if this is how it works, but is it joining the given Associate and looping through all its Sessions in the table? Or is it at this point joining all Associates with all their Sessions (populating their collections)?

Being able to answer silly questions makes us feel smart. :) But really, it wasn't a silly question - it was a very common question.

The associated objects are indeed being loaded on demand, which is very efficient. But be careful, if you go after one of those associated objects after the session has been closed, you'll get a LazyInitializationException. It's the bane of new Hibernate developers, but it's easily fixed if you know what's going on.

I'm not sure if I answered your second question there. Here's a little tutorial I put together on How Hibernate Works. You might find it interesting. It talks about LazyInitializations, the difference between load and get, and a little bit about the Hibernate cache as well.

http://jpa.thebookonhibernate.com/Javacode/learn.jsp?tutorial=07howhibernateworks

Read the tutorial, and it is helpful. I knew much of it already, but there are a few items that I found enlightening.

I guess my question remains...when I go to load a massive list of records into a collection, is there any way to force the Hibernate-generated SQL to filter the results immediately (from the query level) so I dont load up the huge collection into memory and then filter from there.

With the earlier advice, this isnt as crucial as before, but it still seems like I could get the code to run faster if this step is possible.

Thanks for the help already!

You could always do a hql and filter the children you don't like. If I remember correctly you could also have a where clause for sets which means the relation always picks a filtered list of children.

I am myself buffled a little bit on lazy semantics from the many-to-one side. I can't really tell what the difference between proxy and no-proxy is. I have done a small test and proxied many-to-one relations are eagerly fetched while that is not what I expected. Any insights?



Farzad-

I'm seeing this same behavior - have you gotten any insights into why?

Explain your situation.



Farzad-

I believe I'm in a similar situation to the one you described. I have a set of objects. When I initialize the lazy-loaded set, any many-to-one relationships in the set objects are also initialized, instead of being left as proxies.

So, A<1--*>B<*--1>C

If I load the set of B's on A, each of the C's on the B's are loaded as well, each in a separate SQL call. I would expect the C references to be proxies instead.

What is the Bs to Cs mapping?



Farzad-

and also how do you fetch Bs?



Farzad-

Here's the mapping. I've also tried moving the relationships to a standard many-to-one, and mapping the keys separately. No change.

The B's are fetched from a set on A (so a.getBs()).

With this configuration I doubt if lazy loading makes any sense. Key have to be fetched anyways. Best you can do is to use some L2 caching... If you like send me an example of regular <many-to-one...> mapping that doesnt work. I would like to try it myself and see how I can get it to work.



Farzad-

While the keys would need to be fetched, that shouldn't required separate fetches for each object. You could create a proxy for the actual classes.

Here's what else I tried: