Brief summary....
Small camp reservations system has Child, CampWeek (one for each camp week during the summer) and ChildInCampWeek - which represents the fact that a child is in a particular CampWeek (e.g. Camp Foo, Week 6). Since ChildInCampWeek has some additional information (confirmed or not) this is its own entity with 1:n mappings to Child and ChildInCampWeek.

The puzzle is that after I add two children to the same CampWeek, the database looks fine, but the Hibernate collection does not seem to reflect the state of the database. Of course, it is not Hibernate that is confused, it is me! But here is what I see....

Code that generates a list of all the children in the CampWeek:

Results of that code:


But looking at the database tables directly, we see for CampWeek #19 the following information in the ChildInCampWeek table:

In the database, there are clearly two children (10, 15) that have this CampWeek, but they don't show up in the campWeek.getChildInCampWeekSet().

Any ideas as to how to resolve this situation would be greatly appreciated! (and will of course earn credits once resolved).

Hibernate version:
2.1.8

Mapping documents:
Here are the seemingly relevant sections...
From CampWeek.hbm


From ChildInCampWeek.hbm


Thanks again!

PS - I had a recent post on a delete cascade problem that I was having, but I believe the above problem is actually at the core of that one, so I am trying to attack this first. I have tried many many different varieties of code and hbm settings, but my fundamental confusion is if the db has the right information, why does it not show up in the collection.

Most obvious thing to blame would be the hashCodes of the various classes. If they're using member fields instead of properties, then the default values may be being used, instead of the hydrated values (from the DB). hashCodes likes this are bad:This will return hashCode 17 for most objects loaded from the database, meaning that all objects will go into the same hashbucket. And as this is a set, there's only one item per hashbucket.

This can be fixed by using accessors instead of field access.

You mentioned in the other thread that you have some hibernate tool building this code for you, so presumably it's doing the right thing. Turning on the persister-level debugging that I mentioned before should help with determining if it is. Another thing you can do is add a printout to your hashCode methods (to be removed asap, of course :) Have it print out the business key and hashCode: if you find two objects (different business keys) with the same hashCode, you've found your problem.

I turned on the SQL tracing and just finished wading through the 48 queries that are generated. They break down into 7 unique queries and all of them seem to give the right answers when I put them into SQL. The one that appears most relevant to this problem is:


And when I add the 19 for the value of the CampWeek, I get the TWO children that are in the database for that particular CampWeek. Which is totally baffling.

Hence my suggestion that it's hashCode that's at fault. If your hashCode method returns the same number for both loaded objects, they'll go into the same element in the set, even if they have different database keys. It's the business key that determines where in the collection an object goes, not the database key.

Sorry for not noticing your reply!

Here is the hashcode for the ChildInCampWeek - again, I believe this is generated by HibernateSynchronizer. It seems to be using the accessors.



Do you see anything amiss there?

Oh yes. That hashCode is wrong on so many levels that I'd curl your toes (backwards) if I had to fully explain its horrors. To start with, this.getClass() will return a different value when the object is proxied, so when the object is deproxied its hashCode value changes... even though its location in any collection its in doesn't. Eek.

Find all hashCode and equals methods generated by HibernateSynchronizer and nuke them. With at least 20MegaTonnes. Preferably from another continent.

Replace them with simpler ones, based solely off business keys.

Thank you for the informative (and amusing) answer. I will modify the hashCodes and equals in the relevant classes and give it a try. I certainly hopes this solves my very strange problem.

I will report back tomorrow on how it went.

Thanks again!
RB

Tenwit:

Although my first quick attempt to change the equals/hc didn't work, I will keep at it, probably my error. I just wanted to check my understanding of the problem and how it might relate to all three issues I have been having.

1. Problem adding a new child to a CampWeek that has other children (this was noted in my other post).
Potential explanation: My understanding is that since my eq/hc code is wrong, when I try to add the second child to the CampWeek, the add to the set fails because Hibernate views the second ChildInCampWeek object as being equal to the first, and won't add it to the set. Since the second child is actually different from the first, the database adds it fine.

2. Problem with the cascading delete (the original topic of the other post)
Potential explanation: When I delete a camp, all its campWeeks are deleted in the cascade, but Hibernate treats both children as the same, so it only deletes one set of ChildInCampWeeks, creating the problems noted earlier.

3. Problem with retrieving the complete collection (the topic of this thread). Potential explanation: Although Hibernate retrieves two ChildInCampWeek records from the database (as seen by the SQL), when it constructs a set of unique records, it only finds one - since it uses the broken eq/hc code.

Is my understanding generally correct here? I will now get to work on relaunching those nukes to see if I can get the eq/hc code working. I will have a bit of a problem with business keys for the ChildInCampWeek objects, as the only fields are:
* ChildInCampWeekId (can't use that due to the standard problems with using the db id in Hibernate)
* ChildId (foreign key)
* CampWeekId (foreign key)
* Confirmed - boolean property
I was going to start with the combination of ChildId and CampWeekId as those will definitely be unique. They are currently mutable in my application - namely, if a Child1's second week of camp is changed from CampWeek10 to CampWeek17, I was just updating that ChildInCampWeek record from (id, 1, 10, true) to (id, 1, 17, true) - but I can modify my code to drop the (id, 1, 10, true) record and insert a new (id2, 1, 17, true) record - if immutability matters.

Thanks again for all your help - I hope I can pay it forward to someone in the future.

RB

More work on testing the new eq/hc methods, but still no joy.

I replaced the hashcode and equals on ChildInCampWeek as shown below and traced out all calls to the hashcode function - each unique ChildInCampWeek gets a unique has code. Yet, even though two database records, ChildInCampWeek 41 and 52 which both map to CampWeek19 in the db, only one shows up in the collection (#41). I have also instrumented the equals to see if I could find any time that 41 and 52 were being compared and determined to be the same, but I saw no evidence of that.

Here is the new code that I am using, perhaps there is still some problem, as I haven't written this type of code before. I expanded it for ease of debugging.


As always, your helpfulness is greatly appreciated.
RB

Switching back to the flavor of my problem where I can't add a seemingly different ChildInCampWeek item to the set of ChildInCampWeek items, I have the following code - generated by HibernateSynchronizer, but liberally annotated by me:


My outer level code is simply:


And the results are:


I am baffled....
I am trying to add the second child to the same CampWeek that the first child is in. There is one CiCW (ChildInCampWeek) before I do the add. The second CiCW appears different, it has a different id, a different hashCode and the equality test shows that is not equal. Yet the "add" rejects it. So, what equality check is the add using if not the one for the object being added.
I can't even think of a useful next step....

Thanks in advance for any help on this mystery!
RB

Hi Richard,

Any luck since the last post?

Here's a few things to work with:

- Remove the comparator from your set mapping. You can put it back in later, but get rid of it for now. It's only going to make things muddier, and it may be the source of some of your problems.

- Rewrite hashCode from scratch. There are a few things that your modified hashCode still does that it must not do:
This is intending to cache the hashCode, meaning that any any "fixes" you do to an object between the time it has hashCode first calculated and the time it is fully configured and ready for saving are ignored. For example, if you have a Person class whose business key is their name and social security number, and the constructor you use needs only a name parameter, you could do something create a Person, print out its hashCode (instantly calculating and caching its hashCode), set its security number (which should change its hashCode, but doesn't), then add it to a set. It's going to go into the wrong hashbucket. This code also insulates you from a very common bug: adding an item to a set before its business key is fully known. Because the hashCode won't change while you continue to set up its hashCode, bugs that should be fairly easy to detect will go unnoticed.. until some maintenance developer spots the poor hashCode code and fixes it... and then spends another few days finding all the bugs that were previously hidden.This will return different hashCodes depending on whether or not the object has been saved. This is very bad, and is exactly why hibernate-persisted objects should use only their business keys to determine their hashCode.Nothing wrong with the algorithm, but this is a very inefficient way of calculating a hashCode from two ints.. create four temporary strings (week.id.toString(), week.id.toString() + ":", child.id.toString(), and finally hashStr) then get a hashCode from one of those strings. Why not just go with the tried and true "37 + 17xVal1 + 17xVal2 + 17xVal3..."? Or use apache-commons' HashCodeBuilder class.
This is the actual caching of the hashCode. See the earlier comment.

This hashCode has the right intentions, but suffers from a more subtle issue (that the above hashCode also sufferts from):The problem with this one is that hibernate does not guarantee that child objects are loaded in any particular order. Even though the code works perfectly when the objects are user-created, when they're hibernate-created this code will cause IllegalArgumentExceptions (if the objects are immediately put into hibernate-mapped collections). Business keys must be properties of an object, not properties of an object's nested objects.

You need to do one of two things to get this hashCode to work (and hopefully, to solve all of your problems at the same time). Either add a new property to ChildInCampWeek to use as its business key, or store the Child id and the CampWeek id directly in ChildInCampWeek, and use those ids as its business key. I'd favour the second solution: even though it's more complex to map, I think that it's more accurate. In your hibernate mapping, add the two ids to the ChildInCampWeek mapping, and use formula="ChildId"/formula="CampWeekId" (and not column="..."), as this will allow you to retain the existing many-to-one mappings. There's some java overhead, because you need to ensure that the methods that hibernate uses to populate the various attributes aren't the same as the methods available to the API user: you need an internal and an external version of setChild() and setCampWeek(), and an internal-only version of setChildId() and setCampWeekId(). Then you can use getChildId() and getCampWeekId() to work out the hashCode.

Note that the equals method doesn't have to worry about any of this, as it's not called when adding CampInCampWeek objects to hibernate-mapped sets, so it will have access to getChild() and getCampWeek(). hashCode() cannot guarantee that those methods will return sensible values, and thus cannot use them.

Whew. Now to really throw the spanner in the works: a fairly simple redesign of your model should make this problem go away. You could get rid of the ChildInCampWeek object, and instead put a set of Childs directly into CampWeek (it would be a set containing a many-to-many, rather than a one-to-many: your DB schema probably won't change). Then the most dificult hashCode would just disappear.

tenwit:

Thank you so so much for the detailed suggestions. I had some network problems and couldn''t do much testing over the weekend. I will be able to try some of the suggestions tomorrow, but a few quick thoughts....

* Thanks, again - your help is greatly appreciated and way beyond what I or anyone should have any right to expect.
* It is interesting that you mention the redesign idea of having a set of Childs directly in the CampWeek, as a many-to-many. That certainly would work. I know this because I had that exact design (and have kept the code). I then wanted to add the idea that a particular CampWeek was "confirmed" for a particular Child (vs. just being a "tentative" reservation). Thus, I embarked on the new design and ended up where you now see me (lauching nuclear weapons only to have them land on my own toes :) :) ).

I will start with the removal of the Comparator as that may be indeed making a bit of a mess and I will try a few other debugging thoughts and other possible simplifications. It's strange to me that my equals/hashcode return false when comparing the ChildInCampWeek that I want to add to the one that is in the Set, yet the set.add() returns true. Thus, the set.add is using some other equals/hashcode to do the compare. So, I would like to try and find which one it is using.

I will post back after I have done additional investigation - and hopefully have some answers.

Thanks thanks thanks!!!
RB

If you go with the many-to-many option, you can still implement the tentative/confirmed value in the link table. Have a look at section 8.2, "Collections of dependent objects", for a couple of examples.

There are some problems with composite-element mappings, to do with referencing elements inside composite-element mapppings from HQL, or searching for them using Criteria. Loading/saving them is fine. If you think that these might affect you, you can add a new entity, say Booking, so that your eventual model looks like this:

Camp -(set of)-> CampWeek -(set of)->Booking -(many-to-one)-> Child.

If you further wanted to be able to use a single Booking to book in several Childs to several CampWeeks, then you have to use ternary associations. That's moderately advanced SQL: if you know how to do them, then the hibernate solution is easy as pie.

You were right to suspect the Comparator. When I debugged deeply, it turns out that the code that the TreeSet uses to test equality is the Comparator. And sure enough, my Comparator was incorrect (it was the same one I was using for CampWeeks, which didn't include any information about the child). I fixed the Comparator and my tests started working - the second child is getting added to the collection of CampWeeks and the retrieval is also working fine. I will run some additional test scenarios, but I have gone from despondent to optimistic.

Key lessons learned:
1. When using SortedSets, the compare method of the Comparator class is used to test for equality (not the comparesTo() or the equals() or hashcode() methods).

2. Mistakes often lurk in code that was repurposed without much thinking.

3. Debuggers are even more helpful than one would expect, even when one has learned lesson #3 many times in the past.

4. tenwit is a very helpful person :)

Thanks again!
RB