I'm not understanding your object model, because it appears (and I could be wrong) that you're also trying to capture database-like table associations as well.
Ignoring Hibernate for the moment, what do the raw classes look like, and what is the map in UnitDescrption for?
A concrete example like "this is how inches to feet are represented with my classes" would help.
This derived code from your text is confusing to me:
Code:
public class UnitConversion {
UnitDescription from;
UnitDescription to;
}
public class UnitDescription {
Map<Integer, UnitConversion> m = new Map<Integer, UnitConversion>(); // What's this doing?
}