Need help with Hibernate? Read this first:
http://www.hibernate.org/ForumMailingli ... AskForHelp

Hibernate version:
3.1.3


Mapping documents:

Relevent portion:

<list name="allowablePacking" inverse="true" cascade="all,delete-orphan" collection-type="com.foo.domain.commodity.PackingListImplUserCollectionType">
<key column="commodity_id"/>
<index column="position"/>
<one-to-many class="com.foo.domain.commodity.PackingImpl"/>
</list>


Code between sessionFactory.openSession() and session.close():

public Commodity loadCommodity(long id) throws DataAccessException
{
logger.info("loadCommodity - id = " + id);
return (Commodity) getHibernateTemplate().load(CommodityImpl.class,
new Long(id));
}


Full stack trace of any exception that occurs:

java.lang.NullPointerException: Cannot add a null Packing to a PackingList.
at com.foo.domain.commodity.PackingListImpl.add(PackingListImpl.java:166)
at org.hibernate.collection.PersistentList.readFrom(PersistentList.java:362)
at com.foo.domain.commodity.PersistentPackingListImpl.readFrom(PersistentPackingListImpl.java:51)
at org.hibernate.loader.Loader.readCollectionElement(Loader.java:994)
at org.hibernate.loader.Loader.readCollectionElements(Loader.java:635)
at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:580)
at org.hibernate.loader.Loader.doQuery(Loader.java:689)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1919)
at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:36)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:520)
at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:60)
at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1676)
at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454)
at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:755)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:229)
at org.hibernate.loader.Loader.loadEntity(Loader.java:1785)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:47)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:41)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:2730)
at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:365)
at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:346)
at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:123)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:161)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:87)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:862)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:781)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:774)
at org.springframework.orm.hibernate3.HibernateTemplate$3.doInHibernate(HibernateTemplate.java:489)
at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:365)
at org.springframework.orm.hibernate3.HibernateTemplate.load(HibernateTemplate.java:483)
at org.springframework.orm.hibernate3.HibernateTemplate.load(HibernateTemplate.java:477)
at com.foo.dao.HibernateCommodityDao.loadCommodity(HibernateCommodityDao.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:335)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:181)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176)
at $Proxy2.loadCommodity(Unknown Source)
at com.foo.dao.HibernateCommodityDaoTest.testLoadEachCommodity(HibernateCommodityDaoTest.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:478)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:344)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)


Name and version of the database you are using:
DB2 8.1.3


The generated SQL (show_sql=true):

select allowablep0_.commodity_id as commodity3_1_, allowablep0_.packing_id as packing1_1_, allowablep0_.position as position1_, allowablep0_.packing_id as packing1_26_0_, allowablep0_.description as descript2_26_0_ from packing allowablep0_ where allowablep0_.commodity_id=?


Debug level Hibernate log excerpt:

[12313ms,DEBUG,SQL,main] select allowablep0_.commodity_id as commodity3_1_, allowablep0_.packing_id as packing1_1_, allowablep0_.position as position1_, allowablep0_.packing_id as packing1_26_0_, allowablep0_.description as descript2_26_0_ from packing allowablep0_ where allowablep0_.commodity_id=?
Hibernate: select allowablep0_.commodity_id as commodity3_1_, allowablep0_.packing_id as packing1_1_, allowablep0_.position as position1_, allowablep0_.packing_id as packing1_26_0_, allowablep0_.description as descript2_26_0_ from packing allowablep0_ where allowablep0_.commodity_id=?
[12313ms,DEBUG,AbstractBatcher,main] preparing statement
[12313ms,DEBUG,LongType,main] binding '2' to parameter: 1
[12328ms,DEBUG,AbstractBatcher,main] about to open ResultSet (open ResultSets: 0, globally: 0)
[12328ms,DEBUG,Loader,main] result set contains (possibly empty) collection: [com.foo.domain.commodity.CommodityImpl.allowablePacking#2]
[12328ms,DEBUG,CollectionLoadContext,main] uninitialized collection: initializing
[12328ms,DEBUG,Loader,main] processing result set
[12328ms,DEBUG,Loader,main] result set row: 0
[12328ms,DEBUG,LongType,main] returning '2' as column: packing1_26_0_
[12328ms,DEBUG,Loader,main] result row: EntityKey[com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,Loader,main] Initializing object from ResultSet: [com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,AbstractEntityPersister,main] Hydrating entity: [com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,StringType,main] returning 'Bulk' as column: descript2_26_0_
[12328ms,DEBUG,LongType,main] returning '2' as column: commodity3_1_
[12328ms,DEBUG,Loader,main] found row of collection: [com.foo.domain.commodity.CommodityImpl.allowablePacking#2]
[12328ms,DEBUG,CollectionLoadContext,main] reading row
[12328ms,DEBUG,LongType,main] returning '2' as column: packing1_1_
[12328ms,DEBUG,DefaultLoadEventListener,main] loading entity: [com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,DefaultLoadEventListener,main] attempting to resolve: [com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,DefaultLoadEventListener,main] resolved object in session cache: [com.foo.domain.commodity.PackingImpl#2]
[12328ms,DEBUG,IntegerType,main] returning '0' as column: position1_
[12328ms,DEBUG,AbstractBatcher,main] about to close ResultSet (open ResultSets: 1, globally: 1)
[12328ms,DEBUG,AbstractBatcher,main] about to close PreparedStatement (open PreparedStatements: 1, globally: 1)
[12328ms,DEBUG,AbstractBatcher,main] closing statement
[12328ms,DEBUG,JDBCTransaction,main] rollback
[12328ms,DEBUG,JDBCTransaction,main] re-enabling autocommit
[12328ms,DEBUG,JDBCTransaction,main] rolled back JDBC Connection
[12328ms,DEBUG,JDBCContext,main] after transaction completion
[12328ms,DEBUG,ConnectionManager,main] transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
[12328ms,DEBUG,SessionImpl,main] after transaction completion
[12328ms,DEBUG,SessionImpl,main] closing session
[12328ms,DEBUG,ConnectionManager,main] performing cleanup
[12328ms,DEBUG,ConnectionManager,main] releasing JDBC connection [ (open PreparedStatements: 0, globally: 0) (open ResultSets: 0, globally: 0)]
[12328ms,DEBUG,JDBCContext,main] after transaction completion
[12328ms,DEBUG,ConnectionManager,main] transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
[12328ms,DEBUG,SessionImpl,main] after transaction completion
[12328ms,INFO,HibernateCommodityDaoTest,main] Rolled back transaction after test execution




I'm using Hibernate's UserCollectionType feature to persist a custom List implementation. The custom List implementation was created by a domain layer developer here and I have very little, if any, control over its implementation. One of the things is does is a null check in its add() methods to ensure that null objects aren't being added to the list:

--------------------------------------------------------------------------------
package com.foo.domain.commodity;

import java.util.*;

import org.apache.commons.collections.list.AbstractListDecorator;

public class PackingListImpl extends AbstractListDecorator implements
PackingList
{
//...

public boolean add(Object object)
{
if(null == object)
{
throw new NullPointerException("Cannot add a null pointer to a "
+ "PackingList.");
}
//...
}

public void add(int index, Object object)
{
if(null == object)
{
throw new NullPointerException("Cannot add a null pointer to a "
+ "PackingList.");
}
//...
}

//...
}
--------------------------------------------------------------------------------

Similar to the MyList UserCollectionType example in Hibernate's test code, I created the following PersistentList to support my UserCollectionType:

--------------------------------------------------------------------------------
package com.foo.domain.commodity;

import java.sql.ResultSet;
import java.sql.SQLException;

import org.hibernate.HibernateException;
import org.hibernate.collection.PersistentList;
import org.hibernate.engine.SessionImplementor;
import org.hibernate.loader.CollectionAliases;
import org.hibernate.persister.collection.CollectionPersister;

import com.foo.hibernate3.CustomPersistentList;


/**
* This is a supporting class for
* PackingListImplUserCollectionType. This implementation is based
* on the PersistentMyList class in the Hibernate 3 test suite.
*/
public class PersistentPackingListImpl extends PersistentList implements PackingList
{
public PersistentPackingListImpl(SessionImplementor session)
{
super(session);
}

public PersistentPackingListImpl(SessionImplementor session,
PackingList packingList)
{
super(session, packingList);
}
}
--------------------------------------------------------------------------------

For completeness, here is my UserCollectionType, which is also based on the example in Hibernate's test code:

--------------------------------------------------------------------------------
package com.foo.domain.commodity;

import java.util.Iterator;
import java.util.Map;

import org.hibernate.EntityMode;
import org.hibernate.HibernateException;
import org.hibernate.collection.PersistentCollection;
import org.hibernate.engine.SessionImplementor;
import org.hibernate.persister.collection.CollectionPersister;
import org.hibernate.usertype.UserCollectionType;


/**
* This is an implementation of Hibernate's UserCollectionType that provides
* persistence support for PackingListImpl. This implementation is
* based on the MyListType class in the Hibernate 3 test suite.
*
*/
public class PackingListImplUserCollectionType implements
UserCollectionType
{

public PersistentCollection instantiate(SessionImplementor session, CollectionPersister persister) throws HibernateException
{
return new PersistentPackingListImpl(session);
}

public PersistentCollection wrap(SessionImplementor session, Object collection)
{
if ( session.getEntityMode()==EntityMode.DOM4J ) {
throw new IllegalStateException("dom4j not supported");
}
else {
return new PersistentPackingListImpl( session, (PackingList) collection );
}
}

public Iterator getElementsIterator(Object collection)
{
return ((PackingList) collection).iterator();
}

public boolean contains(Object collection, Object entity)
{
return ( (PackingList) collection ).contains(entity);
}

public Object indexOf(Object collection, Object entity)
{
int index = ((PackingList) collection ).indexOf(entity);
if(index < 0)
{
return null;
}
else
{
return new Integer(index);
}
}

public Object replaceElements(Object original, Object target, CollectionPersister persister, Object owner, Map copyCache, SessionImplementor session) throws HibernateException
{
PackingList result = (PackingList) target;
result.clear();
result.addAll((PackingListImpl)original);
return result;
}

public Object instantiate()
{
return new PackingListImpl();
}
}

--------------------------------------------------------------------------------

When I try to rehydrate the PackingList collection, I get the exception shown above, namely:

java.lang.NullPointerException: Cannot add a null Packing to a PackingList.

This exception can be traced back to the implementation of PersistentList.readFrom():

--------------------------------------------------------------------------------
public Object readFrom(ResultSet rs, CollectionPersister persister, CollectionAliases descriptor, Object owner)
throws HibernateException, SQLException {
Object element = persister.readElement( rs, owner, descriptor.getSuffixedElementAliases(), getSession() ) ;
int index = ( (Integer) persister.readIndex( rs, descriptor.getSuffixedIndexAliases(), getSession() ) ).intValue();

//pad with nulls from the current last element up to the new index
for ( int i = list.size(); i<=index; i++) {
list.add(i, null);
}

list.set(index, element);
return element;
}
--------------------------------------------------------------------------------

The line "list.add(i, null)" tries to add a null object to the List, which is not allowed by the custom List.

One idea I had was to extend PersistentList and override readFrom() so that it uses an intermediate temporary List. Here is the code:

--------------------------------------------------------------------------------
package com.foo.hibernate3;

import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;

import org.hibernate.HibernateException;
import org.hibernate.collection.PersistentList;
import org.hibernate.engine.SessionImplementor;
import org.hibernate.loader.CollectionAliases;
import org.hibernate.persister.collection.CollectionPersister;

/**
* Extends Hibernate's PersistentList to workaround some problems caused when
* using PersistentList to implement Hibernate UserCollectionTypes.
* <p>
* For example, PersistentList's readFrom() method throws an exception if the
* wrapped custom List does not allow null elements to be added.
* <p>
* Note that this class' readFrom() implementation may perform quite slowly for
* big result sets relative to PersistentList's original implementation.
* Therefore, you should only extend from this class if you absolutely need to;
* otherwise, please extend from PersistentList.
*
*/
public class CustomPersistentList extends PersistentList
{
public CustomPersistentList(SessionImplementor session)
{
super(session);
}

public CustomPersistentList(SessionImplementor session,
List list)
{
super(session, list);
}

/**
* Overrides PersistentList's implementation to work with wrapped custom
* List implementations that do not allow null elements to be added.
*/
public Object readFrom(ResultSet rs, CollectionPersister persister, CollectionAliases descriptor, Object owner) throws HibernateException, SQLException
{
Object element = persister.readElement( rs, owner, descriptor.getSuffixedElementAliases(), getSession() ) ;
int index = ( (Integer) persister.readIndex( rs, descriptor.getSuffixedIndexAliases(), getSession() ) ).intValue();

// Here is the key...we use a temporary List
List temp = new ArrayList(list);

//pad with nulls from the current last element up to the new index
for ( int i = temp.size(); i<=index; i++) {
temp.add(i, null);
}

temp.set(index, element);
list.clear();
list.addAll(temp);
return element;
}
}

--------------------------------------------------------------------------------

That workaround works except for one major caveat -- it ONLY works if the records in the JDBC ResultSet just happen to be sorted by the List's mapped index column. For example, if the ResultSet looks like this, it works:

my_list_index, (other columns)
0
1
2
3

However, if the ResultSet looks like this, it fails with the same exception (
java.lang.NullPointerException: Cannot add a null Packing to a PackingList):

my_list_index, (other columns)
1
0
3
2

Initially, I didn't notice this possibility because my database just happened to be returning the record in my_list_index order. However, later on when I modified the indexes on the table that the records live in, they starting being returned in a different order, resulting in the exception.

Looking at the SQL that Hibernate generated (see above), I noticed that Hibernate was not appending an "order by" clause. I though to myself, "No problem, I'll just an an order-by clause in the collection mapping."

The problem is, the <list> element does NOT support the "order-by" attribute. (Interestingly, <set>, <map> and <bag> do support it.)

Therefore, I cannot guarantee that this custom collection will always load correctly, without the NullException.

I realize that removing the null check from the custom collection would easily solve the problem BUT the domain layer developer is extremely reluctant to let any of my persistence layer concerns "pollute" his layer. I know, I know, it's not fair.

*****Any ideas for solving this problem????*********

That's not particularly interesting. The whole point of a list is that it has an integer list index that is the lists absolute, immutable and only ordering. What would be the point in allowing a sort="" or order-by="" attribute that you must completely ignore?

To fix your main issue, add not-null="true" to the <key> attribute of the list. I'm not sure that'll work, but hey, it won't hurt.

An alternative is to change your <list> to <bag>. Bag is implemented as a list, so only the mapping has to change. This will allow you to use order-by. The downside is that you'll have to map the position column yourself, instead of letting hibernate map it to a list index.

Tenwit, thank you for your kind comments.



I agree, on the surface it seems like adding an order-by attribute to the <list> element is pointless. However, it wouldn't hurt to add it because PersistentList doesn't expect the records in the ResultSet to be in any particular order -- PersistentList's readFrom() method does the ordering itself (look at the generated SQL - it does NOT order by the list index on the backend). In other words, if an optional order-by attribute was supported by Hibernate, it shouldn't break any existing code but it would solve my problem.



Thanks but it didn't work.



The downside you mention is the precise reason why we didn't try it. Our domain layer developer is vehemently opposed to adding any "artificial" sort order properties to his domain objects. His philosophy is that the domain layer should not have to make any compromises to support the UI layer and the persistence layer. He wants all workarounds put in the UI layer and the persistence layer and he wants the workarounds to be completely transparent to the domain layer. Ya, I know, I know...

So, I ask again, any solutions to this issue, short of quitting my job or killing the domain layer developer?

The problem is that readFrom() is responsible for reading a single collection element from the result set in isolation; meaning it is not aware about other rows in the jdbc result set awaiting to be read. You need to "tie in" at a level that has access to all the collection elements at once. That is, unfortunately, not very easy as of right now.

But, there is a simple solution if you are certain you have no gaps in your indices. Just put a non-null placeholder into the collection.

Ya, I already checked that out. It is not easy.



Ya, I thought of that too. The problem is, these custom Lists are used all over the domain layer. I would have to extend PersistentList once for each custom collection in order to allow the addition of a placeholder that would not be rejected by the checks in the custom List's add() method. For example, PackingList's add() method would only accept an instance of a Packing that had certain values, FooList's add method would only accept an instance of a Foo that had certain values, etc.

Again, I wish to reiterate that the domain layer developer refuses to change his code to support mine.

I have solved the problem with minimal kludginess and minimal intrusion into the domain layer! Even our domain layer developer is happy (which is saying something!).

Below is a description of how I did it. Sure, it's not super elegant but it works for me. I hope other people find it useful (if you do, please let me know so that I can get a warm fuzzy feeling inside...).

First, I created this interface:



Then I created the following abstract base class which all of our custom List implementations will extend. Previously, our custom List implementations were directly extending org.apache.commons.collections.list.AbstractListDecorator.



Next, I changed the custom List implementation, PackingListImpl, so that it extends AbstractDirectAccessListDecorator instead of AbstractListDecorator. That is the ONLY change I had to make in the domain layer so our domain layer developer is happy.



Note that the PackingList interface remains unchanged. In other words, it does NOT extend DecoratedListAccessor.



Next, I changed the implementation of the readFrom() method in CustomPersistentList. The readFrom() method now acts upon the underlying decorated List instead of the custom List implementation, thereby bypassing the validity checks in the custom List implementation's add() method.



As you can see, readFrom() must cast the custom List implementation to DecoratedListAccessor, so that it can get at the underlying decorated List. Of course, any sneaker developer could do the same cast to get at the underyling decorated List. However, we created a policy today that says this readFrom() method is the only place where we will allow casting to DecoratedListAccessor. Furthermore, the business layer is not allowed to declare services that return DecoratedListAccessors. For example, in this case, the business layer can only return PackingList from its services.

Note that I throw an informative exception message if the custom List implementation doesn't implement DecoratedListAccessor. I'm so thoughtful. :-)

By the way, since I solved my own problem, can I please get my credit back? :-)

Why? I mean literally just a placeholder such as a string or random object. Based on the code in your original post, there is no mention of the fact that PackingList's add method only accepts a Packaging instance. Of course this *only works* if the indices are contiguous...

But it looks like you found a solution at any rate. So if you are happy with that go with that...

PackingList's add() method only accepts a Packing instance. It also does some other checks too. We also have oodles of other custom List implementations that do various checks on the object passed to add(). So, a placeholder will be rejected unless it meets the criteria checked in add(). For rehydration purposes, it's easier just to bypass those checks altogether.

Anyway, my solution seems to work. My Unit tests pass and more importantly, the domain developer is happy.