Query relating to the cascade option 'delete-orphan'

I have a table 'TableFoo' having a foreign key relationship with itself.
This is how my hbm file looks like
<set name="children" cascade="all,delete-orphan" inverse="true"
fetch="select" lazy="false">
<key column="parentId" />
<one-to-many class="com.... .hb.FooClass" />
</set>

<many-to-one name="parent"
class="com.... .hb.FooClass"
column="parentId" />

Now, in a session I pull out a particular record with all its children to an object fooObject of type FooClass.
In another session, I remove one of the children from fooObject by fooObject.getChildren().iterator().remove().
and do a saveOrUpdate(fooObject). I suppose this should delete the particular child from the DB. But its not!!

The same scenario works fine for me when a table has a foreign key mapped to another table.
Thanks,