DTO or not DTO, together, find THE solution

anthony · **Posted:** Sun Feb 22, 2004 11:29 am

hi all,
it seems that experts discussing here have the solution for their layered architecture.
For us, we use:
- struts for the controller and we're planning to use JSF later for the presentation layer
- a business (or pseudo service) layer to define our business rules and adapt domain model to views
- a Data access layer to code reusable queries or chained CRUD calls on the application
- of course Hibernate for the persistance (orm)

We all need to "map" the client views with the business model, the fastest method is to design DTOs with attributes named like our java (persistent) objects (= BO = Business Objects) and then use for example the copy proterties... we can copy attributes of n differents BO to one DTO which will be used by the JSP.
Now the consequences are:
- if we want to reuse the business layer, we have to take the same DTO package, so to have the same views logic. The consequence is we can recode all the DTO and organize other copy properties.... it's bad in term of reusability isn't it?
- if the views are reworked, we must also rework the business layer only for copy properties considerations and not for business considerations... too bad no?

So we have worked on two solutions which have the same approach. To resume, the two use a parameter XML file in which is described the composition of the view. For each element, we associate an attribute of a BO like this (simplified example):
<view name="view-client" >
<attribute name="numberPhone" BO="com.test.Client" BOAttr="numberPhone"/>
<attribute name="adress" VIEW="view-adress"/>
...
</view>
<view name="adress">
<attribute name="postCode" BO="com.test.Adress" BOAttr="postCode"/>
<attribute-list name="lines" BO="com.test.Adress" BOAttr="adressLines"/>
...
</view>

And the best, to "create" these views object, we'll call a method like this: myViewManager.flushView("view-client",[BO1,BO2,...]). The rework on the business layer is "smaller".
Somewhere, it looks like an HBM file isn't it? ;)

So where the solutions are different is:
- the first use 100% dynaclasses. The advantage is that we don't need extra DTO classes --> the DTO are 90% unusefull, in fact there are 100% usefull but 90% hidden and dynamiquely managed. But this solution is ugly on the way that we can't detect any errors on the management of these views objects at the compilation, we must always test it at runtime and this could be very dangerous for a developpment team with 10-15 persons.
- the second use the view-bo mapping file + DTO, and only propose a "copy property" api. The advantage is that we can controle more errors at the compilation, but we must code again the DTO...

If someone have another solution please discuss here, i think that the problem is interesting.

Excuse me for my english, i'm only a french advanced hibernate user lol

steve · **Posted:** Sun Feb 22, 2004 2:11 pm

We also use a very similiar architecture. A service layer mediates between the client ui tier and a hibernate-backed DAO tier. The services broker requests to the DAO to manipulate domain entities (managed through hibernate) then offloads pre-defined pieces of data into DTOs for return to the client.

The DTO piece really sucks. If I add a new property to the domain, I need to update not only the domain entity, but also one or more DTOs and the "assemblers" which move the data between the domain entities and the DTOs. This approach works for small projects, but as soon as it is applied to any of our larger projects, it becomes a maintenance nightmare managing all those DTOs and assemblers.

Currently, what I am proposing to my bosses is one of to approaches...

1) Identify the application representing the "system of record" for each piece of information. That application would be built such that the services would return the actual domain entities using hibernate's detached object support. Identitify the pieces of information which should be available to other applications and expose them through both a web-service end-point and a session ejb end-point. The ejb end-point might still use DTOs for brokering with the client (I haven't decded yet), but it would be many, many fewer DTOs to maintain.

2) The applications we build are basically CRM (and some ERP functionality) in nature. Thus they all hit mostly the same DB. My manager is enamoured with the idea of a centralized service layer (using both session ejbs and web-services). If he persists in insisting on this setup, I plan on suggesting (insisting, actually, since maintaining all those DTOs is killing me) the use of CarrierWave. It does essentially the same stuff you mention regarding your "view mapping". Plus it actually generates all the DTOs (their called "images" in CarrierWave-speak) for you given the definition of the domain entity and the "plan" (its mapping file). Very promising stuff.

michael · **Posted:** Sun Feb 22, 2004 4:23 pm

The CarrierWave thing indeed looks interesting. I really wonder however if all that overhead of code generation, icon/image/imageable confusion etc. is really needed ... what keeps us from just using our Hibernate mapped objects as DTOs?

Wouldn't it be possible to create a rather simple framework which allows us to do fetching of specified parts of the object graph, and partition it using the proxy/lazy loading features of Hibernate?

christian · **Posted:** Sun Feb 22, 2004 4:36 pm

CarrierWave _is_ that framework. It is not simple, thats the problem. The ideas and patterns used are.

anthony · **Posted:** Sun Feb 22, 2004 6:16 pm

thanks for CarrierWave, i didn't know this project, we'll try this as soon as possible

christian · **Posted:** Sun Feb 22, 2004 6:19 pm

Warning: It is (too) complex and not really ready for public consumption. The author knows about this and currently tries to release a more user-friendly package.

anthony · **Posted:** Sun Feb 22, 2004 6:28 pm

thanks, we'll continue working on the mapping idea.... i'll give the conclusion quickly

thanks again

RobJellinghaus · **Joined:** Thu Jan 08, 2004 6:17 pm **Posts:** 278

I think you all may be interested in
http://opensource.atlassian.com/project ... key=HB-700

The quick version:

Many Hibernate users use hbm2java to create their basic persistent POJOs from Hibernate mappings. Something like this:

Code:

<hibernate-mapping>

    <class name="my.Parent">
        <meta attribute="generated-class">my.ParentDTO</meta>
        <id name="id" type="long" unsaved-value="null" >
            <generator class="native"/>
        </id>
        <set name="children" inverse="true" cascade="all-delete-orphan">
            <key column="parent_id"/>
            <one-to-many class="my.Child"/>
        </set>
    </class>

    <class name="my.Child">
        <meta attribute="generated-class">my.ChildDTO</meta>
        <id name="id" type="long" unsaved-value="null" >
            <generator class="native"/>
        </id>
        <discriminator/>
        <many-to-one name="parent" column="parent_id" not-null="true"
            class="my.Parent"/>
        <subclass name="my.SpecialChild"
            discriminator-value="SPEC">
            <meta attribute="generated-class">my.SpecialChildDTO</meta>
        </subclass>
    </class>
</hibernate-mapping>

Which can be used with this hbm2java.config.xml:

Code:

<codegen>
    <generate
        package="data"
        renderer="net.sf.hibernate.tool.hbm2java.BasicRenderer"/>
</codegen>

To produce this code (some boilerplate omitted):

Code:

package data;
...
/** @author Hibernate CodeGenerator */
abstract public class ParentDTO implements java.io.Serializable {
    /** identifier field */
    private Long id;
    /** persistent field */
    private Set children;
    /** full constructor */
    public ParentDTO(Set children) {
        this.children = children;
    }
...
}

/** @author Hibernate CodeGenerator */
abstract public class ChildDTO implements java.io.Serializable {
    /** identifier field */
    private Long id;
    /** persistent field */
    private my.Parent parent;
    /** full constructor */
    public ChildDTO(my.Parent parent) {
        this.parent = parent;
    }
...
    public my.Parent getParent() {
        return this.parent;
    }
    public void setParent(my.Parent parent) {
        this.parent = parent;
    }
...
}

/** @author Hibernate CodeGenerator */
abstract public class SpecialChildDTO extends Child implements java.io.Serializable {

    /** full constructor */
    public SpecialChildDTO(my.Parent parent) {
        super(parent);
    }

    /** default constructor */
    public SpecialChildDTO() {
    }

    public String toString() {
        return new ToStringBuilder(this)
            .append("id", getId())
            .toString();
    }

}

Then you implement my.Parent extends data.ParentDTO, my.Child extends data.Child, and my.SpecialChild extends data.SpecialChildDTO. You put all kinds of business behavior and domain knowledge into the my.Parent, my.Child, and my.SpecialChild subclasses. When you revise your data model, you regenerate the DTO superclasses and leave the my.* subclasses mostly alone. And all is well.

Except you can't actually use the DTO superclasses *as* data transfer objects, since they contain all kinds of references to my.* classes which contain your private business logic. i.e. SpecialChildDTO extends my.Child so you can't send out a SpecialChildDTO without also exposing your my.Child business subclass!

So I have submitted a patch to hbm2java to allow you to use this config.xml:

Code:

<codegen>
    <generate
        package="dto"
        renderer="net.sf.hibernate.tool.hbm2java.BasicRenderer">
        <param name="use-generated-classes-only">true</param>
   </generate>
</codegen>

Which, when run against the *exact* same mapping given above, produces *this* code:

Code:

package dto;
...
/** @author Hibernate CodeGenerator */
public class ParentDTO implements Serializable {
    /** identifier field */
    private Long id;
    /** persistent field */
    private Set children;
    /** full constructor */
    public ParentDTO(Set children) {
        this.children = children;
    }
}

/** @author Hibernate CodeGenerator */
public class ChildDTO implements Serializable {
    /** identifier field */
    private Long id;
    /** persistent field */
    private dto.ParentDTO parent;
    /** full constructor */
    public ChildDTO(dto.ParentDTO parent) {
        this.parent = parent;
    }
...
    public dto.ParentDTO getParent() {
        return this.parent;
    }
    public void setParent(dto.ParentDTO parent) {
        this.parent = parent;
    }
...
}

/** @author Hibernate CodeGenerator */
public class SpecialChildDTO extends dto.ChildDTO implements Serializable {
    /** full constructor */
    public SpecialChildDTO(dto.ParentDTO parent) {
        super(parent);
    }
...
}

This new hierarchy maps exactly to the DTO classes in the other hierarchy -- same property names, same field names, and so forth. And it is possible to write a generic DTOMapper that uses reflection to convert any graph of objects from one hierarchy into a graph of objects from the other hierarchy, without losing any information. (I will be writing such a mapper within the next few days.)

This, I think, is a pretty good solution to the evolving-DTO problem, which we also have in our project.

What do you all think? If you like it, please consider voting for HB-700 in JIRA :-)

Cheers!
Rob

RobJellinghaus · **Joined:** Thu Jan 08, 2004 6:17 pm **Posts:** 278

p.s. Yes, I realize this HB-700 proposal does not address the problems of how much loading of the object graph you want to do (which is partly what CarrierWave is all about); all this proposal does is give you a nice way to use hbm2java to really truly generate DTOs that map directly to your domain model.

anthony · **Posted:** Tue Feb 24, 2004 4:09 am

i think this is interesting but unfortunately it doesn't do what we want here.
Actually attribute's name and "data graph" on views doesn't always match with the business object graph and attribute's names.

Think of this, for a "big" application:
- step1: ask the users for what they need to do their work, for this the best is to design a prototype on which the DTO and naming will be reworked but not entirely.
- step2: once step one is validated, you can launch jsp development team, they're going to code the jsp and the DTO they need, then the struts-action for example
- step3: the business rules are examined, you look on your applications which business objects you can reuse, write the domain model (business class diagram). At this step, the attribute's name can be different... here is the problem for us. You also can imagine that to make a DTO, you can need 3 BO: DTO.Attribute1=BO1.Attribute1,DTO.Attribute2=BO2.Attribute1,DTO.Attribute3=BO3.Attribute1
In fact the problem is due because the views are not THE business representation of an application but is A usefull representation of the business.

I d'ont know if i'm really clear with that.. :)

RobJellinghaus · **Joined:** Thu Jan 08, 2004 6:17 pm **Posts:** 278

Yes, you are clear. We're skewing away from defining a client-specific view of the data model since it introduces so much overhead. We may get bitten if our backend data model drifts significantly away from the frontend model, but that's life :-)

In fact it occurs to me that you could keep a Hibernate mapping for the frontend view as well as for the backend data model, and generate code from each... still wouldn't help you with the mapping code though.

Really what you might want is additional meta attributes on the basic Hibernate mapping to help define the view mapping, and then hbm2java extensions to generate the mapping code :-) Then your view mapping and your base data model would both be generated from the Hibernate mapping, which would really help you keep them both consistent.

But that's not what my patch does ;-) ...well, maybe my patch is a *tiny* step in that direction...
Cheers!
Rob

baliukas · **Posted:** Tue Feb 24, 2004 2:30 pm

I can not understand why people need so many models ?

1. DTO model designed for View .
2. Persistent Objects designed for DAO (hidden in DAO layer and not accessable for application)

Is it something wrong to remove 2. Looks like it just garbage and model dublication, is not it ? Why not to design the same model for View and for DAO ? I think it is better to remove something from code than to add, if it doe's the same (It means removed code doe's nothing).

RobJellinghaus · **Joined:** Thu Jan 08, 2004 6:17 pm **Posts:** 278

Here are *all* the divisions that I have heard of:

1) The basic Hibernate classes (i.e. persistent POJOs that Hibernate manages). (If this is all you have, it's potentially the "anemic domain model" as per Fowler.)

2) Subclasses of (or behavior enhancements to) the basic persistent POJOs, to create a richer domain model.

3) Data access objects (DAOs) which contain Hibernate-specific HQL. Typically your domain model methods (from #2) use your DAOs in order to keep Hibernate-specific code from leaking into your domain model. The idea is that #1 and #2 are generic code -- there's actually little or no Hibernate-specific code in #1 or #2 -- so you keep all the Hibernate-specific code in DAOs which are *used* by your business logic (#2).

4) Data transfer objects (DTOs) used to pass data from the domain model out to the view / client layer. Some people think these should look significantly different than your basic persistent objects -- i.e. they think #4 should be very different than #1 -- in order to let you change #1 without needing to change #4. Other people (Gavin, for instance :-) feel that whenever you change #1 you need to also change #4 (or at least change the mapping code that maps #1 to #4), so why bother with #4? Just use #1.

My proposal above highlights some problems with just using #1 to pass data out of your service layer (namely that if you expose #1, you also have to expose #2). So my proposal lets you automatically create a DTO layer (#4) that is structurally identical to your persistent objects (#1), so you only need to write some generic mapping code and never customize it again.

So, basically, the reasons for each layer:

#1: Simple Persistent Beans. You have to have this layer for Hibernate to work at all :-)
#2: Behavior-Enhanced Persistent Beans. You want to add interesting business logic to your Simple Persistent Beans, but you also want to be able to change your data model without having to rewrite your bean code. So you subclass your #1 objects.
#3: Data Access Objects. You want your #1 and #2 layers to stay free of Hibernate-specific code, so you put all the Hibernate-specific code in separate objects which you use from your #2 methods.
#4: DTOs. You want to hide your view clients from the structure and/or the business logic of your #1/#2 objects.

That's why :-)
Cheers!
Rob

baliukas · **Posted:** Tue Feb 24, 2004 2:59 pm

RobJellinghaus wrote:

Here are *all* the divisions that I have heard of:

1) The basic Hibernate classes (i.e. persistent POJOs that Hibernate manages). (If this is all you have, it's potentially the "anemic domain model" as per Fowler.)

2) Subclasses of (or behavior enhancements to) the basic persistent POJOs, to create a richer domain model.

Just enhance behavior using delegation vs subclassing :

class Enhanced {

POJO o;

Enhanced(POJO o){
this.o = o;
}
public getPOJO(){

return o;

}

............... behavior enhancements ............

}

RobJellinghaus · **Joined:** Thu Jan 08, 2004 6:17 pm **Posts:** 278

Delegation doesn't work seamlessly with Hibernate. In your example, it is really Enhanced that is the object you want Hibernate to be creating and returning.

For instance, if you have a Parent object with a set of Enhanced objects, you want Hibernate to create the Enhanced objects when it loads that set from the database. Otherwise how do you manage the creation of the Enhanced objects when Hibernate loads the Set?

The <meta generated-class> technique is designed for exactly this case -- where you want to have rich behavioral domain objects that Hibernate knows how to instantiate and load (and proxy) on demand, but where you still want Hibernate to generate the base code for the data model fields.

Cheers,
Rob