Using an XML external !ENTITY declaration in .hbm.xml files.

eepstein · **Posted:** Mon Jan 26, 2004 7:55 pm

I'm trying to use a component across several classes. So I put the component definition into its own file and use XML's ENTITY include mechanisms to load this file into each .hbm.xml mapping that wants it.

(This is recommended for subclasses here: http://www.hibernate.org/117.html#A19
).

BUT, the xml parser does not use the currently parsed document (the enclosing .hbm.xml file) as the base/root for resolving the external entity reference. It uses the current working directory of whatever is executing (e.g., the directory where I'm running ant, or the servlet container's execution path).

How do I tell hibernate's XML parser to use the current document as the base directory for relative includes?

E.g., for a file named DynamicEnum.hbm.xml with the following:

Code:

<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
    "-//Hibernate/Hibernate Mapping DTD 2.0//EN"
    "http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd" [
   <!ENTITY usageTracking SYSTEM "file:./UsageTrackingComponent.xml">
]>
    
<hibernate-mapping>
  <!-- CVS: $Id: DynamicEnum.hbm.xml,v 1.3 2004/01/19 03:57:29 eepstein Exp $ -->

<class 
    name="com.publishworks.common.DynamicEnum" 
    table="dynamic_enum"
    schema="common"
>

   &usageTracking;

</class>
</hibernate-mapping>

Will fetch th referenced "./UsageTrackingComponent.xml" out of the directory where the DynamicEnum.hbm.xml file lives and not out of the directory where I'm running my program!?

NOTE: I found other posts on this problem but it hasn't been answered. Any help appreciated.

emmanuel · **Posted:** Tue Jan 27, 2004 4:45 pm

this is very specific to the underlying XML parser. Try to find one providing this capability.

eepstein · **Posted:** Wed Jan 28, 2004 2:35 am

emmanuel wrote:

this is very specific to the underlying XML parser. Try to find one providing this capability.

Could you make any recommendations. Also, is there a reason that Hibernate is using as a default an XML parser which has this behavior?

Thanks.

l.masini · **Posted:** Wed Jan 28, 2004 8:15 am

It's a common XML problem, and also I think it's not easy to fix only changing the XML parser.
The fact come from the fact that when you declare your entities, the parser doesn't expand them, until you make a real reference.
But in that case, in the DOM tree, there is not a standard way to know which was the current directory.
So the only sure way to parse XML entities is to write them absolute.
I hope in future xml schema xml:base attribute to solve everything.
Ciao.

gcoghlan · **Joined:** Fri Jan 09, 2004 10:24 am **Posts:** 7

I found a way to get the parser used by hibernate to resolve an external entity using the classpath.
But it's a quite crude ...

I had seen some mention that the dtd for the hibernate xml files are located in the hibernate jar file.
This led me to realise that hibernate configures it's XML parser with an entity resolver. (i.e. I looked at the source code :-)

This hibernate entity resolver examines the System identifier of an entity and if it starts with the string "http://hibernate.sourceforge.net/" then it treats it as a resource with a name starting with "net/sf/hibernate/" to be located on the classpath
e.g.
<!DOCTYPE hibernate-configuration
PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-2.0.dtd">
will mean the hibernate's entity resolve will look for the resource
net/sf/hibernate/hibernate-configuration-2.0.dtd
on the classpath (and find it in the hibernate jar file)

So you could have
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 2.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd" [
<!ENTITY usageTracking SYSTEM "http://hibernate.sourceforge.net/../../../UsageTrackingComponent.xml">
]>
and this will resolve to the resource name
net/sf/hibernate/../../../UsageTrackingComponent.xml
to be found on the classpath.
i.e.
UsageTrackingComponent.xml would need to be in the root of your CLASSPATH

Although this does work I think it is very vulunerable.

It makes the file defining our external references look as though they are related to hibernate.
This would be very misleading for someone else, or another XML tool that does not have the hibernate entity resolver.

I'm looking for a better solution.
any recommendations ?
regards
Grainne

eepstein · **Posted:** Wed Jan 28, 2004 8:10 pm

OK, the source to the XML entity resolver code is all in:
net.sf.hibernate.util.XMLHelper.java and net.sf.hibernate.util.DTDEntityResolver.java. It is indeed true, DTDEntityResolver hard-codes the resolution of http://hibernate.sourceforge.net/ -- a standard and acceptable technique. And XLMHelper hard codes its usage of DTDEntityResolver without any hooks to add another resolver or in any way manipulate the xml reader. All done in the single static method:

Code:

public static SAXReader createSAXReader(String file, List errorsList)

And finally all the different "addXXX()" flavors (addFile(), addResource, addXML(), etc.) all hard-code a call to createSAXReader() -- that code can be found in net.sf.hibernate.cfg.Configuration.java.

So it seems a dimension of missing flexibility is needed. A logical place for it is in the XMLHelper class. If that class's createSAXReader() method could be rewritten to make use of some class parameters (at a minimum) that wuold setup the groupd for how the SAX Reader is built -- e.g., adding custom entity resolvers. This would be a quick fix to the source and would resolve this problem.

gcoghlan · **Joined:** Fri Jan 09, 2004 10:24 am **Posts:** 7

Yes - a way of customizing the EntityResolver configured by hibernate would be most useful.
I have been looking at XML Catalogs (a means of mapping XML public identifiers to URIs other than the system id).
It would be very handy to be able to configure in a catalog understanding EntityResolver.
But in the meantime I have found that extending net.sf.hibernate.cfg.Configuration and overriding the methods
addFile(String)
addXML(String)
addInputStream(InputStream)
doConfigure(InputStream, String)
to allow me to provide my own EntityResolver works. (I'll post the code if asked)

eepstein · **Posted:** Thu Jan 29, 2004 3:05 pm

gcoghlan wrote:

But in the meantime I have found that extending net.sf.hibernate.cfg.Configuration and overriding the methods
addFile(String)
addXML(String)
addInputStream(InputStream)
doConfigure(InputStream, String)
to allow me to provide my own EntityResolver works. (I'll post the code if asked)

Ah, that's great! It wuld be helpful if you did post it.

It also occurred to me that the "standard" way of handling XML processing is via configurable Factory methods that create the parser (SAX or otherwise). You can then plug in your own parser, configured for entity resolution, error handling, etc. This would be another, even more robust way to fix this. I'm not sure why Hibernate didn't adopt this approach as, it is the standard way this is done.

emmanuel · **Posted:** Thu Jan 29, 2004 7:47 pm

Can you add this improvement request to JIRA ?

rdi · **Posted:** Fri Jan 30, 2004 9:24 pm

gcoghlan wrote:

I have found that extending net.sf.hibernate.cfg.Configuration and overriding the methods
addFile(String)
addXML(String)
addInputStream(InputStream)
doConfigure(InputStream, String)
to allow me to provide my own EntityResolver works. (I'll post the code if asked)

Please! I'm doing the same thing as eepstein WRT external entities and this will save us many hours of configuration heartache.

gcoghlan · **Joined:** Fri Jan 09, 2004 10:24 am **Posts:** 7

As noted by emmanual + eepstein it would be preferable to have the EntityResolver configurable in hibernate.
I intend to raise a JIRA enhancement request for this today.

But in the meantime, as requested here's the code:

HibernateConfiguration is a class that extends net.sf.hibernate.cfg.Configuration

Code:

package com.acme.hibernate;

import com.acme.utils.xml.CompositeEntityResolver;

import java.io.*;
import java.util.*;

import org.apache.log4j.Logger;
import org.dom4j.io.SAXReader;
import org.xml.sax.EntityResolver;
import org.xml.sax.InputSource;

import net.sf.hibernate.HibernateException;
import net.sf.hibernate.MappingException;
import net.sf.hibernate.cfg.Configuration;
import net.sf.hibernate.util.XMLHelper;

public class HibernateConfiguration extends Configuration
{
   private static Logger log = Logger.getLogger(HibernateConfiguration.class);

   protected Configuration doConfigure(InputStream stream, String resourceName) throws HibernateException
   {
      org.dom4j.Document doc;
      try
      {
         List errors = new ArrayList();
         SAXReader saxReader = createSAXReader(resourceName, errors);
         doc = saxReader.read(new InputSource(stream));
         if (errors.size() != 0)
            throw new MappingException("invalid configuration", (Throwable) errors.get(0));
      }
      catch (Exception e)
      {
         log.error("problem parsing configuration" + resourceName, e);
         throw new HibernateException("problem parsing configuration" + resourceName, e);
      }

      return doConfigure(doc);
   }

   protected SAXReader createSAXReader(String resourceName, List errors)
   {
      SAXReader saxReader = XMLHelper.createSAXReader(resourceName, errors);
      EntityResolver entityResolver = saxReader.getEntityResolver();
      CompositeEntityResolver compositeResolver = new CompositeEntityResolver();
      compositeResolver.addResolver(entityResolver);
      EntityResolver acmeEntityResolver = new ACMEEntityResolver();
      compositeResolver.addResolver(acmeEntityResolver);
      saxReader.setEntityResolver(compositeResolver);
      return saxReader;
   }

   public Configuration addFile(String xmlFile) throws MappingException
   {
      log.info("Mapping file: " + xmlFile);
      try
      {
         List errors = new ArrayList();
         SAXReader saxReader = createSAXReader(xmlFile, errors);
         add(saxReader.read(new File(xmlFile)));
         if (errors.size() != 0)
            throw new MappingException("invalid mapping", (Throwable) errors.get(0));
         return this;
      }
      catch (Exception e)
      {
         log.error("Could not configure datastore from file: " + xmlFile, e);
         throw new MappingException(e);
      }
   }

   public Configuration addInputStream(InputStream xmlInputStream) throws MappingException
   {
      try
      {
         List errors = new ArrayList();
         SAXReader saxReader = createSAXReader("XML InputStream", errors); 
         add( saxReader.read(new InputSource(xmlInputStream)));
         if (errors.size() != 0)
            throw new MappingException("invalid mapping", (Throwable) errors.get(0));
         return this;
      }
      catch (MappingException me)
      {
         throw me;
      }
      catch (Exception e)
      {
         log.error("Could not configure datastore from input stream", e);
         throw new MappingException(e);
      }
   }

   public Configuration addXML(String xml) throws MappingException
   {
      if ( log.isDebugEnabled() ) log.debug("Mapping XML:\n" + xml);
      try {
         List errors = new ArrayList();
         SAXReader saxReader = createSAXReader("XML String", errors); 
         add( saxReader.read( new StringReader(xml) ) );
         if ( errors.size()!=0 ) throw new MappingException( "invalid mapping", (Throwable) errors.get(0) );
      }
      catch (Exception e) {
         log.error("Could not configure datastore from XML", e);
         throw new MappingException(e);
      }
      return this;
   }

}

CompositeEntityResolver implements EntityResolver.
It contains 0 or more EntityResolvers.
It delegates call to resolve entities these contained EntityResolvers, stopping with the first one that returns a non-null result.

Code:

package com.acme.utils.xml;

import ims.utils.Logging;

import java.io.IOException;
import java.util.Iterator;
import java.util.List;

import org.apache.log4j.Logger;
import org.xml.sax.EntityResolver;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;

public class CompositeEntityResolver implements EntityResolver
{
   private static Logger log = Logger.getLogger(CompositeEntityResolver.class);
   
   List children;
   
   public CompositeEntityResolver()
   {
      children = new java.util.ArrayList();
   }

   /* (non-Javadoc)
    * @see org.xml.sax.EntityResolver#resolveEntity(java.lang.String, java.lang.String)
    */
   public InputSource resolveEntity(String publicId, String systemId) throws SAXException, IOException
   {
      Iterator iterator = children.iterator();
      InputSource source = null;
      while(iterator.hasNext())
      {
         EntityResolver resolver = (EntityResolver) iterator.next();
         source = resolver.resolveEntity(publicId, systemId);
         if (null != source)
            break;
      }
      return source;
   }
   
   public void addResolver(EntityResolver resolver)
   {
      if (!children.contains(resolver))
      {
         children.add(resolver);
      }
   }
   
   public void removeResolver(EntityResolver resolver)
   {
      if (children.contains(resolver))
      {
         children.remove(resolver);
      }
   }
}

ACMEEntityResolver implements EntityResolver to intercept the entities that whose public id starts with a particular prefix. It then uses the ClassLoader to get the entity as a resource from the classpath.
To implement this I followed the example posted at
http://www.dom4j.org/faq.html#cannot-find-dtd

Code:

package com.acme.hibernate;

import ims.utils.Logging;

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;

import org.apache.log4j.Logger;
import org.xml.sax.EntityResolver;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;

public class ACMEEntityResolver implements EntityResolver
{
   private static Logger log = Logger.getLogger(IMSEntityResolver.class);
   
   public static final String ACME_PUBLIC_ID_PREFIX = "-//ACME/";      
   public static final String ACME_RESOURCE_PREFIX = "acme"; 

   public InputSource resolveEntity(String publicId, String systemId) throws SAXException, IOException
   {
      InputSource inputSource = null;
      if (publicId.startsWith(ACME_PUBLIC_ID_PREFIX))
      {
         ClassLoader classLoader = getClass().getClassLoader();
         URL url = new URL(systemId);
         String path = url.getPath();
         String resourceName = ACME_RESOURCE_PREFIX+path;
         InputStream inputStream = classLoader.getResourceAsStream(resourceName);
         if (null == inputStream)
         {
            log.debug(systemId + "not found in classpath");
         }
         else
         {
            log.debug("found " + systemId + " in classpath");
            inputSource = new InputSource(inputStream);
            inputSource.setPublicId(publicId);
            inputSource.setSystemId(systemId);
         }
      }
      return inputSource;
   }

}

To use this code you must instantiate the HibernatedConfiguration rather than the standard net.sf.hibernate.cfg.Configuration
e.g. creating the SessionFactory

Code:

SessionFactory sessionFactory = new HibernateConfiguration().configure().buildSessionFactory();

rdi · **Posted:** Mon Feb 02, 2004 6:56 pm

[quote="gcoghlan"]
But in the meantime, as requested here's the code:
[/quote]

Thanks greatly. Any restrictions on use, copyright attributions that should be observed, etc? I know I'm going to be asked this by my management.

nathanmoon · **Posted:** Thu Jun 23, 2005 6:11 pm

I apologize for raising such an old thread, but I'm trying to use XML entities as suggested in the docs (H3, section 10.1.6 Table per concrete class, using implicit polymorphism). I'm getting an error about relative URIs. This thread talks about this issue, and 'gcoghlan' said he was putting in a JIRA request, but I can't find it.

Is there any way in H3 to customize the EntityResolver in the XML parser, or somehow get my relative URIs resolved? (besides the messy subclassing solution suggested in this thread)

Thanks.

nathanmoon · **Posted:** Fri Jun 24, 2005 1:37 pm

OK, I found the JIRA for this, it was submitted by 'eepstein' - HB-667. Looks like setEntityResolver was added to Configuration in H3.

This means I can create my own entity resolver and have Hibernate use it. The problem being that when running Hibernate Tools from ANT, I don't have any way to set the entity resolver. I can get things working when running my app, but I don't have any way to get hbm2java to be able to parse my mapping files.

It seems like such a basic thing to want to specify relative URIs for the DTD, so I looked into modifying Hibernate to get this to work. I couldn't figure out how to get relative URIs to work but I was able to make minor modifications to DTDEntityResolver to get it to load my xml file as a resource.

For example, if I have a mapping file Test.hbm.xml that's located in com/nathan/Test.hbm.xml from the classpath, and some properties defined in com/nathan/testprops.xml (in the same location), I can put the following in Test.hbm.xml to get those properties included in the mapping file:

Code:

<!DOCTYPE hibernate-mapping PUBLIC 
   "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
   "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd"
   [ <!ENTITY testprops SYSTEM "resource://com/nathan/testprops.xml"> ]>

My new DTDEntityResolver will look for com/nathan/testprops.xml as a resource. Here's the code, since it's not very long:

Code:

   private static final String RESOURCE = "resource://";

   public InputSource resolveEntity (String publicId, String systemId) {
      if ( systemId!=null ) {
         String path = systemId;
         if ( systemId.startsWith(URL) ) {
            log.debug("trying to locate " + systemId + " in classpath under org/hibernate/");
            path = "org/hibernate/" + systemId.substring( URL.length() );
         }
         else if ( systemId.startsWith(RESOURCE) ) {
            log.debug("trying to locate " + systemId + " in classpath");
            path = systemId.substring( RESOURCE.length() );
         }
         // Search for DTD
         InputStream dtdStream = resourceLoader==null ? 
               getClass().getResourceAsStream(path) :
               resourceLoader.getResourceAsStream(path);
         if (dtdStream == null) {
            dtdStream = Thread.currentThread().getContextClassLoader()
                  .getResourceAsStream(path);
         }
         if (dtdStream == null) {
            dtdStream = Environment.class.getClassLoader()
                  .getResourceAsStream(path);
         }
         if (dtdStream==null) {
            log.debug(systemId + " not found in classpath");
            return null;
         }
         else {
            log.debug("found " + systemId + " in classpath");
            InputSource source = new InputSource(dtdStream);
            source.setPublicId(publicId);
            source.setSystemId(systemId);
            return source;
         }
      }
      else {
         // use the default behaviour
         return null;
      }
   }

The Thread.currentThread().getContextClassLoader() was the one that found the resource in my code, but I put the Environment.class.getClassLoader() part in there, too, to match what Configuration does when loading mapping files. I realize that this causes DTDEntityResolver to import org.hibernate.cfg.Environment, and maybe that is undesirable. So that part can be left out if necessary, I think.

Anyway, the only thing I don't like about this is the whole "resource://" thing, but I couldn't figure out a good alternative. If I just put a relative path ("com/nathan/testprops.xml"), the parser doesn't even ask the DTDEntityResolver to resolve it - it just immediately says that it can't resolve relative URIs. I had to add something to make it seem fully qualified so that it would pass it to the entity resolver. If anyone has suggestions on how to make that part cleaner, it seems like this would be a worthwhile addition to Hibernate.

Comments anyone? Alternative solutions? I'd hate to have to run on a home-grown version of Hibernate, but I'd really like to use the XML entity thing in my mapping files and don't see any way to get it to work with Hibernate as it currently stands.

thyrell · **Posted:** Sat Jul 23, 2005 10:13 am

Hi Nathan,

great work. i wanted to try your code but it's missing the URL and resourceLoader variables. Could you post the whole class please? That would rock. I'm trying to use entities for <typedef>s of Java 5 Enum-UserTypes.

Thanks a lot
andi