-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: Table Per Subclass performance concerns for large hierarchy
PostPosted: Wed Jan 23, 2008 8:45 am 
Newbie

Joined: Wed Jan 23, 2008 8:07 am
Posts: 6
Hi,

We have a problem with our use of Hibernate; I don't think it's version or platform specific, but more likely our modeling and mapping that requires fixing.

Hibernate version: 3.1.3

Oracle 10g and MySQL 5.1

HQL

Code:
from WarehouseItemDTO item where item.workspace=:workspaceName and (exists (from WarehouseContentItemDTO ci where ci.id = item.id and ci.nodeId = :id) or exists (from WarehouseCampaignDTO cp where cp.id = item.id and cp.nodeId = :id) or exists (from WarehouseAssetDTO asset where asset.id = item.id and asset.nodeId = :id))


Generated SQL
Code:
select blah
from WHNODE tablei0_
left outer join WHASSET tablei0_1_ on tablei0_.ID=tablei0_1_.ID
left outer join WHAUDIOASSET tablei0_2_ on tablei0_.ID=tablei0_2_.ID
left outer join WHIMAGEASSET tablei0_3_ on tablei0_.ID=tablei0_3_.ID
left outer join WHVIDEOASSET tablei0_4_ on tablei0_.ID=tablei0_4_.ID
left outer join WHCAMPAIGN tablei0_5_ on tablei0_.ID=tablei0_5_.ID
left outer join WHCATEGORY tablei0_6_ on tablei0_.ID=tablei0_6_.ID
left outer join WHCONTENTITEM tablei0_7_ on tablei0_.ID=tablei0_7_.ID
left outer join WHAUDIO tablei0_8_ on tablei0_.ID=tablei0_8_.ID
left outer join WHEXTERNALITEM tablei0_9_ on tablei0_.ID=tablei0_9_.ID
left outer join WHSERVICE tablei0_10_ on tablei0_.ID=tablei0_10_.ID
where
    tablei0_.WORKSPACE=:1
    and (exists (
        select tablec1_.ID
        from WHCONTENTITEM tablec1_
            inner join WHNODE tablec1_1_ on tablec1_.ID=tablec1_1_.ID
        where tablec1_.ID=tablei0_.ID and tablec1_.ITEMID=:2)
    or exists (
        select tablec2_.ID
        from WHCAMPAIGN tablec2_
            inner join WHNODE tablec2_1_ on tablec2_.ID=tablec2_1_.ID
        where tablec2_.ID=tablei0_.ID and tablec2_.CAMPAIGNID=:3)
    or exists (
        select tablea3_.ID
        from WHASSET tablea3_
            inner join WHNODE tablea3_1_ on tablea3_.ID=tablea3_1_.ID
        where tablea3_.ID=tablei0_.ID and tablea3_.ASSETID=:4))


We have used the Table Per Subclass approach mentioned here. So we have a class mapping, with lots of <joined-subclass/> relationships. One aspect that I didn't find mentioned in the documentation is that this will generate relatively expensive queries for large inheritance hierarchies, as per the example above.

The explain plan for Oracle indicates that the exists clauses aren't expensive, but the outer joins are. We'd like to reduce the load on the database, since we're noticing the expense in our testing.

Any suggestions for a better way of modeling the inheritance relationships, that will meet with our goal of optimizing read performance?

Cheers,

Jamesht


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 30, 2008 6:04 am 
Newbie

Joined: Wed Jan 23, 2008 8:07 am
Posts: 6
We've re-jigged the tables to allow a faster query to be executed, and are now executing what should be two fast queries rather than a single slow one. Not great in terms of network traffic, but we're trusting Hibernate to do sensible things with database connections.

We've had to go to native SQL for the first query, but it's now a couple of orders of magnitude faster with the size of data sets that we're testing with. We've encountered a different problem however.

The old, slow query code looked like this:

Code:
        List warehouseItems = hibernateTemplate.findByNamedQueryAndNamedParam(WarehouseItemDTO.class.getName()
                + ".query.findByIDWorkspace", new String[] { "id", "workspaceName" }, new Object[] {
                Integer.valueOf((int) id), workspace });

        if (warehouseItems.size() > 1) {
            // UUID and workspace together form a composite primary key so we only expect one record to be returned
            log.warn("More than one record returned for id '" + id + "' and workspace '" + workspace
                    + "' - check table integrity");
        }

        WarehouseItem warehouseItem = null;

        if (warehouseItems.size() > 0) {
            warehouseItem = (WarehouseItem) warehouseItems.get(0);
        }
       
        return warehouseItem;


The new, faster version looks like this:
Code:
        WarehouseItemDTO warehouseItem = (WarehouseItemDTO) hibernateTemplate.execute(new HibernateCallback() {

            /**
             * {@inheritDoc}
             */
            public Object doInHibernate(Session session) throws HibernateException, SQLException {
                SQLQuery sqlQuery = session.createSQLQuery("SELECT ID FROM WHNODE WHERE "
                        + "NEWID = :newid AND WORKSPACE = :workspace");
                sqlQuery.setInteger("newid", (int) id);
                sqlQuery.setString("workspace", workspace);

                List warehouseNodeIds = sqlQuery.list();

                if (warehouseNodeIds.size() > 1) {

                    /*
                     * newid and workspace together form a composite primary key so we only expect one record to be
                     * returned
                     */
                    log.warn("More than one record returned for id '" + id + "' and workspace '" + workspace
                            + "' - check table integrity");
                }

                WarehouseItemDTO result = null;

                if (!warehouseNodeIds.isEmpty()) {
                        result = (WarehouseItemDTO) session.load(itemClass, (Integer) warehouseNodeIds.get(0));
                }

                return result;
            }
        });

        return warehouseItem;


The problem with this approach is that after a period of time, we start getting the following problem

Code:
java.lang.ExceptionInInitializerError
        at org.hibernate.engine.StatefulPersistenceContext.getCollectionLoadContext(StatefulPersistenceContext.java:146)
        at org.hibernate.type.CollectionType.getCollection(CollectionType.java:488)
        at org.hibernate.type.CollectionType.resolveKey(CollectionType.java:341)
        at org.hibernate.type.CollectionType.resolve(CollectionType.java:335)
        at org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:116)
        at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:842)
        at org.hibernate.loader.Loader.doQuery(Loader.java:717)
        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
        at org.hibernate.loader.Loader.loadEntity(Loader.java:1785)
        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:47)
        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:41)
        at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:2730)
        at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:365)
        at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:346)
        at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:123)
        at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:161)
        at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:87)
        at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:862)
        at org.hibernate.impl.SessionImpl.load(SessionImpl.java:781)
        at org.hibernate.impl.SessionImpl.load(SessionImpl.java:774)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.orm.hibernate3.HibernateTemplate$CloseSuppressingInvocationHandler.invoke(HibernateTemplate.java:1197)
        at $Proxy21.load(Unknown Source)
        at com.example.module.warehouse.dao.WarehouseItemDAOImpl$1.doInHibernate(WarehouseItemDAOImpl.java:175)
        at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:367)
        at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:333)
        at com.example.module.warehouse.dao.WarehouseItemDAOImpl.getWarehouseItemByID(WarehouseItemDAOImpl.java:148)
        at com.example.module.warehouse.WarehouseSynchronizerImpl.synchronize(WarehouseSynchronizerImpl.java:94)
        at com.example.module.warehouse.WarehouseSynchronizerImpl.synchronize(WarehouseSynchronizerImpl.java:67)
        at com.example.module.warehouse.WarehouseEventListener$1.doInJcr(WarehouseEventListener.java:129)
        at com.example.module.springmodule.MIQJcrTemplate.doExecute(MIQJcrTemplate.java:125)
        at com.example.module.springmodule.MIQJcrTemplate.execute(MIQJcrTemplate.java:101)
        at com.example.module.warehouse.WarehouseEventListener.processInterestingPaths(WarehouseEventListener.java:160)
        at com.example.module.warehouse.WarehouseEventListener.onEvent(WarehouseEventListener.java:102)
        at org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:231)
        at org.apache.jackrabbit.core.observation.ObservationDispatcher.run(ObservationDispatcher.java:145)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.commons.logging.LogConfigurationException: org.apache.commons.logging.LogConfigurationException: java.lang.ThreadDeath (Caused by java.lang.ThreadDeath) (Caused by org.apache.commons.logging.LogConfigurationException: java.lang.ThreadDeath (Caused by java.lang.ThreadDeath))
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:543)
        at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
        at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:209)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
        at org.hibernate.engine.CollectionLoadContext.<clinit>(CollectionLoadContext.java:31)
        ... 40 more
Caused by: org.apache.commons.logging.LogConfigurationException: java.lang.ThreadDeath (Caused by java.lang.ThreadDeath)
        at org.apache.commons.logging.impl.LogFactoryImpl.getLogConstructor(LogFactoryImpl.java:397)
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
        ... 44 more
Caused by: java.lang.ThreadDeath
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1229)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)
        at org.apache.commons.logging.impl.LogFactoryImpl$1.run(LogFactoryImpl.java:441)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.commons.logging.impl.LogFactoryImpl.loadClass(LogFactoryImpl.java:435)
        at org.apache.commons.logging.impl.LogFactoryImpl.getLogConstructor(LogFactoryImpl.java:376)
        ... 45 more


Subsequent calls to that method then throw NoClassDefFoundError, since the JVM marks the class as unusable after getting the ExceptionInInitializerError.

This is using Java 5 and Java 6 on Windows Vista and Fedora Core 6, both running Tomcat 5.0.30. I'm going to try some other Tomcat versions and see whether it's some strange container interaction.

No webapp is being deployed or undeployed to Tomcat during the period that the error occurs, and I haven't yet discovered why the ThreadDeath error is happening. Obviously, the faster method listed above isn't a good solution for us going forward, since the web app requires reloading to dispose of the classloader with the bad reference.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Feb 05, 2008 6:52 am 
Newbie

Joined: Wed Jan 23, 2008 8:07 am
Posts: 6
It looks like the ThreadDeath is a known issue with Tomcat and commons-logging versions, and possibly our code as well, in terms of how we're cleaning up commons-logging. We don't use commons-logging, but Hibernate does, so we need to add some code to clean it up properly and see if that fixes our problem.

http://www.qos.ch/logging/thinkAgain.jsp


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.