-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: Batch-loading HOWTO
PostPosted: Sun Jul 18, 2004 4:32 am 
Newbie

Joined: Sat Oct 04, 2003 8:26 am
Posts: 10
Imagine a scenario like: Query is executed but some entities that are part of the resultset are not cached in the second-level cache; the query itself is cached with: query.setCacheable(true);

What Hibernate will do in this case? As the query is already cached, it will just try to load the needed entities. But it will load them with as many SQL requests as the number of non-cached entities.

This, generally, leads to even worse performance than with non-cached queries. Which, in turn, means that unless we know 80-90% of entities that would appear in the resultset, are cached query caching should not be used, at all! Because, if your query returns 100 resulting objects and only 50 of them are in the second-level cache, your single cached query will result in 50 SQL requests to the database - terrible!

Now, it is very hard to have the majority of entities in the cache - JVMs have 2G memory limitation, after all. Even if you have entity-caching enabled, as you need to set some LRU rule, bad things, like described above, may happen quite often and actually slow down your system significantly.

I was advised to use batch-loading to help this problem. The idea is that, batch-size attribute in the hbm mapping forces Hibernate to load all missing objects (or a least the number equal to batch-size) in one SQL, rather then each object individually.

This would really help, if I was able to achieve it, but for some reason - I can not make it happen :(

OK, here is what I am trying: for the simplicity, I have entity caching disabled for my entity altogether - to be sure that these entities are not in the second-level cache (caching enabled and entities not in the cache because of LRU is just a flavor of this situation, right?). I have setCachable(true) for my query. I am firing query twice - first time I see that objects are retrieved with one SQL (query is not cached yet) but on the second run - I see, through Hibernate's show_sql, 5 SQLs going to the database. 5 is the number of objects in the resultset.

Here is my mapping:

Code:
<?xml version="1.0"?>

<!DOCTYPE hibernate-mapping PUBLIC
    "-//Hibernate/Hibernate Mapping DTD 2.0//EN"
    "http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">

<hibernate-mapping>
   <class name="org.digi.Message" table="DG_MESSAGE" batch-size="20" proxy="org.digi.Message">
      <composite-id>
               <key-property name="key" type="java.lang.String" column="MESSAGE_KEY"/>
               <key-property name="locale" type="java.lang.String" column="LANG_ISO" length="2"/>
          <key-property name="siteId" type="java.lang.String" column="SITE_ID" length="100"/>
      </composite-id>
      <property name="message" type="java.lang.String" column="MESSAGE_UTF8"/>
      <property name="created" type="java.sql.Timestamp" column="CREATED"/>
   </class>
</hibernate-mapping>


as you can see, I have batch-size="20" indicated. In the begining, I had only that but then I added proxy="...", too, as I was told that batch-size only works for proxyable objects (??). It did not help.

Each entity is still loaded with an individual query :(

Any help is appreciated.

P.S. following is the Java code used:
Code:
   public static void objQueryList ( Session session ) throws HibernateException{
        Query query = session.createQuery(
            "select m from Message as m where m.key = :key");
        query.setString("key", "login:remember");

        query.setCacheable(true);
        Message msg = null;

        List results = query.list();

        for (Iterator it = results.iterator(); it.hasNext(); ) {
          msg = new Message();
          msg = (Message) it.next();
          System.out.println("Message: " + msg.getMessage() );
        }
  }


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 4:37 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Hum, it looks good, it should work. I even checked the code, and a cached query result set should return proxies.

Try stepping through code.

First, verify that the result set contains proxies.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 4:38 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
oh, btw, how many objects are in the result set?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 5:04 am 
Newbie

Joined: Sat Oct 04, 2003 8:26 am
Posts: 10
Gavin,

thanks for a reply.

There are 5 objects in the resultset.

I tried batch_size = 20, 5 and 3
none of these works :(

Here is the log:

13:01:21,093 INFO [SettingsFactory] cache provider: net.sf.ehcache.hibernate.Provider
13:01:21,109 INFO [Configuration] instantiating and configuring caches
13:01:21,140 INFO [SessionFactoryImpl] building session factory
13:01:21,562 INFO [SessionFactoryObjectFactory] no JNDI name configured
13:01:21,578 INFO [UpdateTimestampsCache] starting update timestamps cache at region: net.sf.hibernate.cache.UpdateTimestampsCache
13:01:21,609 WARN [Configurator] No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: jar:file:/C:/jboss/server/default/deploy/cachetest.war/WEB-INF/lib/ehcache.jar!/ehcache-failsafe.xml
13:01:21,625 WARN [Plugin] Could not find configuration for net.sf.hibernate.cache.UpdateTimestampsCache. Configuring using the defaultCache settings.
13:01:21,656 INFO [QueryCache] starting query cache at region: net.sf.hibernate.cache.QueryCache
13:01:21,656 WARN [Plugin] Could not find configuration for net.sf.hibernate.cache.QueryCache. Configuring using the defaultCache settings.
13:01:21,828 INFO [STDOUT] Hibernate: select message0_.MESSAGE_KEY as MESSAGE_1_, message0_.LANG_ISO as LANG_ISO, message0_.SITE_ID as SITE_ID, message0_.MESSAGE_UTF8 as MESSAGE_4_, message0_.CREATED as CREATED from DG_MESSAGE message0_ where (message0_.MESSAGE_KEY=? )
13:01:21,890 INFO [STDOUT] Message: Remember your email and password?
13:01:21,890 INFO [STDOUT] Message:


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 5:23 am 
Newbie

Joined: Sat Oct 04, 2003 8:26 am
Posts: 10
Sorry,

seems like I was looking at the wrong config file.

Making the entity proxy, helped, indeed.

Maybe the documentation should be more specific what batch-size is for and that it works only for proxiable objects?

thank you


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.