These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 6 posts ] 
Author Message
 Post subject: Slow performance of Hibernate 3.0 Filters
PostPosted: Sat Jul 30, 2005 8:10 pm 
Newbie

Joined: Sat Jul 30, 2005 7:49 pm
Posts: 4
Hi guys,

We recently upgraded to Hibernate 3.0 to be able to use filters for a multitenant application, the idea is to filter the database information based on the user's profile and the application being accessed.

However initial testing of the infrastructure has slowed down our performance test cases to run twice as slow. This is by just enabling a filter for all sessions, but not filtering data just yet.

I checked that the SQL being generated is the same for our app, and profiling reveals that the same time is being spend on the MySQL Drivers code with or without filters. however hibernate and antlr time goes up.

This performance hit of course is not good as it guarantees the application will run at least twice as slow for no apparent reason.

I am definitely expecting a performance hit when filtering most of the queries, but certainly I was expecting no change in performance by just enabling a non-used filter.

So the question is any pointers, recommendations, hints or ideas on how to solve this problem ?

Below a sample program that shows and confirms this behaviour

The results I got here are:

Time no filter = 922
Time with filter = 2797


Hibernate version: 3.0

Mapping documents:
Code:
<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
   "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
   "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">

<hibernate-mapping >

<!--
create table testdata (id bigint not null, name varchar(200), level varchar(200));
create unique index nameidx on testdata (name);

create table hibernate_unique_key (next_hi INTEGER);
insert into hibernate_unique_key values (1);
-->

   <class name="rg.test.TestBean" table="testdata">
       <id name="id" type="long" column="id"  unsaved-value="0">
           <generator class="hilo"/>
       </id>
       <property name="name"   column="name"/>
       <property name="level"  column="level"/>
   </class>

   <filter-def name="levelFilter">
    <filter-param name="currentLevel" type="string"/>
   </filter-def>

</hibernate-mapping>



Test Program and Bean:

Code:
package rg.test;

public class TestBean {

   public TestBean(String name, String level) {
      this.name = name;
      this.level = level;
   }

   private long id;
   private String name;
   private String level;

   public String getLevel() {
      return level;
   }

   public String getName() {
      return name;
   }

   public void setId(long id) {
      this.id = id;
   }

   public void setLevel(String level) {
      this.level = level;
   }

   public void setName(String name) {
      this.name = name;
   }

   public long getId() {
      return id;
   }

   public TestBean() {}
}


Code:
package rg.test;

import org.hibernate.cfg.Configuration;
import org.hibernate.SessionFactory;
import org.hibernate.Session;
import java.util.Properties;
import org.hibernate.cfg.Environment;
import org.hibernate.Transaction;
import java.util.List;
import java.util.Iterator;
import org.hibernate.Query;

public class FilterTest {
   
   public static void main(String[] arg) {
      FilterTest ft = new FilterTest();
      ft.executeTest();
   }

   public FilterTest() {
   }

   public void executeTest() {
      initHibernate();
     
      createTestData();
     
      runTestWithoutFilter();
     
      runTestWithFilter();
     
      closeHibernate();
   }



   SessionFactory sessions;
   long NUM_QUERIES = 1000;

   private void initHibernate() {
      Properties properties = new Properties ();
      properties.setProperty (Environment.DRIVER, "org.gjt.mm.mysql.Driver" );
      properties.setProperty (Environment.URL, "jdbc:mysql://localhost/rgtest");
      properties.setProperty (Environment.USER, "root");
      properties.setProperty (Environment.PASS, "");
      properties.setProperty (Environment.DIALECT, "org.hibernate.dialect.MySQLDialect");

      Configuration cfg = new Configuration();
      cfg.addProperties(properties);
      cfg.addClass(TestBean.class);
      sessions = cfg.buildSessionFactory();
   }

   private void closeHibernate() {
      sessions.close();
   }
   
   private void createTestData(){
      Session session = sessions.openSession();
      Transaction tx = session.beginTransaction();
      Iterator iter  = session.createQuery("from TestBean").iterate();
      while (iter.hasNext()) {
         session.delete(iter.next());
      }
      tx.commit();
     
      tx = session.beginTransaction();
      for (int i=0; i < NUM_QUERIES;  i++) {
         TestBean t = new TestBean("Name " + i, "Level-" + (i / 10));
         session.persist(t);
      }
      tx.commit();
      session.close();
   }

   private void runTestWithoutFilter() {
      long st = System.currentTimeMillis();
      for (int i=0; i < NUM_QUERIES;  i++) {
         Session session = sessions.openSession();
         Query q = session.createQuery("from TestBean where name = ?");
         String name = "Name " + i;
         q.setParameter(0, name);
         List l = q.list();
         TestBean tb = (TestBean)l.get(0);
         session.close();
      }

      System.out.println("Time no filter = " + (System.currentTimeMillis() - st));
   }

   private void runTestWithFilter(){
      long st = System.currentTimeMillis();
      for (int i=0; i < NUM_QUERIES;  i++) {
         Session session = sessions.openSession();

         session.enableFilter("levelFilter").setParameter("currentLevel", "Level-" + (i / 10));       
         Query q = session.createQuery("from TestBean where name = ?");
         String name = "Name " + i;
         q.setParameter(0, name);
         List l = q.list();
         TestBean tb = (TestBean)l.get(0);
         session.close();
      }

      System.out.println("Time with filter = " + (System.currentTimeMillis() - st));
   }

}



Name and version of the database you are using: MySQL: 4.0.23


Top
 Profile  
 
 Post subject: Alternative to use filters to filter data ?
PostPosted: Tue Aug 02, 2005 8:38 pm 
Newbie

Joined: Sat Jul 30, 2005 7:49 pm
Posts: 4
Just in case anybody can provide any help or cares about this issue....

Tracked down the bottleneck to:

Code:
public QueryTranslator[] getQuery(String queryString, boolean shallow, Map enabledFilters)


Where the comment in the code (and the matching implementation) reveals the problem:

Quote:
// if there are no enabled filters, consider cached query compilations,
// otherwise generate/compile a new set of query translators


For an application where the filters are always enabled, the effect is that query compilation is never cached.

Would it be possible to cache the query after the filter has been added to the query by the query compiler? , what about somehow mark filters as "cacheable" at the definition time ? Also I suspect that allowing for filters to be defined at the SessionFactory level rather than the session level could help in identifying filters that will affect queries in such a way that they are cacheable

Again, any pointers ?

Any suggestion for an alternative to provide the same functionality as filters (ideally without having to go and change the app adding the new where clause everywhere)

Thanks
Reinaldo.-


Top
 Profile  
 
 Post subject:
PostPosted: Tue Aug 02, 2005 9:43 pm 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 3:00 pm
Posts: 1816
Location: Austin, TX
The real problem is that not all filters effect all load/query operations. Consider you have filters named customersByRegion and ordersByDateRange attached to entities Customer and Order, respectively. Given the HQL "from Customer", it really does not matter whether ordersByDateRange is enabled or not as it has no bearing on the result.

The same holds true for loaders in general, btw; not just query translators/query-loaders.

We should do this at some point. But I don't see it as a priorityat this time. Enter a feature request in JIRA if you like.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Aug 15, 2005 12:01 am 
Newbie

Joined: Sat Jul 30, 2005 7:49 pm
Posts: 4
Here is my code change to make the filters perform as fast as the non filtered queries. (I do think performance is a priority....)

In the FactorySsssionImpl (I have here version: //$Id: SessionFactoryImpl.java,v 1.86 2005/07/18 22:36:43 epbernard Exp $ )

The idea is to cache the queries attaching the enabled filter names to the key, and then replace the filters to make use of the current vales in the filters:

Code:
   public QueryTranslator[] getQuery(String queryString, boolean shallow, Map enabledFilters)
   throws QueryException, MappingException {

      // if there are no enabled filters, consider cached query compilations,
      // otherwise generate/compile a new set of query translators
      Object cacheKey = null;
      QueryTranslator[] queries = null;

      if ( enabledFilters == null || enabledFilters.isEmpty() ) {
         cacheKey = QUERY_KEY_FACTORY.newInstance(queryString, shallow);
         queries = (QueryTranslator[]) get(cacheKey);
      } else {
         // Include the filters in the cacheKey, this allows for different versions of the queries
         // to be cached if the filters are enabled or not, and also is based on the specific filters
         // that are enabled.
         String filtersKey = "";
         Iterator iter = enabledFilters.keySet().iterator();
         while (iter.hasNext()) {
            filtersKey += ((String)iter.next());
         }
         // Create the cache key with the filter names enabled.
         cacheKey = QUERY_KEY_FACTORY.newInstance(queryString+"|"+filtersKey, shallow);
         queries = (QueryTranslator[]) get(cacheKey);
         if (queries != null) {
            for (int i=0; i<queries.length; i++ ) {
               // Replace the filters to ensure that the current values are used.
               queries[i].getEnabledFilters().putAll(enabledFilters);
            }
         }
      }

      // have to be careful to ensure that if the JVM does out-of-order execution
      // then another threacd can't get an uncompiled QueryTranslator from the cache
      // we also have to be very careful to ensure that other threads can perform
      // compiled queries while another query is being compiled

      if ( queries==null ) {
         // a query that names an interface or unmapped class in the from clause
         // is actually executed as multiple queries
         String[] concreteQueryStrings = QuerySplitter.concreteQueries(queryString, this);
         queries = createQueryTranslators(concreteQueryStrings, cacheKey, enabledFilters);
      }
      for ( int i=0; i<queries.length; i++) {
//         queries[i].compile(this, settings.getQuerySubstitutions(), shallow, enabledFilters);
         queries[i].compile( settings.getQuerySubstitutions(), shallow );
      }
      // see comment above. note that QueryTranslator.compile() is synchronized
      return queries;
   }


So considering that I have limited exposure to the internals of Hibernate I'd like to get some feedback to know if this is just something that will not work, or under which conditions it won't , or other gotchas that I am missing.

Initial testing shows minimal performance degradation when using this version.

Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Aug 15, 2005 1:58 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Please submit a patch (diff format) to JIRA. Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Aug 15, 2005 2:09 pm 
Newbie

Joined: Sat Jul 30, 2005 7:49 pm
Posts: 4
Done as requested.

http://opensource.atlassian.com/project ... se/HHH-864


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 6 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.