JDBC vs Hibernate

ravi · **Posted:** Thu Jan 29, 2004 12:13 pm

greetings,

i did a battery of tests making sure that both hibernate an jdbc generate the *exact* same sql. the test case involves creating objects in various numbers.

Code:

Add objects (INSERT)   
Objects     Hib           JDBC
100          125           47
1000        671          328
5000        2281        1359
10000      4187        2484
      
Find all Objects      
100           16            0
1000         47            16
5000         281          47
10000       350          156
      
Find single Object by id (time in () indicates second run)
100          47 (5)        <15
1000        78 (16)       <15
5000        94 (62)       <15
10000      185 (141)    <15

Find ALL objects by Id one by one      
100           500           78
1000         10516       437
5000         >10 sec    1422
10000       >10 sec    2500

among all these, notice the behaviour with finding a single object by id. as the data set size increases so does the seek time with hibernate. this is unexpected because the id is also primary key and there's an index associated with it in the database. hence any 'select * from table where id=100' is constant time operation (as seen when a JDBC call is made). what is it that hibernate is doing during this extra time.

the new guy
--------------
!try(); do();

Code:

christian · **Posted:** Thu Jan 29, 2004 12:17 pm

What is your question exactly? How to debug your (weird) test case you haven't shown any code for?

bertrand · **Posted:** Thu Jan 29, 2004 12:26 pm

ravi wrote:

... what is it that hibernate is doing during this extra time ?

Besides many other things, Hibernate is creating your object instances from the resultsets... which is what the brain-less JDBC driver can't do !

christian · **Posted:** Thu Jan 29, 2004 12:28 pm

This doesn't take that much time. I'm sure his test is incorrect and not optimized, as all the others that got posted here before. If you are new to Hibernate and don't know how it works, don't try to benchmark it.

ravi · **Posted:** Thu Jan 29, 2004 1:59 pm

thanks for the replies. one of the salient features of hibernate is that the usage of the api is simple to follow. and this not a complex test case that can deviate too much from the usual usage model. so before the benchmarking is dismissed completely, i've included the code that produced these results. comments?

(brenuart, also i've taken the object creation time into account while working with JDBC)

JDBC code for find by id

Code:

         Statement stmt = conn.createStatement();
         ResultSet resultSet = stmt.executeQuery("select * from device where id= '" + id + "'");

         if(!resultSet.next()) System.err.println("Could not find device by id");
         
         Device device;
            
         device = new Device();
         device.setAdminInterfaceIP(resultSet.getString("administrative_interface_ip"));
         device.setBackupStatus(resultSet.getString("backup_status"));
         device.setCreationState(resultSet.getString("creation_state"));
         device.setStatus(resultSet.getString("device_status"));
         device.setBackupEnabled(resultSet.getBoolean("backup_enabled"));
         stmt.close();

hibernate query by id

Code:

      Configuration cfg = new Configuration()
         .addClass(Device.class)
         .addClass(History.class)
         .setProperty(Environment.HBM2DDL_AUTO, "create")
         .setProperty(Environment.DIALECT, "net.sf.hibernate.dialect.MySQLDialect")
         .setProperty(Environment.DRIVER, "org.gjt.mm.mysql.Driver")
         .setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver")
         .setProperty(Environment.URL, "jdbc:mysql:///test")
         .setProperty(Environment.USER, "")
         .setProperty(Environment.PASS, "");
 .
 .
 .

   private void alterpointGetSingleDeviceById(int id)
   {
      try
      {
         Long i = new Long(id);
         List list = s.find("from Device as device where device.ID =" + i);
         
         if (list.size()==0)   throw new IllegalArgumentException("No device for the given id: " + id);
   
         Device device = (Device) list.get(0);
      }
      catch (Exception e) {
         e.printStackTrace();
      }
   }

the times are all in ms. and the factory initilization is surely not included.

R

christian · **Posted:** Thu Jan 29, 2004 2:01 pm

Use load() or get(). Your benchmark has no impact on real-world performance. Also, have you considered what Hotspot does?

gavin · **Posted:** Thu Jan 29, 2004 3:07 pm

FYI, these test results are absolutely absurd. Please study the Hibernate documentation and learn to optimize Hibernate before bothering the list with such stuff. Hibernate includes some test cases in the test package which demonstrate that these numbers are garbage. (The overhead on INSERTs is always *tiny*.)

gavin · **Posted:** Thu Jan 29, 2004 3:26 pm

Hey guys, I am getting really, really sick of these newbie "Hibernate slower than JDBC blah blah blah" posts. What we need to do is clean up the performance tests we already have and package them up nicely, so that we've got something new users can just easily run, to verify our claims about performance, and then they can even use these as a template for their own performance tests.

Its really virtually impossible to debug performance problems from offsite, so these kinds of threads piss /us/ off, since we know it wastes at least an entire day of someones time to track down the problem remotely. And that probably gives the impression that we are somehow unable to back up our claims. Also the new user gets a bad impression of us, because we respond with a nice rude "go solve your own problems" type post.

Since our performance is so great, we really should promote this with actual verifiable numbers!

ravi · **Posted:** Thu Jan 29, 2004 6:41 pm

gavin,

no doubt you guys have done some great work but you must realise that this exercise is the first of the steps that ANYONE would take before stepping into the hibernate pool with both feet. even if there's a prescribed framework to verify performance, we cannot dictate the environment in which the tests will be run.

what i have done is the most simplistic approach to setting up a test case and it may have minor usage deviations but it's not a far cry for a reasonable test framework. in simple terms what is it that is drastically deviant that in a simple test case like mine where i try to find an object by ID from a table of say 1000 records. with the load() and get() i DO get constant time finds on a primary key but the first time they run it's always an order of magnitude longer than an equivalent JDBC call (that includes building the object but not caching it/ with the exact same sql).

that said i'm all up for setting up a package for obtaining quick performance results. i would be glad to help you setup a new ant target for the test cases if you wish. this way newbies can avoid the wrath of the fanatic developer everytime a condradiction is raised.

regards
ravi

emmanuel · **Posted:** Thu Jan 29, 2004 8:12 pm

Hum, you know, these kind of materials will be used to create a petstore like war. And it won't help anybody to see the truth. Some vendors claim over the net a 1000%+ faster than JDBC, we all know this is stupid. This could happen but on very specific and usually non real word samples.
I haven't found the right solution on this however :(