-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 36 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Performance test: Hibernate vs. JDBC - disappointing
PostPosted: Wed Nov 03, 2004 8:47 am 
Newbie

Joined: Wed Nov 03, 2004 7:24 am
Posts: 8
Location: Stuttgart
[b]Hibernate version:version 2.1.6, 9.8.2004 [/b]

You can find at http://www.hibernate.org/15.html, Performance Q&A, the statement: "Generally the overhead is much less than 10% of the JDBC calls". In my performance test app the overhead is between 100% and 2300%. What's wrong with it?

The test inserts/reads 10,000 objects into/from a table with 53 columns. The database engine is a MS SQL Server and I use MS JDBC driver (- yes I should not, but I don't think it's better with another one).

I run the test with different transaction lengths. Here is what I measured:

1. Insert

1.1. HIBERNATE: objects per trans / seconds
100 / 51s
10 / 106s
1 / 703s

1.2. JDBC
100 / 23s
10 / 26s
1 / 30s

2. Read

2.1. HIBERNATE: seconds
46s

2.2. JDBC
22s

Who can explain this result?
Why are Hibernate transactions so much slower than JDBC transactions (I do not use any hibernate.transaction.factory_class)?

I'm sorry that I cannot provide the complete source code of my test project for you. My manager does not allow me to do since it contains too much product specific information.

I use one table with 53 columns. 8 columns are of type varchar, 45 columns are of type int.

The class mapped to the table has a "composite-id": two of the integer columns build a unique row ID. Thus, I have to derive the class from "Serializable" and have to implement "equals" and "hashCode" for it.

public class MyClass implements Serializable {
public int id1;
public void setId1(Integer v) { id1 = v.intValue(); };
public Integer getId1() { return new Integer(v); };
public int id2;
public void setId2(Integer v) { id2 = v.intValue(); };
public Integer getId2() { return new Integer(v); };
...
// the other 51 members

public int hashCode() {
return (id1 << 16) | id2;
}

public boolean equals() {
// ... compare all members
}

};

class mapping:

<hibernate-mapping>
<class name="...MyObject" table="mytable" >

<composite-id >
<key-property name="id1" type="int" column="id1" />
<key-property name="id2" type="int" column="id2" />
</composite-id>

... all other members/columns


public class MainTest {

int nbOfObjs = 10000;
int nbOfObjsPerTrans = 100;

public void testInsert() throws HibernateException {
Session session = HibernateUtil.currentSession();
Transaction tx = null;
boolean beginTrans = true;
boolean endTrans = false;
boolean transOpen = false;

long startTime = System.currentTimeMillis();

for (int i = 0; i < nbOfObjs; i++) {
beginTrans = (i % nbOfObjsPerTrans) == 0;
endTrans = ((i+1) % nbOfObjsPerTrans) == 0;
if (beginTrans) {
tx= session.beginTransaction();
transOpen = true;
}

// create an instance of my object
MyObject obj = new MyObject();
// obj.setId1(1234);
// obj.setId2(4567);

session.save(obj);

if (endTrans) {
tx.commit();
transOpen = false;
}
}

if (transOpen) {
tx.commit();
}

long endTime = System.currentTimeMillis();

System.out.print("testInsert, dt=" + (endTime - startTime));
}

public void testRead() throws HibernateException {
Session session = HibernateUtil.currentSession();
long startTime = System.currentTimeMillis();

for (int i = 0; i < nbOfObjs; i++) {
// create an instance of my object
MyObject obj = new Object();
// obj.setId1(1234);
// obj.setId2(4567);
session.load(MyObject.class, obj);
}

long endTime = System.currentTimeMillis();

System.out.print("testRead, dt=" + (endTime - startTime));
}

public Connection connectJDBC() throws SQLException {
Connection cnn = // open JDBC connection
return cnn;
}

public void testInsertJDBC() throws SQLException {
Connection cnn = connectJDBC();
try {

boolean beginTrans = false;
boolean endTrans = false;
boolean transOpen = false;

long startTime = System.currentTimeMillis();

for (int i = 0; i < nbOfObjs; i++) {
beginTrans = (i % nbOfObjsPerTrans) == 0;
endTrans = ((i+1) % nbOfObjsPerTrans) == 0;
if (beginTrans) {
cnn.setAutoCommit(false);
transOpen = true;
}

// create an instance of my object
MyObject obj = new MyObject();
// obj.setId1(i);
// obj.setId2(i);
// obj.set...

StringBuffer cmd = new StringBuffer();
cmd.append("insert into mytable (").
append("id1,").
append("id2,").
append("..."). // append all other columns
append(") values (");
for (int j = 0; j < 53; j++) {
if (j != 0) cmd.append(",");
cmd.append("?");
}
cmd.append(")");

String s = cmd.toString();
PreparedStatement stmt = cnn.prepareStatement(s);
stmt.setInt(1, obj.getId1());
stmt.setInt(2, obj.getId2());
stmt.setString(3, ...); // set all columns

stmt.execute();
stmt.close();

if (endTrans) {
cnn.commit();
transOpen = false;
}
}

if (transOpen) {
cnn.commit();
}

long endTime = System.currentTimeMillis();
System.out.print("testInsertJDBC, dt=" + (endTime - startTime));

} finally {
if (cnn != null) cnn.close();
}
}

public void testReadJDBC() throws SQLException {
Connection cnn = connectJDBC();
try {

long startTime = System.currentTimeMillis();

for (int i = 0; i < nbOfObjs; i++) {

// create an instance of my object
MyObject obj = new MyObject();
// obj.setId1(i);
// obj.setId2(i);

StringBuffer cmd = new StringBuffer();
cmd.append("select ").
append("id1,").
append("id2,").
append("..."). // append all other columns
append(" from mytable where ").
append(" id1=").append(i).append("and id2=").append(i);

Statement stmt = cnn.createStatement();
ResultSet rs = stmt.executeQuery(cmd.toString());
while (rs.next()) {
for (int j = 0; j < 53; j++) {
rs.getObject(j+1);
}
}

}

long endTime = System.currentTimeMillis();
System.out.print("testReadJDBC, dt=" + (endTime - startTime));

} finally {
if (cnn != null) cnn.close();
}
}

public static void main(String[] args) {
MainTest m = new MainTest();
try {
Connection cnn = m.connectJDBC();
Statement stmt = cnn.createStatement();
stmt.execute("delete from mytable");
stmt.close();
cnn.close();

if (true) {
m.testInsert();
//m.testRead();
}

if (false) {
m.testInsertJDBC();
//m.testReadJDBC();
}
}
catch (SQLException ex) {
System.err.print(ex);
}
catch (HibernateException ex) {
System.err.print(ex);
}
}
}

best regards
wolfgang


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 9:13 am 
Senior
Senior

Joined: Sun Jan 04, 2004 2:46 pm
Posts: 147
This has been asked and answered many times. Look at this and search the forum.

http://blog.hibernate.org/cgi-bin/blosxom.cgi/Gavin%20King/batch.html


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 9:40 am 
Newbie

Joined: Wed Nov 03, 2004 7:24 am
Posts: 8
Location: Stuttgart
Myk, thank you for your answer.

My results proof the opposite of what is explained in the document "Batch processing". Hibernate is "fast" if many objects are inserted. It's slow if only one object is inserted per transaction.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 10:47 am 
Senior
Senior

Joined: Sun Jan 04, 2004 2:46 pm
Posts: 147
OK I'm not sure but I can tell you that internally there is no difference between the JDBC transaction and the hibernate transaction. By default hibernate just delegates the commit and rollbacks to the JDBC transaction it wraps.

You are using the same session object for the entire test so the session will fill up with all the instances you save on it. Hibernate keeps a cache at the session level of all objects attached to it so when one is changed and you call commit() it is automatically updated. This means that for the first call to commit() it just has to do the insert into the db. Next time round it will have one object to insert and the first one still cached. It will dirty check the cached one and do nothing. By the time you're up to 10,000 you've done 9999 + 9998 + 9997 + .... 1 dirty checks which don't need doing.

Easiesy way of saying it is that commit() will call flush() which is the expensive part especially if there's lots of attached instances.

The best way is do use a new session for each transaction or clear() the session after every commit. I'd recommend the former personally, there's no performance hit in creating a session provided you have connection pooling configured correctly.

HTH.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 11:45 am 
Newbie

Joined: Wed Nov 03, 2004 7:24 am
Posts: 8
Location: Stuttgart
Super! Thank you Myk!

I inserted a session.clear() after each transaction.commit().
Now, Hibernet is much faster.

1. Insert
1.1. HIBERNATE: objects per trans / seconds
100 / 46s (before 51s)
10 / 49s (before 106s)
1 / 52s (before 703s)

But the overhead is still significantly greater than 10%, ~100% !
This are the results for direct JDBC access:
100 / 23s
10 / 26s
1 / 30s

regards
Wolfgang


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 3:13 pm 
Senior
Senior

Joined: Sun Jan 04, 2004 2:46 pm
Posts: 147
That's probably fair. You can try changing the jdbc batch size to batch the inserts but this applies to hibernate and jdbc. This should speed both up a fair amount tho.

You have to bear in mind that using hibernate does incur a small amount of overhead and this becomes more pronounced for "faster" operations. A single insert is quick in the scheme of things so the hibernate overhead will appear to be quite large, like the 100% you are seeing. If you took a table with a million rows in, did a select query joining it to other tables and compared the performance the difference would be negligible. The overhead is still there but the time spent in the database doing the query, which will involve a lot of disk IO, will dwarf the overhead so it becomes irrelevant.

For doing bulk insert you CAN use hibernate as it does handle the boring stuff for you, but if speed is paramount you should either use JDBC or even better a bulk load tool such as BCP ( SQLServer ) or SQLLDR ( Oracle ) which are usually several times faster than JDBC.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 03, 2004 6:29 pm 
Beginner
Beginner

Joined: Mon Feb 09, 2004 6:43 am
Posts: 35
inserting data to DB can be very fast under Hibernate. Using Hibernate and Oracle we have achived about 2000 inserts/sec.
But you have to remember about few things:

1) Do not use default connection pool
2) make clear after 100-200 inserts
3) gnerate ID yourself (under Oracle Hibernate do it very very poor)
4) make commit after let say 1000-10000 inserts

After all this changes insert will work very,very fast but we have still perfomance problem with selects :(


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 3:13 am 
Newbie

Joined: Wed Nov 03, 2004 7:24 am
Posts: 8
Location: Stuttgart
Myk,
even if there were a million rows in the table, a database index on the ID columns would provide fast row lookup. Hence, the Hibernate overhead will probably be ~100% for big tables too.

szymic1,
thank you for your reply. This are the results of my test app connected to ORACLE:

1. Insert

1.1. HIBERNATE: objects per trans / seconds
100 / 16s !! faster than JDBC
1 / 43s

1.2. JDBC
100 / 25s
1 / 31s

2. Read

2.1. HIBERNATE: seconds
71s !! very slow

2.2. JDBC
20s

The results confirm your experience.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 5:18 am 
Senior
Senior

Joined: Wed Aug 27, 2003 4:08 am
Posts: 178
Location: Wiesbaden, Germany
szymic1 wrote:
After all this changes insert will work very,very fast but we have still perfomance problem with selects :(


Well, you may have to tune hibernate mapping to do lazy loading,
use proxies ( especially if you have collections )

Sometime small schema changes ( adding index here and there )
does wonders.

I had massive performance problems with outer join queries in my content management system, and was forced to perform selects through my lucene search index , but with strategically placed indexes it got up to 100 times faster

_________________
Got new hibernate xdoclet plugin? http://www.sourceforge.net/projects/xdoclet-plugins/
... Momentan auf der Suche nach neuen Projekt ode Festanstellung....


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 11:12 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
If you want a fair test for select performance, you must compare apples to apples and compare direct JDBC to Hibernate "selct new Foo(...) from ... " style query.

Then you will of course find that HIbernate overhead is trivial, as per website claims.


Top
 Profile  
 
 Post subject: publish the assets to execute "select new Foo()",
PostPosted: Thu Nov 04, 2004 3:56 pm 
Newbie

Joined: Thu Oct 28, 2004 1:33 pm
Posts: 14
Location: Stuttgart
Gavin,
I was following this discussion for some days now...

Although I would love to go with the Hibernate RDB2OO converter, I am getting more and more confused: You and your site tell the world Hibernate is only a bit slower (10%) compared to native JDBC but much more convenient to use. Convenience is of course fine and saves development time in the short run! On the other side, some statements above tell me a deployed Hibernate test app is getting horribly slow for real world select statements of bigger amounts of records. So I fear that short term development convenience is paid by long term performance penalties and end users having cups of Java coffee while waiting for results?

You see: Developers and decision takers are told "the tests published on the site prove the just 10% slower than JDBC" performance numbers of Hibernate compared to JDBC. Unfortunately, the tests are just not comparing apples to other fruits, but make Hibernate looking much better than reality tells afterwards.

In your last post, you mention another "select new Foo()" test the Hibernate customers can apply himself to prove what your site tells us.

Can you please publish the data structures and indexes used, the Java code and the corresponding response time result numbers ? (As mentioned above, the tests published so far do not really help.)

Thanks for help,

Dirk V. Schesmer
Stuttgart/Germany


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 4:01 pm 
Hibernate Team
Hibernate Team

Joined: Mon Aug 25, 2003 9:11 pm
Posts: 4592
Location: Switzerland
What is a "Hibernate RDB200 converter"?

_________________
JAVA PERSISTENCE WITH HIBERNATE
http://jpwh.org
Get the book, training, and consulting for your Hibernate team.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 4:54 pm 
Newbie

Joined: Thu Oct 28, 2004 1:33 pm
Posts: 14
Location: Stuttgart
christian wrote:
What is a "Hibernate RDB200 converter"?


It converts 'RDB' modeling concepts and artifacts contained in Relational Databases (like table, view, colum and key) to ('2') Object Oriented ('OO') concepts and artifacts (classes, interfaces, instances and fields).


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 5:03 pm 
Pro
Pro

Joined: Mon Sep 08, 2003 4:30 pm
Posts: 203
gavin wrote:
If you want a fair test for select performance, you must compare apples to apples and compare direct JDBC to Hibernate "selct new Foo(...) from ... " style query.

Then you will of course find that HIbernate overhead is trivial, as per website claims.


Gavin,

Those who use Hibernate they use mostly session.load() to retrieve, for instance, a large subgraph, like a parent with all it's children. This is not doable with "selct new ..." .

So, we are stuck with very poor performance, if we really want to use Hibernate and not JDBC. And telling people that they should use JDCB query (or "select new Foo()...") if they want performance, defeats the purpose of Hibernate.

Couldn't anything be done to improve performance? It is really slow when doing session.load() on large data models.

Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 04, 2004 5:05 pm 
Hibernate Team
Hibernate Team

Joined: Mon Aug 25, 2003 9:11 pm
Posts: 4592
Location: Switzerland
What is this? A thread on JavaLobby? Guys, "Hibernate is slow when session.load()" is NOT how you describe the bugs in your code.

Please stick to the rules!

_________________
JAVA PERSISTENCE WITH HIBERNATE
http://jpwh.org
Get the book, training, and consulting for your Hibernate team.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 36 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.