it looks like comments are disabled on the blog, or blogpost is having issues. So adding here some thoughs:
1) I liked the historycal insight a lot, it was nice to see this collection of references; a good amount of directions about what everybody should consider.
2) I don't consider it meaningful to measure the number of SQL statements performed per unit of time; as mentioned in many places by experienced people, the ORM "overhead" will hardly be a bottleneck in any real world scenario. What is really affecting real world scenario is the choice of statements, the working of caching, the configuration flexibility, the options to fall-back to JDBC in case of need. It's easy to understand that producing a different kind of JOIN statement could heavily affect the system performance; also in some cases a clever JPA implementation might use less statements than another to perform the business logic and perform better because of this, or others might perform better because they are instead generating more statements, but are simpler for the database to optimize, or have a better cache hit-ration and so perform much better.
It would be great to see a benchmark using a big data set and not measuring the number of statements but the performance of "business methods" using JPA. How many blogposts can I insert per second using Hibernate? So as everybody should do already, you should stress-test your final applications and use your real world data to see what performs better in your specific case.
3) Comparing JPA to JDBC has another issue: they are two different things. I hardly believe the "business layer" is going to interact with ResultSets - the maintenance cost would become very high - so in the JDBC use case to be fair you should still introduce a minimal mapping to minimal DTOs; and think about a fairly complete and flexible code .. not worth to try creating such a benchmark, as every line of code you introduce is hardly similar to what others would do.
4) Business's point of view: performance/$$$: The best performance benchmark should consider that producing software has a cost, it takes time and there's a risk factor associated with custom-build code, and maintenance costs. Let's say you have a budget of X$ to build a new application having a performance requirement. Assuming that using Hibernate you'll have more productive developers, and a less costly overall maintenance, you're saving some part of X compared to the cost of building the same app using direct JDBC, this money is likely able to buy you more hardware and largely compensate for any possible performance issues - taking into consideration that for the 20% borderline critical performance issues you might have when using an ORM you can always fall back to plain SQL, and you can still reuse resource pools and object mappers provided by Hibernate to minimize the effort, the risks in your code and the amount of things to maintain and test. Not to mention that a simpler to maintain software will get you a better time-to-market in future for requirement changes and new features! Of course, that's all hard to "benchmark scientifically", just to point out that using an ORM just wins compared to JDBC, whatever you might measure in any benchmark.
A completely different thing is to find out changes we could apply to Hibernate to improve it even more, that's always interesting to discuss and well-proven improvements are very likely to be accepted.
_________________ Sanne http://in.relation.to/
|