Optimistic locking is appropriate most of the time for my application, but there are certain cases where pessimistic locking is desired.

My Hibernate enabled application is the only mutator of the database, so using the database for pessimistic locking can't be justified, because of the cost of network I/O and database action.

Since this app is the only client of the database, all pessimistic locking should be able to be performed in-memory without any network I/O (assuming a single-JVM solution). In-memory locking should be orders of magnitude faster than going to the database.

Another advantage of in-memory locking is that I may wish to apply pessimistic locking sematics to objects other than database-related objects.

For the case of multiple JVMs, I would need a distributed locking service. Some network I/O is unavoidable in this case, but it still should be possible to avoid any use of disk or databases.

What I'm really looking for is a general purpose distributed synchronization service. One that is optimized for minimal network I/O and no disk I/O.

I once worked with a custom in-house ORM solution that incorporated an extremely fast distributed synchronization service. It utilized lock caches in each JVM to remember previously requested lock states between transactions, so requests for the same locks in subsequent transactions did not incur redundant network I/O. These long life lock caches had the effect that most network I/O would occur only when two or more transactions in different JVMs attempted to lock the same object.

Now I'm looking for a stand-alone open source distributed synchronization/lock service similar to that described above. So far I've been unsuccessfull.

Is anybody here using or know of such a service? Open source, of course.

Thanks.

--Jeff

There is a nice text about this here http://blog.hibernate.org/cgi-bin/blosxom.cgi/2004/01/27#locks

I read Gavin's post. For very large Hibernate-centric, database-centric applications, it would be hard to argue with his points.

On the issue of technology -- a general purpose, scalable distributed synchronization service is definitely a PhD level problem. There are hundreds of papers written by computer scientists on the subject. You'd have to be pretty desperate to consider writing your own full-blown, full-featured service.

But consider the case of small servers that will seldom consist of more than one running JVM instance. In the past five years our group has implemented dozens of these small back-end special purpose servers. They all have a common requirement -- they must exhibit extremely low latency as perceived by the client of the server (usually another server). They also have another thing in common: very complex object models and logic.

Our requirements are such that we scrutinize every single I/O call to the database and try to find ways to eliminate it. We cache enormous amounts of data. We use connection pools that don't have to hit the database or perform any I/O to verify connections (it can be done). We don't even call commit on every transaction, if that transaction only performed selects or only read from the cache. Thus many transactions operate at full memory speed without incurring any I/O whatsoever.

Up to now we've been using our own hand-crafted ORM. It is a bare bones transparent persistence service that offers very few features, and it does not scale well past a single JVM. But it is fast, very fast. And it easily handles 800 concurrent, very low-latency transactions, when running on a large multi-cpu machine. It also does in-memory pessimistic locking at sub-microsecond speeds.

But we've always yearned for a full-featured ORM like Hibernate. But Hibernate was never a contender until it gained the second-level cache and the query cache. These two caches allow us to keep a significant chunk of our object graph in ram, portions of which must appear concurrently in the hundreds of concurrent transactions (so we can't use the attach/detach approach).

Performance testing shows us that Hibernate achieves 85% to 90% overall performance of our custom ORM. It is amazing that a full featured, general purpose ORM can compete with a such a bare-bones, special purpose service. The design of Hibernate and the creative talents of Gavin and company totally blows us away.

There are couple of sticking points. We need pessimistic locking here and there. If we did it using the database, Hibernate would no longer be competitive with our custom ORM. Not even in the same ball park. So we're working on solving this problem (for our specific case of the small server that does not need to scale -- for large, scalable servers, we totally agree with Gavin's philosophy).

We also sometimes have the need to run a small cluster of no more than two or three JVMs. Our custom ORM can do the necessary cache sync and primitive distributed locking orders of magnitude faster than going to Oracle. But it won't scale past four JVMs. So that's the tradeoff.

Hibernate is great and it is scalable to really big applications. Hibernate also scales down really well, but I think more can be done in this direction -- to wring out every gram of performance for the small scale app. Who knows, maybe Hibernate could some day be used in true Real-Time apps, as well as the truly huge apps.

--Jeff

I agree with our points, thats why I implemented the pessimistic offline lock myself with Hibernate. I have to add another reason for locking in the middle tier: If a use case requires a long running exclusive transaction on some objects.

It would be great if you could just open a Wiki page and give your posting a structure.

Well, we already need to solve this problem for CMP (in fact, we pretty much *have* a solution). So, I don't think it is very hard to plug in a locking model (really, all you would need is appropriate callbacks, which I am happy to add).

If you are serious about this, and are happy to provide your own lock manager, I am prepared to add any callbacks you need.

One of the more interesting problems is lock granularity, i.e. how to lock a bunch of objects, not only one by one. For this, the objects have to be in some composite. We have to define this composite (e.g. a "lock group") and we need a mechanism to address this composite in calls to the lock manager.

hmmmm .... are you sure that in-memory locking implementations such as entity beans really do this? I think they probably just acquire a collection of fine-grained locks simultaneously...

I don't know what entity beans do :)

I'm always thinking about a POJO graph. So if we have a lock table, we would not use some object identifier but some composite identifier. Imagine the Item/Bid relationship: We could have metadata that defines this as a lock group, by default. So locking an Item locks all the Bids, which is what we usually want to do. It's more like an optional extension to fine-graind locking...

Hmmm interesting. That is much more sophisticated/complex...

Christian: If we come up with a viable locking solution, I'd be happy to add my notes to the wiki. But any solution we have would only really work for the case of a single running VM with exclusive access to the database. Anything more complicated or more scalable would be beyond the scope of my project (and my budget).

Gavin: I may very well take you up on your offer to add callbacks. Thank you!!

--Jeff

Oh, I have a working exclusive lock manager using Hibernate (it doesn't support the read/write strategy...). It should even work in a cluster, as it uses a database table to synchronize the locks.

The problem is with the database transactions: You would need a serializable transaction to obtain/check a lock in the lock table, but you are usually in the context of an already running database transaction when you aquire a lock. This can be solved for exclusive locks, as we achieve atomicity with a single statement. A read/write lock strategy however requires two SQL statements for a lock check, so we would need the full transaction isolation. Do we start a second serializable transaction on a second database connection? That would break the transaction semantics for the original unit of work...

Interesting stuff.

They may not need db-level locks; it might be enough to simply use memory-level locks.

Well, thats easy, but only in a single VM. A static LockManager would do that.

of course; and it really /is/ quite easy, even to support multiple tx isolation levels

I couldn't wrap my head around that problem: I need a SELECT and (possibly) INSERT in the lock table for the read/write strategy. I'm in a Session and Transaction, with the isolation level used for all database transactions. I need these two operations executed at serializable isolation. How?