-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

View unanswered posts | View active topics


Board index » Hibernate & Java Persistence » Hibernate Users

All times are UTC - 5 hours [ DST ]


Forum locked This topic is locked, you cannot edit posts or make further replies.  Page 1 of 1
  [ 7 posts ] 
  Previous topic | Next topic 
Author Message
tnleeuw
 Post subject: Question about concurrency
PostPosted: Wed May 19, 2004 11:48 am 
Newbie

Joined: Wed Apr 28, 2004 6:44 am
Posts: 15
Location: Amsterdam, Netherlands
Hi,

I have two related classes, Job and Task. Although there's no relation from Job to Task, there's a many-to-one relation from Task to Job. (IE, there's a 1-to-many relationship here but only 1 side has been modelled.)

There are many tasks, and each task operates in it's own thread. The Job also operates in a thread of it's own.
The task records are updated only by the owning task-thread, however the Job record can be updated by any thread. (The Job thread can update a Task record in the database only on creation, when the task-thread doesn't exist, or when it concludes the task-thread has died.)

I'm expecting some degree of contention on the Job record, however I'm getting deadlocks and StaleObjectExceptions on the TASK record from the task-threads!!

Is it possible that when I update the job in thread A, and then try to update the task in thread B, that thread B then gets an error because it sees that the job has been updated? And that the exception then mentions the Task instead of the Job?

I'm really stumped on the cause of these exceptions.

I'm using Hibernate 2.1.3, java-runtime is JDK1.3.1 from IBM's WAS5.0.2.2.

From the mapping document:

Code:
   <class name="BatchJob" table="BATCH_JOB">
      <id
         column="JOBIDX"
         name="jobIndex"
         type="int"
         unsaved-value="0"
      >
         <generator class="native" />
      </id>
      <version
         column="SFTLCK"
         name="softLock"
         type="int"
         unsaved-value="undefined"
       />

      <property
         column="CTRENGREF"
         length="20"
         name="contractEngineRef"
         not-null="true"
         type="string"
       />
      <property
         column="CMPREF"
         length="20"
         name="companyRef"
         not-null="true"
         type="string"
       />
      <property
         column="COLDTE"
         name="collectionDate"
         not-null="true"
         type="java.sql.Date"
       />
      <property
         column="BEGREF"
         length="20"
         name="beginRef"
         not-null="false"
         type="string"
       />
      <property
         column="ENDREF"
         length="20"
         name="endRef"
         not-null="false"
         type="string"
       />
      <property
         column="BEGTIM"
         name="beginTime"
         not-null="true"
         type="timestamp"
       />
      <property
         column="ENDTIM"
         name="endTime"
         not-null="false"
         type="timestamp"
       />
      <property
         column="STA"
         length="20"
         name="status"
         not-null="true"
         type="string"
       />
      <property
         column="TSKHBTTIMITV"
         name="taskHeartBeatTimeInterval"
         not-null="true"
         type="int"
       />
      <property
         column="TSKMAXREF"
         name="taskMaxRef"
         not-null="true"
         type="int"
       />
      <property
         column="TSKMAXDUR"
         name="taskMaxDuration"
         not-null="true"
         type="int"
       />
      <property
         column="JOBCHKTSKTIMITV"
         name="jobCheckTaskTimeInterval"
         not-null="true"
         type="int"
       />
      <property
         column="JOBMAXTSK"
         name="jobMaxTasks"
         not-null="true"
         type="int"
       />
      <property
         column="TSKREG"
         name="tasksRegistered"
         not-null="false"
         type="int"
       />
      <property
         column="TSKINI"
         name="tasksInitiated"
         not-null="false"
         type="int"
       />
      <property
         column="TSKACV"
         name="tasksActive"
         not-null="false"
         type="int"
       />
      <property
         column="JMSMSG_ID"
         length="50"
         name="jmsMessageID"
         not-null="true"
         type="string"
       />
      <property
         column="HBTTIM"
         name="heartBeatTimestamp"
         not-null="true"
         type="timestamp"
       />
      <property
         column="RTR"
         name="retries"
         not-null="false"
         type="int"
       />
      <property
         column="REVWATPER"
         name="reversalWaitPeriod"
         not-null="true"
         type="int"
       />
   </class>
   <class name="BatchTask" table="BATCH_TASK">
      <id
         column="TSKIDX"
         name="taskIndex"
         type="int"
         unsaved-value="0"
      >
         <generator class="native" />
      </id>
      <version
         column="SFTLCK"
         name="softLock"
         type="int"
         unsaved-value="undefined"
       />

      <many-to-one
         column="JOBIDX"
         name="job"
         class="BatchJob"
         not-null="true"
         cascade="none"
       />

      <property
         column="CTRENGREF"
         length="20"
         name="contractEngineRef"
         not-null="true"
         type="string"
       />
      <property
         column="CMPREF"
         length="20"
         name="companyRef"
         not-null="true"
         type="string"
       />
      <property
         column="BEGREF"
         length="20"
         name="beginRef"
         not-null="true"
         type="string"
       />
      <property
         column="ENDREF"
         length="20"
         name="endRef"
         not-null="true"
         type="string"
       />
      <property
         column="BEGTIM"
         name="beginTime"
         not-null="true"
         type="timestamp"
       />
      <property
         column="ENDTIM"
         name="endTime"
         not-null="false"
         type="timestamp"
       />
      <property
         column="POLDONCNT"
         name="nrOfPoliciesProcessed"
         not-null="true"
         type="int"
       />
      <property
         column="TSKSELREF"
         name="taskSelectedRef"
         not-null="true"
         type="int"
       />
      <property
         column="TSKMAXREF"
         name="taskMaxRef"
         not-null="true"
         type="int"
       />
      <property
         column="STA"
         length="20"
         name="status"
         not-null="true"
         type="string"
       />
      <property
         column="JMSMSG_ID"
         length="50"
         name="jmsMessageID"
         not-null="true"
         type="string"
       />
      <property
         column="CURREF"
         length="20"
         name="currentRef"
         not-null="false"
         type="string"
       />
      <property
         column="HBTTIM"
         name="heartBeatTime"
         not-null="true"
         type="timestamp"
       />
      <property
         column="RTR"
         name="retries"
         not-null="false"
         type="int"
       />
   </class>


Exception stack trace:
Code:
net.sf.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) for com.unisys.pdg.common.data.BatchTask instance with identifier: 45
   at net.sf.hibernate.persister.AbstractEntityPersister.check(AbstractEntityPersister.java(Compiled Code))
   at net.sf.hibernate.persister.EntityPersister.update(EntityPersister.java(Compiled Code))
   at net.sf.hibernate.persister.EntityPersister.update(EntityPersister.java(Compiled Code))
   at net.sf.hibernate.impl.ScheduledUpdate.execute(ScheduledUpdate.java(Compiled Code))
   at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java(Compiled Code))
   at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java(Inlined Compiled Code))
   at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java(Compiled Code))
   at com.unisys.pdg.common.dao._RootDAO.update(_RootDAO.java(Compiled Code))
   at com.unisys.pdg.common.dao.base.BaseBatchTaskDAO.update(BaseBatchTaskDAO.java:96)
   at com.unisys.pdg.common.batch.AbstractBatchTaskBean.doProcessPolicy(AbstractBatchTaskBean.java:77)
   at com.unisys.pdg.gi.batch.EJSLocalStatelessGiBatchTask_4106ec18.doProcessPolicy(Unknown Source)
   at com.unisys.pdg.common.batch.AbstractReceiveTaskMessageBean$2.perform(AbstractReceiveTaskMessageBean.java:139)
   at com.unisys.pdg.common.batch.AbstractBatchMessageBean.performWithRetries(AbstractBatchMessageBean.java:61)
   at com.unisys.pdg.common.batch.AbstractReceiveTaskMessageBean.handleMessage(AbstractReceiveTaskMessageBean.java:133)
   at com.unisys.pdg.common.jms.ReceiveMessage.onMessage(ReceiveMessage.java:297)
   at com.ibm.ejs.jms.listener.MDBWrapper$PriviledgedOnMessage.run(MDBWrapper.java:205)
   at java.security.AccessController.doPrivileged(Native Method)
   at com.ibm.ejs.jms.listener.MDBWrapper.callOnMessage(MDBWrapper.java:194)
   at com.ibm.ejs.jms.listener.MDBWrapper.onMessage(MDBWrapper.java:172)
   at com.ibm.mq.jms.MQSession.run(MQSession.java:1043)
   at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:922)
   at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
   at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
   at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
   at java.lang.reflect.Method.invoke(Native Method)
   at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
   at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
   at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
   at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
   at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:593)


As you can see, it's all running inside IBM's websphere... (So I'm not managing my own threads, but WebSphere manages my threads, each task is an MDB instance).

Does anyone have a clue what can be going on here? To what extent can it be Hibernate, to what extent am I mis-understanding concurrency and softlocks in Hibernate? And to what extent is it WebSphere doing bad things and tripping me up?
I'm really stumped here :-(

With regards,

--Tim
Top
 Profile  
 
baliukas
 Post subject:
PostPosted: Wed May 19, 2004 12:59 pm 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
Create new session/transaction per message. One of simple ways is to implemet this infrastucture in base class.

Quote:
public abstract class AbstractMessageListener implements MessageListener{

public void onMessage(Message message)
{
Session session = newSession();
try{
execute(session,message);
}finally{
session.close();
}

}


protected abstract void execute(Message message);

}


You can handle exeptions, demarcate transaction, acknowledge message in this class too.

BTW messages are passed serially to the listenerin the same JMS session, this can help if concurent execution is not a priority (It is the most trivial way to fight with deadlocks)
Top
 Profile  
 
tnleeuw
 Post subject:
PostPosted: Wed May 19, 2004 4:51 pm 
Newbie

Joined: Wed Apr 28, 2004 6:44 am
Posts: 15
Location: Amsterdam, Netherlands
Hi Baliukas,

Sorry, but already done :-( New transaction, with new Hibernate session and terminated by a session.close() for the onMessage, and for calls to session bean methods... Part of the original design.

There are multiple concurrent receivers on the MDBs btw, so there is definately a degree of paralellism.

thanks for the advice,

--Tim
Top
 Profile  
 
baliukas
 Post subject:
PostPosted: Thu May 20, 2004 1:21 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
One of trivial ways to solve this problem is to retrie transaction:

Quote:
public final void onMessage(Message message)
{


for(int i=0; i< MAX_ITERATIONS; i++ ){
tx.begin();
Session session = newSession();
try{

execute(session,message);
tx.commit();
return;

}catch(DeadlockException de){
tx.abort();
}finally{
session.close();
}


}
//ERROR

}


It must be a new transaction per iteration. I think the best way to fight with deadlocks is in DB schema, but It can be not trivial to design deadlock free shema and transactions if it very complex.

BTW: I see from stack trace you are doing somethihg like this too, do not you ?

Quote:
at com.unisys.pdg.common.batch.AbstractBatchMessageBean.performWithRetries(AbstractBatchMessageBean.java:61)
at com.unisys.pdg.common.batch.AbstractReceiveTaskMessageBean.handleMessage(AbstractReceiveTaskMessageBean.java:133


It is very aggressive way ( conservatyve way is to lock tables before the first statement in transaction ), but It must work if implementation is correct "(or unsaved-value mapping was incorrect)"
Top
 Profile  
 
tnleeuw
 Post subject:
PostPosted: Thu May 20, 2004 2:04 am 
Newbie

Joined: Wed Apr 28, 2004 6:44 am
Posts: 15
Location: Amsterdam, Netherlands
baliukas wrote:
One of trivial ways to solve this problem is to retrie transaction:

It must be a new transaction per iteration. I think the best way to fight with deadlocks is in DB schema, but It can be not trivial to design deadlock free shema and transactions if it very complex.

BTW: I see from stack trace you are doing somethihg like this too, do not you ?

Quote:
at com.unisys.pdg.common.batch.AbstractBatchMessageBean.performWithRetries(AbstractBatchMessageBean.java:61)
at com.unisys.pdg.common.batch.AbstractReceiveTaskMessageBean.handleMessage(AbstractReceiveTaskMessageBean.java:133




Yeah, I already do that indeed... Added that recently. But I shouldn't be getting those deadlocks at all, at the Task record. Only on Job, and even that only as an exception.

BTW, the unsaved-value mapping must be right, because it works in general -- just under load it begins to die.

Do you have any advice for me to create deadlock-free schema?
AFAIK, the construction *SHOULD* be deadlock-free because each thread is only updating it's own task-record!

thanks,

--Tim
Top
 Profile  
 
baliukas
 Post subject:
PostPosted: Thu May 20, 2004 3:54 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
"deadlock free shema" implementation sounds very trivial, but it can be not trivial in practice. In theory you need to drop "delete/update" and to use "insert" only and custom garbage collection, archyves. I know two workarounds retie or lock and all of ways can be usefull in practice.
BTW I do almost the same stuff as you, I have used MDB before, but I prefer plain JMS at this time, but all of problems are caused by bugs in application or in external systems, not by concurency control in tools.
Top
 Profile  
 
baliukas
 Post subject:
PostPosted: Thu May 20, 2004 3:59 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
"drop "delete/update", I wanted to say concurent delete/update.
Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  Page 1 of 1
  [ 7 posts ] 

Board index » Hibernate & Java Persistence » Hibernate Users

All times are UTC - 5 hours [ DST ]

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.