We’ve had a problem that we’ve been trying to solve for about a week and I am out of ideas. So I’m posting this to the NHibernate and a MS database forum.
In a nutshell, we periodically get the following message on one of our servers, but not anywhere else:
NHibernate.Exceptions.GenericADOException: could not insert: [CHO.Next.Global.DataObjects.EventTime][SQL: INSERT INTO CHONext.dbo.EventTime (StartTime, EndTime, StartRegDate, EndRegDate, EventId, NextEventTimeId, RecurrenceGroup) VALUES (?, ?, ?, ?, ?, ?, ?); select SCOPE_IDENTITY()] --->
System.Data.SqlClient.SqlException: Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.
It does not seem to be specific to any one (insert, delete, etc.), which will sometimes pass, but message is always about distributed transactions. It is completely inconsistent and hard to reproduce. Sometimes it will work for a few hours and then suddenly every operation will fail.
Architecture notes:
-This is an NT Service which uses MSMQ. So each incoming message used to spawn a new thread, although we’ve just changed it to only have a single thread running at a time to make sure it’s not a threading issue.
-We make almost the same calls through a website and they always work. So it is something specific to running as a service.
-All of the data access is through NHibernate, so I can’t confirm it happens with normal ADO.Net calls.
-One difference compared to other working environments is that the database and web server are on different domains.
-There are a bunch of posts out there about Distributed Transaction Controller settings and I’ve gone through them each one I can find with a fine toothed comb and followed each ones advice if applicable. There is no firewall between the servers, DTC is setup for no No Authentication, I’ve used DTCPing and DTCTester and they pass.
Any thoughts or suggestions are appreciated.
|