Hi guys,

Need some answers to a weird problem here..

I came up with a c# console application to import records from 1 DB to another DB (which uses nhibernate).

The application first selects all records from DB (using ado.net) and then constructs nhibernate objects based on the records and starts saving or updating those objects into the current DB (using nhibernate)

I have also made sure that each step of the whole operation prints out something on the console so that I know its running properly.

Everything runs fine when i execute the application but my question here is, when I first started running it, it imports the records at the rate of 5 record per second.. and as time goes on, the speed decreases exponentially. After the first 10k records, the speed has decreased to 1 record per 30 seconds.

profiling the application tells me that most of the time was spent on nhibernate api functions... task manager tells me the console app is working very hard.. (97% all the time during execution)

I will gladly attach any configuration / source code if it helps to solve this problem...

this is the current configuration code that i have-->

<?xml version="1.0" encoding="utf-8" ?>
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2" >
<session-factory name="*****">
<!-- properties -->
<property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
<property name="connection.driver_class">NHibernate.Driver.SqlClientDriver</property>
<property name="connection.connection_string">data source=****;initial catalog=****;user=****;password=****</property>
<property name="hibernate.connection.isolation">ReadCommitted</property>
<property name="hibernate.default_schema">****.dbo</property>
<property name="show_sql">false</property>
<property name="dialect">NHibernate.Dialect.MsSql2000Dialect</property>
<property name="use_outer_join">true</property>
<property name="query.substitutions">true 1, false 0, yes 'Y', no 'N'</property>
<!-- mapping files -->
<mapping assembly="*****" />
</session-factory>
</hibernate-configuration>

a typical save of objects uses the following code -->

public static bool Save(BizObj bizObj, bool useNewDbSessionTransaction)
{
//if (BizObjTypeHasFlagDelete(bizObj.GetType()))
// bizObj.FlagDelete = 0;

ITransaction tx = null;
try
{
ISession session = NHibernateHttpModule.CurrentSession;
if (useNewDbSessionTransaction)
tx = session.BeginTransaction();

session.Save(bizObj);

if (useNewDbSessionTransaction)
{
tx.Commit();
session.Refresh(bizObj);
}
}
catch (Exception ex)
{
if (tx != null) tx.Rollback();
// handle exception
log.Error(ex.Message);
log.Error("bizObjType : " + bizObj.GetType().FullName + "\n" + ex.StackTrace);
return false;
}

MyClassLibrary.manager.AuditLogManager.InsertToAuditTrail(bizObj, ChangeTypeValue.add, useNewDbSessionTransaction);

return true;
}

Is there any settings i can configure to improve the speed?

Hi Ioba,

i think NHibernate is not the appropriate tool for working with mass data. In your case i would try to implement the import job via plain ADO.NET.

If you are using .NET 2.0 and NHibernate 1.2 you may try to set hibernate.adonet.batch_size to a non-zero value (see NHibernate reference, chapter 15.6).

Regards
Klaus

Yes, i'm using .net 2.0 and nhibernate 1.2 but batch fetching doesnt apply here. The only time when NHibernate is being used is after record has been converted to nHibernate objects and save or updated record by record. As this was done on a per record basis, there's no need for batch transactions of multiple updates.. (IF i'm getting the idea of how it works here correctly)

I think the real issue is that you are leaving your objects in the Session the entire time. NHibernate now must manage all new objects it is saving as well as all objects which were previously saved.

After you have saved and flushed the records to the database make sure you remove them from the session before moving on. Ensuring that the number of objects in the session remains relatively small will drastically improve performance. Try performing your save operations in batches of 500 or maybe 1000 records, then discard the objects and continue.

NHibernate slows down fast as the number of objects in the session grows large.

Hi,

does anyone know how to discard objects in the session?

That's ISession.Evict(Object obj).