Hibernate Community • View topic - system.outofmemoryexception with binaryblob and large file

View unanswered posts | View active topics

Board index » Hibernate & Java Persistence » NHibernate Users

All times are UTC - 5 hours [ DST ]

system.outofmemoryexception with binaryblob and large file

Page 1 of 1

[ 2 posts ]

Previous topic | Next topic

Author

Message

kpatton

Post subject: system.outofmemoryexception with binaryblob and large file

Posted: Sat Jun 03, 2006 8:48 am

Newbie

Joined: Thu Nov 17, 2005 6:37 pm
Posts: 13

We are getting a system.outofmemory exception when saving a 9MB file that has been uploaded by a user to a binaryBlob NH type and SQL Server 2005 image field type. The file uploads fine, but it appears to be getting set into a stringbuilder. I'm wondering how efficient this whole operation is ? The server has 2GB of ram and is windows 2003 server fully patched. We have to do more in depth testing, but was wondering why we may be getting this bottleneck in NH? We need to support large file uploads for logo images that a customer uploads in our application.

Hibernate version:
1.0.1

Mapping documents:

The sub class Image is the one that is relevant here.

<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.0">
<class
name="Logoers.BusinessLogic.File, Logoers.BusinessLogic"
table="dbo.tblFile"
discriminator-value="File"
proxy="Logoers.BusinessLogic.File, Logoers.BusinessLogic">
<id name="ID" type="Int32" unsaved-value="0">
<column name="fileID" sql-type="int" not-null="true" unique="true" index="PK_tblFile"/>
<generator class="native" />
</id>
<discriminator column="fileClassRef" type="string" />
<property name="FileData" type="BinaryBlob">
<column name="fileData" length="2147483647" sql-type="image" not-null="false"/>
</property>
<property name="Name" type="String">
<column name="name" length="200" sql-type="varchar" not-null="true"/>
</property>
<property name="Extension" type="String">
<column name="extension" length="5" sql-type="varchar" not-null="true"/>
</property>
<property name="MimeType" type="String">
<column name="mimeType" length="50" sql-type="varchar" not-null="false"/>
</property>
<property name="Size" type="Int32">
<column name="size" sql-type="int" not-null="false"/>
</property>
<component name="ChangeLog" class="Logoers.BusinessLogic.ChangeLog, Logoers.BusinessLogic">
<property name="CreatedOn" type="DateTime" column="createdOn" not-null="true" />
<property name="CreatedBy" type="String" column="createdBy" not-null="true" />
<property name="LastUpdatedOn" type="DateTime" column="LastUpdatedOn" not-null="true" />
<property name="LastUpdatedBy" type="String" column="lastUpdatedBy" not-null="true" />
</component>
<subclass name="Logoers.BusinessLogic.Image, Logoers.BusinessLogic" discriminator-value="Image" proxy="Logoers.BusinessLogic.Image, Logoers.BusinessLogic">
<property name="Height" type="Int32">
<column name="image_Height" sql-type="int" not-null="false"/>
</property>
<property name="Width" type="Int32">
<column name="image_Width" sql-type="int" not-null="false"/>
</property>
</subclass>
<subclass name="Logoers.BusinessLogic.Document, Logoers.BusinessLogic" discriminator-value="Document" proxy="Logoers.BusinessLogic.Document, Logoers.BusinessLogic">
</subclass>
</class>
</hibernate-mapping>

Code between sessionFactory.openSession() and session.close():

Full stack trace of any exception that occurs:

1) Exception Information
*********************************************
Exception Type: System.Exception
Message: NHibernate SaveOrUpdate Failure: NHibernate Session Flush Failure: Exception of type 'System.OutOfMemoryException' was thrown.
Data: System.Collections.ListDictionaryInternal
TargetSite: Void SaveOrUpdate(System.Object)
HelpLink: NULL
Source: Hyro.NHibernate.Utilities

StackTrace Information
*********************************************
at Hyro.NHibernate.Utilities.SessionFactory.SaveOrUpdate(Object toSaveOrUpdate)
at Logoers.Persistence.EntityPersister.Save(Entity toSave) in C:\VTSProjects\Logoers\Logoers\Logoers.Persistence\EntityPersister.cs:line 32
at Logoers.BusinessLogic.Image.Save() in C:\VTSProjects\Logoers\Logoers\Logoers.BusinessLogic\Image.cs:line 55
at Logoers.BusinessLogic.UserLogo.SaveHiResImage(Image image) in C:\VTSProjects\Logoers\Logoers\Logoers.BusinessLogic\UserLogo.cs:line 105
at Logoers.BusinessLogic.UserLogo.set_HiResImage(Image value) in C:\VTSProjects\Logoers\Logoers\Logoers.BusinessLogic\UserLogo.cs:line 43
at Logoers.AppService.DataSource.UserLogoWorker.Insert(Int32 ID, String savedBy, Int32 clientUserID, String name, String notes, Object image, String pmsInfo, Int32 totalColours) in C:\VTSProjects\Logoers\Logoers\Logoers.AppService\DataSource\UserLogoWorker.cs:line 86

2) Exception Information
*********************************************
Exception Type: System.Exception
Message: NHibernate Session Flush Failure: Exception of type 'System.OutOfMemoryException' was thrown.
Data: System.Collections.ListDictionaryInternal
TargetSite: Void FlushCurrentSession()
HelpLink: NULL
Source: Hyro.NHibernate.Utilities

StackTrace Information
*********************************************
at Hyro.NHibernate.Utilities.SessionFactory.FlushCurrentSession()
at Hyro.NHibernate.Utilities.SessionFactory.SaveOrUpdate(Object toSaveOrUpdate)

3) Exception Information
*********************************************
Exception Type: System.OutOfMemoryException
Message: Exception of type 'System.OutOfMemoryException' was thrown.
Data: System.Collections.ListDictionaryInternal
TargetSite: System.String GetStringForStringBuilder(System.String, Int32, Int32, Int32)
HelpLink: NULL
Source: mscorlib

StackTrace Information
*********************************************
at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
at System.Text.StringBuilder.Append(String value)
at NHibernate.Util.CollectionPrinter.ToString(IDictionary dictionary)
at NHibernate.Impl.Printer.ToString(Object entity)
at NHibernate.Impl.Printer.ToString(IEnumerator enumerator)
at NHibernate.Impl.SessionImpl.FlushEverything()
at NHibernate.Impl.SessionImpl.Flush()
at Hyro.NHibernate.Utilities.SessionFactory.FlushCurrentSession()

Name and version of the database you are using:
SQL Server 2005

Top

anem

Post subject: workaround, use a IUserType

Posted: Fri Sep 28, 2007 6:57 am

Newbie

Joined: Tue Jan 09, 2007 2:07 pm
Posts: 1

I also experienced this problem, however with data around 20Mb but also mapped as BinaryBlob (nHibernate v1.2.0) and varbinary(max) on SQL Server 2005.

The workaround for me was to create an IUserType where the code basically is the implementation I found for BinaryBlob and its super classes. With this implementation I've tried with data up 100Mb, around this level I get DB timeout problems but no out of memory exceptions.

Here the code for the IUserType:

Code:

using System;
using System.Collections.Generic;
using System.Text;
using System.Data;
using NHibernate;
using NHibernate.SqlTypes;
using NHibernate.Util;


namespace Elements.DataPersistanceManager
{
    public class UserBlob : NHibernate.UserTypes.IUserType
    {
        public UserBlob()
        {
        }

        private BinarySqlType sqlType = new BinarySqlType(2147483647);

        #region IUserType Members

        public object Assemble(object cached, object owner)
        {
            if (cached == null)
            {
                return null;
            }
            else
            {
                return DeepCopy(cached);
            }
        }

        public object DeepCopy(object value)
        {
            return (value == null) ? null : DeepCopyNotNull(value);
        }

        private object DeepCopyNotNull(Object value)
        {
            byte[] bytes = (byte[])value;
            byte[] result = new byte[bytes.Length];
            Array.Copy(bytes, 0, result, 0, bytes.Length);
            return result;
        }

        public object Disassemble(object value)
        {
            if (value == null)
            {
                return null;
            }
            else
            {
                return DeepCopy(value);
            }
        }

        public int GetHashCode(object x)
        {
            unchecked
            {
                byte[] bytes = (byte[])x;
                int hashCode = 1;
                for (int j = 0; j < bytes.Length; j++)
                {
                    hashCode = 31 * hashCode + bytes[j];
                }
                return hashCode;
            }
        }

        public bool IsMutable
        {
            get { return true; }
        }

        public object NullSafeGet(System.Data.IDataReader rs, string[] names, object owner)
        {
            int index = rs.GetOrdinal(names[0]);
            int length = (int)rs.GetBytes(index, 0, null, 0, 0);
            byte[] buffer = new byte[length];

            int offset = 0;

            while (length - offset > 0)
            {
                int countRead = (int)rs.GetBytes(index, offset, buffer, offset, length - offset);
                offset += countRead;

                if (countRead == 0)
                {
                    // Should never happen
                    throw new AssertionFailure("Error in UserBlob.Get, IDataRecord.GetBytes read zero bytes");
                }
            }

            return buffer;
        }

        public void NullSafeSet(System.Data.IDbCommand cmd, object value, int index)
        {
            ((IDataParameter)cmd.Parameters[index]).Value = (byte[])value;
        }

        public object Replace(object original, object target, object owner)
        {
            return DeepCopy(original);
        }

        public Type ReturnedType
        {
            get { return typeof(byte[]); }
        }

        public NHibernate.SqlTypes.SqlType[] SqlTypes
        {
            get { return new SqlType[] {sqlType}; }
        }

        public new bool Equals(object x, object y)
        {
            if (x == y)
            {
                return true;
            }
            if (x == null || y == null)
            {
                return false;
            }

            return CollectionHelper.CollectionEquals((byte[])x, (byte[])y);
        }


        #endregion
    }
}

and the equivalent mapping (the length affects the type of db type that is generated when I generate a schema according to the mapping. I use sql server 2005 and sql server compact edition. 8001 gives me nvarbinary(max) and image respectively):