-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 
Author Message
 Post subject: CacheEntry huge serialization size - Production problems
PostPosted: Thu May 11, 2006 12:44 pm 
Newbie

Joined: Thu May 11, 2006 11:46 am
Posts: 1
we use in production hibernate second level cache with tangosol coherence cache provider. we have a cluster with 2 weblogic nodes and we choose a replicated cache to synchronize object state between the nodes.
the cache contains 600.000 entries

when we use one node in the cluster, the cache size on the node is 200Mo (size calculated by yourkit profiler) but when we use two nodes the cache size is ~1Go on each node.

after hours profiling and reading hibernate code, i understood that hibernate puts in the cache a CacheEntry object that contains a "dehydrated" / disassemblade state of the entity or in simpler words it contains a serialieable array of entiy's properties.
more over when a node puts an object in the cache, tangosol puts the instance in the current node but it puts the serialized form in the second one.

is there anyone who faced this problem?

is there any version of the CacheEntry class with an optimized serialization or a cache provider that reduces the CacheEntry's serialization size?


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 25, 2006 9:47 am 
Newbie

Joined: Thu May 25, 2006 9:45 am
Posts: 7
hi,

did you find the solution to your problem?

I am using coherence as hibernate plugin and faced the same problem and in my case we have some hundereds of entries in tangosol cache and its showing outofmemory on weblogic console.

Thanks,
-Amit


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 29, 2007 1:16 am 
Newbie

Joined: Tue Jul 20, 2004 8:52 pm
Posts: 6
For anyone else that is searching for this, I just created a version of org.hibernate.cache.entryCacheEntry with the following additions below, and replaced the original Hibernate version with it. A public default contructor is also required.

Useing a modified serialisation test harness from http://forums.tangosol.com/thread.jspa? ... 8&tstart=0 comparing the original and optimised CacheEntry, and setting the dissasembledState as new Serializable[]{"1234","AWS","Platform",new Long(1000),null,Boolean.TRUE,"Yes", null}
the results were

Serialization -> 455 bytes
ExternalizableLite -> 146 bytes

Serialization 100000 times -> 3005ms
ExternalizableLite 100000 times -> 487ms

which is a fairly substantial improvement

This could be optimised further for your usage, I just did the main TYPE_XXX that came up from profiling our app.

Dan

Code:

   public static final byte TYPE_STRING = 0x01;
   public static final byte TYPE_NULL = 0x02;
   public static final byte TYPE_LONG = 0x03;
   public static final byte TYPE_BOOL = 0x04;
   public static final byte TYPE_DOUBLE = 0x05;
   public static final byte TYPE_UNKNOWN = 0x06;
   public static final byte TYPE_INTEGER = 0x07;
   public static final byte TYPE_CHAR = 0x08;
   public static final byte TYPE_TIMESTAMP = 0x09;
   public static final byte TYPE_OTHER = 0x0A;

   public static final byte VERSION_NULL = 0x00;
   public static final byte VERSION_LONG = 0x01;
   public static final byte VERSION_TIMESTAMP = 0x02;
   public static final byte VERSION_OTHER = 0x03;

   public void readExternal(DataInput in) throws IOException
   {
      int length = in.readShort();
      disassembledState = new Serializable[length];
      for (int i = 0; i < length; i++)
      {
         byte type = in.readByte();
         switch (type)
         {
            case TYPE_STRING:
               disassembledState[i] = in.readUTF();
               break;
            case TYPE_NULL:
               break;
            case TYPE_LONG:
               disassembledState[i] = new Long(in.readLong());
               break;
            case TYPE_BOOL:
               disassembledState[i] = Boolean.valueOf(in.readBoolean());
               break;
            case TYPE_DOUBLE:
               disassembledState[i] = Double.valueOf(in.readDouble());
               break;
            case TYPE_UNKNOWN:
               disassembledState[i] = BackrefPropertyAccessor.UNKNOWN;
               break;
            case TYPE_INTEGER:
               disassembledState[i] = Integer.valueOf(in.readInt());
               break;
            case TYPE_CHAR:
               disassembledState[i] = Character.valueOf(in.readChar());
               break;
            case TYPE_TIMESTAMP:
               disassembledState[i] = new Timestamp(in.readLong());
               break;
            default:
               disassembledState[i] = (Serializable)ExternalizableHelper.readObject(in);
               break;
         }
      }

      subclass = in.readUTF();
      lazyPropertiesAreUnfetched = in.readBoolean();

      byte versionType = in.readByte();
      switch (versionType)
      {
         case VERSION_NULL:
            break;
         case VERSION_LONG:
            version = new Long(in.readLong());
            break;
         case VERSION_TIMESTAMP:
            version = new Timestamp(in.readLong());
            break;
         case VERSION_OTHER:
            version = ExternalizableHelper.readObject(in);
            break;
      }
   }

   public void writeExternal(DataOutput out) throws IOException
   {
      out.writeShort(disassembledState.length);
      for (int i = 0; i < disassembledState.length; i++)
      {
         Serializable ser = disassembledState[i];

         if (ser == null)
         {
            out.writeByte(TYPE_NULL);
         }
         else if (ser instanceof String)
         {
            out.writeByte(TYPE_STRING);
            out.writeUTF(((String)ser));
         }
         else if (ser instanceof Long)
         {
            out.writeByte(TYPE_LONG);
            out.writeLong(((Long)ser).longValue());
         }
         else if (ser instanceof Boolean)
         {
            out.writeByte(TYPE_BOOL);
            out.writeBoolean(((Boolean)ser).booleanValue());
         }
         else if (ser instanceof Double)
         {
            out.writeByte(TYPE_DOUBLE);
            out.writeDouble(((Double)ser).doubleValue());
         }
         else if (ser == BackrefPropertyAccessor.UNKNOWN)
         {
            out.writeByte(TYPE_UNKNOWN);
         }
         else if (ser instanceof Integer)
         {
            out.writeByte(TYPE_INTEGER);
            out.writeInt(((Integer)ser).intValue());
         }
         else if (ser instanceof Character)
         {
            out.writeByte(TYPE_CHAR);
            out.writeChar(((Character)ser).charValue());
         }
         else if (ser instanceof Timestamp)
         {
            out.writeByte(TYPE_TIMESTAMP);
            out.writeLong(((Timestamp)ser).getTime());
         }
         else
         {
            out.writeByte(TYPE_OTHER);
            ExternalizableHelper.writeObject(out, ser);
         }
      }

      out.writeUTF(subclass);
      out.writeBoolean(lazyPropertiesAreUnfetched);

      if (version == null)
      {
         out.writeByte(VERSION_NULL);
      }
      else if (version instanceof Long)
      {
         out.writeByte(VERSION_LONG);
         out.writeLong(((Long)version).longValue());
      }
      else if (version instanceof Timestamp)
      {
         out.writeByte(VERSION_TIMESTAMP);
         out.writeLong(((Timestamp)version).getTime());
      }
      else
      {
         out.writeByte(VERSION_OTHER);
         ExternalizableHelper.writeObject(out, version);
      }
   }


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 3 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.