-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 30 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: FastClassByCGLIB classes won't get garbage collected
PostPosted: Tue Apr 13, 2004 2:41 pm 
Newbie

Joined: Tue Apr 13, 2004 9:54 am
Posts: 2
Environment:

Hibernate: 2.1.2
OS: Windows 2000 5.0
Tomcat: 5.0.19
JVM: SUN 1.4.2_03

Using DBCP connection pooling against Oracle.

While running several of our Hibernate applications through profilers (we've used OptimizeIt, JProfiler and WSAD Profile) we noticed that after our application has been stopped and garbage collection has occurred, the dynamic CGLIB objects MyClass$$FastClassByCGLIB$$ aren't getting garbage collected. We'll have one of these objects for each class mapped to a table. We have also seen this when profiling in WSAD against the WebSphere 5.0 Test Environment.

If we restart our application in the container we get duplicates of these objects.

Showing references to these objects indicate they are owned by net.sf.cglib.reflect.FastClass$Generator via the static field SOURCE.

While these are small objects, our development environment tends to get littered with them quickly as developers repeatedly restart their applications.

Does anyone have any advice on how these dynamically generated classes can get cleaned up - or - can anyone explain why they cannot be even when the application is stopped?

Thank you.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 13, 2004 3:48 pm 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
This is a bug in cglib cache. I will fix it today.


Top
 Profile  
 
 Post subject: has this been fixed
PostPosted: Thu May 20, 2004 4:33 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Has this issue been fixed !

i downloaded Hibernate 2.1.3 (25.04.2004) yesterday !


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 20, 2004 5:57 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
Yes, it is. cglib-2.0.1 uses more weak cache, it must help if cglib is in server/lib. If cglib is in the same class loader as application then probably it will not help (GC must drop cglib itself, if container dereferences classloader). If it doe's not work as I expect then it will be possible to add some static method "clearCache(ClassLoader loader)" to help GC and container, you can test it yourself if you can reproduce this problem.


Top
 Profile  
 
 Post subject: error with new file.
PostPosted: Thu May 20, 2004 6:41 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Team

I get the following error when i put the new jar(2.1.3) file in the web application classpath and removed the old file(2.0.2).

This is happening at application startup with the new version

java.lang.NoClassDefFoundError: net/sf/cglib/beans/BulkBeanException
at net.sf.hibernate.util.ReflectHelper.getBulkBean(ReflectHelper.java:170)
at net.sf.hibernate.type.ComponentType.<init>(ComponentType.java:110)
at net.sf.hibernate.cfg.Binder.bindComponent(Binder.java:899)
at net.sf.hibernate.cfg.Binder.bindRootClass(Binder.java:287)
at net.sf.hibernate.cfg.Binder.bindRoot(Binder.java:1243)
at net.sf.hibernate.cfg.Configuration.add(Configuration.java:249)
at net.sf.hibernate.cfg.Configuration.addFile(Configuration.java:171)

The zip file for 2.1.3 has a src folder that does not contain the net/sf/cglib package as far as i can see. Even the hibernate jar file does not have that
package? It is a 14 MB download from here:-

http://prdownloads.sourceforge.net/hibe ... &sort=desc

hibernate-2.1.3.zip

Does this mean that i should put cglib-full-2.0.1.jar (this comes with the distribution in the lib directory), in the server classpath ? And the hibernate jar file in the web app classpath?

Thanks in advance.

Regards
suchak jani


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 20, 2004 7:00 am 
Hibernate Team
Hibernate Team

Joined: Tue Sep 09, 2003 2:10 pm
Posts: 3246
Location: Passau, Germany
You should put the hibernate jar and all jars required by hibernate in the web-inf/lib folder. And remove all old jars from there.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 20, 2004 11:14 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
cglib 2.0.1 must work with any deployment, 2.0 causes memory leak after redeploy if class loader for generated classes and cglib core classes is not the same.


Top
 Profile  
 
 Post subject: Still not being garbage collected !
PostPosted: Fri Jun 04, 2004 8:10 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Team,

I now have the hibernate 2.1.3 and the cglib full 2.0.1.

Even now there are two clasess per mapping class being generated at startup and not being garbage collected at all.

Myclass$$BulkBeanByCGLIB$$
Myclass$$FastClassByCGLIB$$


This is keeping my heap size very high, i can see 16 24+ bytes per such class in optimizeit.

I know that this is the case and i can send the heap dump to anyone interested by email.

I have put the cglib jar in the server library class path and hibernate jar in the web app classpath.

Baliukas
Maybe we do need a static method "clearCache(ClassLoader loader)" . Please advise !

Here are my tested environments

IBM JDK1.3.1
Tomcat 4.1.27- Windows
Websphere 5.0 - Windows/AS400

Regards
Suchak Jani


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 04, 2004 8:13 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Just as a note ,

I have deleted the cglib-asm.jar from all the systems.

Regards
Suchak Jani


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 04, 2004 9:28 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
I use this test (it simulates redeploy), if you can reproduce "OutOfMemory" with cglib depencies only then I can fix it.

Code:
   


ClassLoader loader =
                          new ClassLoader(this.getClass().getClassLoader()){};
       
        FastClass.create(loader,Simple.class).newInstance();
       
      java.lang.ref.Reference ref =
                                         new java.lang.ref.WeakReference(loader);
       
        loader = null;
        java.util.List list = new  java.util.ArrayList();
       
        for(int i = 0; i < 512; i++  ){
           
            System.gc();
           
            if(ref.get() == null ){
             
                return false;
               
            }
           
          byte[] garbage  =  new byte[ (i + 1)*1004 ];
          list.add(garbage); 
       
         
        }
       
        return true;


Top
 Profile  
 
 Post subject: The Cglib full 2.0.1 jar file
PostPosted: Fri Jun 04, 2004 9:54 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Team,

As far as i can see , the cglib full 2.0.1 jar file has the following classes that reference other jars that are not a part of hibernate 2.1.3 source distribution lib directory and neighter are a part of the binary cglib full 2.0.1 jar.

net.sf.cglib.core.DebuggingClassWriter has :
Quote:
import org.objectweb.asm.util.TraceClassVisitor;

From - asm-util.jar


net.sf.cglib.transform.AbstractClassLoader has :
Quote:
import org.objectweb.asm.util.*;

From - asm-util.jar


net.sf.cglib.transform.hook.AbstractPreProcessor has :
Quote:
import org.codehaus.aspectwerkz.hook.ClassPreProcessor;

From - aspectwerkz-core-0.10.RC1.jar


net.sf.cglib.transform.hook.AsmClassLoaderPreProcessor has :
Quote:
import org.codehaus.aspectwerkz.hook.ClassLoaderPatcher;
import org.codehaus.aspectwerkz.hook.ClassLoaderPreProcessor;

From - aspectwerkz-core-0.10.RC1.jar


Again the above jars are not a part of the hibernate 2.1.3 distribution lib directory nor are are a part of the binary cglib full 2.0.1 jar.

Should we change the xml of the cglib 2.0.1 build.xml file from :

Quote:
<target depends="compile" description="Create binary distribution" name="jar">
<mkdir dir="${dist.home}"/>
<mkdir dir="${build.home}/src"/>
<copy file="LICENSE" todir="${build.home}/classes"/>
<jar basedir="${build.home}/classes" jarfile="${dist.home}/${component.name}-${component.version}.jar" >
<include name="**/*"/>
<exclude name="samples/**" />
</jar>

<jar basedir="${build.home}/classes" jarfile="${dist.home}/${component.name}-full-${component.version}.jar" >
<zipfileset dir="lib" includes="asm*.txt"/>
<zipfileset src="${asm.jar}"/>
<include name="**/*"/>
<exclude name="samples/**"/>
</jar>
</target>



to

Quote:
<target depends="compile" description="Create binary distribution" name="jar">
<mkdir dir="${dist.home}"/>
<mkdir dir="${build.home}/src"/>
<copy file="LICENSE" todir="${build.home}/classes"/>
<jar basedir="${build.home}/classes" jarfile="${dist.home}/${component.name}-${component.version}.jar" >
<include name="**/*"/>
<exclude name="samples/**" />
</jar>

<jar basedir="${build.home}/classes" jarfile="${dist.home}/${component.name}-full-${component.version}.jar" >
<zipfileset dir="lib" includes="asm*.txt"/>
<zipfileset src="${asm.jar}"/>
<zipfileset src="${asm-util.jar}"/>
<zipfileset src="${aspectwerkz-core-0.10.RC1.jar}"/>

<include name="**/*"/>
<exclude name="samples/**"/>
</jar>
</target>


This will ensure that the above four files , if they are important have the valid jars in the class path as far as cglib 2.0.1 is concerned.

I do not know if this would solve the problem of the classes not being garbage collected as described in the above posts though,

Regards
Suchak Jani


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 04, 2004 10:08 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
It will not solve any problems, experimental features and cglib development tools depend on asm-util.jar . There is more serious redeployment problem on Orion, I am not sure it is related, but I need help to reproduce it. It is possible to use workaround (clear cache in context listener), but I want to be sure there is no good way to fix it first.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 04, 2004 10:20 am 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Baliukas

Quote:
FastClass.create(loader,Simple.class)


Fastclass in the version of 2.0.1 src i have does not have a create method with (ClassLoader,Class) as params it just has (class).

So i tried it with

FastClass.create(Simple.class)

Quote:
import java.lang.reflect.InvocationTargetException;

import net.sf.cglib.reflect.FastClass;

public class Test {

public boolean doit(){
ClassLoader loader = new ClassLoader(this.getClass().getClassLoader()){};

try {
FastClass.create(Simple.class).newInstance();
} catch (InvocationTargetException e) {

e.printStackTrace();
}

java.lang.ref.Reference ref =
new java.lang.ref.WeakReference(loader);

loader = null;
java.util.List list = new java.util.ArrayList();

for(int i = 0; i < 512; i++ ){

System.gc();

if(ref.get() == null ){

return false;

}

byte[] garbage = new byte[ (i + 1)*1004 ];
list.add(garbage);


}

return true;

}

public static void main(String[] args) {
Test t = new Test();
System.out.println("Result :::: " + t.doit()) ;
}


}


public class Simple {
}



And here are the results from windows with IBm JDK 1.3.1

Result :::: false

Still then why does the heap still show me all the cglib classes when in the app server after garbage collection ...

Regards
Suchak Jani


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 04, 2004 11:14 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
I am afraid thre is some problem with distribution, I have added method "create(loader,clazz)" in cglib 2.0.1 to make this test and to fix a memory leak (this test fails with cglib 2.0 ). Your modified test does nothing without this parameter, we need to define generated class in "loader" and try to dereference it. "false" means a "good" result (no leak), "true" means "memorey leak or iteration count is too low". You can try cvs version if you have time, I will check distribution.


Top
 Profile  
 
 Post subject: hibernate 213 and 214 both use FastClass.create(class)
PostPosted: Fri Jun 04, 2004 9:31 pm 
Beginner
Beginner

Joined: Wed Sep 24, 2003 8:27 am
Posts: 36
Baliukas

I think if what we are discussing is true then we have found the root of the problem.

I downloded hibernate 213 src and 214 src both and in both fast class is use in 3 classes as shown below and in all three it is
Quote:
FastClass.create(class)
and not
Quote:
FastClass.create(loader,class)
.

Here are the codes

1

Quote:
net.sf.hibernate.util.ReflectHelper
public static BulkBean getBulkBean(Class clazz, String[] getterNames, String[] setterNames, Class[] types){
try {
BulkBean optimizer = BulkBean.create(clazz, getterNames, setterNames, types);
if ( !clazz.isInterface() ) {
//test out the optimizer:
Object instance = FastClass.create(clazz).newInstance();
optimizer.setPropertyValues( instance, optimizer.getPropertyValues(instance) );
}
//if working:
return optimizer;
}catch....


2

Quote:
net.sf.hibernate.type.ComponentType
public ComponentType(
Class componentClass,
String[] propertyNames,
Getter[] propertyGetters,
Setter[] propertySetters,
final boolean foundCustomAccessor,
Type[] propertyTypes,
int[] joinedFetch,
Cascades.CascadeStyle[] cascade,
String parentProperty
) throws MappingException {

this.componentClass = componentClass;
this.propertyTypes = propertyTypes;
getters = propertyGetters;
setters = propertySetters;
propertySpan = propertyNames.length;
String[] getterNames = new String[propertySpan];
String[] setterNames = new String[propertySpan];
Class[] propTypes = new Class[propertySpan];
for ( int i=0; i<propertySpan; i++ ) {
getterNames[i] = getters[i].getMethodName();
setterNames[i] = setters[i].getMethodName();
propTypes[i] = getters[i].getReturnType();
}
if (parentProperty==null) {
parentSetter=null;
parentGetter=null;
}
else {
PropertyAccessor pa = PropertyAccessorFactory.getPropertyAccessor(null);
parentSetter = pa.getSetter(componentClass, parentProperty);
parentGetter = pa.getGetter(componentClass, parentProperty);
}
this.propertyNames = propertyNames;
this.cascade = cascade;
this.joinedFetch = joinedFetch;
constructor = ReflectHelper.getDefaultConstructor(componentClass);
optimizer = !foundCustomAccessor && Environment.useReflectionOptimizer() ?
ReflectHelper.getBulkBean(componentClass, getterNames, setterNames, propTypes) :
null;
fastClass = FastClass.create(componentClass);
}


3

Quote:
net.sf.hibernate.persister.AbstractEntityPersister

protected AbstractEntityPersister(PersistentClass model, SessionFactoryImplementor factory) throws HibernateException {
.....

optimizer = !foundCustomAccessor && Environment.useReflectionOptimizer() ?
ReflectHelper.getBulkBean(mappedClass, getterNames, setterNames, types) :
null;
fastClass = FastClass.create(mappedClass); //line 756



So i think we will also have to change the hibernate code for the garbage collection to work !

Hibernate Team,
Is it ok if i change and test the hibernate code after i get the latest cglib code from Baliukas locally in my systems and then mail it to someone in the hibernate team to test and merge to hibernate ?

Regards
Suchak Jani


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 30 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.