Well, the solution was for me to rebuild the library containing the JBoss Hibernate MBean. In case this happens to anybody else, here's what to do. I did this for 4.0.5 GA; I imagine this would work for 4.0.4 as well.
1. Download the source.
2. Open up <jboss source>\hibernate-int\src\main\org\jboss\hibernate\jmx\Hibernate.java
3. Edit the createService() function so that it looks like this:
Code:
protected void createService() throws Exception
{
log.trace( "forcing bytecode provider -> cglib" );
// todo : really need a much better solution for this...
System.setProperty( Environment.BYTECODE_PROVIDER, "cglib" );
}
4. Save this revised class and do a build of JBoss - you'll find build scripts in <jboss source>\build\ for your platform, which in my case was XP, so I just ran the build.bat file.
5. When the build is complete, grab <jboss source>\hibernate-int\output\lib\jboss-hibernate.jar and replace the one in <your jboss distro>\server\default\lib with the one you just built.
This will switch
all of your Hibernate services to use CGLIB as your bytecode provider. In my case, it cleared up all of the horrendous PermGen OutOfMemory problems I was having with javassist and my application is now happy again. Hopefully, future releases will support a pluggable bytecode provider for Hibernate MBeans.