Hibernate + Tomcat hot deploy = PermGen Memory leaks?

danbst · **Joined:** Mon Jul 07, 2014 11:49 am **Posts:** 3

Hello. I'm trying to make tomcat hot deploy (without restart) of Hibernate application

Here is example of very very simple webapp suitable for tomcat. It just opens a SessionFactory on webapp deploy and closes it on webapp undeploy

Code:

package servlet;

import java.sql.Driver;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Enumeration;

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;

import org.hibernate.SessionFactory;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.cfg.Configuration;
import org.hibernate.engine.jdbc.connections.internal.DriverManagerConnectionProviderImpl;
import org.hibernate.internal.SessionFactoryImpl;
import org.hibernate.service.ServiceRegistry;

@WebListener
public class Listener implements ServletContextListener {
   
   public static SessionFactory createFactory() {
      Configuration c = new Configuration();
      c.setProperty("hibernate.cache.use_second_level_cache", "false");
      c.setProperty("hibernate.cache.use_query_cache", "false");
      c.setProperty("hibernate.pool_size", "0");
      c.setProperty("hibernate.connection.driver_class", "org.postgresql.Driver");
      c.setProperty("hibernate.dialect", "org.hibernate.dialect.PostgreSQL82Dialect");
      c.setProperty("hibernate.connection.url", "jdbc:postgresql://localhost:5432/spaceship");
      c.setProperty("hibernate.connection.username", "lilu");
      c.setProperty("hibernate.connection.password", "multipass");
      
      ServiceRegistry serviceRegistry = new StandardServiceRegistryBuilder()
         .applySettings(c.getProperties())
         .build();
      return c.buildSessionFactory(serviceRegistry);
   }

   @Override
   public void contextInitialized(ServletContextEvent event) {
      event.getServletContext().setAttribute("factory", createFactory());
   }

   @Override
   public void contextDestroyed(ServletContextEvent event) {
      
      Enumeration<Driver> drivers = DriverManager.getDrivers();
      while (drivers.hasMoreElements()) {
         Driver driver = drivers.nextElement();
         try {
            DriverManager.deregisterDriver(driver);
         } catch (SQLException e) {
            e.printStackTrace();
         }
      }

      SessionFactory factory = (SessionFactory) event.getServletContext().getAttribute("factory");
      if (factory != null) {
         SessionFactoryImpl f = (SessionFactoryImpl) factory;
         DriverManagerConnectionProviderImpl prov = (DriverManagerConnectionProviderImpl) f.getConnectionProvider();
         prov.stop();
         
         factory.close();
      }
   }
   
}

I have -XX:+CMSClassUnloadingEnabled in my JVM argument list. I'm using hibernate-core-4.3.5 final, postgresql-9.3-1100 and tomcat-7.0.54

Using JProfiler I've collected list of all the classes that prevent Tomcat's WebappClassLoader to free PermGen space: http://pastebin.com/0gsnzwc0

So

Is this Hibernate problem? I'm using default connection pool and no L2 cache.
Why and what prevents classes from unloading? Maybe singletons (e.g. org.hibernate.type.FloatType.INSTANCE) and other static variables?
Are there ways to purge hibernate classes from classloader in tomcat7?
To be fair, Postgres JDBC driver also has some relation to this problem. I'll create separate ticket to JDBC guys if appropriate.

sanne.grinovero · **Posted:** Mon Jul 07, 2014 7:33 pm

Hi,
why do you want to cleanup PermGen? you don't need to cleanup static constants to be able to restart an Hibernate application.
If you want to physically remove the classes/jars and potentially change the Hibernate version, then you'd need to destroy the classloader. I'm not sure how you would do that in practice, I think people would usually restart Tomcat for that.
It sounds like you actually want a modular container like WildFly.

danbst · **Joined:** Mon Jul 07, 2014 11:49 am **Posts:** 3

sanne.grinovero wrote:

Hi,
why do you want to cleanup PermGen? you don't need to cleanup static constants to be able to restart an Hibernate application.
If you want to physically remove the classes/jars and potentially change the Hibernate version, then you'd need to destroy the classloader. I'm not sure how you would do that in practice, I think people would usually restart Tomcat for that.
It sounds like you actually want a modular container like WildFly.

How would I then solve the "OutOfSpaceException: PermGen space" when hot-deploying? 20 hot-deploys and permgen space is filled by 1GB just with Hibernate classes. Maybe you suggest using Hibernate libraries as shared JARs? That may be good solution up to time when you want to upgrade Hibernate itself.

Yes, currently we are restarting Tomcat to clear classloaders, but I'm searching for softer webapp upgrade solution.

sanne.grinovero · **Posted:** Tue Jul 08, 2014 5:51 am

I can't honestly say for sure that there is no leak in your scenario, but looking at static constants won't help so I suspect there is something more going on.
Some ideas:

Are you sure your contextDestroyed is invoked?
Probably should unload the Driver after stopping the connection pool?
Stop the connection pool after the SeesionFactory?
Use a "real" connection pool, the one you're using is not suitable for production use.
Look for Threads still active, and leaked ThreadLocal instances, not constants.

And again, did you try WildFly? it solves all such problems and is more efficient than Tomcat in pretty much any aspect. Redeploy and modularity for different versions of Hibernate (or anything else) come out of the box, including an high quality connection pool.
(Yes you might consider me biased as my employer also funds WildFly development, but it also funds Tomcat and I have no interest in selling you either one.. I just think Tomcat is rather outdated, especially in these areas you're looking at now)

danbst · **Joined:** Mon Jul 07, 2014 11:49 am **Posts:** 3

sanne.grinovero wrote:

And again, did you try WildFly? it solves all such problems and is more efficient than Tomcat in pretty much any aspect. Redeploy and modularity for different versions of Hibernate (or anything else) come out of the box, including an high quality connection pool.
(Yes you might consider me biased as my employer also funds WildFly development, but it also funds Tomcat and I have no interest in selling you either one.. I just think Tomcat is rather outdated, especially in these areas you're looking at now)

Hey, it is very interesting to try WildFly, but my boss believes Tomcat is the best solution. I need time to make list of advantages (including test for memory leaks when hotdeploying), thats lots of time

As for your suggestions: I am sure contextDestroyed is invoked, no threads are left, no leaked ThreadLocals. I don't use third party connection pool just to minify example (c3p0 may be leaking too). The other advices I'll try someday, but I don't believe they will help freeing classloader.

Quote:

but looking at static constants won't help

but that may be a reason for a leak? Static field INSTANCE holds a hard reference to class, so the class will not be unloaded until we nullify that INSTANCE. It is also final, so we cannot do anything to that class - it will free permgen space only when classloader is destroyed (e.g, tomcat restart).