I've spent a couple of months tracking down a problem in my application, and now it appears that it may be an issue between Hibernate and the pooling mechanisms.
For reference, I'm using Hibernate 2.1.7 and MySQL 4.0.18, although I've also tried numerous other versions of both.
The class HibernateSessionFilter gets instantiated as the top level filter in the chain so that it first opens a connection, and then closes that connection once the request finishes.
Code:
public class HibernateSessionFilter implements Filter {
public void init(FilterConfig filterConfig) throws ServletException {}
public void destroy() {}
...
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws
IOException,
ServletException
{
try {
Session hsession = HibernateHelper.currentSession();
hsession.setFlushMode(FlushMode.NEVER);
// Pass control on to the next filter
chain.doFilter(request, response);
}
catch (HibernateException ex) {
throw new ServletException(ex);
}
finally {
try {
HibernateHelper.closeSession(); // THIS IS HibernateSessionFilter.java LINE 54
}
catch (HibernateException ex) {
throw new ServletException(ex);
}
}
}
...
}
It accomplishes this through HibernateHelper, which uses ThreadLocal to keep things thread-safe:
Code:
public class HibernateHelper {
static SessionFactory sessionFactory;
static final ThreadLocal session = new ThreadLocal();
...
public static SessionFactory getSessionFactory() {
if(sessionFactory == null) {
try {
sessionFactory = new Configuration().configure().buildSessionFactory();
} catch (HibernateException ex) {
throw new RuntimeException("Hibernate Configuration problem: " + ex.getMessage(), ex);
}
}
return sessionFactory;
}
...
public static Session currentSession() throws HibernateException {
Session s = (Session) session.get();
// Open a new Session, if this Thread has none yet
if (s == null) {
s = getSessionFactory().openSession();
session.set(s);
}
return s;
}
public static void closeSession() throws HibernateException {
Session s = (Session) session.get();
session.set(null);
if (s != null && s.isOpen())
s.close(); // THIS IS HibernateHelper.java LINE 53
}
}
The application itself runs fine for awhile, but after a number of hours, it begins to intermittantly throw exceptions:
net.sf.hibernate.exception.GenericJDBCException: Cannot close connection
at net.sf.hibernate.exception.ErrorCodeConverter.handledNonSpecificException(ErrorCodeConverter.java:90)
at net.sf.hibernate.exception.ErrorCodeConverter.convert(ErrorCodeConverter.java:79)
at net.sf.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:30)
at net.sf.hibernate.impl.BatcherImpl.convert(BatcherImpl.java:325)
at net.sf.hibernate.impl.BatcherImpl.closeConnection(BatcherImpl.java:311)
at net.sf.hibernate.impl.SessionImpl.disconnect(SessionImpl.java:3387)
at net.sf.hibernate.impl.SessionImpl.close(SessionImpl.java:584)
at com.somewhere.application.data.HibernateHelper.closeSession(HibernateHelper.java:53)
at com.somewhere.application.data.HibernateSessionFilter.doFilter(HibernateSessionFilter.java:54)
...
Caused by: java.sql.SQLException: No operations allowed after connection closed.
HibernateHelper.java:53 and HibernateSessionFilter.java:54 are marked in the above code snippets.
According to the JDNI datasource HOWTO for Tomcat5, exceptions on close should always be caught and swallowed.
From
http://jakarta.apache.org/tomcat/tomcat ... 20ProblemsCode:
Connection conn = null;
Statement stmt = null; // Or PreparedStatement if needed
ResultSet rs = null;
try {
conn = ... get connection from connection pool ...
stmt = conn.createStatement("select ...");
rs = stmt.executeQuery();
... iterate through the result set ...
rs.close();
rs = null;
stmt.close();
stmt = null;
conn.close(); // Return to connection pool
conn = null; // Make sure we don't close it twice
} catch (SQLException e) {
... deal with errors ...
} finally {
// Always make sure result sets and statements are closed,
// and the connection is returned to the pool
if (rs != null) {
try { rs.close(); } catch (SQLException e) { ; }
rs = null;
}
if (stmt != null) {
try { stmt.close(); } catch (SQLException e) { ; }
stmt = null;
}
if (conn != null) {
try { conn.close(); } catch (SQLException e) { ; }
conn = null;
}
}
However, Hibernate doesn't seem to do this, as can be seen by following the stack trace.
The important code within Hibernate itself goes like this:
Code:
BatcherImpl.java:
public void closeConnection(Connection conn) throws HibernateException {
try {
if ( !conn.isClosed() ) {
try {
JDBCExceptionReporter.logWarnings( conn.getWarnings() );
conn.clearWarnings();
}
catch (SQLException sqle) {
//workaround for WebLogic
log.debug("could not log warnings", sqle);
}
}
factory.getConnectionProvider().closeConnection(conn);
}
catch (SQLException sqle) {
throw convert( sqle, "Cannot close connection" );
}
}
In the case of C3P0, getConnectionProvider().closeConnection() looks like this:
Code:
public class C3P0ConnectionProvider implements ConnectionProvider {
...
public void closeConnection(Connection conn) throws SQLException {
conn.close();
}
...
}
And DBCP:
Code:
public class DBCPConnectionProvider implements ConnectionProvider {
...
public void closeConnection(Connection conn) throws SQLException {
conn.close();
}
...
}
Both C3P0 and DBCP exhibit this behavior.
I'm not sure where to stand on this issue.
In this case, I've managed to track the root cause of the exception to the new mysql jdbc driver, which after version 3.0.9 no longer checks the connection status prior to executing operations upon it (apparently to bring the driver in line with the spec).
The answer is supposed to be to change the connection pool settings to retire a connection before the MySQL timeout (default 8 hours), but I've tried this and it doesn't work.
But I digress...
If exceptions are simply swallowed and ignored when they occur (as suggested by the Tomcat people), doesn't this cause a bad connection to be returned to the connection pool? I do notice that if I let the application run in this bad state for long enough, the pool will eventually stop giving out connections.
I could of course simply swallow exceptions at the HibernateHelper level, but this is for a production system, and I don't want to do something akin to taping the needle on a boiler because the safety systems keep going off.
Which is the correct way to handle this?
For now I can of course just switch back to MySQL driver 3.0.9, but it doesn't give me a warm fuzzy feeling not being able to upgrade the driver, or knowing that any other JDBC issue could bring me right back to this problem again.
Login
FAQ
Search
