Hibernate version: 2.16

Mapping documents:N/A

int rows = 0;
try{
Session as400Session = getHibernateTemplate().getSessionFactory().openSession();

//set up the ora session
HibernateDaoSupport dao = (HibernateDaoSupport) getOraDao();
oraSession = dao.getSessionFactory().openSession();
oraTx = oraSession.beginTransaction();

//delete the vins
int rowsDeleted = getOraDao().deleteVins(oraSession);
log.info("There were " + rowsDeleted + " vins deleted");

//set up the as400 transaction
as400Tx = as400Session.beginTransaction();

rows = ( (Integer) as400Session.iterate("select count(*) from As400Vin").next() ).intValue();

Query q = as400Session.createQuery(VINS);
q.setMaxResults(50000);
ScrollableResults result = q.scroll();
int count = 0;
while ( result.next() ) {
As400Vin as400Vin = (As400Vin) result.get(0);
Vin oraVin = new Vin();
BeanUtils.copyProperties(oraVin, as400Vin);

//save the oracle object
oraSession.save(oraVin);

if ( ++count % 20 == 0 || count == rows) {
log.info("Flushing/Clearing sessions " + count);
as400Session.flush();
as400Session.clear();
oraSession.flush();
oraSession.clear();
}
}

//close Oracle and AS400 sessions
oraTx.commit();
oraSession.close();
as400Tx.commit();
as400Session.close();
}
catch(HibernateException ex){
BaseMigration.handleExceptions(ex, oraTx);
throw new RuntimeException("HibernateException", ex);
}
catch(Exception ex){
BaseMigration.handleExceptions(ex, oraTx);
throw new RuntimeException("Exception", ex);
}
return new Integer(rows);

Full stack trace of any exception that occurs: N/A

Name and version of the database you are using: DB2 / AS400 and Oracle 10g

The generated SQL (show_sql=true): N/A

Debug level Hibernate log excerpt: N/A

I know that using ORM for retrieving and saving bulk data is not the suggested way of accomplishing such task, but I am in a position where I don't have much say for the time being. So having gotten that out of the way, I am following the approach suggest by Gavin at: http://blog.hibernate.org/cgi-bin/blosx ... batch.html

My problem is that I am still getting out of memory errors even though I am flushing my sessions (Please look at code above).

I have about 97,000 rows of data that I am retrieving via a ScrollableResults. Whenever I set q.setMaxResults... to 50,000 all 50,000 records get inserted. If I set q.setMaxResults... to 60,000 I get an out of memory error on the 21360th row. I am closing all open hibernate sessions every 20th trip through the loop, so I don't know what else is causing by out of memory errors. My situation is slightly different than the example used at: http://blog.hibernate.org/cgi-bin/blosx ... batch.html in the sense that I need a Hibernate session for DB2 and a different session for Oracle open at the same time.

Any help is greatly appreciated.

Thanks

So, how much memory have you got allocated to JVM heap? (The default is only 64K, which won't hold much). How large is each object on the heap? Seems to me that even if you objects have a portly footprint of 2K each, that's only 200 MB on the heap. Also, are you caching these objects in the second level cache? If you are, don't. Are you using a profiler? Have you tried deallocating your objects and Thread.sleep(ing) long enough to allow a GC? I seriously doubt that there's a problem with Hibernate dealing with 60,000 objects. I'm using it on tables of millions of records with no problems. (Of course, I'm not trying to load the whole thing into memory either!)

My bad. I meant 64 MB is the default heap size, not 64 K. :D