-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 10 posts ] 
Author Message
 Post subject: Bulk insert(Java heap space error)
PostPosted: Thu Mar 27, 2008 4:19 am 
Newbie

Joined: Tue Mar 18, 2008 2:07 pm
Posts: 10
Hi frndz,

I have problem while inserting bulk number for records. When am inserting more than 1000 records. Am getting..

"org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Java heap space
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919)"



For every record am creating new persistence object. So totally creating 1000 objects.

So am getting java heap space error.

Any body help me.

Thanks in advance.

Regards,
sivaprakash.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 27, 2008 4:28 am 
Regular
Regular

Joined: Mon Aug 06, 2007 10:49 am
Posts: 67
Location: Banska Bystrica, Slovakia
u have to clear session to evict object from session cache

read this:


http://www.hibernate.org/hib_docs/v3/re ... ch-inserts


Top
 Profile  
 
 Post subject: Bulk Insertion(Out of memory Exception)Java Heap Space
PostPosted: Fri Mar 28, 2008 3:17 am 
Newbie

Joined: Tue Mar 18, 2008 2:07 pm
Posts: 10
Hi Frndz,

I have to insert bulk amount of data.

Three tables:
table1
table2
table3

table1(one-to-many)table2(one-to-one)table3

So when am inserting into table1. It inserts in both table2 and table3.

Am inserting more than 10000 records, Even i have applied batch siza as "30" and second level cache is disabled.
And also for even 30 insertion i will clear and flush the session.

For insertion,
for single insertion, am creating three instace. so totally it creates 30000 instace. So am receiving "Out of memory Exception"

java.lang.OutOfMemoryError: Java heap space
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919)
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:476)
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:441)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)

17:26:18,574 INFO [STDOUT] java.lang.OutOfMemoryError: Java heap space
17:26:38,304 ERROR [[dps]] Servlet.service() for servlet dps threw exception
java.lang.OutOfMemoryError: Java heap space



Plz frndz help me

Thankx in advance


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 4:37 am 
Regular
Regular

Joined: Mon Aug 06, 2007 10:49 am
Posts: 67
Location: Banska Bystrica, Slovakia
u keep reference for those object,

so try session.getStatistics().getEntityCount() if u really evict all object from session after 30 saving records

and if yes, than u have to look where do u keep references for all your objects for saving in your app


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 4:47 am 
Newbie

Joined: Tue Mar 18, 2008 2:07 pm
Posts: 10
Thanx for your reply...

Actully i dont know, When i gave the Sysout for following statement.
session.getStatistics().getEntityCount().. Am receiving keep on increasing value.
Can u tell me. whether it correct or not


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 5:05 am 
Regular
Regular

Joined: Mon Aug 06, 2007 10:49 am
Posts: 67
Location: Banska Bystrica, Slovakia
after each 30 insertion u should get same count(+-),
if number is still increasing aritmeticaly after each 30 insertion (e.q. 90,180,270), it is bad


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 5:10 am 
Newbie

Joined: Tue Mar 18, 2008 2:07 pm
Posts: 10
Thanx share ur idea.

May i know.. Why its occuring.

This is my code

if ( i % 0 == 0 ) {
transaction.commit();
session.flush();
session.clear();
transaction=session.beginTransaction();
}


Even am getting increasing value. And am using Hibernate3.jar file. Is it support the Bulk Insert


Regards,
sivaprakash.G


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 5:45 am 
Regular
Regular

Joined: Mon Aug 06, 2007 10:49 am
Posts: 67
Location: Banska Bystrica, Slovakia
are u sure that app will hit those code inside if condition???


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 7:11 am 
Newbie

Joined: Tue Mar 18, 2008 2:07 pm
Posts: 10
Hi,
Sorry... i wrongly sent that ......


if ( i % 300 == 0 ) {
transaction.commit();
session.flush();
session.clear();
transaction=session.beginTransaction();
}

Thanx a lot to share your idea with me.

Now no heap space havent came....

May i know details about batch size in cfg file..

My application is running slowly..

For Single Insertion:
Insertion in three table and two file read and one file write and one file upload into BLOB

Can u give some infor. Regarding improve the performance of the application

Regards,
Sivaprakash.G


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 28, 2008 9:07 am 
Regular
Regular

Joined: Mon Aug 06, 2007 10:49 am
Posts: 67
Location: Banska Bystrica, Slovakia
batch size in cfg should be same as count of records after which u clear session - i use 20 -50 depends on situation, just try and will see

hmm ... and other optimization dunno


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 10 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.