-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 17 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Postgres's memory leak.
PostPosted: Wed Jul 21, 2004 1:31 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
Hello.

I use Hibernate 2.1.4 with Postgres 7.4 1-3 on cygwin.
(using Tomcat, Axis.)

[My project]
(1)one session
(Tomcat boot -> 1 open cofig. 2 create sessionFactory
3 load session 4 connect session)
In my project, session never close. Because we want to Hibernate's session cache.

My Hibernate's mapping file contains proxy, a lot of subclass(The table-per-hierarchy mapping), a lot of list(lazy=true&use EHCache).
(EX)
<list name="ZZZ" access="field" table="XXX"
cascade="save-update" lazy="true">
<cache usage="read-write"/>
<key column="XXX_ID"/>
<index column="INDECS"/>
<many-to-many
class="CLASSXXXX"
column="XXXX_ID"/>
</list>

But, After I loaded object from postgreSQL, I access to object's list.
Then SQL excute in net.sf.hibernate.loader.Loader.doQuery().
At that time, postgreSQL's memory is increase.
(This memory never decrease.)
I checked if ResultSet & PreparedStatement is closed in net.sf.hibernate.loader.Loader.doQuery().
This postgreSQL memory leak is Hibernate bug or PostgreSQL bug or my logic's mistake?

[more infomation]
(1)After loading list, this object commit, but postgreSQL's memory never decrease.
(2)In lazy="false", postgreSQL's memory is more increase.
(2)In no using EHCache, result is same.

Please give me your advice.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 1:37 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Quote:
This postgreSQL memory leak is Hibernate bug or PostgreSQL bug or my logic's mistake?


Well, which do you think is most likely?


Sounds like you forget to commit the transaction.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 2:00 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
>Well, which do you think is most likely?

Now, I don't know.
But I think around proxy logic bug in hibernate or postgresql's memory leak.
My application use a lot of large data.
Client: load a lot of data from Server. And Send a lot of data to server:
Server: mergre data from DB, cleient. --> We want to keep data in session.


>Sounds like you forget to commit the transaction.[/quote]

In this case, we want to get alot of data from DB.
I think that It's no need to commit because of loding.
(It's my mistake?)
But,
Even if I commits the object read from DB, a memory leak does not decrease.

Do you know DB's memory leak with postgreSQL?


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 2:57 am 
Hibernate Team
Hibernate Team

Joined: Thu Dec 18, 2003 9:55 am
Posts: 1977
Location: France
Quote:
I think that It's no need to commit because of loding.

_________________
Anthony,
Get value thanks to your skills: http://www.redhat.com/certification


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 2:58 am 
Hibernate Team
Hibernate Team

Joined: Thu Dec 18, 2003 9:55 am
Posts: 1977
Location: France
never say that again....lol

always use transaction even for commit, and i confirm we had a memory leak in the past because of "not" comitting on a large loading

_________________
Anthony,
Get value thanks to your skills: http://www.redhat.com/certification


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 4:47 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
PostgreSQL cache is global, it is stored in shared memory blocks and it is allocated on backend startup, it never grows. Local memory is used for things like sort and hash join, this memory is released at the end of transaction (postgres manages memory itself), so you must end any transaction to release memory.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 7:20 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
Thank you to answer.

But,
(1)Session session = factory.openSession();
(2)Transaction tx = session .beginTransaction()
(3)Object obj = Session.load(Class theClass, Serializable id); <--obj is proxy.
(4)obj do get list;<--list is proxy.
--->postgreSQL's memory increase.
(5)Transaction.commit(); <-- JDBCTransaction
--->postgreSQL's memory never decrease.
(6)We want to do session.close(). because we want to use Hibernate's cache.

"end any transaction" measns call Transaction.commit()?
but postgreSQL's memory never decrease..


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 7:35 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
>(6)We want to do session.close(). because we want to use Hibernate's cache.
sorry. mistake.

(6)We want to not do session.close(). because we want to use Hibernate's cache.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 7:38 am 
Hibernate Team
Hibernate Team

Joined: Thu Dec 18, 2003 9:55 am
Posts: 1977
Location: France
Hibernate's cache.???

you have to understand 1st level and second level cache

1st level is to use as a unit of work --> it is hibernate session
2nd level cache is what we can say "a real cache" that many unit of work can use... it's jboss cache, swarmcache,...

don't think hibernate session as a global and huge cache system... or you're going into troubles

_________________
Anthony,
Get value thanks to your skills: http://www.redhat.com/certification


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 7:55 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
>Hibernate's cache.???

Hibernate's cache I said is 1st level cache.

>don't think hibernate session as a global and huge cache system... or you're going into troubles[/quote]

I agree with you.
I understand it's no good way to cahce lot of object.
Finally OutOfMemory or swap will ouucers.
but I want to cache large object...

Thanks a lot.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 21, 2004 8:21 am 
Hibernate Team
Hibernate Team

Joined: Thu Dec 18, 2003 9:55 am
Posts: 1977
Location: France
memory leak is not the problem

the problem is about concurrency and stale data! be extremely carefull with this

_________________
Anthony,
Get value thanks to your skills: http://www.redhat.com/certification


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 1:58 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
>the problem is about concurrency and stale data! be extremely carefull with this.

You think I should use evict() or clear() or session.close()?

Then I did session.clear() after (4*n),postgres's memory do not decrease.
-------------------------------------------------------------------------------
(3*n)Object obj = Session.load(Class theClass, Serializable id); <--obj is proxy.
(4*n)obj do get list;<--list is proxy.
--->postgreSQL's memory increase.
-------------------------------------------------------------------------------

http://forum.hibernate.org/viewtopic.ph ... emory+leak
<--Should a session be closed?
but I want to know postgres's memory leak reason...


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 2:25 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
Sorry, but I do not understand you. Do you have problems with postgres, hibernate or with your code ?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 2:46 am 
Newbie

Joined: Wed Jul 21, 2004 1:24 am
Posts: 8
Location: Japan
Thanks baliukas.
I think It's better to do session.close() (it's only place that I worry in my code), prostgres memory is controled by postgres as you said.
Where is place that Hibernate notify postgres?
(ex)JDBCTransaction.commit().

I looked anothor issue about memory leak.
But here, everyone disucuss about Hibernate's memory leak.
And I understood Hibernate's has no memory leak.
How about DB's memory?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 3:32 am 
CGLIB Developer
CGLIB Developer

Joined: Thu Aug 28, 2003 1:44 pm
Posts: 1217
Location: Vilnius, Lithuania
There is a parameter in configuration file "sort_mem", it tells how many memory can use postgresql instance (it creates instance per session), if this parameter is to low then it will alocate memory in file, if it is too high then OS will use swap.
Postgresql must release local memory at the end of transaction (commit/rollback) or reuse if some function calls "pfree" in transaction, all functions use "paloc" to alocate memory and I do not think it is possible to reproduce memory leaks in postrgesql.


Hibernate session is not referenced by singletons, so it can not produce memory leak if you do not hold strong reference on session yourself (Threadlocal holds weak reference), some global cache implementation can cause memory leak, but as I understand you do not use it.

JDBC driver can cause problems, some versions can be broken, pool implementation can be broken too, but I think problem is in your code. Do not assign hibernate session to any field of object if it lives longer than thread and you will not have any problems. If you are going to store hibernate session in http session then you must be very advanced user to make it work and I am not sure it can be possible.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.