Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: StatelessSessions and proxies (or lack thereof)
PostPosted: Tue Jun 06, 2017 7:50 pm 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
I'm trying to switch a process I have that works with lots of data from a standard Session to a StatelessSession, and I'm wondering why the decision was made to not allow proxy loads. Basically my process does something like 40,000 inserts and maybe 2 updates. I don't really need to keep all that in a standard session, but reworking the process to not use any proxies is, to say the least, a daunting task.

Is there some very bad reason why loading items through the proxies is a bad idea when not stored in the session cache? I'm just trying to understand the design decision more thoroughly. Any insight as to why it works the way it does would be appreciated.

Thanks in advance.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:02 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
There is no reason for using a StatelessSession which has lots of limitations.

This is the best way to do batch processing with JPA and Hibernate.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 9:31 am 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
No, that's what I've been doing for years, but it's incredibly slow and error-prone. There is no reason to maintain a persistence context that I throw away. In my persistence context my process probably grabs and maintains some 500,000+ records over time of which, only about 30-200 are needed across the entire process.

Using a stateless session is ideal from a raw performance standpoint. I need to insert all 80,000 records in a single transaction, and I need to be able to maintain certain reference objects that all 80,000 items refer to. Flushing and clearing the persistence context adds extra overhead, and leads to issues where some things are cleared prematurely. Having to reload those items into the cache after every flush/clear is where most of my errors have come from over the years.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 10:38 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
In my persistence context my process probably grabs and maintains some 500,000+ records over time of which, only about 30-200 are needed across the entire process.


Well, that goes against everything I explained in the article that I gave you. Read it, and allow your application to run at warp speed.

Quote:
Using a stateless session is ideal from a raw performance standpoint.


No, it's not. If you want to process data, you have better alternatives:

- do the processing on the DB, on a replica node, so that you avoid the network overhead. There's a very nice chapter on this topic in my book as well.
- do the processing in the application, in very small batches so that you avoid long-running transactions and memory issues

Quote:
I need to insert all 80,000 records in a single transaction, and I need to be able to maintain certain reference objects that all 80,000 items refer to.


Now, that's the problem. I used to work on a project where we had to insert data from real estate agencies into our system, and there is no such thing as rolling back 79.999 good items, just because one has an issue which prevents it from being saved.

It's better to roll back 100 items, instead of 80.000, especially if your DB engine uses MVCC.

Quote:
Flushing and clearing the persistence context adds extra overhead, and leads to issues where some things are cleared prematurely.


Try benchmarking that with the overhead of a long-running transaction and see which one wins. I bet on the flush/clear one. ;)

Quote:
Having to reload those items into the cache after every flush/clear is where most of my errors have come from over the years.


You can still use parent entities in detached mode, just for the sake of populating @ManyToOne or @oneToOne associations.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 11:00 am 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
Quote:
Well, that goes against everything I explained in the article that I gave you. Read it, and allow your application to run at warp speed.


I did read your article. I don't want to save 100 or 200 or 2000 at a time. If I did, then my data would be in an invalid state. If all 80k are not valid, then it needs to rollback. If I wanted shit data in my database, I'd use couchdb instead of a relational database with ACID. In general, I perform validation checks before the process starts, so rollbacks would be extremely rare once the 80k inserts start. I'm not really concerned about rollback performance, I'm worried about commit performance.

Quote:
No, it's not. If you want to process data, you have better alternatives:

- do the processing on the DB, on a replica node, so that you avoid the network overhead. There's a very nice chapter on this topic in my book as well.
- do the processing in the application, in very small batches so that you avoid long-running transactions and memory issues


Sadly, I have no replica nodes. Single DB. Doing the processing in the application is what I'm doing. Read the file, insert the data, perform a single update at the end.

Quote:
Try benchmarking that with the overhead of a long-running transaction and see which one wins. I bet on the flush/clear one. ;)


So far my testing has shown that flush and clear is slower or doesn't work because I get lazy loading exceptions if an object is removed from the cache and then reused later.

I'm sorry that not every real world case fits into your book.

If we could bet back on topic for my actual question that would be nice though.

Quote:
Is there some very bad reason why loading items through the proxies is a bad idea when not stored in the session cache? I'm just trying to understand the design decision more thoroughly. Any insight as to why it works the way it does would be appreciated.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 11:19 am 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
Is there some very bad reason why loading items through the proxies is a bad idea when not stored in the session cache? I'm just trying to understand the design decision more thoroughly. Any insight as to why it works the way it does would be appreciated.


StatelessSession was designed not to work with the st-level cache.

Quote:
A stateless session does not implement a first-level cache nor interact with any second-level cache, nor does it implement transactional write-behind or automatic dirty checking, nor do operations cascade to associated instances. Collections are ignored by a stateless session. Operations performed via a stateless session bypass Hibernate's event model and interceptors. Stateless sessions are vulnerable to data aliasing effects, due to the lack of a first-level cache.


So, no 1st-level cache, no Proxies. You'd either reconsider your design choices or stick to it and see for yourself if that was a good idea or not. ;)

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 11:35 am 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
vlad wrote:
Quote:
Is there some very bad reason why loading items through the proxies is a bad idea when not stored in the session cache? I'm just trying to understand the design decision more thoroughly. Any insight as to why it works the way it does would be appreciated.


StatelessSession was designed not to work with the st-level cache.

Quote:
A stateless session does not implement a first-level cache nor interact with any second-level cache, nor does it implement transactional write-behind or automatic dirty checking, nor do operations cascade to associated instances. Collections are ignored by a stateless session. Operations performed via a stateless session bypass Hibernate's event model and interceptors. Stateless sessions are vulnerable to data aliasing effects, due to the lack of a first-level cache.


So, no 1st-level cache, no Proxies. You'd either reconsider your design choices or stick to it and see for yourself if that was a good idea or not. ;)


Yes, I read the very limited documentation on stateless sessions before going this route. I'm not worried about aliasing or other cache-missing issues as I'm not updating anything that could possibly be aliased. It doesn't explain WHY the proxies don't work.

Do the proxies rely on the other caches? Would they work without the caches, but it was decided not to support them to prevent aliasing? I'm looks for DESIGN decisions regarding this choice, not Javadoc copy-pasting.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:10 pm 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
Yes, I read the very limited documentation on stateless sessions before going this route.


There's enough documentation about StatelessSession, considering its limited usage.

I'm now improving the User Guide, especially for interesting stuff like what's the purpose of every annotation. Unfortunately, StatelessSession is not on my agenda, so, if you want more, just send us a Pull Request, and I'll integrate it.

Quote:
I'm looks for DESIGN decisions regarding this choice, not Javadoc copy-pasting.


When you access a lazy Proxy association, this is how the association gets initialized. See the session parameter? That's why you can't have Proxies for StatelessSession.

Code:
final Object loadedValue = initializer.initializeLazyProperty(
   attributeName,
   target,
   session
);


So, why would it need the Session, right?

Code:
final EntityEntry ownerEntry = session.getPersistenceContext().getEntry( entity );


Code:
session.initializeCollection( collection, false );


Code:
final Serializable id = session.getContextEntityIdentifier( entity );


Makes sense, right? If you are eager to learn more about it, here's the Hibernate package on GitHub.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:19 pm 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
vlad wrote:
Quote:
Yes, I read the very limited documentation on stateless sessions before going this route.


There's enough documentation about StatelessSession, considering its limited usage. If you want more, just send us a Pull Request, and I'll integrate it.

Quote:
I'm looks for DESIGN decisions regarding this choice, not Javadoc copy-pasting.


When you access a lazy Proxy association, this is how the association gets initialized. See the session parameter? That's why you can't have Proxies for StatelessSession.

Code:
final Object loadedValue = initializer.initializeLazyProperty(
   attributeName,
   target,
   session
);


So, why would it need the Session, right?

Code:
final EntityEntry ownerEntry = session.getPersistenceContext().getEntry( entity );


Code:
session.initializeCollection( collection, false );


Code:
final Serializable id = session.getContextEntityIdentifier( entity );


Makes sense, right? If you are eager to learn more about it, here's the Hibernate package on GitHub.



Thanks for that info. I had seen that documentation, but couldn't find the bytecode classes that are used.

for the latest Master (6.0?) here: https://github.com/hibernate/hibernate- ... eptor.java

I'm not seeing any reason why a StatelessSession can't be passed to the proxy loader. It's using the SharedSessionContractImpl for the session parameter.

Does this mean that proxy loading could be available to StatelessSessions in a later release?


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:25 pm 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
for the latest Master (6.0?) here: https://github.com/hibernate/hibernate- ... eptor.java


In git terms, that's called Latest commit 87e3f0f on Apr 19, 2016.

Quote:
I'm not seeing any reason why a StatelessSession can't be passed to the proxy loader. It's using the SharedSessionContractImpl for the session parameter.

Does this mean that proxy loading could be available to StatelessSessions in a later release?


No, it won't be added in a future release. If you use StatelessSession a lot, then you are better off using jOOQ instead.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:40 pm 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
vlad wrote:
Quote:
for the latest Master (6.0?) here: https://github.com/hibernate/hibernate- ... eptor.java


In git terms, that's called Latest commit 87e3f0f on Apr 19, 2016.

Quote:
I'm not seeing any reason why a StatelessSession can't be passed to the proxy loader. It's using the SharedSessionContractImpl for the session parameter.

Does this mean that proxy loading could be available to StatelessSessions in a later release?


No, it won't be added in a future release. If you use StatelessSession a lot, then you are better off using jOOQ instead.


No, I wouldn't be using them a lot. My main use for the stateless session is this bulk loading operation which is a small but intensive part of a very large application. 99+% of the application would still use the standard session management. Switching to jOOQ would be a much larger rewrite than I want to tackle.

It would be nice to not have to refactor so much code though to work around stateless proxy loading issues and to somewhat seamlessly switch between the backing sessions. But then as noted in the docs, with the aliasing, if some tree structure was aliased a lot in session, it could lead to a lot of memory usage that could be mitigated by pre-loading it in bulk as forced to do now without the proxy support.

vlad wrote:
No, it won't be added in a future release.

If the proxy classes support it, why not? Overall, I feel it should be supported with LARGE caveats that it could (but not always) cause lots of extra database traffic due to aliasing instead of loading them manually, but I think for users who have (apparently) fringe cases like mine, it would be a large time saver for development.

Thanks.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:49 pm 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
Switching to jOOQ would be a much larger rewrite than I want to tackle.


Why do you think you need to use either Hibernate or jOOQ. You can easily combine them so that you use Hibernate for write operations, and jOOQ for advanced SQL queries and bulk processing (e.g. MERGE, UPSERT, MySQL batch inserts with IDENTITY column).

Quote:
If the proxy classes support it, why not? Overall, I feel it should be supported with LARGE caveats that it could (but not always) cause lots of extra database traffic due to aliasing instead of loading them manually, but I think for users who have (apparently) fringe cases like mine, it would be a large time saver for development.


If you think it's possible, send us a Pull Request, and let's then discuss its actual impact.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 12:58 pm 
Newbie

Joined: Tue Jun 06, 2017 6:43 pm
Posts: 7
vlad wrote:
Quote:
Switching to jOOQ would be a much larger rewrite than I want to tackle.


Why do you think you need to use either Hibernate or jOOQ. You can easily combine them so that you use Hibernate for write operations, and jOOQ for advanced SQL queries and bulk processing (e.g. MERGE, UPSERT, MySQL batch inserts with IDENTITY column).


At the moment all my business logic is linking to my DAOs rather statically. In theory it should not be a problem, but on a large code base over 12 years old, it could lead to a lot of refactoring in order to support 2 different data layers between jOOQ and Hibernate/QueryDSL that I use now.

For my codebase, it would probably be easier for me to add a second set of QueryDSL generated classes that used direct SQL instead of hibernate as the backend.

vlad wrote:
Quote:
If the proxy classes support it, why not? Overall, I feel it should be supported with LARGE caveats that it could (but not always) cause lots of extra database traffic due to aliasing instead of loading them manually, but I think for users who have (apparently) fringe cases like mine, it would be a large time saver for development.

If you think it's possible, send us a Pull Request, and let's then discuss its actual impact.


Thanks, and duly noted. I may consider doing that, but at the moment I don't have the expertise in the hibernate codebase to do that quickly. If I have some downtime, I'll look into it.


Top
 Profile  
 
 Post subject: Re: StatelessSessions and proxies (or lack thereof)
PostPosted: Wed Jun 07, 2017 1:18 pm 
Hibernate Team
Hibernate Team

Joined: Thu Sep 11, 2014 2:50 am
Posts: 1263
Quote:
and Hibernate/QueryDSL that I use now.


If you use QueryDSL already, no need for jOOQ. Use QueryDSL for all those use cases when you now use a StatelessSession.

_________________
If you liked my answer, you are going to love my High-Performance Java Persistence book and my blog as well.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.