In the first case, we are looking up 5000 entities by looping on the client, meaning that we start a new session 5000 times.

In the second case, the lookup loop has been moved into the session bean. Under the second scenario, all of these 5000 lookups are performed within the context of the same session.

Against all logic, the second scenario performs much worse than the first one. The time to process each record appears to increase geometrically, until it litterally gets to a crawl.

Also, the second scenario generates massive paging to disk (which does not make any sense, since we are only performing lookups).

We were wondering if this was not a level-2 cache issue, so we totally disabled the level 2 cache by switching to the NoCacheProvider in our persistence.properties file, and this did not help the performance issue.

Consequenty, we are now wondering if the level-1 cache is not trying to page to disk !!!

(I know this sounds crazy, but remember that all we are doing are lookups, and that looking up these 5000 records takes about 20 times longer when looping within the session bean).

How do you lookup your entities ?
I had such kind of problem with a lot of objects in one transaction. I was performing query to lookup my objects. In fact, after a lot of efforts, I discovered that the problem was due to my query. Before each query, a flush is automatically performed. My entities had a lot of collections with a CascadeType set to persist. In that case, flush becomes very time consumming. I removed the CascadeType.persist and performance becomes quite good.

Maybe something you can investigate ! I'm interesting to know if your problem looks like my one.