Hi,

I used Hibernate for my object model to persist. I needed to import raw data from legacy database to my object model database. I had to deal with a huge number of rows (1 - 15 million rows)

In this process, to import an object, I had to check for the existence of the object and do either insert/update (save or update).

The lookups are by a property name (account name). So I created look up methods.

I have the second-level cache and query cache also enabled. My lookups were slower when I had single session (provides first level cache) for many transactions (includes lookup and save), whereas it was quite faster when I had a new session for each transaction.

Most of the time, my lookups may not find the right candidate in cache because it is an import service to import new data.

So does having a single session with lot of objects in the cache (for example 10,000) and not finding a match causes overhead?

Can anyone help me figure out the reason for slow performance to do the lookups?

Thanks in advance.

Are you using ehcache?
If so, do you have overflowToDisk set to true?
If you have lots of records in your cache it’s quite possibly overflowing them to disk, which as I understand it just does a standard java object serialization to the file system. Deserializing them off the file system would probably be slower than just reading them in form the DB.
That’s the only thing I can think of off the top of my head.