I want some help on improving perfomance while using hibernate to read large volume of data from the oracle database (upwards of 100 Billion records). I read the data from the database by executing scalar queries, loop thru the resultset and write the records to the java object. Subsequently, i upload the objects to the cache server. till now i have tested with 1.8 mln records, it takes almost 18 minutes to do this operation. i am target this to come down to few seconds. Can anyone please help

The first step would be to break down this to see how much time the different parts of the code uses. To begin with, you could for example measure the following:

* The time it takes to execute the query
* The time it takes to loop through the resultset without creating the result objects.
* The time it takes to look through the resultset when you create the result objects.
* The time it takes to upload the objects to the cache server

Depending on the result of this you may have to subdivide some parts further. To measure the time you can for example use System.currentTimeMillis() and/or System.nanoTime().

I already done this. out of 18 minutes, query takes less than a minute (that can be tuned to bring down to few seconds). most of the time is taken in looping thru the recordset and populating the object and writing that to the cache server. need help in reducing this cycle.

I guess this rules out the query as the source of the performance problem. What about the other three points? Have you tried to separate the time used for each one of them?

Yes. Looping is consuming most of the time. i have used currentTimeMillis() to measure the time taken. I need to understand if there are some configurations which can help me in reducing time. My application related to finance domain where we will be doing the simulations on multiple tenors which will generate large volume of data. when the simulation is complete on one tenor, i will populate that into the java objects which inturn will be loaded to the cache server.