caching problems

anoopb · **Posted:** Tue Aug 31, 2004 4:17 pm

I have the hibernate EHCache configured. I expect this works as a second-level process cache, independent of Sessions. However, I noticed , through logging in entity object constructors and finalizers, as well as through hash code outputs, that a new object is being created for the same DB record. A query is also being fired each time.
What would I be doing wrong that is causing this? Looks like secondary-level caching isnot working at all....
Not sure what I'm doing wrong..went through all forum questions and documentation too without any luck.. Greatly appreciate any help!

Configuration, source and logs are below:

Hibernate version:2.1

Mapping documents:
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-2.0.dtd">

<hibernate-configuration>

<session-factory>
<property name="hibernate.connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
<property name="hibernate.connection.url">jdbc:oracle:thin:@172.25.44.194:1521:fkt2</property>
<property name="hibernate.connection.username">madmax</property>
<property name="hibernate.connection.password">cubby</property>
<property name="dialect">net.sf.hibernate.dialect.OracleDialect</property>
<property name="show_sql">true</property>
<property name="transaction.factory_class">
net.sf.hibernate.transaction.JDBCTransactionFactory
</property>
<property name="hibernate.cache.provider_class">
net.sf.hibernate.cache.EhCacheProvider
</property>

<mapping resource="../eg/org/hibernate/auction/PlanWorksheet.hbm.xml"/>

</session-factory>
</hibernate-configuration>

<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 2.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">
<hibernate-mapping
package="org.hibernate.auction">

<class name="PlanWorksheet" table="maxdata.planworksheet" >
<cache usage="transactional" />
<id name="id" column="planworksheet_id">
<generator class="native"/>
</id>

<timestamp name="last_update_date"/>
<property name="planworksheet_id" insert="false" update="false" />
<property name="planversion_id" />
<property name="planmaster_id" />
<property name="plangroup_id" />
<property name="description" />
<property name="from_merch_name" />
<property name="create_date"/>

<property name="plan_count"/>
<property name="name"/>

</class>

</hibernate-mapping>

Code between sessionFactory.openSession() and session.close():
public void viewAllPlanGroups() throws Exception {
System.out.println("Viewing all plan groups info thru Hibernate");

Session s = factory.openSession();
setMemMarker();
Transaction tx=null;
try {
tx = s.beginTransaction();

List worksheets=null;Iterator iter=null;
long currTime=0;

currTime=System.currentTimeMillis();
worksheets = s.createCriteria(PlanWorksheet.class).add( Expression.eq("planworksheet_id", new Long(10636))).list();
eet_id=10636");
System.out.println("Num of ws ="+worksheets.size());

iter = worksheets.iterator();
while ( iter.hasNext() ) {
PlanWorksheet ai = (PlanWorksheet) iter.next();
System.out.println("hash code="+ai.hashCode());
System.out.println(ai);
}

System.out.println("Total query time="+(System.currentTimeMillis()-currTime));
outMemory(worksheets.size());
tx.commit();

}
catch (Exception e) {
if (tx!=null) tx.rollback();
throw e;
}
finally {
s.close();
}
}

public static void main(String[] args) throws Exception {

final Main test = new Main();

Configuration cfg = new Configuration();
cfg.configure();
test.factory = cfg.buildSessionFactory();
test.viewAllPlanGroups();

//expect cache to be hit on second call-but goes to DB again....
test.viewAllPlanGroups();

}

Full stack trace of any exception that occurs:

Oracle 8.1.7 driver

org.hibernate.auction.Main
15:56:06,094 INFO Environment:469 - Hibernate 2.1.6
15:56:06,094 INFO Environment:498 - hibernate.properties not found
15:56:06,109 INFO Environment:529 - using CGLIB reflection optimizer
15:56:06,109 INFO Environment:540 - JVM does not support Statement.getGenerated
Keys()
15:56:06,109 INFO Environment:551 - JVM does not support LinkedHasMap, LinkedHa
shSet - ordered maps and sets disabled
15:56:06,109 INFO Environment:554 - using workaround for JVM bug in java.sql.Ti
mestamp
15:56:06,125 INFO Configuration:895 - configuring from resource: /hibernate.cfg
.xml
15:56:06,125 INFO Configuration:867 - Configuration resource: /hibernate.cfg.xm
l
15:56:06,500 INFO Configuration:331 - Mapping resource: ../eg/org/hibernate/auc
tion/PlanWorksheet.hbm.xml
15:56:06,625 INFO Binder:229 - Mapping class: org.hibernate.auction.PlanWorkshe
et -> maxdata.planworksheet
15:56:06,719 INFO Configuration:1053 - Configured SessionFactory: null
15:56:06,719 INFO Configuration:627 - processing one-to-many association mappin
gs
15:56:06,719 INFO Configuration:636 - processing one-to-one association propert
y references
15:56:06,734 INFO Configuration:661 - processing foreign key constraints
15:56:06,750 INFO Dialect:82 - Using dialect: net.sf.hibernate.dialect.OracleDi
alect
15:56:06,765 INFO SettingsFactory:63 - Use outer join fetching: true
15:56:06,765 INFO DriverManagerConnectionProvider:42 - Using Hibernate built-in
connection pool (not for production use!)
15:56:06,765 INFO DriverManagerConnectionProvider:43 - Hibernate connection poo
l size: 20
15:56:06,781 INFO DriverManagerConnectionProvider:77 - using driver: oracle.jdb
c.driver.OracleDriver at URL: jdbc:oracle:thin:@172.25.44.194:1521:fkt2
15:56:06,781 INFO DriverManagerConnectionProvider:78 - connection properties: {
user=madmax, password=cubby}
15:56:06,781 INFO TransactionFactoryFactory:31 - Transaction strategy: net.sf.h
ibernate.transaction.JDBCTransactionFactory
15:56:06,797 INFO TransactionManagerLookupFactory:33 - No TransactionManagerLoo
kup configured (in JTA environment, use of process level read-write cache is not
recommended)
15:56:07,125 INFO SettingsFactory:103 - Use scrollable result sets: true
15:56:07,125 INFO SettingsFactory:106 - Use JDBC3 getGeneratedKeys(): false
15:56:07,125 INFO SettingsFactory:109 - Optimize cache for minimal puts: false
15:56:07,125 INFO SettingsFactory:115 - echoing all SQL to stdout
15:56:07,125 INFO SettingsFactory:118 - Query language substitutions: {}
15:56:07,125 INFO SettingsFactory:129 - cache provider: net.sf.hibernate.cache.
EhCacheProvider
15:56:07,125 INFO Configuration:1116 - instantiating and configuring caches
15:56:07,297 INFO SessionFactoryImpl:118 - building session factory
******pw obj created ****cnt=1
Finalizing object of id 0
15:56:07,922 INFO SessionFactoryObjectFactory:82 - Not binding factory to JNDI,
no JNDI name configured
Viewing all plan groups info thru Hibernate
free memory=133157816
Hibernate: select this.planworksheet_id as planwork1_0_, this.last_update_date a
s last_upd2_0_, this.planworksheet_id as planwork1_0_, this.planversion_id as pl
anvers3_0_, this.planmaster_id as planmast4_0_, this.plangroup_id as plangrou5_0
_, this.description as descript6_0_, this.from_merch_name as from_mer7_0_, this.
create_date as create_d8_0_, this.plan_count as plan_count0_, this.name as name0
_ from maxdata.planworksheet this where this.planworksheet_id=?
******pw obj created ****cnt=2
Num of ws =1
hash code=3426001
id=10636 description=null create_date=2004-04-01 00:00:00.0
plan_count=4211649 last_update_date=2004-04-01 00:00:00.0 from_merch_name=
Company : Merchandise Total - TOTAL
Total query time=78
free memory=133119912
memory used=38 KB
memory used per record=38312 bytes
Viewing all plan groups info thru Hibernate
15:56:08,281 WARN SessionImpl:3400 - unclosed connection
free memory=132974216
Finalizing object of id 10636
Hibernate: select this.planworksheet_id as planwork1_0_, this.last_update_date a
s last_upd2_0_, this.planworksheet_id as planwork1_0_, this.planversion_id as pl
anvers3_0_, this.planmaster_id as planmast4_0_, this.plangroup_id as plangrou5_0
_, this.description as descript6_0_, this.from_merch_name as from_mer7_0_, this.
create_date as create_d8_0_, this.plan_count as plan_count0_, this.name as name0
_ from maxdata.planworksheet this where this.planworksheet_id=?
******pw obj created ****cnt=3
Num of ws =1
hash code=7346727
id=10636 description=null create_date=2004-04-01 00:00:00.0
plan_count=4211649 last_update_date=2004-04-01 00:00:00.0 from_merch_name=
Company : Merchandise Total - TOTAL
Total query time=32
free memory=133099296
memory used=-124 KB
memory used per record=-124672 bytes
15:56:08,437 INFO SessionFactoryImpl:534 - closing
15:56:08,437 INFO DriverManagerConnectionProvider:143 - cleaning up connection:

[code][/code]

anthony · **Posted:** Tue Aug 31, 2004 4:43 pm

Quote:

ehcache doen't support transactionnal
change to read or read-write

anoopb · **Posted:** Tue Aug 31, 2004 4:48 pm

Thanks for the reply.
I was actually trying out all possible caching usage parameters to get caching working. It still doesn't work for me even if I use read or read-write. I observed that average memory consumption does seem to go up with a cache-usage parameter specified, so I guess something is going in the cache....though why a SQL execution/object creation happens is what I am not quite sure of..

anthony · **Posted:** Tue Aug 31, 2004 5:27 pm

show ehcache configuration file

anoopb · **Posted:** Wed Sep 01, 2004 8:38 am

Here is the ehcache.xml..thanks.

Code:

<ehcache>

    <!-- Sets the path to the directory where cache .data files are created.

         If the path is a Java System Property it is replaced by
         its value in the running VM.

         The following properties are translated:
         user.home - User's home directory
         user.dir - User's current working directory
         java.io.tmpdir - Default temp file path -->
    <diskStore path="java.io.tmpdir"/>


    <!--Default Cache configuration. These will applied to caches programmatically created through
        the CacheManager.

        The following attributes are required for defaultCache:

        maxInMemory       - Sets the maximum number of objects that will be created in memory
        eternal           - Sets whether elements are eternal. If eternal,  timeouts are ignored and the element
                            is never expired.
        timeToIdleSeconds - Sets the time to idle for an element before it expires.
                            i.e. The maximum amount of time between accesses before an element expires
                            Is only used if the element is not eternal.
                            Optional attribute. A value of 0 means that an Element can idle for infinity
        timeToLiveSeconds - Sets the time to live for an element before it expires.
                            i.e. The maximum time between creation time and when an element expires.
                            Is only used if the element is not eternal.
                            Optional attribute. A value of 0 means that and Element can live for infinity
        overflowToDisk    - Sets whether elements can overflow to disk when the in-memory cache
                            has reached the maxInMemory limit.

        -->
    <defaultCache
        maxElementsInMemory="10000"
        eternal="true"
        timeToIdleSeconds="120"
        timeToLiveSeconds="120"
        overflowToDisk="true"
        />

    <!--Predefined caches.  Add your cache configuration settings here.
        If you do not have a configuration for your cache a WARNING will be issued when the
        CacheManager starts

        The following attributes are required for defaultCache:

        name              - Sets the name of the cache. This is used to identify the cache. It must be unique.
        maxInMemory       - Sets the maximum number of objects that will be created in memory
        eternal           - Sets whether elements are eternal. If eternal,  timeouts are ignored and the element
                            is never expired.
        timeToIdleSeconds - Sets the time to idle for an element before it expires.
                            i.e. The maximum amount of time between accesses before an element expires
                            Is only used if the element is not eternal.
                            Optional attribute. A value of 0 means that an Element can idle for infinity
        timeToLiveSeconds - Sets the time to live for an element before it expires.
                            i.e. The maximum time between creation time and when an element expires.
                            Is only used if the element is not eternal.
                            Optional attribute. A value of 0 means that an Element can live for infinity
        overflowToDisk    - Sets whether elements can overflow to disk when the in-memory cache
                            has reached the maxInMemory limit.

        -->

    <!-- Sample cache named sampleCache1
        This cache contains a maximum in memory of 10000 elements, and will expire
        an element if it is idle for more than 5 minutes and lives for more than
        10 minutes.

        If there are more than 10000 elements it will overflow to the
        disk cache, which in this configuration will go to wherever java.io.tmp is
        defined on your system. On a standard Linux system this will be /tmp"
        -->
    <cache name="my.package.Class"
        maxElementsInMemory="10000"
        eternal="false"
        timeToIdleSeconds="300"
        timeToLiveSeconds="600"
        overflowToDisk="true"
        />
        

    <!-- Sample cache named sampleCache2
        This cache contains 1000 elements. Elements will always be held in memory.
        They are not expired. -->
    <cache name="my.package.Class.collection"
        maxElementsInMemory="1000"
        eternal="true"
        overflowToDisk="false"
        /> -->

    <!-- Place configuration for your caches following -->
<cache name="org.hibernate.auction.PlanWorksheet"
        maxElementsInMemory="10000"
        eternal="true"
        timeToIdleSeconds="300"
        timeToLiveSeconds="600"
        overflowToDisk="true"
        />    

</ehcache>

anoopb · **Posted:** Wed Sep 08, 2004 1:29 pm

Hi,

Can anyone point me as to what I'm doing wrong? Caching across sessions in hibernate does mean no new object instances are created for already fetched data, right(discounting expiry issues and/or 'staleness')?

Thanks,
Anoop

zoglmannk · **Joined:** Wed Sep 08, 2004 5:59 pm **Posts:** 5

anoopb wrote:

Hi,

Caching across sessions in hibernate does mean no new object instances are created for already fetched data, right(discounting expiry issues and/or 'staleness')?

Thanks,
Anoop

Hey,

I'm no expert, but I'll let you know what I have learned by experience.

*If you call a query several times (even within the same session) it will execute it every time and only pull the primary key for the objects you wish to load. If there is a miss, it will make another another call to the db (which could cause n+1 selects the first time a query is ran).

*typically queries retrieving only primary keys run exceptionally quick because often times the query can be executed without reading anything but the b-tree indexes.

*You can cache query results preventing the query from being re-executed on the database, but you'll need to be absolutely sure that the query results never change.

*Secondary caching results in substantial gains when you're ratio between reads to writes is high.

*Be aware of using an object who's session is closed or disconnected. you will run into problems if you make a call that wants to instantiate a lazily loaded collection. the answer is to re-load the object for the new session.

And now more towards your original problem..

*If you are using secondary cache, you do have to ensure that no two objects with the same identifier are associated with the session (or at least you don't try to save one of them). If for some reason you get this, you'll need to call evict or re-work you application logic/flow.

*It is my understanding that a new object IS created for each session when it makes a successful cache hit. The reason being is that multiple sessions (possibly in different threads) maybe accessing that object. It would result in bizarre behavior if each session did not have its own copy to muck with before a particular session called saveOrUpdate/update.

gavin · **Posted:** Wed Sep 08, 2004 8:28 pm

Quote:

*You can cache query results preventing the query from being re-executed on the database, but you'll need to be absolutely sure that the query results never change.

This is not true - the Hibernate query cache is aware of changes made to data in the query cache.

caleb_phillips · **Joined:** Thu Sep 02, 2004 2:31 pm **Posts:** 12

I am also not an expert, but my experience over the last few days has lead me to believe that the ehcache caches objects by id, and so is used only when loading by id. I had enabled caching on several classes, but did not see any performace gains in repeated querying. I was querying by a field other than the id, but I noticed in the logs the ehcache was creating entries by class name and id.
As a test, I created a map that mapped the field I was querying by to the id and did a load instead of a find. In this case it was very obvious that the object was being loaded from memory without a trip to the database. This is obviously not what you want to do, but I was hoping it might shed some light on your situation.
Please correct me if I am wrong about the way the cache works.
By the way, I did not having query caching turned on.

gavin · **Posted:** Wed Sep 08, 2004 10:55 pm

use the query cahce, Query.setCacheable(true)

zoglmannk · **Joined:** Wed Sep 08, 2004 5:59 pm **Posts:** 5

gavin wrote:

Quote:

*You can cache query results preventing the query from being re-executed on the database, but you'll need to be absolutely sure that the query results never change.

This is not true - the Hibernate query cache is aware of changes made to data in the query cache.

I didn't compose that statement well. Yes, if you change an object returned from a cached query, call save on that object, and re-run the query again, the results of the query will reflect your change.

However, if you change data that effects which objects are returned from a cached query, this is where things can go wrong. Use cached queries when you know that the objects returned will not vary. Right?

christian · **Posted:** Thu Sep 09, 2004 9:04 am

No, it doesn't go wrong. Hibernate has some mechanisms to know what has been modified and reacts accordingly. If your statement means "if _someone else_ modifies the data in the database" than yes, you should not use any non-transaction scoped caching, such as the second-level and the query cache in Hibernate.

anoopb · **Posted:** Thu Sep 09, 2004 9:19 am

Sorry if I missed the answer in this thread, but my original question: does Hibernate re-use object instances in its secondary-level cache when the data is not presumed stale? If so, what must I do to make this happen(I tried Query.setCacheable(true)-didnt seem to work, plus I understand a Query object is anyway alive only during the lifetime of a Session?).

It's pretty critical for us in our prototyping to prove that Hibernate will not recreate object instances on subsequent queries if the data is there in the secondary cache.

Thanks for any help...

christian · **Posted:** Thu Sep 09, 2004 9:25 am

It will of course create objects (the second level cache stores objects in a disassembled format), but Hibernate will not hit the database if a) the query itself is cached in the query cache and b) all data returned by the query is in the normal second-level cache. If only a) is true (the query cache holds only the _identifiers_ of entities) you will get database hits.

Note that there is nothing wrong with this system, its just flexible.

For a full explanation, read chapters 5 and 7 of Hibernate in Action.

zoglmannk · **Joined:** Wed Sep 08, 2004 5:59 pm **Posts:** 5

christian wrote:

No, it doesn't go wrong. Hibernate has some mechanisms to know what has been modified and reacts accordingly. If your statement means "if _someone else_ modifies the data in the database" than yes, you should not use any non-transaction scoped caching, such as the second-level and the query cache in Hibernate.

Wow! That is really cool. I might have to go poking around at the source code to see how you guys implemented that. You have peaked my interest. At the minimum I'll check into using this in my own application!

Sorry for being a newbie. I was implying modifying data within a given process and not by "someone else". I made the assumption that it was very difficult to ensure that objects returned from a cached query were the correct results without having to re-query the database. And thus the cached query mechanism was not implemented to ensure that the objects returned were always the correct ones. I thought that it was only good for queries that read from "read only" data and the query criteria were specified on "read only" data. It's very cool to be wrong in this case. :)

Besides a queries cache results expiring because of time or space, what are the circumstances that will cause hibernate to requery the database?