Indexes sometimes not created when saving an entity

sanne.grinovero · **Posted:** Sun Mar 28, 2010 12:54 pm

http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-476

emmanuel · **Posted:** Mon Mar 29, 2010 11:26 am

Hi,
I've implemented a fix for your problem.
http://fisheye.jboss.org/viewrep/Hibern ... 2&r2=19126

Can you try and apply it and let me know if that works for you?

I've also implemented an optimization on a related problem that I had in my todo list for a while but that requires more testing
http://fisheye.jboss.org/viewrep/Hibern ... 3&r2=19127

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

thank you emmanuel! Some basic quick testing shows that your patch works! The only problem for us is that your patch requires the latest beta of Hibernate Search and Hibernate Core... That's not an option for us in production and we use Spring 3.0. I'm not sure how compatible those are even if we were able to use a beta in production.

I'm fine waiting until the final versions are released and spring supports them. Is there a way to just disable indexing for a group of entities about to be persisted?

emmanuel · **Posted:** Wed Mar 31, 2010 8:10 am

I think this patch can be applied as is to the branch 3.1
http://anonsvn.jboss.org/repos/hibernat ... ntity.java

The addWorkForEmbeddedValue method is there and has not changed since.

mschipperheyn · **Joined:** Wed Nov 05, 2003 7:22 pm **Posts:** 211

I'm having similar problems. I can only get an object to be indexed if I either batch index the database or explicitly create a transaction on the FullTextSession. However, this is nog getting me into a transaction conflict due to the fact that I'm using the Spring HibernateTemplate:

Code:

try {
         FullTextSession fts = Search.getFullTextSession(this
               .getSessionFactory().getCurrentSession());
         Transaction tx = fts.beginTransaction();
         fts.index(offer);
         tx.commit();
      } catch (TransactionException te) {
         log.error(te);
      }

Can you give me the Jira Id, so I can monitor progress?

Kind regards,

Marc

sanne.grinovero · **Posted:** Tue Apr 13, 2010 6:14 pm

Quote:

Can you give me the Jira Id, so I can monitor progress?

this thread was referring to HSEARCH-476
which is solved in Hibernate Search 3.2.CR1, released yesterday

But I don't understand why you think this is related to the same issue?
Transaction issues with Spring are an indicator of a misconfiguration of transactions in Spring: please check Spring Hibernate Search Wiki

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

emmanuel wrote:

I think this patch can be applied as is to the branch 3.1
http://anonsvn.jboss.org/repos/hibernat ... ntity.java

The addWorkForEmbeddedValue method is there and has not changed since.

Hi Emmanuel, thank you for the patch. We ended up using hibernate 3.5 and spring 3.0.2 when they became compatible and final shortly after you provided the patch above. However, we can't get hibernate 3.5 to work properly (works fine for everything, all our unit/functional tests pass, but is orders of magnitude slower than 3.3.1 when inserting many entities, rendering it unusable). Rather than waste more time on hibernate 3.5, we have to revert back to hibernate 3.3.1.

So that means I need to use your patch above. Your patch above is for a branch, not a release right? So I have to compile code not yet even in beta status? That's probably going to be an issue for us (hibernate 3.5 is even final and isn't working for us). I could probably use and compile the release code if all I had to do was add an if (id != null). Would that work for the addWorkForEmbeddedValue method in hibernate search 3.1.1?

emmanuel · **Posted:** Thu Apr 15, 2010 3:26 am

Yes the patch is off the branch. But the branch is almost identical to Hibernate Search 3.1.1: there should not be any issue here if you apply it off 3.1.1.

Have you investigated a bit on the slowness during batch inserts: we have not experienced that but maybe there is a bug lying here or more probably some subtle change in JPA 2's semantic (and thus our impl) that leads to this slowness.
Anyway it would be good to investigate. If you can isolate the culprit part of your domain model and attach it to JIRA, we might find a clue.

sanne.grinovero · **Posted:** Thu Apr 15, 2010 4:42 am

FYI there's another thread referring to degraded performance on 3.5:
https://forum.hibernate.org/viewtopic.php?f=1&t=1003886

going to ask more there.

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

So it seems our hibernate 3.5 issue is really with hibernate search 3.2. I'm very surprised we didn't think of this before, it just seemed like a hibernate issue because the slowdown and problems occurred with the database.

Basically we have these extreme slowdowns when saving more than a thousand or 2 entities in batches of 20. But when we remove the hibernate search eventListeners from hibernate, everything is superfast again. The extreme slowdown occurs whether we use lucene or jms as the backend.

We use Spring 3.0.2 to manage our transactions and hibernate, so maybe the issue lies there as well? Normally Spring closes the session and the transaction when finishing the method that was declared to initiate a transaction.
I'm guessing hibernate search 3.2 makes hibernate/spring not close the session?

Also, this issue does not occur if the entities are simple, that is they don't have relationships to common entities that are cascaded in the save. So what we do is each person connects to an event. 2 people can connect to the same event. The first person saved also saves the event. The second person saved doesn't need to save the event. If we remove these event relationships, then hibernate search 3.2 doesn't cause these extreme slowdowns.

We'll try to post a minimal test case soon.

emmanuel · **Posted:** Fri Apr 16, 2010 3:40 am

The test will help. We will wait for it before Hibernate Search 3.2 Final.

emmanuel · **Posted:** Fri Apr 16, 2010 3:42 am

Which version of Hibernate Search where you trying btw? CR1?

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

emmanuel wrote:

Which version of Hibernate Search where you trying btw? CR1?

We tried on both beta1 and CR1, also tried both hibernate 3.5 and 3.5.1.

amin-mc · **Joined:** Wed Oct 03, 2007 2:31 pm **Posts:** 205

Hi

Are you using spring managed JTA?

Cheers
Amin

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

amin-mc wrote:

Hi

Are you using spring managed JTA?

Yes, specifically here are the relevant lines of my application-context.xml

Code:

    <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource"/>
        <property name="eventListeners">
            <map>
                <entry key="post-update"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-insert"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-delete"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-recreate"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-remove"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-update"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
            </map>
        </property>
        <property name="mappingResources">
            <list>
                <value>repository/my_entitities.hbm.xml</value>
            </list>
        </property>
        <property name="hibernateProperties">
            <value>
                hibernate.dialect=org.hibernate.dialect.MySQLInnoDBDialect
                hibernate.search.default.indexBase=indexes
                hibernate.search.worker.execution=async
                hibernate.search.worker.jms.connection_factory=java:comp/env/jms/ConnectionFactory
                hibernate.search.worker.jms.queue=java:comp/env/queue/hibernatesearch
                hibernate.search.worker.backend=jms
                hibernate.search.default.directory_provider=org.hibernate.search.store.FSSlaveDirectoryProvider
                hibernate.search.default.sourceBase=indexes_source
            </value>
        </property>
    </bean>

    <bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionFactory"/>
    </bean>