Mass Indexer + JTA not allowing for lazily loaded entities

astralbodies · **Posted:** Fri Sep 02, 2011 12:41 pm

I am attempting to use the max indexer against my project which works great until I change one of the entities to lazily load the entity. I get the following exception:

Code:

2011-09-02 08:58:59,095 ERROR [org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer] (Hibernate Search: collectionsloader-2) error during batch indexing: 
org.hibernate.search.bridge.BridgeException: Exception while calling bridge#objectToString
        class: com.project.domain.candidate.CandidateLocation
        field bridge: pk
        at org.hibernate.search.bridge.util.ContextualExceptionBridge.buildBridgeException(ContextualExceptionBridge.java:78)
        at org.hibernate.search.bridge.util.ContextualException2WayBridge.objectToString(ContextualException2WayBridge.java:84)
        at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.index(EntityConsumerLuceneworkProducer.java:157)
        at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.indexAllQueue(EntityConsumerLuceneworkProducer.java:128)
        at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.run(EntityConsumerLuceneworkProducer.java:99)
        at org.hibernate.search.batchindexing.OptionallyWrapInJTATransaction.run(OptionallyWrapInJTATransaction.java:107)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)
Caused by: org.hibernate.LazyInitializationException: could not initialize proxy - no Session
        at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:167)
        at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:215)
        at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:190)
        at com.project.domain.lucene.CandLocationPKBridge.objectToString(CandLocationPKBridge.java:21)
        at org.hibernate.search.bridge.util.ContextualException2WayBridge.objectToString(ContextualException2WayBridge.java:77)
        ... 7 more

The entity that I am indexing has a field bridge, and that is where the failure is occurring. I have Hibernate set up to use JTA (via Spring) and am injecting the JtaTransactionManager into the AnnotationSessionFactoryBean. The Hibernate JTATransactionFactory returns false for isTransactionManagerRequired prevents org.hibernate.search.batchindexing.OptionallyWrapInJTATransaction.run() from creating a transaction around the indexing. Inside of wrapTransaction the following block clearly says don't wrap:

Code:

      final TransactionFactory transactionFactory = factory.getSettings().getTransactionFactory();
      if ( !transactionFactory.isTransactionManagerRequired() ) {
         //Today we only require a TransactionManager on JTA based transaction factories
         log.trace( "TransactionFactory does not require a TransactionManager: don't wrap in a JTA transaction" );
         return false;
      }

Is this a bug? If my session is being held in JTA and the mass indexer is running multiple threads, how does it know to use the current transaction that I started before the mass indexer was called?

astralbodies · **Posted:** Fri Sep 02, 2011 1:01 pm

Forgot to mention, I am using Hibernate 3.6.6.Final, Hibernate Search 3.4.1.Final and Spring 3.0.5.

sanne.grinovero · **Posted:** Fri Sep 02, 2011 1:08 pm

Hi, welcome.

clearly since isTransactionManagerRequired is false it should be self-explanatory that a transaction is not needed?
But this check is only related to a container verifying that the resources are properly accessed inside the transaction scope, it is not related with options such as "my session is being held in JTA".

Could you please elaborate what you mean by that? I'm not familiar with Spring related patterns.

I think the problem is not related to JTA, but more likely related to
org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.indexAllQueue(Session)

The MassIndexer is a multithreaded pipeline; in that method the to-be-indexed entity is being re-attached to a new Session, and that somehow fails to re-attach to the new session also the failing proxy.

Is the main indexed entity (the root of the @Indexed graph) eagerly loading collections or other proxies? Could you try making those associations lazy?

astralbodies · **Posted:** Fri Sep 02, 2011 1:32 pm

Spring likes to manage the transaction demarcation and delegates to JTA as the overall transaction manager. Normally when using JTA you specify hibernate.current_session_context_class=jta but in this case Spring provides org.springframework.orm.hibernate3.SpringSessionContext to be able to look up the current transaction for Session.getCurrentSession().

The main indexed entity has a primary key object and two Long values. The primary key is being indexed via a FieldBridge. The primary key object has two @ManyToOne associations in it, and I am attempting to set these entities to lazy loading since I only need the IDs for indexing. When I set either to lazy, I get the error about there being no session.

It is possible that the problem lies with the session factory being controlled by Spring as a local session. Even though I'm using JTA it's through Spring's transaction manager. Hibernate knows the JTATransactionManager but delegates to Spring because of the current_session_context_class. I'm assuming, then, that in a normal JTA (non-Spring) environment that each thread being spun off for the indexer is magically able to get the current session directly out of JTA without using a transaction manager?

astralbodies · **Posted:** Fri Sep 02, 2011 1:42 pm

I did just manually set hibernate.current_session_context_class=jta and I am getting the same results.

I am using JBoss EAS 5.0 if that makes any difference.

sanne.grinovero · **Posted:** Sun Sep 04, 2011 2:53 pm

It would be useful if I could reproduce your issue.

Would you be able to create a test case using Hibernate Search ?

If you check out the source code, you can add a test in the main module, which is preferable, assuming you can remove Spring and still reproduce.

If Spring is really a requirement to reproduce this, you can add a test to the hibernate-search-integrationtest Maven module, which has Spring dependencies.

I'm not sure yet how to classify this, it's either a bug in the session association code, in which case the peculiarity is in the mapping you're using, or it's an issue with the Spring+Search+JTA Mix, which is not covered enough by unit tests. Either way, adding your test to our continuous integration tests should make sure this is fixed and stays that way in the future.

Could you please create an issue on JIRA and attach your test there?

astralbodies · **Posted:** Tue Oct 18, 2011 12:47 pm

It looks like the culprit is a combination of Spring and the container I am using (JBoss EAS 5). I believe the JTA resource being pulled from the container was causing a conflict because it is JTA 1.0 not 1.1. I had included the JTA 1.1 JAR in my project but when transaction synchronization would occur with Spring in the mix, it wasn't holding onto the transaction long enough.

I switched over to a Spring Local Transaction manager since I'm really only using a single data source and a JMS connection factory. Now it appears the transactions are being handled correctly and are not getting any lazy load exceptions.

sanne.grinovero · **Posted:** Wed Oct 19, 2011 4:55 am

I'm glad you fixed your problem. Indeed having different copies of JTA is problematic to handle; you could use Tattletale (jboss.org/tattletale) to get a scan of your classpath and check for duplicate class definitions.

astralbodies · **Posted:** Wed Oct 19, 2011 10:06 am

Awesome thanks for that suggestion!!

mschipperheyn · **Joined:** Wed Nov 05, 2003 7:22 pm **Posts:** 211

I'm also running into this problem

Code:

org.hibernate.LazyInitializationException: could not initialize proxy - no Session
   at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:167)
   at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:215)
   at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:190)
   at nl.msw.compraventa.model.URLPlace_$$_javassist_44.toString(URLPlace_$$_javassist_44.java)
   at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:167)
   at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:215)
   at org.hibernate.search.util.HibernateHelper.unproxy(HibernateHelper.java:62)
   at org.hibernate.search.engine.impl.HibernateStatelessInitializer.unproxy(HibernateStatelessInitializer.java:48)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:459)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:552)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:552)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:552)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:443)
   at org.hibernate.search.engine.DocumentBuilderIndexedEntity.createAddWork(DocumentBuilderIndexedEntity.java:380)
   at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.index(EntityConsumerLuceneworkProducer.java:160)
   at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.indexAllQueue(EntityConsumerLuceneworkProducer.java:128)
   at org.hibernate.search.batchindexing.EntityConsumerLuceneworkProducer.run(EntityConsumerLuceneworkProducer.java:99)
   at org.hibernate.search.batchindexing.OptionallyWrapInJTATransaction.run(OptionallyWrapInJTATransaction.java:107)
   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)

The problem seems to occur on the Lazy ManyToOne associations with optional = true. when optional = false, there is no problem.
The relationship is set up as follows:

Code:

@Entity
@Indexed
public class User implements Serializable{

   @Embedded
   @JsonIgnore
   @IndexedEmbedded(prefix="location.")
   public Address getAddress() {
      return address;
   }
}

@Embeddable
@Indexed
public class Address implements Serializable{

   @ManyToOne(cascade = CascadeType.MERGE, fetch = FetchType.EAGER)
   @JoinColumn(name = "FK_CityID")
   @IndexedEmbedded
   public City getCity() {
      return city;
   }


@Entity
@Table(name="City")
@Cache(usage=CacheConcurrencyStrategy.NONSTRICT_READ_WRITE,region=CacheRegion.CITY)
@Indexed
public class City implements Serializable{
//OK
   @Override
   @ManyToOne(optional=false,cascade=CascadeType.MERGE,fetch=FetchType.LAZY)
   @Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE,region=CacheRegion.NEVERCHANGE)
   @JoinColumn(name="FK_CountryID", nullable=false)
   @IndexedEmbedded(depth=1)
   public Country getCountry() {
      return country;
   }
//OK
   @OneToMany(cascade=CascadeType.ALL,fetch=FetchType.LAZY,mappedBy="city")
   @BatchSize(size=25)
   @Fetch(FetchMode.SUBSELECT)
   @Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
   @JsonIgnore
   @IndexedEmbedded
   public Set<Barrio> getBarrios() {
      return barrios;
   }

//BOMB
   @ManyToOne(optional=true,cascade={CascadeType.PERSIST,CascadeType.MERGE},fetch=FetchType.LAZY)
   @Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE,region=CacheRegion.CITY)
   @JoinColumn(name="FK_regionID", nullable=true)
   @IndexedEmbedded(depth=1)
   public Region getRegion() {
      return region;
   }

mschipperheyn · **Joined:** Wed Nov 05, 2003 7:22 pm **Posts:** 211

This sounds similar to this https://hibernate.onjira.com/browse/HSEARCH-1030

astralbodies · **Posted:** Tue Jan 31, 2012 3:19 pm

That does sound pretty close to the problem. I ended up scrapping JTA in our project and went back to a Spring-managed transaction manager scenario. It seemed to have fixed the lazy loading issue in the bridged fields.