Hibernate Search 3.2.1 Indexes Unchanged Collections

pushedbytime · **Posted:** Thu Aug 18, 2011 2:53 pm

I've read on the various blogs and release notes that Hibernate Search 3.3 made lots of performance improvements, and some related to collections. But the wording is a little vague from what I've read. On Emmanuel's blog I see:
"if an entity changes in Hibernate Core but none of its indexed properties change, we won't trigger indexing anymore."

But what about if a particular property isn't changed? Right now I have a collection on an entity that isn't touched during a transaction, but another simpler property(integer) is. This property is loaded lazily. The depth on the collection property is also pretty deep, so the triggered joins can be expensive.

Its slightly more complicated than that since the property that is changed is directly on the entity being saved, while the collection that gets a load triggered is a within an entity referenced by the entity being saved. To be clear the model is:

Code:

class A{
    @ManyToOne(fetch=FetchType.LAZY, cascade = {CascadeType.MERGE, CascadeType.PERSIST})
    @IndexEmbedded(depth=5)
    private B bee;

    @Field
    private Integer foo;

}

class B{
    @ManyToMany(fetch=FetchType.LAZY, 
   cascade = { CascadeType.MERGE, CascadeType.PERSIST})
      @JoinTable(
      name="join_table", 
      joinColumns={@JoinColumn(name="some_col")}, 
      inverseJoinColumns={@JoinColumn(name="som_col_2")} 
   )
    @IndexedEmbedded(depth = 2) 
    private Set<C> cSet;
}

And we are essentially just doing the following with async indexing enabled:

Code:

A someA = session.get(id);
someA.setFoo(newValue);
sess.merge(someA);

So someA.bee is never touched. Yet the embedded indexing still happens even though we never dirtied bee. Re-indexing cSet is unnecessary and expensive.

In short my questions are:
1. Was this a known problem in hibernate search 3.2.X?
2. Will upgrading to 3.3.X for sure fix this?

I just want to ask on the forum first since the last time I toyed with the idea of upgrading core to 3.6 a lot of things broke, so it will take some effort to even try to upgrade, before I can even verify in reality that the performance improvements will be beneficial in this situation.

Thanks in advance!

sanne.grinovero · **Posted:** Thu Aug 18, 2011 5:17 pm

Hi,
we had many performance improvements in 3.3, even more in 3.4 and 4.0 (Alpha1 released today). Some of these improvements are related to similar issues as what you mention, BUT from what I see you're really changing a directly indexed field: in this case we will always need to rebuild the Lucene Document for this entity - together with all it's IndexedEmbedded elements - as the Lucene API doesn't support updating a single field so we have to rewrite it completely.

If performance is a problem, I'd suggest enabling a second level cache so that the loading of these additional elements is not as expensive. You could also improve performance by getting advantage of the other new features in more up-to-date Search versions, but I don't think it will solve this specific problem.

About issues with Hibernate Core 3.6 .. I hope you reported them, we don't want to have people stuck on older versions!

pushedbytime · **Posted:** Thu Aug 18, 2011 5:43 pm

Sanne,

That makes perfect sense now(the need to re-write the entire document). We do have a lot of really deep embedded indexing, many of which are collections of entities, so we could probably benefit from upgrading. Bulk re-indexing in particular right now is fairly slow right now for relatively few documents( < 1 million takes > 30 minutes).

Thanks for the prompt reply