Indexes sometimes not created when saving an entity

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

I've implemented hibernate search and everything seems to be working fairly well in general. But I just noticed that if I loop through many entities saving them 1 at a time, the first several might not get indexed. The first one might be indexed and the next 2 are not, then all of the rest are indexed perfect... If I batch them into let's say 20 at a time in 1 transaction, the first 20 are not indexed and the rest of the batches index perfectly. This is all very predictable and repeatable. But I have no idea why the indexes aren't always created in the beginning.

The entities are always saved perfectly and the code specified on page 154 of "Hibernate Search in Action" works perfectly to run through every entity and manually index them. It's the automated indexing through hibernate which is faulty...

Any ideas what's going on? I tried logging debug statements and there's no debugging output from hibernate search or hibernate during the entity saving process.

I'm using Spring 3.0.1 with Hibernate 3.3.1. Are these incompatible with the latest version of Hibernate Search (3.1.1)?

I save the new entities that are sometimes not indexed in the beginning of saving many of them with:

Code:

sessionFactory.getCurrentSession().save(entity);

The above works perfect when I'm just saving 1 entity at a time through the web... it's when doing them fast through a batch process that the beginning doesn't get indexed. They ALWAYS get saved to the database though, that works perfect.

Since I'm not using annotations, I've had to set these up in spring:

Code:

    <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource"/>
        <property name="eventListeners">
            <map>
                <entry key="post-update"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-insert"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-delete"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-recreate"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-remove"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
                <entry key="post-collection-update"><bean class="org.hibernate.search.event.FullTextIndexEventListener"/></entry>
            </map>
        </property>

hardy.ferentschik · **Posted:** Fri Mar 12, 2010 8:13 am

Which transaction manager are you using. One of the biggest problems people have using Spring and Hibernate Search is configuring the transaction manager correctly. See for example viewtopic.php?f=9&t=998155
If you do a search you will find many more similar posts.
However, I would expect in this case that no automatic indexing would occur, but you seem to imply that only the first few entities are not indexed and afterwards it works.
I would check transaction management first. Also you should be able to get some Hibernate debug trace. Don't you see any Hibernate log messages?

--Hardy

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

hardy.ferentschik wrote:

Which transaction manager are you using. One of the biggest problems people have using Spring and Hibernate Search is configuring the transaction manager correctly. See for example viewtopic.php?f=9&t=998155
If you do a search you will find many more similar posts.
However, I would expect in this case that no automatic indexing would occur, but you seem to imply that only the first few entities are not indexed and afterwards it works.
I would check transaction management first. Also you should be able to get some Hibernate debug trace. Don't you see any Hibernate log messages?

I think I have a little different situation from the post you give above in that the vast majority of my entities do get indexed. It's just the first several entity-saving transactions that sometimes don't index the entities. I'm using the following transaction manager:
org.springframework.orm.hibernate3.HibernateTransactionManager

I would guess this is an issue where hibernate is not calling the "post-insert" event listeners always in the beginning? Seems strange though why it wouldn't... Again, manually indexing the entities always works perfect. And saving entities individually through a webpage always seems to work perfect as well as far as indexing goes.

This is really a big problem for us as our data set is very large and we can't continuously manually index every entity.

For the hibernate log messages, this is what I see at the moment I insert one of the entities I want indexed:

Code:

2010-03-12 15:55:28,294 DEBUG [http-8080-1] SQL - insert into Person (Deleted, Primary_Name_Id, Email, Male, Description, Media_Hosting, FamilySearch_User, Fami
lySearch_Session, FamilySearch_Session_Used, FamilySearch_Id, Activity_Score, Activity_Added_People, Activity_Changed_People, Activity_Added_Email, Activity_Sco
re_Past_Month, Activity_Added_People_Past_Month, Activity_Changed_People_Past_Month, Activity_Added_Email_Past_Month, Activity_Posts, Activity_Remembrances, Log
in_Password, Login_Token, Login_Enabled, Login_Created_Date, Login_Verified_Date, Last_Login_Encouragement_Notification_Date, Profile_Image_in_Id, Language_Id, 
Birth_Event_Id, Birth_Order, Death_Event_Id, Change_Id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2010-03-12 15:55:28,303 DEBUG [http-8080-1] IdentifierGeneratorFactory - Natively generated identity: 97
2010-03-12 15:55:28,303 DEBUG [http-8080-1] AbstractBatcher - about to close PreparedStatement (open PreparedStatements: 1, globally: 1)
2010-03-12 15:55:28,303 DEBUG [http-8080-1] EventSourceTransactionContext - No FullTextIndexEventListener was registered
2010-03-12 15:55:28,303 DEBUG [http-8080-1] EventSourceTransactionContext - No FullTextIndexEventListener was registered
2010-03-12 15:55:28,307 DEBUG [http-8080-1] AbstractSaveEventListener - executing identity-insert immediately

So the "No FullTextIndexEventListener was registered" log statement is after every single entity, even the ones whose indexes were created properly. Don't know what it means... Is this a clue to what's going on?

Finally, is it possible hibernate search doesn't work well with Spring 3.0.1?

sanne.grinovero · **Posted:** Sat Mar 13, 2010 10:32 am

Quote:

I think I have a little different situation from the post you give above in that the vast majority of my entities do get indexed.

Your setup might be right, but please double check as it's possible for a wrong configuration to fail inconsistently: it might work *sometimes*; that makes developers very unhappy ;-)

Quote:

EventSourceTransactionContext - No FullTextIndexEventListener was registered

This debug message is a bit misleading - I just fixed the message - what it was meant to say is "FullTextIndexEventListener was not registered as FlushEventListener".

Search really requires proper usage of transactions, but so many people where having trouble with this that we introduced a sort of "safety net" in 3.1.1 which does some attempts to a) work around the flushing b) log some warnings
see: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-178

What I guess from your logging message is that this safety net is being triggered, but the FullTextIndexEventListener listener was not registered as FlushEventListener.

When the workaround for missing transactions kicks in && this listener is not found then your changes are essentially dropped, which is what happens to you.. so you should:
1) verify transactions setup as Hardy mentioned - the fact it works often doesn't mean it's ok
2) verify you always are using a transaction
3) optionally: register the listener, as explained on reference docs, to be able to workaround cases you missed in step 2 (shouldn't be needed)

Cheers,

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

Thank you for your response Sanne. I've registered the FullTextIndexEventListener as the flush listener. Now, I'm not getting the "EventSourceTransactionContext - No FullTextIndexEventListener was registered" warning anymore. So hibernate is firing the flush event and from the logs it looks like hibernate is firing it for EVERY single entity we're trying to index. But still the entities are not indexed correctly (most of them are indexed and some are not). So it appears the bug is with Hibernate Search.

I highly doubt this is a transaction issue. We're always using transactions in the standard Spring way. The transaction manager is configured like this:

Code:

   <bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionFactory"/>
    </bean>

    <aop:config>
        <aop:advisor pointcut="execution(* com.example.account.*Listener.onApplicationEvent(..))" advice-ref="txListenersAdvice"/>
        <aop:advisor pointcut="execution(* com.example.*.*Manager.*(..))" advice-ref="txAdvice"/>
    </aop:config>

    <tx:advice id="txListenersAdvice" transaction-manager="transactionManager">
        <tx:attributes>
            <tx:method name="onApplicationEvent"/>
        </tx:attributes>
    </tx:advice>

    <tx:advice id="txAdvice" transaction-manager="transactionManager">
        <tx:attributes>
            ....
            <tx:method name="add*"/>
            <tx:method name="delete*"/>
            <tx:method name="*" read-only="true"/>
        </tx:attributes>
    </tx:advice>

and we 're using an addEntity() method to add the entities. So the transaction manager should work correctly. We've never had a problem before with transactions, that all works fairly smooth.

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

I think I just figured out why Hibernate Search is silently breaking on us. So let me be more concrete with what we're doing.

We need to search Person entities. We also need to import people, a lot of people. The importing needs to happen in batches, otherwise the transactions get way too big very fast. 20 people at once seems to be the sweet spot. But these people have relationships to other people in the imported set. For example (this is simplified and not exactly how we implement it):
personParent.getChildrenEvents().contains(personChild.getBirthEvent())

So the parent may be in 1 batch and the child in another batch. That's ok, the event will be persisted along with the first person. When the other person is persisted, they'll be connected to the same event.

Works great.

Now comes Hibernate Search. I need to be able to make searches like: parent.name:"John Smith"
So in order to make those kinds of searches happen, I have to set @IndexedEmbedded on the birthEvent field of a person and then the parents field of an event.

I think Hibernate Search breaks (without even a debug message) because the @IndexedEmbedded is pointing to an object that has not yet been persisted. That's why this only happens sporadically in the beginning of an import. And it also explains why this is always easily reproducible.

So how can we make this work? I guess Hibernate Search needs the Id of the parent if a child was persisted first and we can't have that Id yet. Right? Seems to me that the solution has to be to tell Hibernate Search not to index these people that are about to be imported. And then manually index them later? Hmm, that's not ideal certainly as I have to figure out which people were not indexed yet. If something goes wrong in the middle or immediately after importing people, I could end up with some people not indexed.

What do you guys think? Is the above scenario really a problem for Hibernate Search? Is there a good workaround?

sanne.grinovero · **Posted:** Sun Mar 14, 2010 12:50 pm

Quote:

I think Hibernate Search breaks (without even a debug message) because the @IndexedEmbedded is pointing to an object that has not yet been persisted.

Hibernate Search doesn't care if the object is persisted, managed or not; it will fail to receive automatic events from Hibernate Core if the objects aren't persisted, but if in the end you got them all persisted we can assert that all needed events where fired (Core itself also persists entities listening to the same events).

Even if it was possible that Search was missing some events, the @IndexedEmbedded links to other objects are followed independently from the events, it's the root entity which needs the event.
It's more likely that you didn't set some bi-directional reference from "the other side": the framework won't follow "null" references ;-)

Quote:

So the parent may be in 1 batch and the child in another batch. That's ok, the event will be persisted along with the first person. When the other person is persisted, they'll be connected to the same event.

Works great.

I totally lost you here. Could you create a minimal test? Feel free to create an issue on JIRA and attach it there.

It's not an option to batch-import without indexing and then batch-index the entities?

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

s.grinovero wrote:

Quote:

I think Hibernate Search breaks (without even a debug message) because the @IndexedEmbedded is pointing to an object that has not yet been persisted.

Hibernate Search doesn't care if the object is persisted, managed or not; it will fail to receive automatic events from Hibernate Core if the objects aren't persisted, but if in the end you got them all persisted we can assert that all needed events where fired (Core itself also persists entities listening to the same events).

Even if it was possible that Search was missing some events, the @IndexedEmbedded links to other objects are followed independently from the events, it's the root entity which needs the event.
It's more likely that you didn't set some bi-directional reference from "the other side": the framework won't follow "null" references ;-)

I wasn't suggesting Hibernate Search isn't getting the flush or post-insert hibernate events. I'm suggesting Hibernate Search breaks when indexing entities with relationships which are not yet persisted.

s.grinovero wrote:

Quote:

So the parent may be in 1 batch and the child in another batch. That's ok, the event will be persisted along with the first person. When the other person is persisted, they'll be connected to the same event.

Works great.

I totally lost you here. Could you create a minimal test? Feel free to create an issue on JIRA and attach it there.

It's not an option to batch-import without indexing and then batch-index the entities?

batch-import without indexing and then batch-index does work and it looks like that's what I'll have to do as a workaround, but it's not ideal... because then I have to worry about what happens to indexing should any issues arise during import. Mistakes in that situation can be big problems.

I'll write up a minimal test soon. The issue is pretty simple, it would be something like this:

Code:

class Person {
 @DocumentId
 Integer Id;
 @Field
 String name;
 @IndexedEmbedded
 Event birthEvent;
 @ContainedIn
 Set<Event> ChildrenEvents;
}

class Event {
 Integer Id;
 @IndexedEmbedded
 Set<Person> parents;
 @ContainedIn
 Set<Person> children;
}

now set
personA.birthEvent = eventA;
personB.parentEvents.add(eventA);

personB is the parent of personA. We save personA within a hibernate transaction. This, I'm pretty sure, chokes Hibernate Search. The problem is personB was not yet persisted, yet the @IndexedEmbedded path made Hibernate Search try to read personB. I'm guessing this is a problem for Hibernate Search because it needs personB's Id, but the Id has not yet been set? I just notice in Luke that the Ids are indexed.

Does that make sense? Is this a problem for Hibernate Search? Is Hibernate Search just not designed to work with this type of situation?

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

I also noticed I get 4 of these lines when starting tomcat:
WARN [main] DocumentBuilderContainedEntity - @DocumentId specified on an entity which is not indexed by itself. Annotation gets ignored. Use @Field instead.

There is only 1 entity with an @DocumentId annotation and that's the Person entity I'm indexing. I'm guessing the reason I see those warnings is because through @IndexedEmbedded's the Person entity is referenced 2 times. Maybe some backtracking (I use @ContainedIn's where appropriate) causes that number to double?

Anyway, I mentioned this because maybe it helps track down the above issue...

sanne.grinovero · **Posted:** Mon Mar 15, 2010 5:20 am

The @DocumentId is needed on the root entity, the embedded object's id is ignored while indexing the root entity - right this warning is misleading.

We now that having the object not persisted should not raise problems as the "JBoss Cache Searchable" is an edition of Hibernate Search which indexes annotation mapped pojos which are stored in the cache, so as I said before

Quote:

Hibernate Search doesn't care if the object is persisted, managed or not

- that is only relevant to fire the appropriate events, not for graph navigation.

I'll be happy to help, but we need the test.

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

You're right that Hibernate Search was not having problems just because some IndexedEmbedded objects were not persisted. It's pretty strange why Hibernate Search just doesn't write the index sometimes without any debugging output or warning. But we've been able to reproduce it in a much smaller test case. I wasn't sure if JIRA is the right place to post this since I'm still not 100% convinced this is a Hibernate Search bug. Certainly seems like it is, but what I'm doing is very simple...

So here's the test code. There are 3 objects; Person, Event and ParentOfBirthEvent. A Person is connected to their parents through person.getBirthEvent().getParentsOf(), then for each ParentOfBirthEvent call parentOfBirthEvent.getParent(). Likewise, a Person is connected to their children through person.getParentOfBirthEvents(), then for each ParentOfBirthEvent call parentOfBirthEvent.getEvent().getChildren().

Code:

@Indexed
public class Person implements Serializable {

    @DocumentId
    private Long id;
    @ContainedIn
    private Set<ParentOfBirthEvent> parentOfBirthEvents;
    @IndexedEmbedded(depth = 4)
    private Event birthEvent;

    public Person() {
        birthEvent = new Event();
        birthEvent.getChildren().add(this);
        parentOfBirthEvents = new LinkedHashSet<ParentOfBirthEvent>();
    }

    ...getters and setters for all the fields
}

public class Event implements Serializable {

    protected Long id;
    @IndexedEmbedded
    protected Set<ParentOfBirthEvent> parentsOf;
    @ContainedIn
    protected Set<Person> children;

    public Event() {
        parentsOf = new HashSet<ParentOfBirthEvent>();
        children = new HashSet<Person>();
    }

    ... getters and setters for all the fields
}

public class ParentOfBirthEvent implements Serializable {
    private Long id;
    @IndexedEmbedded
    private Person parent;
    @ContainedIn
    private Event event;

    public ParentOfBirthEvent(Person parent, Event event) {
        this.parent = parent;
        this.event = event;
    }

    ... getters and setters for all the fields
}

here are the hibenate mappings:

Code:

<hibernate-mapping>
    <class name="people.Person" table="Person">
        <id name="id" column="Id" type="long"><generator class="native"/></id>
        <many-to-one name="birthEvent" class="people.Event" column="Birth_Event_Id" cascade="save-update" not-null="true"/>
        <set name="parentOfBirthEvents" table="Parent_of_Birth_Event" cascade="save-update">
            <key column="Parent_Id"/>
            <one-to-many class="people.ParentOfBirthEvent"/>
        </set>
    </class>
</hibernate-mapping>

<hibernate-mapping>
    <class name="people.Event" table="Event">
        <id name="id" column="Id" type="long"><generator class="native"/></id>
        <set name="parentsOf" table="Parent_of_Birth_Event" inverse="true">
            <key column="Event_Id"/>
            <one-to-many class="people.ParentOfBirthEvent"/>
        </set>
        <set name="children" inverse="true">
            <key column="Birth_Event_Id"/>
            <one-to-many class="people.Person"/>
        </set>
    </class>
</hibernate-mapping>

<hibernate-mapping>
    <class name="people.ParentOfBirthEvent" table="Parent_of_Birth_Event">
        <id name="id" column="Id" type="long"><generator class="native"/></id>
        <many-to-one name="parent" class="people.Person" column="Parent_Id" unique-key="Relationship_Constraint" insert="false" update="false"/>
        <many-to-one name="event" class="people.Event" column="Event_Id" unique-key="Relationship_Constraint" cascade="save-update" not-null="true"/>
    </class>
</hibernate-mapping>

here's the code to test the indexing:

Code:

public class PersonDaoHibernate {
    protected SessionFactory sessionFactory;

    public long addPerson(Person person) {
        return (Long) sessionFactory.getCurrentSession().save(person);
    }

    public SessionFactory getSessionFactory() {
        return sessionFactory;
    }

    public void setSessionFactory(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }
}

public class PersonManager implements Serializable {

    private PersonDaoHibernate personDao;

    public void savePeople(Person[] people) {
        for (Person person : people)
            personDao.addPerson(person);
    }

    public void setPersonDao(PersonDaoHibernate personDao) {
        this.personDao = personDao;
    }
}


public class PersonManagerTests extends AbstractDependencyInjectionSpringContextTests {

    private PersonManager personManager;

    @Override
    protected String[] getConfigLocations() {
        return new String[] {"classpath:config/application-context.xml"};
    }

    public void testSavePeople() {
        personManager.savePeople(createParentAndChild());
    }

    private Person[] createParentAndChild() {
        Person[] people = new Person[2];
        Person parent = new Person();
        Person child = new Person();
        connectChildToParent(child, parent);
        people[0] = parent;
        people[1] = child;
        return people;
    }

    private void connectChildToParent(Person child, Person parent) {
        Event birthEvent = child.getBirthEvent();
        child.setBirthEvent(birthEvent);
        ParentOfBirthEvent parentOfBirthEvent = new ParentOfBirthEvent(parent, child.getBirthEvent());
        parent.getParentOfBirthEvents().add(parentOfBirthEvent);
    }

    public void setPersonManager(PersonManager personManager) {
        this.personManager = personManager;
    }
}

The PersonDaoHibernate methods are executed in a transactional context and the PersonManager methods are not. What happens is a parent and then his child are saved, in separate transactions. This works great for hibernate, both parent and child get persisted. But Hibernate Search only indexes the child. Now, if I change the order in which the Person objects are saved, namely if I save the child first and then the parent, both are indexed correctly.

There are many more oddities, but the above is as simple as we could make an example for. Anything simpler basically works.

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

Can anyone see what might be the issue above? We've got Hibernate Search fully working otherwise and would absolutely love to use it... but I need to know ASAP if we can actually get it to work completely or we have to migrate to Solr.

The example case above is simple enough, I have a difficult time believing Hibernate Search is unable to deal with such a common situation. So I'm guessing my annotations are wrong or there is some other issue I've overlooked.

sanne.grinovero · **Posted:** Sat Mar 27, 2010 7:29 am

Hi, I've converted your test in a testcase I could run without spring - makes it much simpler, you can find many examples of simple tests in the source code of Hibernate Search itself.

I'm experiencing problems with cascading, you didn't seem so specify any cascade strategy and expect everything to be saved, while you're only saving the Person. So I enabled cascade-persist to make sure that related objects (as Event and ParentOfBirthEvent) where saved together with the Person.
After I enabled the cascade strategies on all relations (shouldn't be necessary, just choose a root entity and select a cascading path), I still had a problem: when persisting the first Person I also have to persist the second one, as they are related this needs to be done, or in alternative I should cut the relation in the first transaction. So when you ask to persist the second person, it's failing as you're asking to persist an already-persisted entity - because the first transaction did.

You should make a clear choice: either you persist the whole graph in a single transaction, or you persist a first group of self-contained entities in a first transaction, and add changes in a second transaction - but the self-contained transaction shouldn't contain references to non-persistent entities.
I think all your trouble is coming from the fact that you have the transactions wrapping the dao only - if this is still related to the batch work you mentioned in first post I'd suggest you to store all persons in a first batch, avoiding relations, and add all relations in a second pass - the alternatives are to write the whole object graph in a single transaction (that might not be an option) or to clearly identify independent sub-graphs to be written one in each transaction.

I'm not sure to why you don't get proper exceptions about the cascading, as I mentioned I had adapted your testcase to remove Spring from the scenario, but hope the reasoning about the transactions is understandable and framework independent.

mueller · **Joined:** Mon Mar 10, 2008 6:40 pm **Posts:** 114

Hi Sanne, thank you so much for looking at this in detail. Though I'm not sure I understand what you mean when you said I "didn't seem so specify any cascade strategy and expect everything to be saved, while you're only saving the Person." The cascade statements are in the hibernate mapping files above... That should be pretty clear. Basically when you save a Person, the birth event and the ParentOfBirthEvent objects it refers to are saved as well. But the cascading stops at Event. So saving one person NEVER saves another person.

Again, hibernate works perfect. Hibernate Search is the problem. Everything gets saved correctly in the database. But not everything gets indexed.

Saving everything in 1 transaction is not an option because there are way too many to save. And typically they're all connected in 1 huge graph, so separating them into disjoint graphs is not an option either.

The reason I made this a Spring example minimal test case is because Hibernate Search is failing for us in a Spring environment... and maybe the problem is Hibernate Search is incompatible with Spring 3.0?

sanne.grinovero · **Posted:** Sun Mar 28, 2010 9:57 am

hi mueller,
sorry for the rant on cascading I didn't see them in your xml - I'm used to annotations and didn't pay attention.
I'm now able to see that indeed something is going wrong: core is generating an unexpected sequence of events, I'll investigate more and ping someone on that.

It might be possible to provide a workaround in Search, could you help in creating an appropriate testcase which could follow the Hibernate Search conventions?
I'll follow up with my version of your test converted into jpa annotations, but this isn't asserting anything. Could you add some expectations to it, to have it fail and possibly green as a check for a workaround? Or is the single smoke assert I inserted good enough to cover your needs?
Also please check the generated schema is matching your xml mapping, I didn't dedicate much time on it.

I'm going to create an issue on JIRA and attach my patch there, I'll post a link here.