[HS 5.2.0] Index update not triggered when entity is deleted

pawel.predki · **Posted:** Sat May 23, 2015 11:47 am

According to the documentation:

Quote:

By default, every time an object is inserted, updated or deleted through Hibernate, Hibernate
Search updates the according Lucene index

This makes much sense. However, I noticed that the Lucene index is not updated when I'm deleting entities from my code.

I tried using both CriteriaDelete and NativeNamedQuery methods but in both cases the index is not updated. I haven't tried the EntityManager.remove method but I don't want to use it because it is very limited.

Code:

   public void removeTestEntity(long id) {
      CriteriaDelete<TestEntity> cd = cb
            .createCriteriaDelete(TestEntity.class);
      Root<TestEntity> root = cd.from(TestEntity.class);
      cd.where(cb.equal(root.get(TestEntity_.id), id));
      em.createQuery(cd).executeUpdate();
   }

Code:

        @NamedNativeQueries({ @NamedNativeQuery(name = "@SQL_DELETE_TEST_ENTITY", query = "DELETE FROM test_entity WHERE id = :id") })

   public void removeTestEntity(long id) {
      Query q = em.createNamedQuery("@SQL_DELETE_TEST_ENTITY");
      q.setParameter("id", id);
                q.executeUpdate();
   }}

Below is the config I'm using:

Code:

                        <!-- Properties for Hibernate -->
         <property name="hibernate.show_sql" value="false" />
         <property name="hibernate.jdbc.batch_size" value="100" />
         <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
         <property name="hibernate.cache.use_second_level_cache"
            value="true" />
         <property name="hibernate.cache.use_query_cache" value="true" />

         <!-- Properties for Hibernate Search -->
         <!-- Define Lucene version -->
         <property name="hibernate.search.lucene_version" value="LUCENE_CURRENT" />

         <!-- appropriate directory provider -->
         <property name="hibernate.search.default.directory_provider"
            value="filesystem" />

         <!-- local master location -->
         <property name="hibernate.search.default.indexBase" value="/opt/wildfly/luceneIndex" />

         <!-- We don't need synchronous index updates - speed is not that important 
            to us -->
         <property name="hibernate.search.default.worker.execution"
            value="async" />

         <!-- Enable statistics MBean -->
         <property name="hibernate.search.jmx_enabled" value="true" />

pawel.predki · **Posted:** Mon May 25, 2015 8:32 am

I tested the problem with

Code:

void javax.persistence.EntityManager.remove(Object entity)

and the index update is triggered.

This is bad news. How can I then update the index if I want to delete entities according to some parameters in batch? e.g. older than a certain date? Would I need to load all of them using some query and then iterate in a loop calling em.remove? This seems very wasteful.

pawel.predki · **Posted:** Mon May 25, 2015 9:38 am

I also tried HQL:

Code:

      
Query hqlQuery = em.createQuery("DELETE TestEntity WHERE id IN :ids");
hqlQuery.setParameter("ids", ids);
return hqlQuery.executeUpdate();

But it also doesn't trigger the index update.

emmanuel · **Posted:** Tue May 26, 2015 5:01 am

Hibernate Search is notified only if you use entityManager.remove(object) or session.delete(object).
If you bypass the persistence context and update the database directly, Hibernate Search cannot know about it.

pawel.predki · **Posted:** Tue May 26, 2015 6:00 am

emmanuel wrote:

Hibernate Search is notified only if you use entityManager.remove(object) or session.delete(object).
If you bypass the persistence context and update the database directly, Hibernate Search cannot know about it.

This is most unfortunate... Loading all the entities from the database and then removing them in a loop seems to include a lot of unnecessary overhead. Is there a way to manually delete the index entries using similar techniques as Criteria Query, HQL, etc.? Or is the only way to do the looping?

sanne.grinovero · **Posted:** Mon Jun 01, 2015 7:34 am

Hi Pawel,
there is a way to perform a "bulk delete" using a Lucene Query. It was introduced only recently and it's not exposed as a public API as it requires a bit of expertise.

It's also experimental, we will probably polish the API - and would love to hear your feedback on that as well!

https://hibernate.atlassian.net/browse/HSEARCH-1765

Here is an example:

https://github.com/hibernate/hibernate-search/blob/7f77072741863fa0c767e28bb0cac26fafc2aff8/engine/src/test/java/org/hibernate/search/test/backend/DeleteByQueryTest.java#L83

this test doesn't use the Hibernate Session but you can do the same via a SearchIntegrator

Code:

SearchIntegrator si = searchFactory.unwrap( SearchIntegrator.class );
org.hibernate.search.spi.SearchIntegrator.getWorker().performWork( ... );

Please keep in mind that at this stage it only supports org.hibernate.search.backend.spi.SingularTermDeletionQuery (for good reasons - see below); to make the most out of it you might want to index ad-hoc tokens to use for the bulk delete definitions.

We can discuss adding more types and to eventually make this into a stable public API but that's a complex subject:
queries would also need a stable serialization strategy which won't break easily at version upgrades, and also allowing more complex queries introduces some reordering concerns on the backend processing (it gets complex to figure which reodering of Work operations is valid, and we do currently quite some reordering to optimise performance of the index writing component).

pawel.predki · **Posted:** Wed Jun 03, 2015 10:42 am

sanne.grinovero wrote:

Hi Pawel,
there is a way to perform a "bulk delete" using a Lucene Query. It was introduced only recently and it's not exposed as a public API as it requires a bit of expertise.

It's also experimental, we will probably polish the API - and would love to hear your feedback on that as well!

https://hibernate.atlassian.net/browse/HSEARCH-1765

Here is an example:

https://github.com/hibernate/hibernate-search/blob/7f77072741863fa0c767e28bb0cac26fafc2aff8/engine/src/test/java/org/hibernate/search/test/backend/DeleteByQueryTest.java#L83

this test doesn't use the Hibernate Session but you can do the same via a SearchIntegrator

Code:

SearchIntegrator si = searchFactory.unwrap( SearchIntegrator.class );
org.hibernate.search.spi.SearchIntegrator.getWorker().performWork( ... );

Please keep in mind that at this stage it only supports org.hibernate.search.backend.spi.SingularTermDeletionQuery (for good reasons - see below); to make the most out of it you might want to index ad-hoc tokens to use for the bulk delete definitions.

We can discuss adding more types and to eventually make this into a stable public API but that's a complex subject:
queries would also need a stable serialization strategy which won't break easily at version upgrades, and also allowing more complex queries introduces some reordering concerns on the backend processing (it gets complex to figure which reodering of Work operations is valid, and we do currently quite some reordering to optimise performance of the index writing component).

Hey Sanne,

I rewrote my methods to use the SessionManager and I currently don't have the time to look into this experimental API, unfortunately. It's good to know that something like that is in the making. When things slow down at work and we have more time to experiment, I'll come back to this issue, thanks!