HibernateSearch multi-tenancy and master/slave JMS

yge45 · **Posted:** Tue Apr 08, 2014 3:07 am

Hello,

I try to implement multi-tenant application using Hibernate Search dynamic sharding (version 4.4.2.Final). I've got expected behaviour when I use lucene backend but if I switch to master/slave JMS it doesn't work.

I would expect my master to be able to update index for many tenants. From slave, message is posted with index name x.y.z.MyClassName.ShardID (seems OK) but on master side AbstractJMSHibernateSearchController can't find matching indexManager.

So, as illustrated with code bellow, I extend AbstractJMSHibernateSearchController to create indexManagers on startup based on initial shard names provided by ShardIdentifierProvider. It seems to behave as I expect.

Because of using reflection to get HS private field, I'm not very happy with this solution. Do you see a better way to do it. Is there a built-in solution I've missed ?

Thanks for help.

Code:

   @Override
   public void afterPropertiesSet() throws Exception {
      SearchFactoryImplementor factory = ContextHelper
            .getSearchFactory(getSession());

      Map<Class<?>, EntityIndexBinding> indexBindings = factory.getIndexBindings();

      // iterate over all entityIndexBinding load on start up
      for (EntityIndexBinding entityIndexBinding : indexBindings.values()) {

         // if binding is dynamic sharding one then get one IndexManager per
         // shard (initial ones) in order to create it => they wil exist on
         // message consumption
         if (entityIndexBinding instanceof DynamicShardingEntityIndexBinding) {
            // get inital shards
            Set<String> shards = entityIndexBinding.getShardIdentifierProvider().getAllShardIdentifiers();

            // get indexBaseName by reflection... beurk
            Field field = DynamicShardingEntityIndexBinding.class.getDeclaredField("rootDirectoryProviderName");
            field.setAccessible(true);
            String indexBaseName = (String) field.get(entityIndexBinding);

            // create indexManager for all shards
            for (String shard : shards) {
               factory.getIndexManagerHolder().getOrCreateIndexManager(
                           indexBaseName,
                           shard,
                           (DynamicShardingEntityIndexBinding) entityIndexBinding);
            }
         }
      }
   }

sanne.grinovero · **Posted:** Tue Jun 02, 2015 6:58 am

Hi,
sorry I missed this. If I may summarize, what you've found is that the AbstractJMSHibernateSearchController does a "getIndexManager( indexname )" but doesn't trigger an IndexManager initialization, for those IndexManagers which should be defined by a DynamicShardingStrategy, correct?

That seems like a bug, I've filed https://hibernate.atlassian.net/browse/HSEARCH-1886

Your workaround seems correct. An alternative workaround would be to run a full-text query on all types, for example you could use Object.class as a target type. Running the query should trigger the initialization of all missing IndexManager instances, but I didn't try that.
An open problem would be that you should guarantee that this initialization happens before the backend controller starts processing incoming messages; maybe you could run such a FullTextQuery in the initialization block of the AbstractJMSHibernateSearchController implementor?

emmanuel · **Posted:** Thu Jun 04, 2015 5:03 am

Sanne, I am not sure your query proposition for indexing will work.

If you run this query when the instance is just started, the list of shard will be empty (as it seems to use dynamic shards).
If you run this query for each MDB message, you would need to detect the new shard and have it identified by the shard identifier provider before execution the query?

Is that what you had in mind?

emmanuel · **Posted:** Thu Jun 04, 2015 5:06 am

Nevermind, I did not see that it's shard identifier provider has the list of shards fromt he get go and that Yoann's changes were executed at init time.