Hello,

I am migrating to Hibernate Search 5.6.1 in order to try using the database-to-Elasticsearch syncing capabilities of Hibernate Search.
I have already integrated with my ElasticSearch cluster and am indexing and querying using the low-level REST API, however I am searching for a managed syncing solution (only) and since you guys are working on it I figured it's better to leave it to the people with the know-how.

I already have my entity data indexed in ES. I split it into 3 indices (LIVE 1, LIVE 2 and ARCHIVE) where the two live indices hold more recent and relevant data (split into two halves for query performance reasons), and the archive index holds all old data. I need to be able to index an entity into one of these three indices, selectively, depending on my own custom logic. I know I can use @Indexed(index="INDEX_NAME") but how cat I choose the index name at index time?

Additionally, I use the 'routing' parameter for the ES Document APIs when doing CRUD operations against ES, and I configured my indices so that the 'routing' parameter is required. I use 'routing' to control in which shard the document should be placed, which is good for search performance. I use the value of a property in the document ast the value of the routing parameter. Can I use routing with Hibernate Search somehow?

Any help is appreciated!

Hello,

splitting the data across indexes can be done using Hibernate Search's sharding capabilities - not to be confused with the sharding applied by Elasticsearch.

In particular see "Dynamic Sharding" as it allows you to define the name of the index you want to target for each operation:
- https://docs.jboss.org/hibernate/search ... c-sharding

The "routing" option is not supported yet. It seems like a good idea to add it! Please open a feature request:
- https://hibernate.atlassian.net/projects/HSEARCH/issues

Looks like we should consider matching the "dynamic sharding" capabilities of Hibernate Search to map directly to the routing API. I'd be most interested in your suggestions on this: if you could describe your use case on the feature request that will be a great help.

Thanks!

Thank you, it looks like dynamic sharding could work for selecting a single index to put the data in.

I created a feature request with my use case here: https://hibernate.atlassian.net/browse/HSEARCH-2634

thanks that feature request, that's very useful feedback for our team.

I hope we can include this in the 5.8 version, as that's what we're working on. That will imply that you'll have to upgrade to Hibernate ORM 5.2.x, I hope that's not a problem?

This version will also support both Elasticsearch 2.x and Elasticsearch 5.x, and switch to the new Elasticsearch client so it should be worth it.

Unfortunately I don't think I will be able to upgrade to Hibernate 5.2 just yet, since it requires me upgrading to Java 8 and that is a project on its own. Any chance ES 5.x and this '_routing' change will be backported to 5.6.1?

Definitely looking forward to ES 5! I will give more feedback as I test

I'm sorry we won't be able to backport these things, that's way too much work and we're a small team.

We spent a lot of energy to make sure that people would have a very smooth experience to upgrade to Java 8; if we're missing anything there which could help please let us know that as well.

For example - while I don't know about other libraries which you might be using - the versions you mentioned on HSEARCH-2634 are all already compatible with Java8.

Thanks!