I am still trying to find an answer to a very simple question:

Is it possible to conditionally index an entity? If I have millions of rows in a database and I only want to have a subset appear in an index, how is this possible? The simplest case is that I have dates on entities and I only want entities within a specific date range to be located within an index. I don't mean 'queried'. They can't be in the index because the indexing will take far to long, and too large when a system has been brought down, or initially started.

I would like to keep a subset of data, so I can use a RAMDirectory too. Hopefully this all works well with EHCache since I have distributed cache.

Does this make much sense?

It seems there is something vital missing, like a simple 'index query' that could be provided to the indexer for the specific entity. Perhaps there is such a mechanism?

Thanks for your time,

Briggs.

Hi,

Not yet, please join the brainstorming with your use case and suggestions: HSEARCH-471


Depending on if you're concerned on the event processing (real time index update) or the initial index creation, maybe you're more interested in HSEARCH-499 ?
But writing some code which populates initial indexes using your favourite criteria is trivial, just add some conditions before creating the scrollableresult using the old approach instead of the MassIndexer : search-batchindex-flushtoindexes


That seems a good fit for the new Infinispan backend, so that you also have a distributed index.

Thank you very much for the response. I plan on spending some time in the morning researching your suggestions.

Edit: BTW, I never noticed the createCriteria(String) method on the fullTextSession(). I believe that is a huge part of my solution. I'd always been creating the index the 'the old way', so it is hardly a big addition to my API. That will get me around server restarts too I believe. Though I see how fast that mass indexer is. It was a huge performance boost for my tests earlier today.

yes, it's supposed to be lots faster. So you should try and see if indexing all with the MassIndexer or indexing a part of it using the criteria API, what is faster depends on too many factors.

Of course, providing a patch to combine them would be the best approach: http://opensource.atlassian.com/project ... SEARCH-499
The code is trivial, it wasn't done yet as I've an hard time trying it out: so if someone could contribute it, it would be better.