Automatic optimization slows down manual indexing

byte · **Posted:** Fri Sep 11, 2009 4:27 am

Hello *,

is there a way to disable automatic optimization for manual indexing?

Greetings Matthias

hardy.ferentschik · **Posted:** Fri Sep 11, 2009 5:40 am

Hi,

At the moment there is no way to turn automatic optimization off while manual indexing. There is a new batch inserting backend in the trunk of Search now implemented by Sanne. Not sure how things are handled there.

There is a set- and getOptimizerStrategy in SearchFactoryImpl. You could try to get the current OptimizerStrategy and replace it with a NoOpOptimizerStrategy for the time of the manual indexing. That said, I haven't checked if there are any implications doing this. You definitely have to investigate this closer before going down this path.

--Hardy

byte · **Posted:** Fri Sep 11, 2009 6:32 am

Thank you for your answer hardy! :)
I think i have to live with this for now and will check your recommendation at the time it is really required.
Have a nice weekend!

Greetings Matthias

sanne.grinovero · **Posted:** Fri Sep 11, 2009 8:39 am

Hi,
when using the new MassIndexer there are some methods to setup this behaviour:

Code:

MassIndexer m = fulltextSession.createIndexer()
   .optimizeAfterPurge(false)
   .optimizeOnFinish(false);

the "purge" phase is at the beginning, right after the deletion of all older documents; the finish phase is after all new
documents have been added.
The default is true for both, but optimization is always disabled during intermediate steps for top performance (ignoring
automatic optimization settings).

In current stable versions (3.1.x) the default is to not optimize automatically; optimization is enabled if
optimizer.operation_limit.max and/or optimizerStrategy are set in configuration.
So if you want to disable automatic optimization *globally* make sure you don't set this values.

As a temporary hack, and not completely correct, you could cast the SearchFactory to SearchFactoryImpl,
doing so you will be able to expose

Code:

public addOptimizerStrategy(DirectoryProvider<?> provider, OptimizerStrategy optimizerStrategy)

and change the optimizerStrategy for a specific Directory to NoOpOptimizerStrategy.
I was mentioning this as not completely correct as there is no correct synchronization in this use case (the class is sealed at framework's startup): there is no guarantee another thread will "see" the change, but you'll probably use the indexing from the same thread so this should work fine for you.

A more interesting alternative is to implement your own OptimizerStrategy and write the code you like most,
whet I'd recommend however is to try the new MassIndexer.