With blackhole I expect a strong performance improvement in any environment. Could you verify it is being enabled via debugging/logging/profiling ? I've just realized that in certain conditions the "blackhole" option is being ignored and no warning is given - now sure why, have to look into it.
Sorry probably I'm missing something: should I rebuild the index with "blackhole" enabled? I suppose no, since I've read it discards indexing changes, however in the log I see
02 May 2012 14:36:53,723 -- [LoggerFactory] -- initialized "blackhole" backend. Index changes will be prepared but discarded!
During search there aren't concurrent modifications to the index.
I've seen that java/lang/OutOfMemoryError class is loaded but in the log I can't see any exception/error thrown: do you think there can be a memory problem?
If one happens in the backend workers, since they are asynchronous, we would not be able to "throw it back" to the application but it should at least be logged. You could also configure a custom error handler to make sure your application deals with these exceptional cases: http://docs.jboss.org/hibernate/search/ ... e/#d0e2443
My problem is during the search phase, not while indexing is in progress: can I use a custom error handler also for the search phase?
What JVM are you using? Any chance you could try a different JVM ? as far as I remember IBM provides several JVMs, some are free some are commercial, it might be worth trying a different one.
As far as we know the only available JVM for iSeries is the one shipped with the operating system.
Can you check used memory? The numbers from your document look like reasonable, but I don't know how much memory is being used/free in the JVM or if the operating system is swapping too much.
I've made a simple memory profiling using Runtime.getRuntime().totalMemory() before and after the query: before the query totalMemory() is about 157MB and after the query is about 350 MB, with a maximum heap size of 1GB. Increasing the available heap size to 2 GB didn't change anything.
Now we've asked IBM support: we'll better check the JVM behavior and the availability of a different JVM. I'll let you know if we find a problem in the JVM settings.