I've read page 154 of Hibernate Search in Action where Listing 5.22 gives an example of going through all Item entities and indexing them one by one. I did this and it worked... when my tables were small. Unfortunately as I've outlined in this stackoverflow question about ScrollableResults, it seems that MySQL loads all rows for a ScrollableResult set into RAM before proceeding... so OutOfMemoryErrors are inevitable:
http://stackoverflow.com/questions/2826319/using-hibernates-scrollableresults-to-slowly-read-90-million-records

What's the best approach for this?

The problem with the ScrollableResults is really a db problem in this case. Using pagination is a potential solution, but it seems that has not worked for you either.

I recommend you try the mass indexer api which got introduced with the latest release of Search (3.2.0.Final). Check the online documentation for this new API and some usage examples.

--Hardy