Hi.
I'm trying to create a lucene index of about 3.4 million entities. No matter what I've tried, I still have some memory leaks. How should I optimize this index's creation?
I'm running Hibernate Search with Seam 2.1.1.GA.
My versions:
Hibernate 3.3.1.GA
Hibernate EntityManager 3.4.0.GA
Hibernate Search 3.1.1.GA
I'm running a Postgres database.
Here's the code I'm using
Code:
int step = 5000;
int indexed = 0;
// Get the min and max Id
long minId = ((Number) entityManager.createQuery("select min(id) from Book b").getSingleResult()).longValue();
long maxId = ((Number) entityManager.createQuery("select max(id) from Book b").getSingleResult()).longValue();
boolean fetch = true;
FullTextSession fullTextSession = Search.getFullTextSession((Session) entityManager.getDelegate());
while (fetch) {
List<Book> books = entityManager.createQuery(
"select b from Book b where id >=:minId and id <:maxId and role=:role").setParameter("minId", minId).setParameter("maxId", minId + step)
.setParameter("role", Role.STOCK).getResultList();
// Index every books
for (Book book : books) {
fullTextSession.index(book);
indexed++;
}
books = null;
fullTextSession.flushToIndexes();
fullTextSession.clear();
entityManager.clear();
minId += step;
// Every books were fetched
if (minId > maxId) {
fetch = false;
}
}