[Hibernate Search] Filter by entity class issue

digi9ten · **Posted:** Mon Oct 15, 2007 1:03 am

Hibernate version:
Hibernate-Core: 3.2.5, Hibernate-Annotations: 3.3.0, Hibernate-Search: 3.0.0

Mapping documents:
<property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSDirectoryProvider</property>
<property name="hibernate.search.default.indexBase">lucene-indexes</property>
<property name="hibernate.search.default.refresh">600</property>
<property name="hibernate.search.reader.strategy">shared</property>
<property name="hibernate.search.autoregister_listeners">true</property>

The Location class:
@Indexed(index = "Location")
public class Location extends AbstractAuditable implements Auditable, Deletable, Ownable<AdminUser>, Comparable<Location> {

@IndexedEmbedded(depth = 1)
private AdminUser owner;

@Fields({
@Field(index = Index.TOKENIZED),
@Field(name = "name_forSort", index = Index.UN_TOKENIZED, store = Store.YES)})
private String name;
...
}

Code between sessionFactory.openSession() and session.close():
org.apache.lucene.search.Query luceneQuery = new QueryParser("name", new StandardAnalyzer()).parse("unknown");
FullTextSession fullTextSession = Search.createFullTextSession(session);
FullTextQuery query = fullTextSession.createFullTextQuery(luceneQuery, Location.class);

================================

OK, I (re)Index the Location data before making the call above, and expect to see the 2 records that match the query. Instead I see zero records.

While debugging this issue, I replaced the Location.class parameter in the createFullTextQuery method call, with "new Class[0]". When I do this, I get the 2 expected records, and another entity that simply refers to one of the expected results. Based on your documentation, this is as designed. After making that call, if I execute the original query, or any other valid query after that, and I get my expected results.

Is it possible that my initial indexing isn't taking until I make this 'find all matching classes' query? If so, why?

digi9ten · **Posted:** Mon Oct 15, 2007 9:43 pm

I dug a bit deeper and found something in

Code:

QueryLoader's load(EntityInfo... entityInfos)

, lines 84 and 85.

The method asks the current session to load the given entity by Id, and then a

Code:

Hibernate.isInitialized( element )

call is made. If that is successful, the item is added to the result list being returned.

If I set a breakpoint at that line and manually run that session.load call, it works. So, if the these entities do exist in the system, could the issue be my initial indexing of the entities?

Just to be clear, the ObjectLoader also implements the same interface, Loader, and does not face the same issue, beacause it calls load, and then attempts to initialize the loaded item, and then adds that item to the result list it returns. Shouldn't the QueryLoader pretty much do the same? In as far as being more forgiving of the value returned by session.load.

digi9ten · **Posted:** Tue Oct 16, 2007 6:31 pm

OK...

I modified my local instance of Hibernate Search's QueryLoader. Here's the new List load(EntityInfo... entityInfos) that worked for me.

Code:

    public List load(EntityInfo... entityInfos) {
      final int maxResults = entityInfos.length;
      if ( maxResults == 0) return EMPTY_LIST;
      if (entityType == null) throw new AssertionFailure("EntityType not defined");
      if (criteria == null) criteria = session.createCriteria( entityType );

      DocumentBuilder builder = searchFactoryImplementor.getDocumentBuilders().get( entityType );
      String idName = builder.getIdentifierName();
      int loop = maxResults / MAX_IN_CLAUSE;
      boolean exact = maxResults % MAX_IN_CLAUSE == 0;
      if (!exact) loop++;
      Disjunction disjunction = Restrictions.disjunction();
      for (int index = 0 ; index < loop ; index++) {
         int max = index*MAX_IN_CLAUSE + MAX_IN_CLAUSE <= maxResults ?
               index*MAX_IN_CLAUSE + MAX_IN_CLAUSE :
               maxResults;
         List ids = new ArrayList(max - index*MAX_IN_CLAUSE);
         for (int entityInfoIndex = index * MAX_IN_CLAUSE ; entityInfoIndex < max ; entityInfoIndex++) {
            ids.add(entityInfos[entityInfoIndex].id);
         }
         disjunction.add( Restrictions.in( idName, ids ) );
      }
      criteria.add( disjunction );
      criteria.list(); //load all objects

      //mandatory to keep the same ordering
      List result = new ArrayList(entityInfos.length);
      for (EntityInfo entityInfo : entityInfos) {
         Object element = session.load(entityInfo.clazz, entityInfo.id);
         Hibernate.initialize(element);
         result.add(element);
      }
      return result;
   }

emmanuel · **Posted:** Thu Oct 18, 2007 10:06 pm

Your solution is not good and lead to some other bugs (ie when the index is not completely up-to-date with the database), but clearly something is going wrong :)
It might be due to the hierarchy, I don't know, can you cut out a reproducible test case (as simple as possible if you can) and post that to JIRA, I'll need to have a deeper look

digi9ten · **Posted:** Fri Oct 19, 2007 11:29 pm

I was about to enter a bug, but after thinking about it some more, I realized it was not a bug, but working as designed.

Looking at section 2.3 of Search's documentation, I am told to that within a transaction, I need to call flush for Search to see my changes.

Now that did not work, for me, but I am using Spring to manage my transactions. So, this may have an affect. I have found that during unit tests, setting the hibernate.search.worker.batch_size=1 solves the problem.

emmanuel · **Posted:** Sat Oct 20, 2007 11:56 am

Ahh, interesting, yes Hibernate Search does not propagate the Lucene changes until after the tx is commited.
What is your usecase for needing to search something that you've just inserted and before the tx commit?
I can see some ways to provide that feature, but nobody came with a decent use case so far.

digi9ten · **Posted:** Sat Oct 20, 2007 12:31 pm

Well it really only should happen during a integration test, when you've saved an entity, and immediately after are looking for that saved item to ensure it was saved as expected.

In terms of real-world applications, you may have a service that adds a new set of entities via hibernate, and then applies a query to retrieve a subset of those newly added items. This may all happen within one transactional call. Some pseudo-code to better explain. Note, by the time a call to findBestPossibleLabGroups is made, all the new students have to be properly indexed.

Code:

public Map<String, Set<Person>> addNewStudentsAndReturnBestPossibleLabGroups(String collegeCourseId, Set<Person> nuStudents) {
   // iterate over nuStudents: nuStudent
      // call to hibernate via a dao to 'enroll' nuStudent in collegeCourseId
   
   
}

public Map<String, Set<Person>> findBestPossibleLabGroups(String collegeCourseId) {
   // iterate over all students enrolled in collegeCourseId, 
      // and find groups that balance out based on current GPAs, and name the group 
      // by the person w/ the highest GPA. 

   // return built groups
}

emmanuel · **Posted:** Tue Oct 23, 2007 1:23 pm

But such a query is best handled by HQL, I am looking for a Full Text use case.
As for the integration test, I usually do my changes, commit and then test that changes has been properly committed, it's a better *integration* test :)