Hi

We are going to implement the master slave configuration that comes out of the box of Hibernate Search. We are in the final stages of getting the proof of concept accepted (90% there). I do have a couple of questions which I was hoping that some one might be able to help with.

1) Should we use JTA for the ayschronous indexing? Is this performant? I read on the Hiberate Search online documentation that

Is it possible to get more information on this and how it applies to Lucene?

2) Is there a batch update/add operation that I can perform using the JMS configuration. We are using ActiveMQ and have add some problems with the JMS provider and have some concerns around the number of document payload that will be put on the queue everytime an entity is updated.

Any help would be highly appreciated.

The recommendation is definitely to work within a transaction. JTA sounds good to me. In transactional mode the indexing jobs are adding to the queue when the transaction commits. If the transaction fails nothing gets indexed. This way you can avoid to a certain degree inconsistent data in the index. This is really independent of the synchronous vs asynchronous work mode. We actually recommend to start with the synchronous mode and test its performance.


You could to you batch indexing completely on the master bypassing the JMS queue completely. Besides, if you let the slaves to the batch indexing you would have to come up with a way to split the indexing work.

--Hardy

Hi

Thanks for your reply.



I presume the transactional stuff is configured using the jms provider jndi properties according to the documentation?

Due to the complex and crazy domain model we have I've decided to create a single lucene document that contains the data we need to perform our search. I know this has a downside that i can't create custom queries but I didn't not want to create seperate lucene documents to mirror our domain model.

I have created a custom classbridge from the top level object. My manager is concerned about the performance implication on the indexing side. From what I can understand, any updates to the object will kick of a lucene work item and that a lucene document is then put on to the queue not the domain object. I can understand that making a change to the embedded object(s) in the top level object will cause the entire object to be re indexed but I believe it's a price to pay otherwise we would have a complicated lucene document structure.

Sorry i think I had a bit of a rant! We have clustered set up so we're going with the master slave configuration and I think this is the appropriate solution to go with.

There was just recently a discussions about custom bridges and having one "do it all" class bridge. If you re aware of the implications it is a valid approach.


That's correct.


I guess you have to setup some performance tests. For example, how often will your entities change? If it is a rare event there is no problem.
I guess you understand that in your case a reindex only occurs when the top level entity changes. Since you are not relying on @IndexedEmbedded and @ContainedIn reindexing will not occur when any of the associated entities change.

--Hardy

Ah. I did not encounter this in the Hibernate In Action book. I though that the classbridge would be invoked when any changes (either to the top level object or any associated embedded objects). Can I use @ContainedIn/@IndexEmbedded with a classbridge? Basically I want to avoid the seperate lucene document scenario and just have one lucene document. My ClassBridge basically navigates through and creates fields for each of the entities inside the top level object.

Basically my object graph looks something like this

class AbstractProject
- User owner

class Project extends AbstractProject
- Programme
- Set<SubProject> subProject

class SubProject
- Risk

I therefore created a classbridge to handle the object graph. Should I use indexembedded for all the objects inside the top level object?

I realise I have gone off topic from my original question and I do apologise. I can create a seperate thread if required.

Cheers
Amin

You cannot use @IndexedEmbedded and @ContainedIn in your custom class bridge. These things are unrelated.

That said, I think you might misunderstand how many Lucene documents are getting created. You get one document for each instance which is marked with @Indexed.
In a object graph situation you only have to mark your top level entity (the one you want to retrieve as part of the search) as @Indexed. Marking a reference of this top level entity with @IndexedEmbedded (and the other side of the relationship with @ConstainedIn) does not mean that there will be a separate Lucene document created for this referenced object. in contrary. The referenced object gets added to the document of the main entity. It seems you replicated in your custom class bridge what Hibernate Search does by default.

--Hardy

Oh joy. I think the reason for creating the custom bridge was to group certain objects and place then in a custom field. Basically I wanted to group certain data and place them in a field like project.data which would use the embedded objects and group the data to gether. I presume this is possible to this with a custom field bridge and use @IndexEmbedded.

So effectively:


class AbstractProject
- User owner

@Indexed (top level entity and one i want to create a lucene document)
class Project extends AbstractProject

@IndexEmbedded
- Programme

@IndexEmbedded
- Set<SubProject> subProject

class SubProject
@ContaineIn
- Project
- Risk

<What should this entity be marked?>
- Programme
@Field (name="project.programme")
- reference
@Field (name="project.programme")
- description
@ContainedIn
Set<Project> projects

I have noticed that the field project.programme is being indexed as programme.project.programme which I don't want to do. Not sure how to remove this.

Not sure if the above made sense? Thanks for helping me out with this! Much appreciated.

The @IndexedEmbedded has a prefix attribute which defaults to the property name, you can set this to "" (empty string)
to change it to project.programme