Hello All,
Iam using HibernateSearch to search contacts based on FirstName and lastName. I could do everything except one thing. The question is how to do Accent based search .

Please help me out in solving this..feel free to ask extra info on this regard

Hello,
some issues:

it looks like you're defining the "customanalyzer" using @AnalyzerDef, but you didn't use the definition on the entity or on the fields. you still need to use @Analyzer(definition = "customanalyzer") on the class level or on the specific attributes (see examples in docs)

To be case insensitive your Analyzer needs to use the lowercase filter as well:

(again see examples in the docs)

finally, instead of creating a different analyzer by doing

you should ask the framework to get a reference to the proper analyzer:
so this way you're sure that you're going to create the query using the same kind of text analysis that you're using during indexing.

Another hint: if you use the QueryBuilder API instead of the parser you don't have to care for analysis during search time as it will automatically apply the proper analysis.
http://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#search-query-querydsl

Still unable to do accent based search. After my previous thread i got to change my searching logic on firstname and lastname. My requirement is to do search on firstname and lastname by keeping constraints on them. For example it should be like this.. "~like *kalyan* AND ~Like *raj*" by having accent based search also. I could get the logic to retrieve case insensitive contains operator search except accent searching. Below is my code used
...... .... ....
//created indexes,then
FullTextSession fullTextSession = Search.getFullTextSession(sessionForQuery);

ContactDetails contactWithDob = new ContactDetails();

contactWithDob.setFirstName(firstName);

contactWithDob.setLastName(lastName);

Example example = Example.create(contactWithDob).enableLike(MatchMode.ANYWHERE).ignoreCase();

emailList= fullTextSession.createCriteria(ContactDetails.class).add(example).list();

The above logic is not hibernate search logic ,where queries hit indexes. Above one is hibernate core logic it hits DB and gets results. If i go for that logic i think i cannot make use of any analyzer to do accent based search.Can any one suggest relavant hibernate search code where i can achieve above logic and accent based search.

Below is my relavant pojo code

@Indexed(index = "fulltext")
@Entity
@Table(name = "contact_details", catalog = "myschema")
@AnalyzerDef(name="customanalyzer",
tokenizer = @TokenizerDef(factory =StandardTokenizerFactory.class),
filters = {
@TokenFilterDef(factory = FrenchStemFilterFactory.class)

}
)
public class ContactDetails implements Serializable{
@Column(name = "FIRST_NAME", length = 100)
@Field(index=Index.TOKENIZED, store=Store.YES)
@Analyzer(definition = "customanalyzer")
public String getFirstName() {
return this.firstName;
}

public void setFirstName(String firstName) {
this.firstName = firstName;
}
@Column(name = "LAST_NAME", length = 100)
@Field(index=Index.TOKENIZED, store=Store.YES)
@Analyzer(definition = "customanalyzer")
public String getLastName() {
return this.lastName;
}

public void setLastName(String lastName) {
this.lastName = lastName;
}
}

Hi,
it's not clear to me if you tried my previous suggestions. it works if you do.

Hello ,

Well can you suggest me a piece of code where i can do case insensitive and contains operator functionality while searching for records.

For example i have a record whose firstname and lastname are abcKALYANxyz and abcRAJxyz . User gives kalyan ,raj as inputs ,i should be able to show the record to user.

All that i could achieve above functionality is using below code

contactWithDob.setFirstName(firstName);

contactWithDob.setLastName(lastName);

//contactWithDob.setDateOfBirth(dateOfBirth);
// and so on
Example example = Example.create(contactWithDob).enableLike(MatchMode.ANYWHERE).ignoreCase();

emailList= fullTextSession.createCriteria(ContactDetails.class).add(example).list();

But above code hit DB but not lucene indexes. Then i tried with some other code but in vain

MultiFieldQueryParser q=new MultiFieldQueryParser(Version.LUCENE_31,fieldArray,new StandardAnalyzer(Version.LUCENE_31));

Query luceneQueryForContactDob=q.parse("firstName:\\*"+firstName+"* AND lastName:\\*"+lastName+"* AND dateOfBirth:"+dateOfBirth+"*");

FullTextQuery hsQueryForContactWithDob = fullTextSession.createFullTextQuery(luceneQueryForContactDob,ContactDetails.class);
// execute search
contactWithDobList = hsQueryForContactWithDob.list();

The above functionality is returning me empty

Is there any way i could achieve my above requirement using either dsl or lucenequery So that as a part of next step i think of accent based searching

If you know that you analyzed the names lowercasing during indexing, then you don't need the analyzer during searching as long as you make sure your search parameters are lowercased as well:

I found as well no exemple in the code where both @AnalyzerDef and @Analyzer are present at the same time, and I it might be misleding, I faced the same issue.

Thank you for pointing it out; there's an example, named
"Example 1.10. Using @AnalyzerDef and the Solr framework to define and use an analyzer"
in http://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#d0e389
But I guess that's not explicit enough. Adding a sentence about it.

It's when I saw the @Anayliser on the field for this exemple that I realised @AnalyserDef wasn't enough for the entity.

Maybe adding a @Analyser(class=StandardAnalyser.class) could do it also.

Firstly iam very much indebted to all who helped me in sorting out this case insensitive Accent based search. It is working fine now..

Before we implement this in our project i would like to confirm one thing. The problem is .. Our client maintains one DB which we hit using webservices to get data. Since searching using webservices is bit slow and also accent based searching is not allowed , we thought of maintaining one local DB where we synchronize the client DB using one Batch program(which runs for every 5mins). As of today there are more than 6 lakhs records in DB ,which is can be incremental. On server start up or once i will do batch indexing manually. My question is whether hibernate search automatically synchronizes the local DB data to indexes or not. I mean when i synchronize data to local DB using Batch, will hibernate search automatically detects changes applied on DB and does update its indexes or not every time without fail. Iam using annotations based approach in my pojo class. I heard if we use annotations based approach hibernate search will automatically synchronize the data between DB and indexes.

Can any one confirm on this? My batch is a normal standalone java application which will be scheduled on the server.

No problem I'm happy to help.


Hibernate Search will not be able to synch changed about which it doesn't know: it only listens to hibernate events; so it depends what do you mean by batch synchronizations. If it is implement by SQL or stored procedures, the indexes won't be updated. This is explained on the reference docs, I'd suggest to read them instead of relying on rumours you have heard of.

So, why do you need to hit it with webservices? if you have direct access - needed for any batch operation - then I don't understand why you would need webservices at all to interact with data: as you suggest alternative access yourself, it seems that this is an option you have?

I'd suggest three alternatives:
- implement search over webservice
- avoid using the webservices but connect directly to the remote database - would perform way better for other operations too
- have the webservice server index the data, and then copy over the index regularly as you would do with the database synch. Again, read the documentation as it provides already helpers to periodically synch indexes across shared filesystems.

Hibernate Team :Hibernate Search will not be able to synch changed about which it doesn't know: it only listens to hibernate events; so it depends what do you mean by batch synchronizations. If it is implement by SQL or stored procedures, the indexes won't be updated. This is explained on the reference docs, I'd suggest to read them instead of relying on rumours you have heard of.

KalyanRaj: So what i understood is hibernate search cannot listen to updates happened through sql or stored procedures. So if i write one java application which synchronizes my client data with my local DB using Hibernate, indexes will get updated.

HibernateTeam :So, why do you need to hit it with webservices? if you have direct access - needed for any batch operation - then I don't understand why you would need webservices at all to interact with data: as you suggest alternative access yourself, it seems that this is an option you have?

KalyanRaj : Well , our client data is maintained by other company, which has exposed the DB through webservices. We dont have direct access to it. Using WSDL file they exposed and using stub less approach (xml based request and response) we hit their DB and get what ever is required for us.

Hibernate Team: : implement search over webservice
- avoid using the webservices but connect directly to the remote database - would perform way better for other operations too
- have the webservice server index the data, and then copy over the index regularly as you would do with the database synch. Again, read the documentation as it provides already helpers to periodically synch indexes across shared filesystems.


KalyanRaj : I hope you understood why we are using webservices. Since using webservices is very much costly interms of performance while searching and also accent based searching is not possible , we thought of keeping whole data in our local DB and keep synchronizing with 3rd party DB.

It sounds interesting regarding search over webservice. Iam very much keen to see how hibernate can interact with remote DB which has been exposed via webservice.

How can i index the webservice exposed data ? If it is possible can i have link to that documentation which helps in synchronizing the indexes across shared filesystems.

exactly, but I have no clue about how you think to be able to synch the remote database, especially since you need to do it via this webservice then it's not going to be a very fast operation. You could delete your local copy of the database, and then copy over all data from the source.
I understood now why you have to use webservices, but if you can't update the webservice to also provide search over webservice, then having the webservice provide an index is likely not an option.


that wasn't done before, so no documentation exists.

Thanks for your reply.. Well the logic behind the synchronizing the clients data to our local DB is simple. For the first time we export all the latest data from clients DB to our local DB. For each and every record in the client DB table we do have Modified date column. Based on we query all the records which were modified after last batch run date (which will be kept in some file) from our clients DB and merge with our local DB. Now the point is how hibernate search indexes will get updated upon modifying the local DB (which is the source for index files) . So what i understood is if i do that synchronization between client DB and local DB using Hibernate core (which will have hibernate search), hibernate search will automatically synchronize index files with local DB assuming both my synchronization batch application and other application which uses this indexes and does searching are in same server. am i right?

Also i assume as of now it is not possible to do hibernate search over data which is exposed via webservice. can you confirm this?

As far as our requirement concerned ,i solved all the problems except this synchronization between local DB and index files by hibernate search.

that depends on who created and designed this webservice.