Hello ,

I am struggling for a while on a issue of custom sorting. How can custom sorting be done in Lucene, any code example will be helpful and secondly is there a way that fields can be merged so that it can be sorted as a single field using some kind of custom sort?

Hi,
paragraph 4.1.2: http://docs.jboss.org/hibernate/search/ ... e/#d0e2888

Or you can "make up" additional fields:



You can also customize the scoring formula, if you need to sort by relevance but have need to customize the scoring:
http://docs.jboss.org/hibernate/search/ ... e/#d0e7140

More about that is described in chap. 12 of the book.

Hope it helps? Otherwise could you explain better what you're looking for.

Thanks Sanne, however my problem use case is slightly different. Let me explain

1. There are 3 fields of an entity per record.
Field1 --> author_name
Field2 --> book_name
Field3 ---> publisher

2. In my index I can see from Luke that this has been indexed correctly

3. In the UI the condition is that display all author and book names in a single column and sorted alphabetically, so I am using Lucene sort like this :

org.apache.lucene.search.Sort sort=new Sort(new SortField("author_name",SortField.STRING,true),new SortField("book_name",SortField.STRING,true));

4. Sample data indexed
id author_name book_name publisher
1. peter alpha pub1
2. james cook book pub2
3. cathy gardening net


The sorting for above comes like this in the UI
Names
--------
cathy
james
peter
alpha
cook book
gardening

But what I am expecting is this
Names
----------
alpha
cathy
cook book
gardening
james
peter

it's definitely possible to do as you are expecting, you must have some error. A common one is to double check that the index field you are sorting on is not tokenized (analyzed); could you check in Luke that the there is a single value per document on the sorted-on fields?
That's why all examples have Analyze.NO; technically using a field with Analyze.YES is possible, but either you have a custom analyzer which produces a single token, or sorting is unpredictable.

If it seems all tokenizers are correct, please provide me a runnable test (complete) so I can go through it.

Hi Sanne,

Here is the complete scenario



Step 1:Create a table : author
TABLE : author
-------------------------------------------
| id | NUMBER (NOT NULL) |
| author_name | varchar2(10) |
| book_name | varchar2(10) |
| publisher | varchar2(10) |
--------------------------------------------

Step 2: Insert values into the table

insert into author values (1,'phil','book1','pub1');
insert into author values (2,'rick','garden','pub2');
insert into author values (3,'joseph','city life','pub2');
insert into author values (4,'amanda','rose','pub2');
insert into author values (5,'lee','库控专员','pub3');
insert into author values (6,'yong','Dama 专员','pub4');


Step 3: Create entity
@Entity
@Table(name = "author")
@Indexed
public class Requisition {

private int id;
private String author_name;
private String book_name;
private String publisher;

@Id
@DocumentId
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}

@Field(index = Index.TOKENIZED, store = Store.YES)
public String getAuthor_name() {
return author_name;
}
public void setAuthor_name(String author_name) {
this.author_name = author_name;
}

@Field(index = Index.TOKENIZED, store = Store.YES)
public String getBook_name() {
return book_name;
}
public void setBook_name(String book_name) {
this.book_name = book_name;
}

@Field(index = Index.TOKENIZED, store = Store.YES)
public String getPublisher() {
return publisher;
}
public void setPublisher(String publisher) {
this.publisher = publisher;
}

}

Step 4: Index the entity by the clause : select * from author

Step 5: In the search class

public class SearchDAO {

@PersistenceContext(unitName = "SAMPLE_ENTITY")
protected EntityManager entityManager;

private FullTextSession fullTextSession;

public EntityManager getEntityManager() {
return entityManager;
}

public void search(String keyword) {
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(getEntityManager());
Session session = (Session) fullTextEntityManager.getDelegate();
this.fullTextSession = org.hibernate.search.Search.getFullTextSession(session);
BooleanQuery boolquery = new BooleanQuery();
org.apache.lucene.search.Query[] queries = new org.apache.lucene.search.Query[2];

queries[0] = new TermQuery(new Term("author_name", keyword));
queries[1] = new TermQuery(new Term("book_name", keyword));

org.apache.lucene.search.Query termQueryAll = new TermQuery(null);
org.apache.lucene.search.Query holder = termQueryAll.combine(queries);
booleanQuery.add(holder, BooleanClause.Occur.MUST);
org.hibernate.search.FullTextQuery hibernateQuery= fullTextSession.createFullTextQuery(query, Author.class);
hibernateQuery.setSort(new Sort(new SortField("author_name",SortField.STRING), new SortField("book_name",SortField.STRING)))
List<Object[]> results =hibernateQuery.list();

}
}

The results object do not seem to sort the way I would expect.

just have a look at what sanne allready suggested.

Sorry my mistake as a part of copy paste. All my sort fields are UN_TOKENIZED .

The issue I am trying to resolve is if there is a way that the sort fields can be merged into a single field while sorting, the problem that I see is Lucene is following a sort order first by Field1 and then by Field2 treating them as separate. Wondering if we can override that and use our own custom sort order so that the sort fields are merged.

You're correct, if you use different fields, Lucene will treat them "separate" as you say.

A common solution is to create a third field which contains the string you want to sort on, as in the example of my first post.

Can this 3rd field be assigned dynamically at runtime? I heard that Lucene supports dynamic fields. Wonder how it is done.

This is what I want :

Doc 1 --> Field 1 : book1, Field 2 : book2
Doc 2 --> Field 1 : sam, Field 2 : dover

While querying I want Field 3 to take value of Field 1 or Field 2 dynamically

No, you'll have to add all the combinations you need.

So if you need two different order, you'll have a field3 and a field4, combining the values in different ways.


It just means you don't have to pre-define the schema of fields before being able to create the values.
(you don't have to create the columns as in a database)

You can not have a field defined as a "virtual field" on othe fields; it's not a database..

Sorry Sanne, being little naive here. Can you explain a bit more detail. Here is the scenario

My documents will have 2 fields one with English version and other in say Chinese

@Fields ({
@Field(index = Index.UN_TOKENIZED,store = Store.YES,name="author_en")
})
public String getAuthor_en() {
return author_en;
}
@Fields ({
@Field(index = Index.UN_TOKENIZED,store = Store.YES,name="author_zh")
})
public getAuthor_zh() {
return author_zh;
}

Now I want to sort in such a way that author_en, author_zh should be behaving as a single field. How would field 3 and field 4 used as combination here?

Also this is much better, since you can actually tokenize the first two fields, making a better search experience.

Ok thanks Sanne. Let me try and post you back the outcome.

Hi Sanne,

Need to thank you a tonne! This worked as a charm. I had to do few changes though, but without your hint I was reaching no where.

The forum is awesome .... !

Thanks