I have entity Student and Course, in a many-to-many association.
The linking entity is Payment.
All this entity has surrogated id as primary key.

I want to get all the students that are not associated to a specific course.
How to do it in HQL?

I tried something like

select s from Student s, Course c, Payment p where c = :course and (p.course <> c and p.student = s)

but doesn't return the student with no entry in payment table.

This is not peculiar to HQL. You need to explicitly outer join if you are looking for rows that don't exist.