cassandra and hibernate

mascix · **Joined:** Mon Jun 15, 2009 4:31 am **Posts:** 2

I know cassandra is not a relational database and its not set of db engines of hibernate. but this could be cool:

Code:

@SuppressWarnings("serial")
@Entity("cassandra")
public class Message extends EntityData{
    Long messageId;
    Long parentMessageId;
}

I specifically gave that example becasue when people start messaging each other in a web site. it can be huge and gets huger day by dy. so moving those data into cassandra would be very profitable for performance. because its not a big data. just hold messageId and userId and sendedUserId or stuff like that. no need to have foreign keys or onetomany connections between tables. so here comes my question. do you know any way to work cassandra with hibernate ?

thanks in advance.

sanne.grinovero · **Posted:** Fri Nov 19, 2010 11:20 am

Hi,
we recently started a proof of concept, it's really in early stages of development, to have the whole model stored in a key-value store.
It's called HibernateOGM (Object Grid Mapper), it's being built to use Infinispan as a data storing grid, but Infinispan can be easily configured to be a cache in front of cassandra: the Cassandra CacheStore was recently constributed to Infinispan and I know of some clever developers building a product on top of this combination.

We're looking for help and testers, if you want to join making this that would be great.
Start by having a look into:
https://github.com/emmanuelbernard/hibernate-core-ogm
It supports basic CRUD operations already, we're working towards query capability and have the final goal to support all of JPA2.

Cheers,
Sanne

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

Hi there,

Sorry for bumping a bit of an old topic, but it seemed more appropriate than starting a new thread.
Is there anywhere I can find more information about the hibernate OGM? Also, how do I actively get involved with development?
I'm extremely interested in ditching the MySQL layer of my application and replacing with Cassandra, but I would very much like to keep the Hibernate DAO layer, with the minimal amount of changes.
Will hibernate tools support reveng on OGMs?

sanne.grinovero · **Posted:** Tue Dec 21, 2010 10:31 am

Hi Adam,
you're more than welcome to join us, the code is on github currently living as a fork of Hibernate Core https://github.com/emmanuelbernard/hibernate-core-ogm/tree/ogm
(please note the branch name "OGM").

Help is highly appreciated, in any form, as besides exploring options we need to maintain and bugfix existing projects.

Of course as soon as it gets in better shape it will either be merged in core or have a proper module, maybe we can accommodate changes needed in core to work as an extension, we will see what fits best.
To get in direct touch, drop in on irc: http://www.hibernate.org/community/irc
or even better, on mailing list: http://www.hibernate.org/community/mailinglists

I highly recommend to use hibernate-dev mailing list for this purpose, you're free to combine it with IRC if you want to discuss something in a more interactive fashion, but then it's always nice to have a wrap-up on the public mailing list as reference for who wasn't attending on IRC.

RE automatic reveng, it's likely too early to forecast anything, but I don't think there is enough metadata in a schemaless key/value store. Maybe there's enough metadata for it in Cassandra, but the hard part of the project is to leverage advantages of different grid and key/value stores while still providing integration with many different implementations. If you have suggestions, we'll be interested in it.

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

Hi Sanne,

Thanks very much, it would be really exciting to contribute to something like this, I can really see it taking off in 2011 and beyond as we start to see the demise of the relational database in production applications. (And just when I really started to like RDB+Hibernate ._.)

I've not really used IRC before, I tried to join your channel on (irc://irc.freenode.net/#hibernate-dev) but it just refuses me. :( There's a good possibility I'm blocked at work though.
I've also registered to join the hibernate-deb mailing list, are there any threads of note regarding Cassandra? I tried to google search the mailing lists but only found spurious mentions.

I've not used Git/Hub before so I'm just familiarising myself now. I've registered with Hub, and downloaded msysGit - I'll try to get the project up and running in Git.

I did already get a copy of the code, but I'm wondering what setup (Cassandra etc) I need to get it working? Any notes you guys have been making on this, that could bring me up to speed would be awesome. :)

I can see it's far too much in the early stages to be thinking about modifying tools to accommodate for NoSQL and the like. I suspect the direction that needs to go in will become much more clear once this is out of the conceptual phase.

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

Sanne, any ideas?

Code:

[INFO] Scanning for projects...
[WARNING] The POM for net.sf.docbook:docbook:jar:1.72.0 is missing, no dependency information available
[WARNING] The POM for net.socialchange.doctype:doctype-changer:jar:1.1 is missing, no dependency information available
[WARNING] The POM for net.sf.docbook:docbook:jar:1.74.0 is missing, no dependency information available
[ERROR] The build could not read 4 projects -> [Help 1]
[ERROR]
[ERROR]   The project org.hibernate:hibernate-manual:3.6.0-SNAPSHOT (D:\workspace\play\hibernate-cassandra\hibernate-core-ogm\documentation\manual\pom.xml) has 2 errors
[ERROR]     Unresolveable build extension: Plugin org.jboss.maven.plugins:maven-jdocbook-plugin:2.3.3 or one of its dependencies could not be resolved: The following artifacts could not be resolved: net.socialchange.doctype:doctype-changer:jar:1.1, net.sf.docbook:docbook:jar:1.74.0: Failure to find net.socialchange.doctype:doctype-changer:jar:1.1 in https://m2proxy.atlassian.com/repository/public was cached in the local repository, resolution will not be reattempted until the update interval of
central has elapsed or updates are forced -> [Help 2]
[ERROR]     Unknown packaging: jdocbook @ line 15, column 16
[ERROR]
[ERROR]   The project org.hibernate:hibernate-envers-manual:3.6.0-SNAPSHOT (D:\workspace\play\hibernate-cassandra\hibernate-core-ogm\documentation\envers\pom.xml) has 2 errors
[ERROR]     Unresolveable build extension: Plugin org.jboss.maven.plugins:maven-jdocbook-plugin:2.3.3 or one of its dependencies could not be resolved: The following artifacts could not be resolved: net.socialchange.doctype:doctype-changer:jar:1.1, net.sf.docbook:docbook:jar:1.74.0: Failure to find net.socialchange.doctype:doctype-changer:jar:1.1 in https://m2proxy.atlassian.com/repository/public was cached in the local repository, resolution will not be reattempted until the update interval of
central has elapsed or updates are forced -> [Help 2]
[ERROR]     Unknown packaging: jdocbook @ line 15, column 16
[ERROR]
[ERROR]   The project org.hibernate:hibernate-quickstart:3.6.0-SNAPSHOT (D:\workspace\play\hibernate-cassandra\hibernate-core-ogm\documentation\quickstart\pom.xml) has 2 errors
[ERROR]     Unresolveable build extension: Plugin org.jboss.maven.plugins:maven-jdocbook-plugin:2.3.3 or one of its dependencies could not be resolved: The following artifacts could not be resolved: net.socialchange.doctype:doctype-changer:jar:1.1, net.sf.docbook:docbook:jar:1.74.0: Failure to find net.socialchange.doctype:doctype-changer:jar:1.1 in https://m2proxy.atlassian.com/repository/public was cached in the local repository, resolution will not be reattempted until the update interval of
central has elapsed or updates are forced -> [Help 2]
[ERROR]     Unknown packaging: jdocbook @ line 37, column 16
[ERROR]
[ERROR]   The project org.hibernate:devguide:3.6.0-SNAPSHOT (D:\workspace\play\hibernate-cassandra\hibernate-core-ogm\documentation\devguide\pom.xml) has 2 errors
[ERROR]     Unresolveable build extension: Plugin org.jboss.maven.plugins:maven-jdocbook-plugin:2.3.3 or one of its dependencies could not be resolved: The following artifacts could not be resolved: net.socialchange.doctype:doctype-changer:jar:1.1, net.sf.docbook:docbook:jar:1.74.0: Failure to find net.socialchange.doctype:doctype-changer:jar:1.1 in https://m2proxy.atlassian.com/repository/public was cached in the local repository, resolution will not be reattempted until the update interval of
central has elapsed or updates are forced -> [Help 2]
[ERROR]     Unknown packaging: jdocbook @ line 37, column 16
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

sanne.grinovero · **Posted:** Tue Dec 21, 2010 12:39 pm

Quote:

Thanks very much, it would be really exciting to contribute to something like this, I can really see it taking off in 2011 and beyond as we start to see the demise of the relational database in production applications. (And just when I really started to like RDB+Hibernate ._.)

I'm sure RDBS won't disappear, they are still the best tool for 90% of use cases, but for the amazing results in the 10% case in which a database wasn't fitting well people got very excited. So as always, we need to be able to use the right tool for each problem, and as most cases you don't figure that out initially so it would be great if you could build your app on a common API like Hibernate and then have the flexibility of choice on the stores.

Quote:

I've not really used IRC before, I tried to join your channel on (irc://irc.freenode.net/#hibernate-dev) but it just refuses me. :( There's a good possibility I'm blocked at work though.

Try connecting to #hibernate , that's an open channel, if it refuses you then yes you're blocked by your corporate firewall. If it works, to join #hibernate-dev you need to create a free account on freenode, we only allow authenticated people to minimize spammers.

About github, start by forking emmanuel's project, and then clone that repository to your developer station so that you can commit to your own repo and we can look at patches and discuss them. It's like you having your own dedicated repository, so you can mess and commit as often as you like, and you'll see it gets easier to then ask for specific changes to be pulled in the upstream project.

Quote:

I did already get a copy of the code, but I'm wondering what setup (Cassandra etc) I need to get it working? Any notes you guys have been making on this, that could bring me up to speed would be awesome. :)

I'm not sure about Cassandra, are you mostly interested in that store only or in a more general K/V store? I believe Emmanuel is testing with Infinispan, and there should be unit tests ready: nothing to setup.
In case of Cassandra, we can ping some experts to get some help, but I don't know much myself.

sanne.grinovero · **Posted:** Tue Dec 21, 2010 12:40 pm

About your dependency issues, you have to setup the JBoss maven repository:
http://community.jboss.org/wiki/MavenGe ... rted-Users

sanne.grinovero · **Posted:** Tue Dec 21, 2010 12:43 pm

Oh, for developing you likely need this one, especially if you're going to release stuff ;) http://community.jboss.org/wiki/MavenGettingStarted-Developers

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

s.grinovero wrote:

About your dependency issues, you have to setup the JBoss maven repository:
http://community.jboss.org/wiki/MavenGe ... rted-Users

That's fixed it thanks :)

Do you know if all the tests are written? is it just a case of getting them working?

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

s.grinovero wrote:

... most cases you don't figure that out initially so it would be great if you could build your app on a common API like Hibernate and then have the flexibility of choice on the stores.

Yes this is exactly what I'm looking to do with an application I'm working on. I think it would be a huge boon for Hibernate to be able to offer an easy transition from RDBS to non-RDBS if your system is really taking off.

Looks like I am blocked at work, but I'll see if I can connect to IRC on G3 via my netbook later.

s.grinovero wrote:

About github, start by forking emmanuel's project, and then clone that repository to your developer station so that you can commit to your own repo...

Yep, done this, mvn is running nicely now, and I see the test that is failing.

s.grinovero wrote:

I'm not sure about Cassandra, are you mostly interested in that store only or in a more general K/V store? I believe Emmanuel is testing with Infinispan, and there should be unit tests ready: nothing to setup.
In case of Cassandra, we can ping some experts to get some help, but I don't know much myself.

Infinispan can be configured to act as a Façade for Cassandra, so it's essentially the same thing. Although having Hibernate communicate directly with Cassandra would be really nice too.
I'm looking to use Infinispan/Cassandra as a NoSQL datastore for a rich web application that needs to scale to 10s of millions of users.

AdamGibbons1982 · **Joined:** Fri Nov 19, 2010 8:57 am **Posts:** 11

One more quick thing -- if I've forked from emmanuel's code and checked out my own fork with git, how do i get updates from emmanuel's branch? ._.

Thanks!

sanne.grinovero · **Posted:** Wed Dec 22, 2010 9:21 am

so, basically you'll be dealing with three repositories, one on your workstation, one on your github account, one on emmanuel's github account.
You can pull changes directly from emmanuels's github repo to your workstation, and then push them to your github account.
(and yes I was surprised too, it's simple but surprising for users of SVN and similar - I promise you will love it, after you got these concepts)

Feel free to experiment with all git commands, worst thing that can happen you (other than loosing your mind) is that you could loose some of your
own work - you can't directly write on the repository of somebody else: the concept is that you just ask him to take changesets from yours, so we'll be there to comment
and review it. So take some time to play with it before doing really interesting work, we all had to.

Recommended readings:
http://in.relation.to/Bloggers/HibernateMovesToGitGitTipsAndTricks
http://progit.org/

emmanuel · **Posted:** Wed Dec 22, 2010 10:31 am

Hi Adam,
Thanks for your interest.

A couple of comments. Cassandra is a column based NoSQL which means we could do a bit better than simply using it as an Infinispan back-end (ie map somehow the concepts of columns between JPA and Cassandra). That would be probably better than using Cassandra as a dump K/V engine. That being said, I haven't thought deeply about the problem as this was not on my initial road map.

But support for column-based families is certainly something I had in my mind. If you're feeling like experimenting we could explore that together and bump up the priority.

The first step maybe would be for you to get familiar with the ogm module. All the tests are in there and should run. I use an in-memory Infinispan today. Remember that's very early stage so there are a lot of things to do and polish :)
Maybe you could try and tackle an unrelated task like supporting more types (more GridTypes to implement). That would give you a feel of how OGM does things. Once you feel more comfortable we can try and discuss how to add the appropriate abstractions to support:
- other key/value pairs engines
- column based engines like Cassandra

Let me know what you think. As Sanne said, the best is for you to join IRC on #hibernate-dev on freenode.

Emmanuel