Ye gods! Punish me not, Apollo, for using Cassandra, a data store written at Facebook and adopted at Reddit and Digg, and tried at Twitter, for I have made the required sacrifice of many hectatombs. All right, I will end with the pagan prayer and get to the technology.
This data store is confusingly called a column-store, but I will avoid such terminology as that is confusing. Really, it should be called a multi-dimensional array store, as it allows an arbitrary number of key value pairs to be stored and retrieved with one key. Why not call this a column? A column is already a term in relational DBs, and “column-store” can be confused with those RDBMS that store their data by column rather than by row in their physical structure to allow analytical queries (sums) to be rapidly executed.
Cassandra was originally inspired by Dynamo, the first modern NoSQL datastore targeted toward large Internet applications, designed and built by Amazon. The source code for this application was never released, but the original creators released a paper describing how they created and managed this system. Facebook took this paper, built a similar application, then released the source, and Apache took on the project of extending and maintaining it. It has many features in common with its private cousin: built-in clustering, gossip protocol, reliance on replication for durability, and a key-value type of storage. With the 0.7 version comes secondary indices, so I don’t know if we can call this a KV store, but close enough.
The largest container in Cassandra is the keyspace. Think of this as a database. The next container is a column family, which is somewhat analogous to a table. You can either store key value pairs in here, or you can store a set of key value pairs within the context of another structure called a SuperColumn. Plenty of sites already talk about the model of Cassandra, but there are few key notes: you don’t specify your data model upfront, but you do specify your collation for your data buckets (column-families) beforehand. One more note, SuperColumns cannot store more SuperColumns, so no wishing for more wishes. By which I mean that you can’t have more than a five-dimensional array.
Okay, this all might not much sense, but this was the only data store that took me more than an hour to understand the model. It is trippy but it works.
There are several Java clients, but I chose Hector after reviewing it because it seems like it is better written, and it has more community support. The Cassandra project owners recommend using a high-level client rather than the bare Thrift API. I implemented a mapper that translated an object into the key value pairs, and I serialized any complex objects within an object, storing it in one column. This feels like a bad solution, but it worked for me.
Cassandra is fairly easy to install on Windows 7 and on Linux (Ubuntu 10.10 Server) as long as you have Java and you set your path. The most complex part is setting up your data folders and then specifying them in your config files. Watch out for the differences between 0.6 and 0.7 because the configs were changed from XML to YAML, and the settings were moved around.
Creating a cluster was also trivial, and took only a few line changes in the config files. The clustering worked like a charm, and I could write to one cluster and read from the other without trouble. I used two Ubuntu VMs on VMware Server to do this.
After finishing the model, it took almost no time to implement the DAL. The longest time was spent on retrieving objects by their secondary indices. I did it the 0.6 way by creating a secondary CF to store the relationship, and then selecting the objects by a range. I found out later that in 0.7 you can add a secondary index beforehand.
Cassandra is the most widely-used NoSQL store, and for good reason: it has the most production time as well as the design to be able to handle lots of data and hold up under most conditions, including high-latency public VM clouds. Please read about Reddit’s and Digg’s experience, because it would be a shame to reproduce the same failures again and again. Would I implement it as a main data store? No, but that’s just because of that big fat goose-egg that stands at the front of the version number that is representative of a lack of maturity in administrative features as well as stability, and because it is much harder to develop for. Unlike some other NoSQL stores, there is still an impedance mismatch between database and language, but that can be handled with a mapper. The way I did it, data cannot be shared outside of a language.
Setup was fast, clustering was easy, and the performance is good, especially on updates. Be careful with her, and you might have great success. But let not your hubris grow, else the data gods strike you down.