Thomas v James.com — Relax & Unwind with a little Redis

Relax & Unwind with a little Redis

Written by . Posted at 9:22 pm on February 23rd, 2012

In a previous post, Unwind: Sequences in the cloud, I wrote about one of the challenges of moving from the relational database world to the non-relational database world. The issue of generating numerical sequences.

This is an important issue for a number of reasons:
1) Numerical sequences are the de-facto standard for identity column generation.
2) Numerical sequences are easy to display to the end user
3) Easy for the end user to remember and recant

The GUID/UUID data type is great for replacing the numerical ID of a record with something that can stand up to the challenges of distributed data, but they are not very suitable for use by the end-user. In some applications you really do still want to be able to generate a reliable numerical sequence number, such as for an invoice number.

So how to we do this?

Introducing Redis, another NoSQL brethren of the Key-Value store type but with a whole lot more. It’s stable, fast and supports a number of data type operations on the server-side.

The operation we care about is one of the simplest:
String INCR

We’ll use Redis with this operation to keep track of our applications counters. This way we have the distributed power of CouchDB combined with the stability and speed of Redis.

It may seem like overkill to add an additional component to our software stack just to handle the task of keeping a counter, but Redis fits the mould perfectly in this case. Its also simple to setup and use, not adding much to the solution’s overhead. Especially considering that, like CouchDB, the redis instance is “in the cloud”.

Within the sample Unwind application, the implementation is as simple as implementing the counter interface with the ServiceStack Redis client.


using (var redis = new RedisClient("Host", 1234)){
	redis.Password = "Password";
	
	var counter = redis.IncrementValue("UnwindNextLinkCounter");
}

Can it get any harder?

2 comments.

  1. But redis is a single point of failure in distributed id generation, when redis dies you are no longer able to get new incremental identification numbers.
    Even redis master-slave replication will not help you, while it is asynchronous: when the master dies and the slave is not up to date you will generate an id twice, no more unique ids !

    Look for twitters snowflake algorithm ;) It’s implementable in your application, it’s distributed (no SPOF) and has natural sorting for ids.


  2. Thomas James

    Tobias, you are absolutely right, redis becomes a single point of failure in the system, but as with any system it becomes a tradeoff. For something like generating invoice numbers you could live with the problem, arguing that you can still check the master database for clashes and roll the slave forward until ids no longer overlap.

    I think i’ve failed to articulate in the blog post, that coming from a relational world there might be the temptation to just have a document in couch that contains the counter, which would be terrible on system performance and result in many document conflicts.

    Snowflake looks to be a pretty interesting way to solve the problem, especially at the scale of twitter. I’ll look into it… might be overkill for this little app but it would certainly be interesting to implement.