Redis Transactions

Redis is a pretty awesome piece of software. If you’re not familiar with it, Redis is a in-memory key-value store, similar to Memcached. Like Memcached, Redis has get, set, increment, and decrement commands for string value keys. Unlike Memcached, it supports numerous other data type primitives including hash maps, sets, sorted sets, and lists. It also supports persistence, replication, and isolated and atomic transactions. Today I’m going to show you how Redis transactions work.

First, note that Redis is single-threaded. This simplifies the implementation of transactions in Redis server code since a batch of operations for a transaction can be performed in isolation from other operations without the need for locking keys. The lack of key locks means that Redis does not have to worry about deadlock detection or avoidance.

However, providing isolation without locking keys implies that the operations within a transaction must be performed in batch. There is no opportunity for a client to read a key within a transaction, get the result, and perform alternative Redis operations within the same transaction. After beginning a transaction with the MULTI command, rather than returning the result of any operation directly to the client, Redis returns a status of QUEUED, indicating the operation has been queued in memory by the server for later execution. Once all the operations to be performed within the transaction have been queued, the client can execute them atomically with the EXEC command. If the transaction is successful, a list containing the results of the operations performed will be returned to the client.

Allow me to present an example using redis-cli, the command line Redis client utility:
$ redis-cli> MULTI
OK> GET foo
QUEUED> SET foo bar
1) (nil)
2) OK>

This simple use of Redis transactions is fine when all that needs to be done is atomically perform multiple operations. But what if you need to get the value of a Redis key and then update it? If all you do is read and then write, the write will be atomic, so there’s not even a need for a transaction in that case. But how do you ensure that your read and write are not racing with some other read and write trying to make the same change? Or some conflicting change?

For example, what if a feature of a distributed application works with three Redis keys. Let’s call them A, B, and C. Keys A and B are strings that can have any value and key C holds a hash of A and B. When either A or B is updated, the application must read the other and update C. If when updating key A, the application just reads key B, calculates the new hash and updates key A and C atomically in a transaction, it may be that another concurrent process has updated keys B and C between the time the first process reads B and writes A and C.

This is an example of an application storing denormalized data. It turns out that any time an application stores denormalized data that this type of concurrency issue is possible.

It turns out that Redis provides a way to handle this case correctly even though Redis won’t return key values between MULTI and EXEC. The solution lies in the WATCH command.

The way the WATCH command works is that if after watching a key, it changes before a transaction is committed with an EXEC command, the transaction fails. In fact, this is the only way for a Redis transaction to fail without communication being severed between client and and server. If the WATCH command is performed before the read operation, it won’t be possible to wind up with inconsistent data in Redis.

Here’s an example using redis-cli again using the same concurrent example. Here we prepare for the transaction in a first terminal:> WATCH B

Now in a second terminal, we watch and get A and then update C:> WATCH A
"foo"> MULTI
OK> SET B baz
QUEUED> SET C 80338e79d2ca9b9c090ebaaa2ef293c7
1) OK
2) OK>

Now back in the first terminal, we complete the first transaction:> MULTI
OK> SET A baz
QUEUED> SET C c3c23db5285662ef7172373df0003206

The result of nil indicates that the transaction failed.

Assuming the desired behavior is still to set the key A to “baz”, the application should respond to a failed transaction by retrying the entire transaction from the beginning, including the WATCH and the GET. If the application is making heavy use of a key in this way resulting in frequent transaction collisions, it will increase the workload of both Redis and the application server, but it will never wind up with inconsistent denormalized data in the Redis database.