Redis beyond a cache: counters, leaderboards, simple locks

Many designs need a count, a ranking, or a rule that only one program runs a job at a time. Interviewers ask how you build these. A weak answer reads a number, adds one, and writes it back. A strong answer uses an atomic step (one that nothing can interrupt) and says where it can fail. At SDE-3 level, know the limits of the lock.

Redis is a key-value store (lesson 17: you put a value under a key and get it back by that key). It keeps its data in memory, so it is fast. You can use it as a cache (lesson 31). It has three more common jobs.

1. Counter. Two requests read a count of 10. Each adds 1 and writes 11. The right answer is 12, so one add is lost (lesson 21: a lost update). The command INCR adds 1 in one atomic step (lesson 22: nothing can interrupt it), so no add is lost. If the key is missing, INCR starts it at 0.

2. Sorted set. A sorted set holds unique items. Each item has a number called a score. Redis keeps the items in score order. ZADD adds an item or changes its score. ZINCRBY adds to a score. Each takes about log N steps, where N is the number of items. The log of N is how many times you can halve N until one is left. It grows very slowly: about 20 steps for 1,000,000 items, because 2 to the power of 20 is 1,048,576. ZRANGE key 0 2 REV returns the top three (REV means highest score first). Its time is log N plus the number of items returned. (ZREVRANGE is the old name; Redis deprecated it in 6.2.)

3. TTL lock. Sometimes only one worker (a program that runs jobs) may run a job. SET key token NX PX 30000 sets the key only if it does not exist (NX), and removes it after 30,000 ms (PX). The reply OK means you hold the lock. nil means someone else does. If the worker crashes, the TTL frees the lock. A token is a random value that only you know. Release the lock only if the key still holds your token. Do not use two commands, GET then DEL: the lock could change owner between them (the gap from lesson 22). Send one short script instead (Redis scripts use the Lua language). Redis runs a whole script as one atomic step.

The game. The answer to the predict box is a sorted set and ZINCRBY.

ZADD board 120 asha 90 ravi 100 meena      # add three players with scores
ZINCRBY board 50 asha                      # asha is now 170
ZRANGE board 0 2 REV WITHSCORES            # asha 170, meena 100, ravi 90
ZREVRANK board ravi                        # 2: rank 0 is the top player
INCR views:product:42                      # a plain counter: 1, then 2, then 3

The top 10 of 1,000,000 players is ZRANGE board 0 9 REV WITHSCORES: about 20 steps plus 10 items.

The lock. A nightly report must run on one worker only.

SET lock:report 7f3a NX PX 30000           # OK: this worker holds the lock
# release: delete only if the value is still 7f3a (one script, so check and delete are one step)
EVAL "if redis.call('get',KEYS[1])==ARGV[1] then return redis.call('del',KEYS[1]) else return 0 end" 1 lock:report 7f3a

Assumption: the job takes 20 seconds and the TTL is 30 seconds, so the lock holds. Now one run takes 45 seconds. The lock ends at 30 seconds and a second worker takes it. The two workers run together for up to 45 - 30 = 15 seconds. At 45 seconds the first worker releases. The key holds the second worker's token, so the script deletes nothing. A plain DEL would remove the second worker's lock.

• Counter and sorted set solve shared counts and rankings with no lost updates. They cost memory, and data is lost if the server stops without a saved copy.
• The lock solves duplicate work. It costs safety: a job longer than the TTL gives two holders, and a lost key lets another worker in.
• I would change it when a double run would damage data or money. Then I also check in the database with a unique constraint or a version check (lesson 22).

"A lock with a TTL guarantees one worker." It does not. It works only if the job ends within the TTL and the key survives.

"I release a lock with DEL." If your lock already ended and another worker took it, DEL removes their lock. Check the token first, in one script.

1. Why is "read the count, add 1, write it back" wrong when many requests run together? What do you use?
2. A lock has a TTL of 30 seconds. A job takes 20 seconds, but one run takes 50 seconds. For how many seconds can two workers run together?
3. Write the commands to add 30 points to player dev and then show the top 5 players with scores.