Redis in Practice: Patterns and Pitfalls

In interviews, I like to ask one question: "What do you use Redis for?" Nine times out of ten, the answer is, "Caching, storing some sessions, and occasionally a counter." Ask how they use zset, how they release distributed locks, or whether they enable AOF or RDB, and the answers usually stop there. Redis is the kind of tool that works as soon as it is installed and that everyone can use for SET/GET, but most people only study its limits and pitfalls after something breaks in production. This article explains the Redis patterns and problems that actually appear frequently in production, using "able to handle real traffic" as the standard.

1. Redis Is More Than a Cache

Treating Redis as a "HashMap with expiration times" is its biggest waste. At its core, it is an in-memory, single-threaded (for command execution), data-structure-rich KV store. "In-memory" makes it fast, with microsecond-level latency; "single-threaded" eliminates concurrent command contention and makes commands atomic; and "data-structure-rich" lets it do far more than caching.

Caching is only its most common use. Redis is also frequently used for:

• Counters / rate limiters: atomic INCR with millisecond-level latency;
• Distributed locks: SET NX EX for cross-process mutual exclusion;
• Leaderboards: a natural use case for zset;
• Message queues: list with LPUSH/BRPOP, or stream since 5.0;
• Deduplication / tags: set;
• Cardinality counting: hyperloglog can count hundreds of millions of unique visitors in 12KB.

In short: when you need state that is shared across processes, low-latency, and persistent, Redis is almost always the most cost-effective choice. The trade-off is that it is expensive, because memory costs an order of magnitude more than disk. Deciding what to store and how to store it therefore requires judgment. Let us start with choosing the right data structure.

2. Choosing Data Structures: The First Step Toward Using the Right Tool

Redis derives its power from its data structures. Choose the right one and your code will be concise and efficient. Choose the wrong one and the code will either be ugly or bring Redis to its knees.

string: The most basic type, and binary-safe, so it can store images, JSON, and serialized objects. Suitable for cached objects, counters, and distributed-lock tokens. Counters are where it shines: INCR / INCRBY provide atomic increments and are ideal for likes, inventory, and API call counts:

INCR post:123:likes           # Likes +1
INCRBY stock:sku:99  -1       # Decrement inventory

A single string key is limited to 512MB, so do not actually put large files in it.

hash: Stores multiple fields under one key and is suitable for objects. For user profiles, a hash is often better than a JSON string:

HSET user:1001 name "张三" age 28 email "[email protected]"
HGET user:1001 name
HINCRBY user:1001 age 1

The benefit is field-level reads and writes. Changing an email address does not require reading and rewriting the entire object. A single hash supports roughly 4.2 billion fields, but in practice it should be split once it grows beyond a few thousand.

list: Ordered, allows duplicates, and supports pushing and popping at both ends. Its classic use case is a simple queue:

LPUSH task:queue "{...job...}"
BRPOP task:queue 30          # Blocking consumption with a 30-second timeout

BRPOP is the blocking variant, so idle consumers do not burn CPU. However, list does not prevent a task from being lost after a consumer failure because the task disappears as soon as it is popped. Use stream, described below, when you need a reliable queue.

set: Unordered and unique. Deduplication and tags are its common use cases, such as calculating "unique users who viewed this article" or "users who have both categories of tags."

SADD post:123:viewers uid:1 uid:2 uid:3
SINTER tag:go tag:redis      # Intersection: people who follow both go and redis
SUNION  tag:go tag:redis     # Union
SDIFF   tag:go tag:redis     # Difference

zset (sorted set): A set whose members have scores and are ordered by those scores. This is Redis's most distinctive structure, and leaderboards are its irreplaceable use case:

ZADD leaderboard:weekly 950 "user:1" 870 "user:2" 920 "user:3"
ZREVRANGE leaderboard:weekly 0 9 WITHSCORES   # Top 10
ZINCRBY leaderboard:weekly 10 "user:2"        # Add points
ZRANK  leaderboard:weekly "user:1"            # Rank, starting from 0

The equivalent SQL is ORDER BY score DESC LIMIT 10. Once the dataset grows, that requires a full-table scan plus index maintenance and is nowhere near as fast as zset's skip list.

stream: Introduced in 5.0, with message IDs, Consumer Groups, and an ACK mechanism. Use it instead of list for messaging scenarios that require reliable delivery and retries after failed consumption. The XADD / XREADGROUP / XACK trio can support an asynchronous task flow before you need to deploy a full-scale MQ system.

hyperloglog: Cardinality estimation with a fixed 12KB memory footprint, capable of estimating hundreds of millions of unique values with an error rate of about 0.81%. For metrics such as unique visitors where an estimate is sufficient, it uses hundreds of times less memory than set. The trade-off is that it can only estimate the count and cannot tell you which users were included.

The one-line selection guide is: use string/hash for cached objects, list/stream for queues, set for deduplication, zset for leaderboards, and hyperloglog for distinct counts. Choose the right structure before discussing performance.

3. Production Pattern 1: Caching Strategies

Caching is Redis's most common role, but "adding Redis" does not mean you have a caching strategy. In production, you must decide at least three things: the update strategy, expiration, and null values.

The most common update strategy is Cache Aside: read the cache first, query the DB on a miss, and then populate the cache. On writes, update the DB first, then delete the cache rather than updating it.

Read: GET cache → miss → query DB → SET cache
Write: UPDATE DB → DEL cache

Why delete instead of update? Updating the cache creates concurrency problems. If two write requests interleave, an old value may remain in the cache. Deletion lets the next read rebuild it and keeps the semantics simple. The order also matters between "update the DB, then delete the cache" and "delete the cache, then update the DB." With the latter, a read can repopulate the cache with the old value after the cache is deleted but before the DB is updated, leaving stale data in place for a long time. The production default is "update DB → delete cache," with delayed double deletion as a fallback for extreme concurrency.

Write Through takes a different approach: update the cache and DB together, with consistency guaranteed by Redis or the client. It is complex and amplifies writes, so ordinary applications rarely use it. It is suitable for configuration data that is rarely written, frequently read, and requires very strong consistency.

A TTL is mandatory. A cache without a TTL is a buried hazard: data accumulates without limit, memory fills up, an eviction policy is triggered, and unpredictable keys are removed. Choose TTL values according to the acceptable window of inconsistency: a few minutes for user profiles, a few minutes for leaderboards, and a few hours for popular articles. Remember: the cache accelerates access; it is not the source. The DB is the source.

Caching null values prevents cache penetration, discussed in Section 6. When a key does not exist, cache a null value with a short TTL so malicious requests cannot repeatedly hit the DB.

4. Production Pattern 2: Distributed Locks

Distributed locking is Redis's second most common use case and the one most frequently implemented incorrectly. A usable distributed lock must provide at least mutual exclusion, freedom from deadlocks, releasability, and safe release.

Correct pattern for a single instance:

-- Acquire the lock atomically: set it only if the key does not exist, with an expiration
SET lock:order:1001  <token>  NX  EX 30

<token> is a unique random string generated by the client, such as a UUID. This is critical, as explained below. NX EX 30 makes lock acquisition and expiration assignment atomic. The old two-step SETNX + EXPIRE pattern creates a permanent deadlock if the process crashes between the two commands, a classic production incident.

Lock release must use Lua. Calling DEL lock:order:1001 directly is wrong. A GC pause might let your lock expire and another client acquire it; your subsequent DEL would then delete someone else's lock. The correct approach is to delete only when the token matches. The check and deletion must be atomic, so Lua is required:

if redis.call("GET", KEYS[1]) == ARGV[1] then
    return redis.call("DEL", KEYS[1])
else
    return 0
end

ARGV[1] is the token used when acquiring the lock. Only the owner of a lock can release it. That is the purpose of the token.

The lock expiration time (EX 30) is a trade-off. If it is too short, another client can acquire the lock before the operation finishes. If it is too long, the lock remains unavailable for a long time after the owner's process crashes. Production systems usually set a conservative upper bound, such as 30s, and renew it with a watchdog. While the lock is held, the watchdog periodically runs EXPIRE to extend the TTL and stops renewing it when the operation finishes.

One final warning: a distributed lock on a single Redis instance can fail during failover. If the primary goes down before the lock is replicated and a replica is promoted, two clients may acquire the same lock. For scenarios where correctness is critical, especially those involving money, use Redisson's RedLock or switch to a CP system such as ZooKeeper/etcd. Redis represents a trade-off that leans toward AP while incorporating some CP behavior; do not treat it as a strongly consistent lock service.

5. Production Pattern 3: Rate Limiting and Leaderboards

Rate limiting is lightweight to implement with Redis. There are two mainstream algorithms.

Fixed window, simple version: INCR + EXPIRE.

INCR ratelimit:uid:1001
EXPIRE ratelimit:uid:1001 60   # Set expiration on the first request
-- Reject when the returned value is > 100

Its weakness is a burst at the window boundary. If 100 requests arrive at both the 59-second and 61-second marks, the system can receive 200 requests within one second.

A more accurate option is a sliding window implemented with zset. Store each request timestamp as a score, remove records outside the window on every request, and then count the entries still inside it:

local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])

redis.call("ZREMRANGEBYSCORE", key, 0, now - window)
local count = redis.call("ZCARD", key)
if count < limit then
    redis.call("ZADD", key, now, now)   -- use now as the member; approximately unique
    redis.call("EXPIRE", key, window / 1000)
    return 1
else
    return 0
end

A token bucket is better when bursts are allowed, such as at an API gateway. Its implementation is similar, with tokens replenished according to elapsed time.

A leaderboard is simply a zset; the basic commands appeared earlier. Two finishing steps matter in production. First, use separate keys for each time period (leaderboard:weekly:2026W29, leaderboard:daily:20260717) and expire old ones. Second, merge them with ZUNIONSTORE. For example, aggregate four weekly leaderboards into a monthly leaderboard without recalculating everything. Leaderboard data is usually small, but at tens of millions of users you must watch zset memory usage. If necessary, cache only the Top N and query the DB on demand for the long tail.

6. Persistence: RDB and AOF

Redis stores data in memory, so power loss means data loss. Persistence is required for data safety. Redis offers two approaches, and you must understand their trade-offs before choosing.

RDB (snapshots): Periodically writes a full binary snapshot of memory to dump.rdb. Its advantages are compact files, fast recovery, and little performance impact, because a forked child process writes asynchronously. Its disadvantage is that data written between snapshots is lost. The default configuration may lose several minutes of data.

AOF (logging): Appends every write command to a log and replays it on restart. Its advantage is less data loss: with appendfsync everysec, it flushes once per second and loses at most one second. Its disadvantages are larger files and slower recovery. Long-running instances rely on BGREWRITEAOF for compaction.

The production default is to enable both. RDB provides fast cold starts, while AOF minimizes data loss. Since Redis 4.0, hybrid persistence (aof-use-rdb-preamble yes) has been available. During an AOF rewrite, the first part uses RDB binary data for fast recovery and the second appends incremental commands. This combines the advantages of both and is the current recommended approach.

One point bears emphasis: persistence protects against "process restarts / power loss," not "operator error." Run FLUSHALL and AOF will faithfully record it. Disable dangerous commands in production (rename-command FLUSHALL "") and regularly back up RDB files to an off-site location.

7. High Availability: Replication, Sentinel, and Cluster

A single Redis instance is an incident waiting to happen. Production requires high availability. There are three options, in increasing order of complexity.

Primary-replica replication: One primary has multiple replicas; writes go to the primary and reads can go to replicas. A single REPLICAOF master 6379 configuration command is enough. This provides read scaling and data redundancy, but primary failure still requires manual intervention, so it is not true high availability.

Sentinel: Adds a group of independent Sentinel processes to primary-replica replication. They monitor primary and replica health, perform automatic failover, and notify clients. If the primary fails, Sentinel promotes a replica and broadcasts the change to clients. It is suitable for caches, sessions, and similar workloads that tolerate interruptions lasting a few seconds. The service is briefly unavailable during failover. This is the most common choice for medium-scale systems.

Cluster: Shards data across multiple shards, each with its own primary and replicas. Redis Cluster has 16,384 hash slots, and the CRC16 of a key determines its slot. It addresses both capacity limits and read/write scaling. When a single machine lacks enough memory or CPU, Cluster is the only way to scale horizontally. The cost is complexity: cross-slot operations are restricted, multi-key commands require all keys to be in the same slot, operations demand more expertise, and failover behavior is harder to debug.

The short selection rule is: use Sentinel when the dataset is below a few dozen GB and interruptions of a few seconds are acceptable; use Cluster for larger datasets that require horizontal scaling. Migration between the two is not seamless, so make the choice early. Retrofitting it later is expensive.

8. Classic Pitfalls: Penetration, Breakdown, and Avalanche

These three terms are often memorized, but few people have actually identified and handled them in production. They share one trait: after a cache miss, traffic reaches the DB. What differs is the reason for the miss.

Cache penetration: Requests target a key that does not exist at all, such as a negative ID or a maliciously constructed user ID. The cache always misses, so every request penetrates through to the DB. Attackers love this technique.

• Solution 1: Cache null values. If the DB finds nothing, cache a null value with a short TTL, such as 60s, so the next request hits the cache.
• Solution 2: Bloom filter. Put a Bloom filter in front of the cache and load all valid IDs into it. Requests check the filter first; if an ID is absent, reject it without touching Redis or the DB. Bloom filters have false positives, meaning they may report that an absent item exists, but no false negatives: if they say an item is absent, it is definitely absent.

Cache breakdown: A hot key suddenly expires, causing tens of thousands of simultaneous cache misses. They all hit the DB to rebuild the cache, overwhelming it.

• Solution 1: Mutex-based rebuilding. After a cache miss, allow only one request to query the DB and rebuild the cache. Other requests wait briefly and retry the cache. The distributed lock described above can implement this.
• Solution 2: Never expire hot keys; refresh them proactively with a background task. This is more complex but eliminates breakdowns entirely.

Cache avalanche: Many keys expire simultaneously, or Redis itself goes down, and the DB is instantly flooded with requests.

• Solution 1: Spread expiration times. Add random jitter to TTLs (ttl = base + random(0, 300)) to prevent simultaneous expiration.
• Solution 2: Use multiple cache levels. Add a local cache, such as Caffeine or an in-process cache, as a fallback. If Redis goes down entirely, the local cache can absorb traffic for a while.
• Solution 3: Use Redis high availability, Sentinel/Cluster, to address the case where Redis itself fails.

Penetration, breakdown, and avalanche may have elaborate names, but all three mean the same thing at a fundamental level: the cache is the gate in front of the DB, and once it leaks, the DB faces traffic directly. The core of mitigation is always to let less traffic through with null values or Bloom filters, prevent it from going through all at once with jitter or mutexes, and provide a fallback when it does get through with local caching, rate limiting, or graceful degradation.

9. Big Keys and Hot Keys

The last two commonly overlooked problems are chronic issues that emerge only after the system has been running for a while.

Big key: A single key has an excessively large value, such as a string containing several MB of JSON or a hash/list containing hundreds of thousands of elements. The consequences include blocked reads and writes because a large command monopolizes the single thread, saturated network egress, a slot becoming stuck during cluster migration, and the main thread blocking for several seconds when DEL removes a big key.

• Detection: redis-cli --bigkeys or MEMORY USAGE key.
• Remediation: Split it up. Divide a large hash into multiple smaller keys, paginate a large list, and bucket a large zset by time or category. Delete big keys with UNLINK, which is asynchronous, instead of DEL.

Hot key: One key receives far more traffic than others, such as the cached product page for a celebrity-endorsed bestseller. This can max out one shard's CPU because a cluster cannot distribute a single key across shards, concentrating traffic on one Redis node.

• Detection: redis-cli --hotkeys, which requires the LFU eviction policy, or client-side instrumentation.
• Remediation: Local caching. Cache a copy inside each application process with Caffeine or go-bigcache and use a very short TTL of a few seconds, so most requests never reach Redis. This is essentially a multi-level cache that distributes a single hotspot across application instances.

Summary

• Redis is more than a cache: It is an in-memory, single-threaded, data-structure-rich KV store. string/hash/list/set/zset/stream/hyperloglog each have distinct strengths, and choosing the right structure is the first step toward using Redis well.
• Remember three production patterns: Cache Aside caching (update DB → delete cache + TTL); distributed locks (SET NX EX + unique token + Lua release, while RedLock/etcd provide strong consistency); and rate limiting plus leaderboards (zset sliding windows and time-partitioned zset leaderboard keys).
• Persistence and high availability: RDB is fast and AOF is comprehensive. Production usually enables both with hybrid persistence. Sentinel handles failover with interruptions lasting a few seconds, while Cluster handles capacity and scaling through sharding at the cost of heavier operations.
• Penetration / breakdown / avalanche correspond to "requesting nonexistent data / a hot key expiring / mass expiration": null-value caching plus Bloom filters, mutex-based rebuilding, and randomized expiration times all keep traffic from flooding the DB at once.
• Split big keys and distribute hot keys: Use UNLINK for asynchronous deletion and local caching for hotspot protection. Detect both problems proactively with monitoring (--bigkeys / --hotkeys) instead of waiting for production latency spikes.

In one sentence: Redis is powerful because it is "fast + structurally rich + atomic," and dangerous because it is "fast + single-threaded + entirely in memory." The more casually you use it, the more spectacularly it fails. Choose the right data structures, configure expiration and eviction properly, and monitor big and hot keys. That is how Redis goes from "it runs" to "it holds up under load."