Redis Persistence: RDB vs. AOF and Hybrid Mode

Redis keeps all data in memory while it runs. It really is fast, but memory has one catch: lose power and it is gone.

So "persistence" fundamentally answers one question: If the process crashes, the machine goes down, or power fails, how much data can I recover after a restart? How quickly can I recover it? What will it cost?

This article explains Redis's three persistence options, RDB, AOF, and the RDB+AOF hybrid mode, and presents the selection logic I use in real production environments.

1. Why Redis Needs Persistence

Many people's first impression of Redis is "an in-memory database." That is only half right. Redis's runtime is indeed entirely in memory, but Redis does not "live only in memory."

Why persist data? There are three very practical scenarios:

1. Preserve data across restarts. Whether the process exits because of a release restart, a configuration change, or an OOM kill, everything in memory disappears. A cache can simply rebuild from its source, but losing semi-critical application data such as sessions, message queues, counters, rate limits, or leaderboards is a production incident.
2. Recover from outages. A power failure, kernel panic, or hardware fault causes all in-memory data to disappear during an ungraceful shutdown. Without a persistence file, Redis restarts as an empty instance.
3. Perform full primary-replica synchronization. A new replica, or one reconnecting after a disconnection, must begin full synchronization from an RDB snapshot created by the primary. The persistence file carries the replication data.

The question is therefore not whether to persist, but what kind of persistence you need. Can you trade several minutes of data for performance, or can you not afford to lose even one second? The answer determines whether you choose RDB, AOF, or hybrid mode.

Redis provides two independent persistence mechanisms:

• RDB (Redis Database): Periodically takes a "photograph" of all data in memory and stores it in a snapshot file named dump.rdb.
• AOF (Append Only File): Appends every write command to a log file named appendonly.aof; replaying the log reconstructs the data.

You can enable either one independently or enable both. Let us examine them separately.

2. RDB: Periodic Full Snapshots

How It Works

The RDB approach is straightforward: at regular intervals, dump every key-value pair currently in memory into one compact binary file. That file is named dump.rdb.

Snapshots can be triggered in several ways:

• Manual commands: SAVE dumps from the main thread and blocks every request, so never use it in production. BGSAVE forks a child process to dump in the background and is recommended.
• Automatic configuration triggers: Use save rules in redis.conf to define "trigger BGSAVE when M modifications occur within N seconds."
• shutdown: When Redis shuts down normally without AOF enabled, it automatically performs an RDB dump.
• Primary-replica replication: When a replica connects for the first time, the primary triggers BGSAVE and sends the resulting RDB to it.

Configuration Example

# RDB settings in redis.conf

# save rules: trigger BGSAVE when any one of these conditions is met
save 3600 1     # At least 1 key changes within 3600 seconds
save 300 100    # At least 100 keys change within 300 seconds
save 60 10000   # At least 10000 keys change within 60 seconds

# To disable RDB entirely, use:
# save ""

# RDB file name and directory
dbfilename dump.rdb
dir /var/lib/redis

# Whether to stop accepting writes if the previous BGSAVE failed
# Default: yes, to protect data consistency
stop-writes-on-bgsave-error yes

# Whether to compress RDB files with LZF
# Default: yes, trading CPU for disk space
rdbcompression yes

# Whether to add a CRC64 checksum to the file header
# Default: yes, to detect file corruption
rdbchecksum yes

Forked Child Processes and Copy-on-Write

How does BGSAVE obtain a consistent snapshot without blocking the main thread? The answer is fork() + copy-on-write (COW):

1. The primary process calls fork() to create a child process.
2. The child and parent share the same physical memory pages through COW.
3. The child traverses the shared memory and writes the data to the RDB file.
4. Meanwhile, the parent continues processing client requests. If the parent modifies a memory page, the kernel copies that page for the parent to write, while the child continues seeing the old version.

This ensures that the child dumps a complete, consistent snapshot from one point in time, while the main thread remains unblocked throughout, apart from the instant when fork itself runs, which is discussed later.

RDB Advantages and Disadvantages

Advantages:

• Compact files. RDB uses a binary format. LZF compression keeps files small and makes them convenient to back up, transfer, and replicate across data centers.
• Fast recovery. On restart, Redis loads the RDB directly into memory, which is much faster than replaying an AOF. A 10GB instance might recover from RDB in tens of seconds, while AOF replay can take several minutes.
• Little impact on the main thread. After forking the child, the main thread only pays a small copying cost when writes trigger COW. Reads are unaffected.
• Suitable for disaster backups. RDB files can be uploaded periodically to S3 / object storage in another location.

Critical disadvantages:

• Writes between snapshots are lost. With a rule such as save 300 100, a power failure at the worst possible moment loses all writes from the preceding five minutes. That is unacceptable for many applications.
• Forking a large instance can cause a pause. A dedicated section below explains this.

In short: RDB is suitable for a "full backup from a specific point in time today," not for strong persistence where "not even one second can be lost."

3. AOF: Recording Every Write Command

How It Works

AOF takes a completely different approach: append every command that changes data (SET, DEL, INCR, EXPIRE, HSET, and so on) verbatim to a log file. Replaying those commands in order during restart reconstructs the data state from that time.

AOF appears far more "reliable" than RDB because every write reaches the disk. But "reaching the disk" is subtle at the operating-system level: write() only writes data into the kernel's page cache. Persisting it to the physical disk requires fsync(). AOF therefore needs an fsync policy that controls how often the log is flushed to physical storage.

fsync Policies: always / everysec / no

This is AOF's most important setting because it directly determines how much data can be lost during a power failure:

# redis.conf
appendonly yes
appendfilename "appendonly.aof"
appenddirname "appendonlydir"   # Since Redis 7.0, AOF uses a directory structure

# Choose one of the three
appendfsync everysec

Comparison of the three policies:

always is a trap. It looks safest, but every write must wait for disk I/O to finish. A single fsync takes milliseconds on a mechanical drive, so QPS can fall by an order of magnitude. Almost no production workload can tolerate that.

everysec is the sweet spot. A background thread performs fsync once per second, while the main thread only appends commands to the AOF buffer. Performance is essentially unaffected, and the worst-case loss is one second of data. That trade-off is acceptable for 99% of applications.

AOF Advantages and Disadvantages

Advantages:

• High data safety. With everysec, at most one second is lost, far better than the several minutes RDB can lose.
• Readable format. AOF uses the plain-text RESP protocol format. During an incident, you can run tail -f appendonly.aof to inspect recent activity or even edit the file manually to remove a bad command, such as an accidental FLUSHALL.
• Fine-grained durability. Unlike a full RDB dump, each write only appends a few lines.

Disadvantages:

• Much larger files than RDB. For the same dataset, AOF is typically two to five times larger because it records every intermediate state.
• Slow recovery. Restarting requires replaying commands one by one, which can block a large instance for several minutes.
• Continuous disk writes. AOF is append-only, so write-heavy instances place more disk I/O pressure on the system than RDB does.

In short: AOF solves the "data loss" problem but introduces the new problems of "large files and slow recovery." That is why AOF rewrites are necessary.

4. AOF Rewrite: Compacting an Inflated Log

Because AOF only appends, the file grows without limit over time. Suppose you execute INCR counter 10000 times. The AOF contains 10000 INCR counter commands, even though the final state is only counter = 10000. The other 9999 commands are redundant.

An AOF Rewrite reconstructs a new AOF from the actual current state in memory using the fewest possible commands, then replaces the inflated old log.

Triggers

# redis.conf

# Trigger when the AOF has grown 100% since the previous rewrite
auto-aof-rewrite-percentage 100

# Require the AOF to be at least 64MB to avoid frequent rewrites of small files
auto-aof-rewrite-min-size 64mb

You can also trigger or tune it manually:

• BGREWRITEAOF: Rewrite asynchronously in the background.
• CONFIG SET auto-aof-rewrite-percentage 50: Adjust the threshold dynamically.

Rewrite Process

Like BGSAVE, AOF rewriting also uses fork():

1. The primary process forks a child process.
2. The child traverses the current in-memory data and writes each key's current value to a new AOF with one command, such as SET k v or HSET k f1 v1 f2 v2.
3. While the child performs the rewrite, the parent continues processing new writes. Those incremental commands are temporarily stored in an aof_rewrite_buffer.
4. When the child finishes, it notifies the parent. The parent appends the buffered incremental commands to the new AOF and atomically replaces the old file.

The main thread is almost never blocked, apart from the instant of fork, but both the old and new AOF files exist temporarily, so disk usage can double.

A rewrite can reduce the AOF to between 1/2 and 1/5 of its original size and significantly improve recovery speed. However, rewriting is both I/O-intensive and sensitive to fork latency. Large instances should run it during off-peak hours, never at peak traffic.

5. Hybrid Mode: RDB + AOF (Recommended for Production)

At this point, a contradiction becomes apparent:

• RDB recovers quickly but may lose several minutes of data;
• AOF protects data but produces larger files and recovers slowly.

Can Redis be both fast and safe? Since Redis 4.0, the answer has been hybrid persistence (Hybrid AOF).

The Idea

Split the AOF file into two parts:

• First part: a full snapshot in RDB format. This is a binary dump of current memory.
• Second part: incremental AOF commands. These are all write commands issued after the snapshot point.

During an AOF Rewrite, instead of translating in-memory data into textual AOF commands, the child process writes RDB binary data directly to the beginning of the AOF file, then appends the incremental commands issued during that period.

During recovery, Redis first loads the RDB portion quickly and then replays the much smaller AOF portion. This combines RDB's recovery speed with AOF's data completeness.

Configuration

Enable it with one setting. Redis 4.0 introduced it, and it has defaulted to yes since 5.0:

# Enable RDB-AOF hybrid persistence
aof-use-rdb-preamble yes

# AOF must also be enabled
appendonly yes
appendfsync everysec

# Usually retain RDB as well for off-site backups
save 3600 1
save 300 100

Why It Is Recommended for Production

Hybrid mode assigns each job intelligently:

• Full recovery uses RDB speed, so a large instance can start in tens of seconds.
• AOF fills in incremental changes, limiting data loss to one second.
• File size remains manageable, because a rewritten AOF is roughly one RDB plus a short incremental segment.
• The operational model stays simple, because you only need to manage one file, appendonly.aof, rather than coordinate separate RDB and AOF files.

In my production experience, Redis 5.0+ instances use hybrid mode by default whenever AOF is enabled. It is the only option that performs reasonably across recovery speed, data safety, and file size, and it is the community's consensus production default.

6. Fork Latency and Memory: Hidden Killers for Large Instances

Both RDB BGSAVE and AOF Rewrite depend on fork(). Fork is also where large Redis instances most commonly run into trouble.

Time Spent in fork Itself

On Linux, fork() uses copy-on-write, so the child does not actually copy all of the parent's memory. It only copies the page tables. But the page tables themselves can be enormous for a large-memory instance:

• A 24GB Redis instance may have page tables measuring tens of MB.
• Fork must copy the entire page table, making it an O(memory size) operation.
• In real measurements, forking a 10GB instance can take 100-300ms, while a 30GB instance can take 1-2 seconds.

During those 1-2 seconds, the main thread is blocked, and every request waits. This is "fork latency." For latency-sensitive workloads such as real-time quotes, risk control, and ad bidding, it is a major cause of P99 jitter.

Risk of Doubled Memory Usage: The Other Side of COW

After fork, the parent and child share memory pages, and COW normally makes this memory-efficient. However:

• If the primary process keeps writing frequently during the dump, every modified page causes the kernel to copy a physical page.
• In the extreme case, the primary modifies every page, and physical memory usage nearly doubles.
• This can cause an OOM. For example, if Redis is configured for 24GB and uses all of it, another 24GB of COW copies during BGSAVE can exceed a machine's 48GB of memory.

Operational guidance:

• Keep Redis instance memory usage below 50-60% of physical memory so COW has headroom during fork.
• Monitor used_memory_rss in info memory; it rises significantly during BGSAVE.
• For large instances over 20GB, consider sharding with Redis Cluster and keep each instance smaller.

Improving Fork Performance

# Disable transparent huge pages (THP), which can make fork 2-3 times slower
# Run this on the host:
# echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Control rewrite/BGSAVE frequency so they do not run during peak traffic
# Use CONFIG SET to trigger them manually during off-peak periods

In short: fork is a "hidden tax" on large Redis instances. The larger the instance, the more carefully you must manage fork frequency and COW memory.

7. Recovery-Speed Trade-offs: Which One Should You Choose?

The following is a representative comparison for a 10GB dataset on identical hardware:

Recovery-speed ranking: RDB ≈ hybrid > AOF (pure)

Data-safety ranking: always > hybrid ≈ AOF everysec > RDB > AOF no

Overall value ranking: hybrid mode > AOF everysec > RDB

8. Selection Advice: It Depends on How You Use Redis

There is no "best option," only the "most suitable option." Here is how I classify common production uses.

Scenario 1: Pure Cache (Rebuild from the Source After Keys Expire)

Recommendation: Disable persistence or enable only RDB.

save 900 1              # Low-frequency RDB as a fallback only
appendonly no

Reason: If a cache is lost, rebuild it from its source. There is no need to pay the disk I/O and fork costs of stronger persistence. If a cold start would be too slow, enable a low-frequency RDB snapshot.

Scenario 2: Semi-Critical Application Data (Sessions, Counters, Rate Limits, Leaderboards)

Recommendation: AOF + everysec.

appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes   # Enable hybrid mode
save 3600 1                # Retain RDB for off-site backups
save 300 100

Reason: Losing one second of data is acceptable, but losing several minutes is not. Hybrid mode also keeps recovery fast enough.

Scenario 3: Core Data Storage (Orders, Payments, Messages)

Recommendation: Do not use Redis as the primary store. Use a real persistent database such as MySQL / PostgreSQL / MongoDB. Redis should be a cache or acceleration layer at most, rebuilt through Canal/binlog subscriptions.

Reason: Redis persistence is "best effort," not an ACID guarantee. Storing core application data in Redis is an architectural mistake.

Scenario 4: Large-Scale Instances (>20GB)

Recommendation: Sharding (Redis Cluster) + hybrid mode.

# Enable hybrid persistence on every shard instance
appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes

Keep each instance between 8-16GB to avoid the fork latency and doubled COW memory risks of large instances. Also configure primary-replica replication + Sentinel HA.

9. Operations Checklist

Review these points before launch to prevent 80% of persistence incidents:

1. Is appendfsync set to everysec? Do not use always or no.
2. Is aof-use-rdb-preamble yes enabled? It defaults to yes in Redis 5.0+, but verify it on older versions.
3. Is disk space monitored? AOF rewrite temporarily creates two AOF files, doubling disk usage.
4. Is stop-writes-on-bgsave-error yes enabled? This prevents Redis from accepting further writes after an RDB failure, avoiding silent data loss.
5. Is THP disabled? Run echo never > /sys/kernel/mm/transparent_hugepage/enabled; otherwise, fork latency can double.
6. Is there enough memory headroom? Keep used_memory_rss below 60% of physical memory.
7. What if the AOF file is corrupted? Repair it with redis-check-aof --fix appendonly.aof, which truncates the corrupted section.
8. How do you validate an RDB file? Use redis-check-rdb dump.rdb.
9. Are there off-site backups? Periodically upload RDB files to object storage to survive an entire data-center failure.
10. Have you rehearsed disaster recovery? An untested backup is no backup.

10. Monitoring Metrics

These are the most important metrics in Prometheus + redis_exporter:

Pay particular attention to redis_rdb_last_bgsave_time_sec and fork frequency. If fork often occurs during peak traffic, raise the save thresholds or switch to purely manual triggering.

Summary

The key lessons about Redis persistence are:

1. RDB is a snapshot, AOF is a log, and hybrid mode combines them. Understanding these fundamental differences matters more than memorizing configuration options.
2. appendfsync everysec is the production default. always is a performance trap; no is a gamble with data.
3. Hybrid mode in Redis 4.0+ is the preferred production option. One AOF file contains a full RDB in the first part and incremental commands in the second, providing both fast recovery and data safety.
4. Fork is a hidden tax on large instances. The larger the instance, the slower fork becomes and the greater the risk that COW doubles memory usage. Keep a single instance between 8-16GB and reserve 40% memory headroom.
5. Choose according to use: pure caches need no persistence, semi-critical application data should use hybrid mode, and core data does not belong in Redis.

One final thought:

Persistence is not a switch; it is a commitment. How much CPU, disk, and fork jitter are you willing to pay to preserve data across restarts? Only after understanding the cost of that commitment can you say, "My Redis is reliable."