一、Redis RDB 持久化
1、RDB 工作原理
RDB（Redis Database）基于时间进行生成数据快照，默认只保留当前最新数据状态，优点时执行速度较快，但上次数据保存点到当前时间点之间的数据修改存在丢失风险
bgsave工作流程
1、当触发bgsave操作时，Redis 从Master主进程fork出一个子进程
2、使用写时复制机制，子进程将内存中数据dump为一个临时rdb文件（temp_pid.rdb）
3、当数据保存完成后，覆盖替换原有的rdb文件
由于bgsave在用临时rdb文件进行覆盖替换正式rdb文件过程中可能发生断电或主机故障，导致文件未完成保存而中断，可能造成数据丢失情况。后期可以手动对生成的rdb文件进行备份保存，确保最大化数据保存。
2、RDB 实现模式
save 手动执行：save期间会阻塞其他请求执行
bgsave 手动执行：后台异步执行，不会阻塞其他请求
自动执行（默认）：根据配置文件中 save 规则自动触发RDB操作
3、RDB 优缺点对比
优点
RDB保存某个时间节点的全部数据，可以通过脚本执行redis bgsave（非阻塞模式） 或 save（阻塞模式）命令，自定义备份时间点，保留多份数据备份。可恢复到不同时间点数据版本，并且数据格式可支持多种第三方工具进行数据分析，很适合作为备份方案。
RDB 最大化发挥Redis性能，执行rdb备份操作时，由主进程fork出的子进程进行数据备份的所有操作，主进程无需执行磁盘I/O操作。
RDB 在加载较大容量数据时（几个G），数据载入速度比AOF模式更快
缺点
无法实时保存数据，可能丢失上次RDB备份到当前时间的内存数据
如果对数据安全性要求极高的情况下，不适用将RDB作为主要数据保障方案，虽然redis支持自定义多条规则触发RDB备份操作，但是由于RDB每次都是保存完整数据集内容，整个备份过程并不轻松，当数据量较大时此操作会持续多达几分钟以上，在这个过程中如果发生故障，将丢失这期间的数据内容。
当数据量非常大时，从主进程fork出子进程将内存数据保存至rdb文件整个过程需要耗费一些时间，可能达到 毫秒或秒 级，依赖于磁盘I/O性能。
在父进程fork子进程时，fork()操作可能会非常耗时，会触发服务短时中止处理客户端请求，如果数据量非常巨大，fork()操作将耗费更多时间，同时CPU资源非常紧张的话，就会造成秒级的服务中止。虽然AOF rewrite操作也会fork()，但无论AOF rewrite 执行语句持续有多长时间，数据持久性不会受到任何损失
4、RDB 配置详解
################################ SNAPSHOTTING ################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving completely by commenting out all "save" lines.
#
# It is also possible to remove all the previously configured save
# points by adding a save directive with a single empty string argument
# like in the following example:
#
# save "" save 900 1 #在 900秒 内有大于等于 1 个key的值发生改变触发RDB备份
save 300 10 #在 300秒 内有大于等于 10 个key的值发生改变触发RDB备份
save 60 10000 #在 60秒 内有大于等于 1000 个key的值发生改变触发RDB备份
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes #当出现空间不足等原因导致RDB文件保存错误时，禁止redis写入，减少发生故障时的数据丢失量，会造成业务中断。不禁止写入，数据丢失风险增大，但当前不会造成业务中断 默认yes,生产环境建议修改为 no # Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes #持久化生成RDB文件进行压缩，默认 yes # Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes #对备份文件进行CRC64校验，默认yes # The filename where to dump the DB
dbfilename dump.rdb #RDB文件名（无需携带路径）。触发RDB完成后服务自动生成，服务重启后未开启AOF模式情况下，会直接通过此文件加载数据，默认 dump.rdb # The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./ #备份文件存放目录 默认为 ./，注意目录权限问题
注意：每次执行RDB时。新生成的rdb文件会覆盖原有文件，因此要注意对历史rdb文件进行定期备份
二、Redis AOF 持久化
1、AOF 工作原理
AOF(AppendOnlyFile) ：AOF按照操作顺序依次将操作记录追加至aof文件末尾
AOF和RDB一样都采用了写时复制机制，AOF默认每秒fsync一次，即将执行的操作记录追加至文件末尾，这样当发生故障时最多导致1秒钟内的数据丢失。也可设置为always模式，即每次发生写入操作时同步执行fsync，fsync将会在后台执行，因此主进程可以继续正常处理客户端请求，而不会受到写入AOF时的磁盘I/O影响。
同时启用AOF和RDB时，加载数据进行恢复时，默认将优先载入AOF保存的数据。
注意：AOF默认为关闭状态，当第一次启用AOF并且重启服务后，由于aof文件优先级高于rdb文件，而会导致原数据丢失。
2、AOF rewrite 重写
AOF rewrite机制可以将aof文件中 重复的，可合并的、无效过期的数据重新整理后，写入一个新的aof文件，从而节约AOF备份占用的空间容量，也可以加速数据载入恢复的过程。
AOF rewrite 工作流程
1、当主进程接收到bgrewriteaof命令后，执行fork出一个子进程
2、同时创建出2个buffer缓冲区，将临时新增的写请求数据存放在缓冲区内，aof_buffer 数据写入旧的aof文件中，aof_rewite_buffer 临时存放
3、子进程将生成临时aof文件，并进行重写
4、子进程重写完成后通知主进程整理结束，主进程将 aof_rewrite_buffer 数据合并至临时aof文件中
5、用临时aof文件覆盖旧的aof文件
AOF rewrite实现模式
手动执行 bgrewriteaof 命令
配置自动rewrite策略
3、AOF 优缺点对比
优点
数据安全性相对较高
突发故障宕机，不会影响破坏原有内容，服务恢复后不影响原有数据载入（记录写入过程中出现宕机，可通过 redis-check-aof 解决数据一致性问题）
文件大小可通过重写命令进行整理，重写过程中突发故障也不会影响原有数据的保存
aof文件格式清晰可读，误操作时可通过修改aof文件删除误操作相关记录，重载数据进行恢复
缺点
重复操作会导致记录部分无效或过期数据，aof文件大小会大于rdb文件
AOF在载入大数据时会比RDB载入速度慢
根据fsync策略不同，AOF速度可能会比RDB慢
出现bug的可能性更高
4、AOF 配置详解
############################## APPEND ONLY MODE ############################### # By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information. appendonly no #启用AOF,优先级高于RDB，对于已存在数据的redis应手动rewriteaof生成aof重写文件后，再修改配置文件重启服务 # The name of the append only file (default: "appendonly.aof") appendfilename "appendonly_6379.aof" #AOF文件名称，存放于 dir 参数配置的目录下，默认 appendonly.aof # The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec". # appendfsync always
appendfsync everysec #AOF同步策略,默认 everysec
# no 不通过执行fsync，由操作系统保证将数据同步至磁盘，Linux默认fsync策略为30s,最多会导致丢失30s数据
# everysec 每秒执行一次fsync操作，最多导致丢失1s的数据，对性能压力较小，安全性较高，生产环境建议值
# always 每次写入数据都执行fsync操作，对服务器性能负担大，性能最差，但安全性最高
# appendfsync no # When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no #执行bgrewriteaof期间，对新增append数据，暂缓使用AOF同步策略，默认 no
#当同时执行bgrewriteaof操作和主进程写入aof时，由于bgrewriteaof会占用大量磁盘操作，会导致主进程执行aof时出现阻塞情况。主要影响磁盘IO和请求阻塞时间
# no : 不暂缓写入，aof记录依旧立即同步至磁盘，不会发生数据丢失情况，但需承担请求阻塞风险
# yes : 暂缓写入aof记录（appendfsync no），临时将数据写入缓冲区内，由操作系统调度写入磁盘，如果此时发生redis宕机会造成缓冲区数据丢失，丢失数据量为操作系统的fsync策略（Linux默认30s），性能较好不会触发请求阻塞
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
#rewriteaof 可以保证AOF文件的精简及完整，启动时更快的加载数据
auto-aof-rewrite-percentage 100 #自动触发rewriteaof的增长百分比，设置为0时表示不触发自动rewrite aof, 默认 100
auto-aof-rewrite-min-size 64mb #自动触发rewriteaof的最小文件容量，默认 64mb # An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes #允许加载由于某些情况（AOF写入过程中，进程被Kill 或 断电）导致末尾文件异常的AOF文件，默认 yes ,建议 yes # When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
# [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes #同时启用AOF和RDB持久化配置，启用后 AOF重写文件将同时包含AOF格式及RDB格式，RDB格式文件用于保存历史存量数据，AOF格式文件用于记录近期增量数据，能够更快速生成重写文件，并在出现问题时快速载入数据，默认 no
5、启用AOF策略的正确顺序
#手动查看redis aof配置是否启动
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning config get appendonly
1) "appendonly"
2) "no"
#启动aof策略，并查看当前进程及目录文件情况
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning config set appendonly yes;pstree | grep redis-server ; ll /app/redis/data/
OK
|-2*[redis-server---3*[{redis-server}]]
|-redis-server-+-redis-server
| `-3*[{redis-server}]
total 127072
drwxr-xr-x 2 redis redis 4096 Apr 15 03:46 ./
drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../
-rw-r--r-- 1 redis redis 0 Apr 15 03:46 appendonly_6379.aof
-rw-r--r-- 1 redis redis 128895906 Apr 15 03:12 dump_6379.rdb
-rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb
-rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
-rw-r--r-- 1 redis redis 1208320 Apr 15 03:46 temp-rewriteaof-4547.aof
[root@Redis-Ubuntu-1804-p22:~]# pstree | grep redis-server ; ll /app/redis/data/
|-3*[redis-server---3*[{redis-server}]]
total 251768
drwxr-xr-x 2 redis redis 4096 Apr 15 03:46 ./
drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../
-rw-r--r-- 1 redis redis 128895906 Apr 15 03:46 appendonly_6379.aof
-rw-r--r-- 1 redis redis 128895906 Apr 15 03:12 dump_6379.rdb
-rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb
-rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
#修改配置文件内的设置 appendonly yes
[root@Redis-Ubuntu-1804-p22:~]# sed -i 's/^appendonly no/appendonly yes/' /app/redis/etc/redis_6379.conf
[root@Redis-Ubuntu-1804-p22:~]# cat /app/redis/etc/redis_6379.conf | grep "^appendonly"
appendonly yes
#查看当前redis数据，进行重启验证对比
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning info Keyspace
# Keyspace
db0:keys=4531650,expires=0,avg_ttl=0 #重启redis,验证aof是否正常启用，并校验数据是否准确
[root@Redis-Ubuntu-1804-p22:~]# systemctl restart redis_6379.service
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning info Keyspace
# Keyspace
db0:keys=4531650,expires=0,avg_ttl=0
[root@Redis-Ubuntu-1804-p22:~]# ll /app/redis/data
total 251772
drwxr-xr-x 2 redis redis 4096 Apr 15 03:57 ./
drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../
-rw-r--r-- 1 redis redis 128895906 Apr 15 03:46 appendonly_6379.aof
-rw-r--r-- 1 redis redis 128895906 Apr 15 03:57 dump_6379.rdb
-rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb
-rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
[root@Redis-Ubuntu-1804-p22:~]# #日志
3849:signal-handler (1681502220) Received SIGTERM scheduling shutdown...
3849:M 15 Apr 2023 03:57:00.659 # User requested shutdown...
3849:M 15 Apr 2023 03:57:00.659 * Calling fsync() on the AOF file.
3849:M 15 Apr 2023 03:57:00.659 * Saving the final RDB snapshot before exiting.
3849:M 15 Apr 2023 03:57:13.133 * DB saved on disk
3849:M 15 Apr 2023 03:57:13.133 * Removing the pid file.
3849:M 15 Apr 2023 03:57:13.133 # Redis is now ready to exit, bye bye...
4846:C 15 Apr 2023 03:57:13.195 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4846:C 15 Apr 2023 03:57:13.195 # Redis version=5.0.14, bits=64, commit=00000000, modified=0, pid=4846, just started
4846:C 15 Apr 2023 03:57:13.195 # Configuration loaded
4846:C 15 Apr 2023 03:57:13.195 * supervised by systemd, will signal readiness
4846:M 15 Apr 2023 03:57:13.202 * Running mode=standalone, port=6379.
4846:M 15 Apr 2023 03:57:13.203 # Server initialized
4846:M 15 Apr 2023 03:57:13.204 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
4846:M 15 Apr 2023 03:57:13.204 * Reading RDB preamble from AOF file...
4846:M 15 Apr 2023 03:57:17.447 * Reading the remaining AOF tail...
4846:M 15 Apr 2023 03:57:17.447 * DB loaded from append only file: 4.243 seconds
4846:M 15 Apr 2023 03:57:17.448 * Ready to accept connections
6、手动执行bgwriteaof
# 执行 redis-cli -a redis --no-auth-warning bgrewriteaof 并查看进程及文件目录状态
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning bgrewriteaof ; pstree | grep redis-server ; ll -h /app/redis/data/
Background append only file rewriting started
|-2*[redis-server---3*[{redis-server}]]
|-redis-server-+-redis-server
| `-3*[{redis-server}]
total 247M
drwxr-xr-x 2 redis redis 4.0K Apr 15 04:32 ./
drwxr-xr-x 7 redis redis 4.0K Apr 14 11:39 ../
-rw-r--r-- 1 redis redis 124M Apr 15 04:26 appendonly_6379.aof
-rw-r--r-- 1 redis redis 123M Apr 15 04:29 dump_6379.rdb
-rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb
-rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
-rw-r--r-- 1 redis redis 312K Apr 15 04:32 temp-rewriteaof-4924.aof #对比看到appendonly_6379.aof 经过rewriteaof后文件变小，但不会小于RDB文件
[root@Redis-Ubuntu-1804-p22:~]# ll /app/redis/data -h
total 245M
drwxr-xr-x 2 redis redis 4.0K Apr 15 04:32 ./
drwxr-xr-x 7 redis redis 4.0K Apr 14 11:39 ../
-rw-r--r-- 1 redis redis 123M Apr 15 04:32 appendonly_6379.aof
-rw-r--r-- 1 redis redis 123M Apr 15 04:29 dump_6379.rdb
-rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb
-rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
[root@Redis-Ubuntu-1804-p22:~]# #日志
4846:M 15 Apr 2023 04:32:05.605 * Background append only file rewriting started by pid 4924
4846:M 15 Apr 2023 04:32:23.890 * AOF rewrite child asks to stop sending diffs.
4924:C 15 Apr 2023 04:32:23.890 * Parent agreed to stop sending diffs. Finalizing AOF...
4924:C 15 Apr 2023 04:32:23.890 * Concatenating 0.00 MB of AOF diff received from parent.
4924:C 15 Apr 2023 04:32:23.891 * SYNC append only file rewrite performed
4924:C 15 Apr 2023 04:32:23.897 * AOF rewrite: 0 MB of memory used by copy-on-write
4846:M 15 Apr 2023 04:32:23.950 * Background AOF rewrite terminated with success
4846:M 15 Apr 2023 04:32:23.950 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
4846:M 15 Apr 2023 04:32:23.951 * Background AOF rewrite finished successfully
三、RDB 与 AOF 选择
如果进作为缓存使用，可接受部分数据丢失情况，可单纯启用RDB（默认开启）即可
如果对数据需要永久保存，无法接受数据丢失情况，建议AOF与RDB模式一同启用，通常不建议单独使用AOF作为数据备份方案
