Redis lazyfree 參數有那些及用途?
lazyfree 有5個參數可以配置,其中4個是在 Redis 4 以上版本才有支援,而 lazyfree-lazy-user-del 是在 Redis 6 才支援,而這些參數主要用於非同步去刪除數據,以避免當刪除到特定大鍵值(big-key)時,造成命令延遲增加的狀況發生。
Published in
8 min readSep 19, 2022
!!! 請注意 !!!
若您的 Redis 集群,有 “大鍵值(big-key)” 的狀況發生時,請優先在業務端去拆分這些鍵值,而不是調整這些參數,來減緩影響的程度、這是本末倒置的。
Bigkey issue- 什麼是bigkey?
■ 在Redis中,一個字符串最大512MB,一個二級數據結構(例如hash、list、set、zset)可以存儲大約40億個(2³²-1)個元素,但實際上中如果下面幾種情況,我就會認為它是bigkey。Bigkey 會造成的問題?
■ 操作延遲。由於Redis單線程的特性,操作bigkey的通常比較耗時,也就意味著阻塞Redis可能性越大,這樣會造成客戶端阻塞或者引起故障切換,它們通常會出現在慢查詢中。
■ 絡擁塞。bigkey也就意味著每次獲取要產生的網絡流量較大,假設一個bigkey為1MB,客戶端每秒訪問量為1000次,那麼每秒產生1000MB的流量。
■ bigkey過期刪除時造成命令阻塞。 若bigkey設置了過期時間,當它過期後,會被刪除,如果沒有使用Redis 4.0的過期異步刪除(lazyfree-lazy-expire yes),就會存在阻塞Redis的可能性,而且這個過期刪除不會從主節點的慢查詢發現。
■ 刪除一個大Key造成主庫較長時間的阻塞,並引發同步中斷或主從切換。(改善方式及理由同上。更新版本至 4.0.10 以上。)怎麼產生的?
■ 一般來說,bigkey的產生都是由於程序設計不當。沒有對Key中的成員進行合理的拆分,造成個別Key中的成員數量過多(大Key)。如何解決?
■ 對大Key進行拆分,在業務面去將鍵值拆分。
■ 對 bigkey 進行拆分,拆成多個 key,然後用MGET取回來,再在業務層做合併。
開啓這些配置後,Redis 會在您刪除數據後,用非同步(子程序)的方式,來清除所佔用的內存空間,所以主要的影響應該會在您刪除大鍵值(bigkey)後,內存釋放的時間不會這麼”即時”,而這些參數在 ElastiCache Redis 參數組中,預設是關閉的,而原生 redis.config 這些參數也都是關閉的。
以下相關外部文檔、參數組資訊、redis.config 配置及說明。
[+] Redis-specific parameters:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ParameterGroups.Redis.html
lazyfree-lazy-eviction
■ Performs an asynchronous delete on evictions.
■ 當內存使用量達到 maximum 時,在執行 Eviction 的刪除鍵值方式。
lazyfree-lazy-expire
■ Performs an asynchronous delete on expired keys.
■ 執行淘汰過期鍵值 Reclaim 的刪除鍵值方式。
lazyfree-lazy-server-del
■ Performs an asynchronous delete for commands which update values.
■ 執行刪除鍵值方式。
slave-lazy-flush
■ Performs an asynchronous flushDB during slave sync.
■ 執行清除數據庫 FLUSHDB | FLUSHALL 的刪除鍵值方式。
lazyfree-lazy-user-del (added in 6.0)
■ When the value is set to yes, the DEL command acts the same as UNLINK.
■ 刪除鍵值方式,會將 DEL 命令,改成使用 UNLINK 的方式來執行。
redis.config:
# It’s up to the design of the application to understand when it is a good
# idea to use one or the other. However the Redis server sometimes has to
# delete keys or flush the whole database as a side effect of other operations.
# Specifically Redis deletes objects independently of a user call in the
# following scenarios:
#
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
# in order to make room for new data, without going over the specified
# memory limit.
# 2) Because of expire: when a key with an associated time to live (see the
# EXPIRE command) must be deleted from memory.
# 3) Because of a side effect of a command that stores data on a key that may
# already exist. For example the RENAME command may delete the old key
# content when it is replaced with another one. Similarly SUNIONSTORE
# or SORT with STORE option may delete existing keys. The SET command
# itself removes any old content of the specified key in order to replace
# it with the specified string.
# 4) During replication, when a replica performs a full resynchronization with
# its master, the content of the whole database is removed in order to
# load the RDB file just transferred.
#
# In all the above cases the default is to delete objects in a blocking way,
# like if DEL was called. However you can configure each case specifically
# in order to instead release memory in a non-blocking way like if UNLINK
# was called, using the following configuration directives.lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no# It is also possible, for the case when to replace the user code DEL calls
# with UNLINK calls is not easy, to modify the default behavior of the DEL
# command to act exactly like UNLINK, using the following configuration
# directive:lazyfree-lazy-user-del no