Redis memory fragmentation
內存破碎(memory fragmentation)形成的原因? 如何降低內存破碎率？ 內存破碎是否會低於1？ 高內存破碎是問題嗎？
Q: What’s is Memory Fragmentation?
Fragmentation Ratio = Used Memory RSS / Used Memory
The actual mapping between Redis’ memory and physical memory is handled by the operating system’s virtual memory manager through the mediation of a memory allocator. For example, while Redis would benefit from contiguous physical memory pages for a 1GB database, no such contiguous segment may be available so the operating system will have to map that 1GB into multiple smaller memory segments.
Q: What’s fragmentation? 什麼是內存碎片?
內存碎片是由於計劃申請的空間比空閒的連續空間小，導致這部分小內存空間無法被使用，無法被使用的內存空間則可稱為內存碎片。
Q: 什麼是 used_memory? 什麼是 used_memory_rss?
The used_memory and used_memory_rss metrics both include memory allocated for:
★★★ used_memory ★★★
used_memory = used_memory_overhead + used_memory_dataset
User-defined data: memory used to store key-value data. 由Redis 分配器分配的内存总量，包含了redis进程内部的开销和数据占用的内存。
Internal overhead: inernal Redis informatin used to represent different data types
★★★ used_memory_rss ★★★
Resident Set Size (RSS) and is the amount of physical memory the operating system has allocated to your Redis instance, 操作系統分配給Redis實例的內存大小，表示該進程所佔物理內存的大小。
used_memory_rss = user-defined data + internal overhead + memory consumed by memory fragmentation
Internal overhead: internal Redis information used to represent different data types
Q: Used_memory_rss can be lower than used_memory? (It’s not possible)
The used_memory and used_memory_rss metrics both include memory allocated for:
• User-defined data: To store key-value data
• Internal overhead: Internal Redis information used to represent different data type Additionally, the used_memory_rss metric includes
• memory consumed by memory fragmentation.
Q: Redis的內存碎片是如何形成?
Redis產生內存碎片主要由以下2點原因導致；
• 內存分配器機制: jemalloc 內存分配方式為 按照一系列固定大小分配內存空間，jemalloc 按照申請的內存大小分配最接近的內存空間。
• Redis數據的修改和刪除引發空間的擴容和釋放。
• Memory fragmentation is caused by inefficiencies in physical memory allocation/deallocation by the operating system. 內存碎片是由操作系統的物理內存分配/解除分配效率低下造成的。
• It is possible that the engine couldn’t find enough continuous pages to allocate the items, consequently causing fragmentation. 引擎可能找不到足夠的連續頁面來分配項目，從而導致碎片化。
Q: 內存為何不釋放? allocator_allocated vs used_memor
Redis有自己的內存分配器，當數據刪除後，釋放的內存空間由Redis自己的內存分配器管理，並沒有立即將內存返回給操作系統，所以對於操作系統而言，仍然認為Redis佔用了內存。這樣的好處是，減少Redis向系統申請內存分配的次數，提升Redis自身性能。
• Redis will not always free up (return) memory to the OS when keys are removed.
• Allocators are smart and can reuse free chunks of memory.
Memory Fragmentation Ratio 高低的差別!!
• 1.5> ratio >1: low memory fragmentation and no memory swapping (OS free up memory lazily, so ratio > 1)
• >1.5: significant memory fragmentation
• <1: operating system may be swapping, but it is possible there is no swapping.
Q: Way to correct the problem? 如何降低內存碎片率?
• Upgrade Redis Engine (upgrade allocator) 較新版本的 Redis Engine 會持續改善內存管理方式(New memory allocator)，可以有效率地去降低內存碎片率。
• Utilize active defragmentation parameters (Redis 4+). Redis can now defragment memory while running which allows more efficient utilization of memory for customer data. This is off by default but you can modify the parameter group to turn it on.
activedefrag: Enabled active defragmentation. (Default: no) (Redis 4.0.10+)
active-defrag-ignore-bytes: Minimum amount of fragmentation waste to start active defrag. (Default: 104857600 100MB) (Redis 4.0.10+)
active-defrag-ignore-bytes 100mb 當碎片達到 100MB時，開啟內存碎片整理。(Default: 104857600)
首先，我們內存碎片整理是在主線程中執行的，通過源碼發現，內存碎片整理操作會 scan （通過迭代進行）整個 redis 節點，並進行內存複製、轉移等操作，因為 redis 是單線程的，所以這肯定會導致 redis 性能下降。
•Restart the Redis: 安全重啟 在集群環境下，可以主從節點切換，通過重啟服務來重載內存以提高內存使用率。(這個動作只能在從節點上執行，若在主節點上執行，會造成數據丟失。 重啟從節點上的 Redis 服務，可以重新跟主節點要求數據同步。)
• 應用程序邏輯上的調整: 數據對齊，盡量使用 ”固定長度” 的數據類型，保持內存塊對齊，也就是說盡可能物盡其用，用多少佔用多少。
ElastiCache Redis 參數 reserved-memory vs reserved-memory-percent 比較?
若客戶的帳號是建立於 2017–05–16 前建立的預設參數組是使用 reserved-memory。且預設參數組中的參數，是無法改成 reserved-memory-percent。
解法，升級版本用不同的預設參數組。### Redis 3.2.4 ###
[] Redis-specific parameters — New parameters for Redis 3.2.4 :
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ParameterGroups.Redis.html#ParameterGroups.Redis.3-2-4.New
—
reserved-memory-percent
Default: 25
—### Redis 2.6.13 ~ 2.8.24 ####
[] Redis-specific parameters — Redis 2.6.13 parameters :
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ParameterGroups.Redis.html#ParameterGroups.Redis.2-6-13
—
reserved-memory
Default: 0
—https://github.com/awsdocs/amazon-elasticache-docs/blob/master/doc_source/redis/redis-memory-management.md
—
As of March 16, 2017, Amazon ElastiCache for Redis provides two mutually exclusive parameters for managing your Redis memory, reserved-memory and reserved-memory-percent. Neither of these parameters is part of the Redis distribution.
—
