Oct 15, 2019
Sharing: Redis get pipeline vs mget
Background
In redis, the simplest command to read a key is GET.
However, there is use case to read multiple keys at same time. Using GET with loops could be costly (especially network IO in cloud), and Redis at least provides two way to speed up: MGET and PIPELINE.
Then it is nature to ask how is it different for MGET and PIPELINE, in terms of use case and performance.
Benchmark
Similar to redis/redis-rb/tree/master/benchmarking, we create a benchmark for mget and pipeline (source code see appendix). In real world, our system need to fetch thousands of keys at a time, so I choose 10k as the total number of records.
user system total real
get 1.46 0.580226 2.050099 ( 12.701514)
pipeline with 10 each 0.74 0.300464 1.041347 ( 2.975966)
pipeline with 100 each 0.54 0.219881 0.766023 ( 1.368800)
pipeline with 1000 each 0.51 0.200081 0.716371 ( 1.403288)
pipeline with 10000 each 0.47 0.200489 0.672237 ( 1.201756)
mget with 10 each 0.22 0.061788 0.289068 ( 2.317538)
mget with 100 each 0.08 0.008353 0.092111 ( 0.240111)
mget with 1000 each 0.06 0.001894 0.063780 ( 0.085815)
mget with 10000 each 0.05 0.001459 0.057561 ( 0.070013)From the result real, we can find that for this use case, PIPELINE is 12x faster then GET, and MGET is 180x faster than GET.
(ps: the gap between total and real should related to the IO)
Pros and Cons
Pros: MGET is faster than PIPELINE, but PIPELINE is more flexible, it supports combining different types of commands.
Cons: MGET and PIPELINE use more memory, because Redis caches all the responses in memory before returning them to the client in bulk.
Appendix
