May 20, 2022
Working With List on Redis With .Net 6.0
Today we will talk about how to work with cumulative data in Redis. First of all, Redis is not a cache manager Tool. It is an In-Memory DB. Redis means “Remote Dictionary Service”. There are five different data types in Redis. Today we will talk about “Lists”. But in the end we will have to change our Redis Data Type for a new busines rules.
Scenario: We have several different categories. A new one can be added and any of them can be removed. The category model seems as below. We will save all categories on Redis to improve our service’s performance.
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
public int ParentID { get; set; }
public DateTime CreateDate { get; set; }
}
We will use ServiceStack Library for Redis in .Net 6.0
First Method:
When I checked the junior developer’s code, I saw, that he set Category List to the Redis as JSON data with Key-Value pair. When he adds a new item to the Category List, he gets JSON data, deserializes it adds a new category item to the list then serialize again and set List to the Redis as JSON. This is unacceptable. For big data, this is a performance killer.
*This method is similar to taking off all books from the library, adding the new book, and putting all taken books back into the library again :)
- We created some dummy categories and added to “categoryList”
2. We set Redis connection server and got Redis Client.
3. We set “CategoryList” to the Redis as JSON as seen above. And got “CategoryList” again.
4. We removed the Category Id=2, added new Category Id=4 and finally set “CategoryList” to the Redis as JSON again.
5. Finally we got all “CategoryList” from Redis and printed them all in a loop as seen below.
Program.cs:
List<Category> categoryList = new();categoryList.AddRange(new List<Category>() {
new Category() { Id=1,Name="Category1",CreateDate=DateTime.Now},
new Category() { Id=2,Name="Category2",CreateDate=DateTime.Now},
new Category() { Id=3,Name="Category3",CreateDate=DateTime.Now}});RedisEndpoint conf = new RedisEndpoint { Host = "127.0.0.1", Port = 6379, Password = "test" };RedisClient redisClient = new RedisClient(conf);redisClient.Set("CategoryList", categoryList);var list = redisClient.Get<List<Category>>("CategoryList");list.Remove(list.First(item => item.Id == 2));list.Add(new Category() { Name = "CategoryBora", Id = 4, CreateDate = DateTime.Now });redisClient.Set("CategoryList", list);var finalList = redisClient.Get<List<Category>>("CategoryList");foreach (var item in finalList)
{
Console.WriteLine(JsonSerializer.Serialize(item));
}
Second Method:
Now it is time to give a chance to “Lists Data Type” in Redis. We don’t have to deal with the all data in Redis when we add, update or remove an item from the list. Dealing with only one item, could be improve the performance.
- We created “CategoryList” on Redis. The type of List is “Category”. It is like “List<Category>”.
2. We Bulk Inserted all Categories to “CategoryList” on Redis. And finally added a new “Category4” without touching to Current List :)
3. Removing an Item from the Redis List is very painful :) You have to find the Index of the item which will be removed. I got (value, index) with the “Single” filter keyword which “Id==3"
I took removedCategory, change the name to “CategoryBora” and put it back to “CategoryList” with the same Index number.
Not:(removedCategory.Index-1) Because one item is removed so “current Index number” => 1 dropped.
4. We got “CategoryList” from the Redis. And we printed all items in a loop as seen below.
Program.cs:
List<Category> categoryList = new();categoryList.AddRange(new List<Category>() {
new Category() { Id=1,Name="Category1",CreateDate=DateTime.Now},
new Category() { Id=2,Name="Category2",CreateDate=DateTime.Now},
new Category() { Id=3,Name="Category3",CreateDate=DateTime.Now}});RedisEndpoint conf = new RedisEndpoint { Host = "127.0.0.1", Port = 6379, Password = "test" };RedisClient redisClient = new RedisClient(conf);//Create CategoryList on Redis and add all items to the list..
IRedisTypedClient<Category> categoryClient = redisClient.As<Category>();var categoryRedisList = categoryClient.Lists["CategoryList"];categoryRedisList.AddRange(categoryList);//Add new Category4 to the CategoryList
categoryRedisList.Add(new Category() { Id = 4, Name = "Category4", CreateDate = DateTime.Now });//Remove "Category Id=3" from the List with it's Index.
var removeCategory = categoryRedisList.Select((Value, Index) => new { Value, Index }).Single(p => p.Value.Id == 3);categoryRedisList.RemoveAt(removeCategory.Index);//Change the Name of Removed Category and put it back
removeCategory.Value.Name = "CategoryBora";
categoryRedisList.Insert(removeCategory.Index - 1, removeCategory.Value);//Get the CategoryList from the redis and print all items in a loop
var getCategoryList = categoryRedisList.GetAll();
getCategoryList.ForEach(item =>{
Console.WriteLine(JsonSerializer.Serialize(item));
});
Third Method:
Later, we got a new case from the customer. “Every category must have an expiration time.” So the current Redis Lists Data type solution has become useless at the end. Because in the List, we can not set an expiration time for each of the category. Only we can set an expiration time for the group of category in Redis Lists. So Bulk expiration time was not acceptable for the customer. At the end, last time we changed our Data Type in Redis.
1-) We decided to grow vertically in Redis. We added all categories one by one under the “Category:” keyword with their Ids. “Category” is the name of the group and “id” is the unique identity of every category.
“Category:{item.Id}
2-) We add a new “Category4” under the “Category:” keyword with its “4” Id.
3-) We easily found “Category3” by its keyword. Against the Lists data type, we don’t need any Index of the item for removing it. We changed the name of “Category3” with “Bora” and put it back by using it’s “Category:3” keyword.
4-) We select all Categories with the “Category*” keyword. We call it Lua Scripting. We filtered all Redis Items, which is starting with the “Category” keyword. We checked if there were any items with the start “Category” keyword. If the result is yes, we got all of them and ordered by their “Id”. And finally, we printed every one of them into the screen.
Program.cs:
List<Category> categoryList = new();categoryList.AddRange(new List<Category>() {
new Category() { Id=1,Name="Category1",CreateDate=DateTime.Now},
new Category() { Id=2,Name="Category2",CreateDate=DateTime.Now},
new Category() { Id=3,Name="Category3",CreateDate=DateTime.Now}});RedisEndpoint conf = new RedisEndpoint { Host = "127.0.0.1", Port = 6379, Password = "test" };RedisClient redisClient = new RedisClient(conf);//We added all categories under the "Category:" with theirs Ids.
categoryList.ForEach(item =>
{
redisClient.Add($"Category:{item.Id}", item);
});//We add new "Category4" to under the "Category:" keyword.
redisClient.Add("Category:4", new Category() { Id = 4, Name = "Category4", CreateDate = DateTime.Now });//We Found “Category3” And Update It.
var category3 = redisClient.Get<Category>("Category:3");
category3.Name = "Bora";
redisClient.Set("Category:3", category3);//All Categories found with "Category*" keyword and printed one by one to the screen.
var keys = redisClient.SearchKeys($"Category*");
if (keys.Any())
{
List<Category> getAllCategoryList = redisClient.GetAll<Category>(keys).Values.OrderBy(item => item.Id).ToList(); getAllCategoryList.ForEach(item =>
{
Console.WriteLine(JsonSerializer.Serialize(item));
});
}
Conclusion:
Redis is not the only key/value memory management tool. It is an In-Memory DB. You can keep your data in several ways in Redis. Growing vertically is not a performance problem for Redis. But keeping Big data as Json in Redis could be problem when high traffic. Serialize and Deserialize are high cost operations. What I am trying to explain with this article is, “If you spliting your data and puting them under a group with their unique identity, you could rid of your performance problem”
See you later until the next article. Bye.
“If you have read so far, first of all, thank you for your patience and support. I welcome all of you to my blog for more!”
Source:
