Jun 6, 2022
Bitwise Implementation on .Net 6
Hi, today we will talk about the advantages and disadvantages of Bitwise Operators and Operations in a real-life scenario.
Scenario: We will send a campaign message to Dealers. And there are lots of users under these dealers. We will try to record who read the message and who did not.
Classic Way
These are MsSqlDB tables: “Dealer”, “Message” relation table “UserMessage” and of course “User” table.
In a classic way, we kept who read the message data in the “UserMessage” table, which is surrounded by the green rectangle as seen below.
1-) Let’s create .Net 6.0 Web Api Project “BitwiseDealers”. And Add DB ClassLibrary Project to the solution.
Add below Libraries by using Nuget to the DB Project.
Create Entity and DBContext
DB already exists so we will use DB First for this solution. Open Terminal and move into the DB Folder with “cd DB” command. We call below “scaffold” command. So we will create Entity and BitwiseContext automatically.
dotnet ef dbcontext scaffold "Server=.;Database=BitwiseDB;Trusted_Connection=True;"
Microsoft.EntityFrameworkCore.SqlServer -o Entities --context-dir "Entities\DbContexts"
--no-pluralize -c BitwiseContext -f
BitwiseDealers/appsettings.json: We added DB connection settings to appsettings.json for the WebApi service.
*Don’t forget to encrypt all config text for the release version.
"ConnectionStrings": {
"DefaultConnection": "Data Source=.;initial catalog=BitwiseDB;Trusted_Connection=True;"
}2-) Create Service Layer: Create “BitwiseService” ClassLibrary Project.
BitwiseService/IUserService: We will get the User list of, who read the specific message.
using Core.Model;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;namespace BitwiseService
{
public interface IUserService
{
List<UserMessageViewModel> GetUserListByMessageID(int messageID);
}
}
BitwiseService/UserService: We will return custom “UserMessageViewModel” from this “GetUserListByMessageID()” method. We will get User FullName, DealerName and ReadStatus. And return it to the Controller.
using Core.Model;
using DB.Entities;
using DB.Entities.DbContexts;
using System.Linq;namespace BitwiseService
{
public class UserService : IUserService
{
BitwiseContext _context;
public UserService(BitwiseContext context)
{
_context = context;
}
public List<UserMessageViewModel> GetUserListByMessageID(int messageID)
{
var data = (from um in _context.UserMessage
join u in _context.User on um.UserId equals u.Id into us
from u in us.DefaultIfEmpty()
join d in _context.Dealar on
um.DealerId equals d.Id into de
from d in de.DefaultIfEmpty()
join m in _context.Message on
um.MessageId equals m.Id into me
from m in me.DefaultIfEmpty()
where um.MessageId == messageID
select new UserMessageViewModel
{
UserName = u.Name,
UserSurname = u.Surname,
DealerName = d.Name,Message = m.Text,
IsRead = (bool)um.IsRead
}).ToList();
return data.Count > 0 ? data : new List<UserMessageViewModel>();
}
}
}
Program.cs: Don’t forget to add userService to program.cs as below.
builder.Services.AddTransient<IUserService, UserService>();3-) Create Core Class Library Project and add Model Folder.
Core/Model/UserMessageViewModel: This is custom UserMessageViewModel
namespace Core.Model
{
public class UserMessageViewModel
{
public string UserName { get; set; }
public string UserSurname { get; set; }
public string Message { get; set; }
public bool IsRead { get; set; }
public string DealerName { get; set; }
}
}4-) BitwiseDealers/Controllers/DealerController: We will get UserList from BitwiseDealers Controller.
using BitwiseService;
using Core.Model;
using Microsoft.AspNetCore.Mvc;namespace BitwiseDealers.Controllers
{
[ApiController]
[Route("[controller]")]
public class DealerController : ControllerBase
{
IUserService _userService;
public DealerController(IUserService userService) { _userService = userService; } [HttpGet("GetUserListByMessageID/{messageID}")]
public IEnumerable<UserMessageViewModel> GetUserListByMessageID(int messageID)
{
return _userService.GetUserListByMessageID(messageID);
}
}
}
In this way, we have to keep all user’s records one by one for every Dealers. If you have 10K users, for every message you have to repeat this. So at the end of the month, you have to keep millions of rows for every message. This is unacceptable to me :)
How to keep the same data with much fewer rows for optimization and performance ?
Bitwise Way
1-) Firstly, we will add “BitwiseID” column to User table. And we will create a new “UserMessageBitwase” table as seen below. After all, we will use the “UserMessageBitwise” table instead of the “UserMessage”
*Don’t forget to call the “scaffold” command again. Because of Update the Backend DBContext and Entities. We will update User Entitiy and add UserMessageBitwise table.
The user’s BitwiseID will be increased +1 with the “2^n” formula.
Example: “1, 2, 4, 8, 16”. We will set “BitwiseID” per user group by DealerID. If DealerID changes, we will start BitwiseID from 1 again as seen below.
We will add new data to “UserMessageBitwise” table.
The trick is we will sum the User’s “BitWiseID”, who read the message in the same group with DealarID.
For example : If you check the “UserMessage Table” for MessageID =1 and in a DealerID =1, group users are “1,2 and 4"
We will sum the User’s “BitWiseID” whose DealerID=1 and User Id = 1,2 and 4 as seen below. It is 1 + 2 + 8= “11” and will put this total number to the “TotalBitwiseId” column of “UserMessageBitwise” table as seen above. So we will write 1 row instead of 3 rows.
2-)Let’s change the Backend “GetUserListByMessageID()” by Using BitwiseID
Redis Implementation
Firstly we need Redis for getting all User “BitwiseId” from memory for every Dealer answer data to improve performance. Our main goal is to increase performance by transferring the load on SQL to Redise.
Create “Caching” folder under “Core” folder. And add “RedisCacheService” as seen below.
Core/Caching/IRedisCacheService:
using System;
using System.Collections.Generic;
using System.Text;namespace Core.Caching
{
public interface IRedisCacheService
{
T Get<T>(string key, long db = 0);
IList<T> GetAll<T>(string key, long db = 0);
void Set(string key, object data, long db = 0);
void Set(string key, object data, DateTime time, long db = 0);
void SetAll<T>(IDictionary<string, T> values, long db = 0);
void Remove(string key, long db = 0);
}}
BitwiseDealers/appsettings.json: Add Redis config to appsetting.json. Don’t forget to encrypt all config text for the release version.
.
.
"BitwiseConfig": {
"RedisEndPoint": "127.0.0.1",
"RedisPort": "6379",
"RedisPassword": "t@stPassword"
},Core/Configuration/BitwiseConfig: We will use BitwiseConfig class, for getting configuration from appsettings.json.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;namespace Core.Configuration
{
public class BitwiseConfig
{
#region Props
public string RedisEndPoint { get; set; }
public string RedisPort { get; set; }
public string RedisPassword { get; set; }
#endregion
}
}
BitwiseDealer/Program.cs: Don’t forget to add “BitwiseConfig” class to program.cs as seen below. So with this, we will match the “appsettings.json/BitwiseConfig” section to the BitwiseConfig class.
.
.
builder.Services.Configure<BitwiseConfig>(builder.Configuration.GetSection("BitwiseConfig"));Core/Caching/RedisCacheService: This is a basic simple RedisCache service. You can Add, Get and Remove base operations by using this class.
using Core.Configuration;
using Microsoft.Extensions.Options;
using Newtonsoft.Json;
using ServiceStack.Redis;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;namespace Core.Caching
{
public class RedisCacheService : IRedisCacheService
{
#region Fields
public readonly IOptions<BitwiseConfig> _bitwiseConfig;
private readonly RedisEndpoint conf = null;
#endregion //config set requirepass fl@rp1$19C23public RedisCacheService(IOptions<BitwiseConfig> bitwiseConfig)
{
_bitwiseConfig = bitwiseConfig;
conf = new RedisEndpoint {
Host = _bitwiseConfig.Value.RedisEndPoint,
Port = Convert.ToInt32(_bitwiseConfig.Value.RedisPort),
Password = _bitwiseConfig.Value.RedisPassword };
}
public T Get<T>(string key, long db = 0)
{
try
{
conf.Db = db;
using (IRedisClient client = new RedisClient(conf))
{
return client.Get<T>(key);
}
}
catch
{
throw new Exception("Redis Not Available");
}
} public IList<T> GetAll<T>(string key, long db = 0)
{
try
{
conf.Db = db;
using (IRedisClient client = new RedisClient(conf))
{
var keys = client.SearchKeys(key);
if (keys.Any())
{
IEnumerable<T> dataList = client.GetAll<T>(keys).Values;
return dataList.ToList();
}
return new List<T>();
}
}
catch
{
throw new Exception("Redis Not Available");
}
} public void Set(string key, object data, long db = 0)
{
Set(key, data, DateTime.Now.AddMinutes(60), db);
}
public void Set(string key, object data, DateTime time, long db = 0)
{
try
{
conf.Db = db;
using (IRedisClient client = new RedisClient(conf))
{
var dataSerialize = JsonConvert.SerializeObject(data, Formatting.Indented, new JsonSerializerSettings{
PreserveReferencesHandling = PreserveReferencesHandling.Objects
});
client.Set(key, Encoding.UTF8.GetBytes(dataSerialize), time);
}
}
catch
{
throw new Exception("Redis Not Available");
}
} public void SetAll<T>(IDictionary<string, T> values, long db = 0)
{
try
{
conf.Db = db;
using (IRedisClient client = new RedisClient(conf))
{
client.SetAll(values);
}
}
catch
{
throw new Exception("Redis Not Available");
}
} public void Remove(string key, long db = 0)
{
try
{
conf.Db = db;
using (IRedisClient client = new RedisClient(conf))
{
client.Remove(key);
}
}
catch
{
throw new Exception("Redis Not Available");
}
}
}
}
BitwiseDealer/Program.cs: Don’t forget to add “RedisServices” class to program.cs as seen below.
builder.Services.AddTransient<IRedisCacheService, RedisCacheService>();3-)Next, We will create “UserRedisModel” as a model which is kept in Redis. We will keep user Detail data in the Redis with this model.
Core/Model/UserRedisModel:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;namespace Core.Model
{
public class UserRedisModel
{
public int Id { get; set; }
public string? Name { get; set; }
public string? Surname { get; set; }
public long? BitwiseId { get; set; }
public int? DealarId { get; set; }
public string DealarName { get; set; }
public DateTime? CreatedDate { get; set; }
}
}
We made all Setup. Next, we will change GetUserListMessageID() method with the Bitwise algorithm as seen below
- In the beginning, we will get all who read the message data and set the “answerData” parameter.
- We will get the message text by “messageID” and set the message variable.
- “resultList” is our return List of the model. We will create an empty model ad the beginning. And we will start to loop in “answerData”.
- We will try to get user data with a specific dealerID from the Redis. If it is null will get all data from SqlDB.
- If Redis is null, we will get user data from DB with LINQ as seen below. We will join the User and Dealer table and return UserRedisModel. And finally, save this dealer’s user data to the Redis.
- *This is the most important part. After we got user data from the Redis or SqlDB, we will loop in UserList. We will check if “User.BitwiseID” is in the total answered “TotalUserBitwiseId”. It means, is this user read this message or not.
user.BitwiseID[4] = (TotalBitwiseID[8] & user.BitwiseID[4]) => result is “Yes”
- If it is true, we will add this user in to the “resultUserList”. And finally return it.
BitwiseService/UserService (Version 2):
We got all the users, who read the MessageID=1 by using “Bitwise” as seen below.
Conclusion:
You can use Bitwise Algorithms for lots of parent-child scenarios. For example Tree Menu, State-City or Product-SubProduct. You can use bitwise for Max 63 subitems. Because the processor can not recognize bigger than “2^63" numbers. But I think 63 is enough for most cases. You can write online 1 line instead of 63 lines. The trick is to use subcategories with a group. As in this article, we grouped users under the Dealer. So for different Dealer, we started User BitwiseID from 1 again. And we earned one more 63 users for the new Dealer.
Don’t worry about looping all users to check whether their’s BitwiseId is in the TotalBitwiseID or not. Because checking bitwise is a very performance operation.
See you until the next article.
“If you have read so far, first of all, thank you for your patience and support. I welcome all of you to my blog for more!”
Source Code
- Github: https://github.com/borakasmer/Bitwise
- BitwiseDB Script: https://borakasmer.com/projects/BitwiseScript.sql
