Outbox pattern in .Net
In this article, the outbox pattern which is a design pattern will be discussed, and figured out how it is possible to use background services to perform some logic that will be used in an outbox pattern. This article will demonstrate the implementation of the outbox pattern in .net 7.
Prerequires
- Basic C# knowledge
- Entity framework core
- Dependency injection
- Visual Studio
- SQL Server Management studio
Before working on the code, let’s find out what is the outbox pattern, why it is important, and where it could be used.
What is the outbox pattern?
Outbox pattern is a reliable method that guaranteed the data is never lost in the Monolithic or Microservices architecture. So, what it is? Let’s assume that in a microservices architecture, some data want to share between other microservices using Kafka, RabbitMQ, or any other message brokers. But suddenly, the message broker interrupted and could not send the data to other microservices. However, the data has been stored in the database table. What could we do to send the data automatically to the microservices when message brokers connected again? Here outbox pattern solved this problem.
As is shown in the above diagram, if the message broker is interrupted, the message should be stored in a table that is called the outbox table in this example. The significant section in this diagram is the Background Service and here all messages could be sent based on the application logic. The above diagram is an example of microservices. Furthermore, MassTransit is a service bus software that easily manages the outbox pattern and other microservices patterns like SAGA. But in this article, the outbox pattern is not going to use in microservices architecture and it will be used in a monolithic architecture. However, it is completely compatible with microservices architecture.
Where the outbox pattern is operable in monolithic architecture?
In monolithic applications, there are a lot of services that are used to send information to the client like email services or SMS services, etc. for instance, when a user is shipping an order, an email will be sent to the user that contains the order information. Now let’s assume that the email service is interrupted and after the order is shipped, the email will not be sent to the user. Here, the outbox pattern stores the information in the outbox table and the background service, based on the logic, the email will be sent to the user when the email service is connected again.
In this article, the email outbox pattern will be covered and, in the future, the outbox pattern in the microservices architecture will be covered.
After the above explanation, Let’s get started coding.
Step 1:
Create a .net core API. In this article, .net 7 has been used and all of the necessary packages like EF core would be on version 7.
Step 2:
Add the following packages using the NuGet package manager.
- Install-Package Microsoft.EntityFrameworkCore -Version 7.0.5
- Install-Package Microsoft.EntityFrameworkCore.Design -Version 7.0.5
- Install-Package Microsoft.EntityFrameworkCore.SqlServer -Version 7.0.5
- Install-Package Microsoft.EntityFrameworkCore.Tools -Version 7.0.5
Step 3:
Create a new folder and call it Models and add the following classes as the requirement models.
Order.cs
public class Order
{
public string Id { get; set; } = Guid.NewGuid().ToString();
public string ProductName { get; set; }
public int Quantity { get; set; }
public decimal Price { get; set; } = decimal.Zero;
public string Email { get; set; }
}EmailOutbox.cs
public class EmailOutbox
{
public string Id { get; set; } = Guid.NewGuid().ToString();
public string OrderId { get; set; }
public Order Order { get; set; }
public DateTime CreatedDate { get; set; } = DateTime.Now;
public bool Success { get; set; }
}Step 4:
In this step, we need to add our DbContext class. So, in the Models folder, add the below class as a DbContext class.
public class AppDbContext : DbContext
{
public AppDbContext(DbContextOptions<AppDbContext> options) : base(options)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Order>().Property(o => o.Price).HasColumnType("decimal(18,4)");
base.OnModelCreating(modelBuilder);
}
public DbSet<Order> Orders { get; set; }
public DbSet<EmailOutbox> EmailOutbox { get; set; }
}Now, add a connection string in the appsettings.json file.
"ConnectionStrings": {
"DbConnection": "server=.;database=OutboxPattern;Trusted_Connection=true;MultipleActiveResultSets=True"
}After adding a connection string, our DbContext needs to register in the program.cs file. So add the following codes to the Program.cs file.
// Register AppDbContext
var connectionString = builder.Configuration.GetConnectionString("DbConnection");
builder.Services.AddDbContextPool<AppDbContext>(options => options.UseSqlServer(connectionString));Step 5:
In this step, migration will be required to create the database and tables based on the model.
Add-Migration InitialMigration
Update-Database
Step 6:
In this step, services will be required, and based on the scenario, order services need to be added. So, add a folder in the root of the project and call it OrderService and add the below class and interface.
IOrderService.cs
public interface IOrderService
{
Task<Order> AddOrder(Order order);
}OrderService.cs
public class OrderService : IOrderService
{
private readonly AppDbContext _appDbContext;
public OrderService(AppDbContext appDbContext)
{
_appDbContext = appDbContext;
}
public async Task<Order> AddOrder(Order order)
{
if(order is not null)
{
await _appDbContext.Orders.AddAsync(order);
await _appDbContext.SaveChangesAsync();
}
return order;
}
}Now, register the above services in Program.cs file.
builder.Services.AddScoped<IOrderService, OrderService>();Now, this service could be used in our controller to add a new order.
Step 7:
Add a new controller and call it OrderController.cs and add the following codes inside of it.
Note: To keep the example simple, the base model will be used directly inside the controller and it is not going to use any DTO object.
OrderController.cs
[Route("api/[controller]")]
[ApiController]
public class OrderController : ControllerBase
{
private readonly IOrderService _orderService;
public OrderController(IOrderService orderService)
{
_orderService = orderService;
}
[HttpPost]
public async Task<IActionResult> Post(Order order)
{
var result = await _orderService.AddOrder(order);
if (result is not null)
{
return Ok(outbox);
}
return BadRequest();
}
}Now an order could be created. Postman or Swagger could be useful to send a request to store some data.
Step 8:
In this step, based on the scenario, an email should be sent to the user after creating a new order. So, add a new folder in the root of the project and call it EmailService and add the following classes and interface.
IMailService.cs
public interface IMailService
{
bool Send(string sender, string subject, string body, bool isBodyHTML);
}MailService.cs
public class EmailService : IMailService
{
private readonly EmailSettings _emailSettings;
// Constructor
public EmailService(IOptions<EmailSettings> emailOptions)
{
_emailSettings = emailOptions.Value;
}
// Send email
public bool Send(string sender, string subject, string body, bool isBodyHTML)
{
try
{
// Create an object based on the MailMessage class
MailMessage mailMessage = new MailMessage();
// Assign properties that are required for sending an email
mailMessage.From = new MailAddress(_emailSettings.Email);
mailMessage.Subject = subject;
mailMessage.Body = body;
mailMessage.IsBodyHtml = isBodyHTML;
// Send email to ...
mailMessage.To.Add(new MailAddress(sender));
// Generate a SmtpClient object
SmtpClient smtp = new SmtpClient();
// smtp.Host = smtp.gmail.com;
smtp.Host = "mail.vahidalizadeh7070.ir";
// If using gmail, set it true
smtp.EnableSsl = false;
// Set username and password of our email
NetworkCredential networkCredential = new NetworkCredential();
networkCredential.UserName = mailMessage.From.Address;
networkCredential.Password = _emailSettings.Password;
// If using gmail, set it true
smtp.UseDefaultCredentials = false;
smtp.Credentials = networkCredential;
// If using google set it 587
smtp.Port = 587;
// Send
smtp.Send(mailMessage);
return true;
}
catch
{
return false;
}
}
}EmailSettings.cs
// This class is going to set all information inside the appsettings.json file to the Program.cs file
public class EmailSettings
{
public const string SectionName = "EmailSettings";
public string Email { get; set; }
public string Password { get; set; }
}Now, this should be registered in the Program.cs file.
var emailSettings = new EmailSettings();
builder.Configuration.Bind(EmailSettings.SectionName, emailSettings);
builder.Services.AddSingleton(Options.Create(emailSettings));
builder.Services.AddSingleton<IMailService, EmailService>();Finally, in this step, the configuration of the email should be added to the appsettings.json file. I have my server and use my configuration, however, there are a lot of services like Mailgun or Sendgrid that could be very helpful to send an email.
"EmailSettings": {
"email": "info@vahidalizadeh7070.ir",
"password": "******"
},In the next step, this service should be injected into the order controller to send an email after creating a new order.
Step 9:
Inject EmailService into the order controller to send an email. So, add the following codes to the order controller.
[Route("api/[controller]")]
[ApiController]
public class OrderController : ControllerBase
{
private readonly IOrderService _orderService;
private readonly IMailService _mailService;
public OrderController(IOrderService orderService, IMailService mailService
{
_orderService = orderService;
_mailService = mailService;
}
[HttpPost]
public async Task<IActionResult> Post(Order order)
{
var result = await _orderService.AddOrder(order);
if (result is not null)
{
// Send email if order store in the database
var send = _mailService.Send(result.Email, "Order is completed", "Your order has been saved in the database", false);
if (send is true)
{
return Ok(result);
}
else
{
// store in the email outbox
}
}
return BadRequest();
}
}Now, send a request again using Postman and retrieve a new email.
As shown in the above image, an email has been sent to the email address after creating a new order. But, if the email service is interrupted, the order will be saved but after restarting the email service, the user could not receive the email. So, here a service will be needed to store the outbox record and send an email after restarting the email service.
Step 10:
Add a new folder in the root of the project and call it EmailOutboxService and add another folder inside of it and call it Service. Then inside this Service folder, add an interface and a class.
IEmailOutbox.cs
public interface IEmailOutbox
{
Task<EmailOutbox> Add(EmailOutbox emailOutbox);
Task<EmailOutbox> Update(EmailOutbox emailOutbox);
IEnumerable<EmailOutbox> GetAll();
}EmailOutbox.cs
public class EmailOutboxes : IEmailOutbox
{
private readonly AppDbContext _appDbContext;
public EmailOutboxes(AppDbContext appDbContext)
{
_appDbContext = appDbContext;
}
public async Task<EmailOutbox> Add(EmailOutbox emailOutbox)
{
if(emailOutbox is not null)
{
await _appDbContext.EmailOutbox.AddAsync(emailOutbox);
await _appDbContext.SaveChangesAsync();
}
return emailOutbox;
}
public IEnumerable<EmailOutbox> GetAll()
{
return _appDbContext.EmailOutbox.Include(o=>o.Order).OrderByDescending(o=>o.CreatedDate).Where(o=>o.Success==false).ToList();
}
public async Task<EmailOutbox> Update(EmailOutbox emailOutbox)
{
var model = await _appDbContext.EmailOutbox.FirstOrDefaultAsync(o=>o.Id ==emailOutbox.Id);
if(model is not null)
{
model.Success = true;
await _appDbContext.SaveChangesAsync();
}
return emailOutbox;
}
}Now, this service should be registered in the Program.cs file.
builder.Services.AddScoped<IEmailOutbox, EmailOutboxes>();Now, inject the email outbox service into the order controller and store the data if the email service was enabled.
OrderController.cs
public class OrderController : ControllerBase
{
private readonly IOrderService _orderService;
private readonly IMailService _mailService;
private readonly IEmailOutbox _emailOutbox;
public OrderController(IOrderService orderService, IMailService mailService, IEmailOutbox emailOutbox)
{
_orderService = orderService;
_mailService = mailService;
_emailOutbox = emailOutbox;
}
[HttpPost]
public async Task<IActionResult> Post(Order order)
{
var result = await _orderService.AddOrder(order);
if (result is not null)
{
// Send email if order store in the database
var send = _mailService.Send(result.Email, "Order is completed", "Your order has been saved in the database", false);
if (send is true)
{
return Ok(result);
}
else
{
// store in the email outbox
EmailOutbox emailOutbox = new EmailOutbox
{
OrderId = result.Id,
Success = false
};
var outbox = await _emailOutbox.Add(emailOutbox);
return Ok(outbox);
}
}
return BadRequest();
}
}Step 11:
In this step, this email outbox service should be used inside the background service. But background services are not able to inject services like controllers and they should be used in Scoped factory. So, add a new folder in the EmailOutboxService folder and call it BackgroundServices and add the below class and interface.
IEmailBackgroundServices.cs
public interface IEmailBackgroundServices
{
void Send();
}EmailBackgroundServices.cs
public class EmailBackgroundServices : IEmailBackgroundServices
{
private readonly IServiceScopeFactory _serviceScopeFactory;
public EmailBackgroundServices(
IServiceScopeFactory serviceScopeFactory)
{
_serviceScopeFactory = serviceScopeFactory;
}
public void Send()
{
using (var scope = _serviceScopeFactory.CreateScope())
{
var emailService = scope.ServiceProvider.GetRequiredService<IMailService>();
var emailOutboxService = scope.ServiceProvider.GetRequiredService<IEmailOutbox>();
var allOutboxResult = emailOutboxService.GetAll();
if (allOutboxResult.Any())
{
foreach (var item in allOutboxResult)
{
var res = emailService.Send(item.Order.Email, "Order is completed", "Your order has been saved in the database", false);
if(res)
{
var updateResult = emailOutboxService.Update(item).Result;
}
}
}
}
}
}As shown in the above codes, IServiceScopeFactory has been injected into this service. Why it should be injected? (1) Because for each background task, a scope will be required. it ensures that each task has its own set of dependencies and services, which prevents conflicts and unexpected behavior. (2)
Now in the EmailOutboxService folder add a class and inherit it from BackgroundService and add the following codes to it.
EmailOutboxService.cs
public class EmailBackgroundService : BackgroundService
{
private readonly IEmailBackgroundServices _emailBackgroundServices;
private readonly ILogger<EmailBackgroundService> _logger;
public EmailBackgroundService(IEmailBackgroundServices emailBackgroundServices, ILogger<EmailBackgroundService> logger)
{
_emailBackgroundServices = emailBackgroundServices;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var timer = new System.Timers.Timer();
timer.Interval = TimeSpan.FromMinutes(1).TotalMilliseconds;
timer.Elapsed += Timer_Elapsed;
timer.Start();
await Task.Delay(Timeout.Infinite, stoppingToken);
}
private async void Timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
_logger.LogInformation("Messages are sending.");
_emailBackgroundServices.Send();
await Task.Yield();
}
}Finally, these services should be registered in the Program.cs file.
builder.Services.AddSingleton<IEmailBackgroundServices,EmailBackgroundServices>();
builder.Services.AddHostedService<EmailBackgroundService>();Result
Finally, we can test the API. So, change the password of the email configuration or any other things that disabled your email service and send a request to the order controller. This makes a demand on the service to store data to the outbox pattern.
As shown in the above images, the order has been stored in the both EmailOutbox and Order table and the value of the Success field in the EmailOutbox is False. Now, if you start your email service, the background service will be executed and send a new email address and update the value that has been stored in the EmailOutbox table.
References
(1)https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes
(2)https://learn.microsoft.com/en-us/dotnet/core/extensions/scoped-service
