Transactional Outbox Pattern

Dual Write Problem

The dual write problem means it’s hard to keep different data sources working together when you write or save things. We need to make sure that all the writes are done correctly and the same in all places. If one write operation fails or only partially succeeds, it can lead to data integrity issues.

Dual Write Problem Diagram

The Transactional Outbox pattern helps solve the Dual Write Problem. It adds a middle step to keep writing actions separate.

How the Pattern Works

• Let’s assume that we also need to send an event to the notification service when a customer is created. It doesn’t immediately tell another service. Instead, it puts the important details in a special table called an outbox table.

  Customer savedCustomer = customerRepository.save(new Customer(customerDto.username(), customerDto.email()));

  //customerProducerService.sendCustomerCreatedEvent(savedCustomer);

  OutboxEvent outboxEvent = new OutboxEvent();
  outboxEvent.setEventPayload(new Gson().toJson(savedCustomer));
  outboxEvent.setEventType("CUSTOMER_CREATED");
  outboxEvent.setEventStatus(EventStatus.PENDING);
  outboxEvent.setCreateDate(LocalDateTime.now());

  outboxEventRepository.save(outboxEvent);

• The outbox table usually has fields like event details, status, and extra info for processing.

@Table
@Entity
public class OutboxEvent {

 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 private Long id;

 private String eventType;

 private String eventPayload;

 @Enumerated(EnumType.STRING)
 private EventStatus eventStatus;

 private LocalDateTime createDate;

• Publishing events can happen separately from the main transaction, known as Message Relay. This makes sure that event publishing doesn’t slow down the service or affect how quickly it responds.
• If publishing fails, the system can try again using a set plan. If it keeps failing, the messages can be put in a special queue for checking or fixing later.

Transactional Outbox Pattern Process

Message Relay

It is a mechanism for delivering messages from the OutboxEvent to consumers.

There are two patterns for implementing the Message relay:

• The Transaction log tailing pattern
• The Polling publisher pattern

I created a sample diagram for polling publisher implementation. Message relay poll the OutboxEvent table periodically. If a message is found, the consumer processes it and updates its data store.

Message Relay Diagram

medium-samples/outbox at master · omernaci/medium-samples

Microservices Pattern: Transactional outbox