Incredible Kafka Integration with AWS S3 using Connectors
Many people think that Kafka is just like an ordinary message broker such as RabbitMQ. Undeniably, integrating among system services is a common use case of Kafka. For example, the system logic to transfer money in the diagram below sends a transaction record to Kafka and then another logic is triggered for account balance calculation. Kafka in this use case serves as a middleware that facilitates the communication between these 2 independent system logic.
Kafka can be replaced by any other message broker if it is just used for the system integration.
However, the storage of historical messages is the unique and powerful feature that makes Kafka so prevailing. Going back to the use case above, the balance is calculated based on transaction history. The balance might need to be re-calculated in case of any adjustments. Then, the updated balance can be derived by replaying all previous transaction records from Kafka.
Kafka also serves as a data store that maintains transaction history in sequence.
