Kafka interopability in Oracle Database 23c
Oracle Database 23c Free — Developer Release was recently made available to developers free of charge, to give them the opportunity to explore the next major release of Oracle and test out its new features before it becomes available for production use.
One of those new features is Kafka APIs for Transactional Event Queues (TxEventQ). This capability was introduced in Oracle Database 23c to bridge the use of Kafka and TxEventQ for Event Driven Application Architecture.
It has three main parts:
- The ability for Java applications written using the Kafka API for Java to use the Oracle Database as a Kafka broker,
- The ability for the Oracle Database to provide Kafka REST APIs, and
- Connectors that allow the propagation of events between Kafka and the Oracle Datbase, and vice versa.
Oracle Database has had queueing for many years (since Oracle 8). The older implementaiton was (and still is) called Oracle Advanced Queuing (or AQ). TxEventQ is a newer implementation that is optimized for much higher throughput and lower latency, and it provides transactional semantics (hence the name). There are clients in all common languages, and you can set up propagation between databases, even across long distances.
When you use TxEventQ with the Kafka APIs, Kafka topics and partitions map to TxEventQ’s partitioned queue tables and event streams, a unique capability among the various JMS messaging providers.
Developers can build applications in many languages and use Kafka APIs or TxEventQ APIs to use the message broker built in to the database. The Kafka Java Client for Transactional Event Queues (available in Maven Central) allows existing Java application that use the Kafka Java Client to work with TxEventQ with only two very minor changes — the import for KafkaProducer or KafkaConsumer has to be changed to an Oracle wrapper version that understands a couple of extra properties, and the bootstrap.servers property needs to be changed to the address of the Oracle Database.
The TxEventQ REST APIs (which are provided in Oracle REST Data Services) are very simliar, in many cases identical to the Confluent Kafka REST APIs. There are many APIs available for functionality like topics, partitions, consumers, consumer groups, offsets, and so on.
Current Oracle Advanced Queuing (AQ) users can migrate to TxEventQ, and also use Kafka Java Client, Kafka REST APIs, and source and sink connectors between Kafka and TxEventQ, for higher throughput requirements, combining point-to-point messaging and pub/sub streaming.
You can learn more about these features, including code samples, in these articles: