The Connected Car with Oracle Event Hub Cloud Service

Introduction

We have been recently working with a few of the worlds largest car manufacturers , car dealerships and car insurance companies who are now more than ever excited to provide a connected car experience with applications, analytics and services as part of their mobility ‘in car’ experience for their users.

Oracle Event Hub Cloud Service has been the default data ingestion mechanism for all such use cases. Oracle Event Hub Cloud service is built on the basic foundations of Open Source Kafka and thanks to the very vibrant and active development community it has a gotten tremendous momentum in this space.

This blog describes about a very common design pattern as envisioned and implemented on Oracle PAAS by most of the players in the connected car space.

The Use Case

GPS enabled devices built into cars are enabled to not only provide a Wifi hotspot experience for dashboard installed applications but also to interpret driving patters, provide car diagnostic information and provide roadside assistance.

These devices are 4G/5G wireless capable and send data over to Oracle Event Hub Cloud Service over Kafka’s Native (and light weight) APIs that are based off TCP/IP and not HTTPS (while that is possible).

Some car manufacturers have chosen to use MQTT if their wireless coverage is expected to be spotty, they require very low power consumption on their devices and if their data sets are expected to be small. For all other scenarios — Kafka Native APIs is the choice for transport.

MQTT works in conjunction with Kafka and an example architecture (not to be mistaken as the exact blueprint) is as below :

MQTT Broker deployed as part of Oracle IOT Cloud Service terminates the MQTT connection and a MQTT Kafka Connector deployed on Event Hub Cloud Service’s built in Kafka Connect Run time consumes the messages and pushes them to a Kafka Topic.

Multiple MQTT brokers can be deployed with each catering to a specific geographic region.

Kafka decouples producers from consumers and is specially useful in this scenario where hundreds and thousands of cars are sending data every few seconds generating a high velocity dataset that can be consumed by consumers which have different date rate ingestion needs.

Data sent by connected cars typically includes diagnostic OBDII trouble codes. If you have your check engine light ON then typically one more more of the codes are “active”.

Example code : P0301 : Misfire Condition on Cylinder #1

A real time dashboard implemented in an application running on ACCS would in real time show active trouble codes/maintenance needs for a set of vehicles when we are dealing with a commercial fleet management use case. Data from the same Kafka topic can be pushed into Oracle Data Hub Cloud Service (Cassandra) a highly scalable NoSQL data base or into a data lake (Oracle Object Storage) for post processing for Business Intelligence or Spark Analytics ( Oracle Big Data Cloud) .

A connected car use case like the one above needs to handle petabyte scale data and Event Hub Cloud Service was built keeping that in mind.

A Highly Available Kafka cluster is not only desirable but often table stakes in the connected car use case. Kafka natively offers high availability with built in support for Broker/ZooKeeper Clustering and automatic failover with topic/partition replication. Oracle Event Hub Cloud Service goes a step beyond by implementing a stretch cluster that splits a Kafka cluster across availability domains that can account for entire Availability Domain failures.

In addition, an optional DR solution using Kafka Mirror Maker ensures protection against regional data center failures.

A well known Netflix implemented hierarchal kafka clustering approach is also possible with Event Hub Cloud Service — Dedicated where the entire, private and configurable Kafka Cluster with SSH access is deployed on Oracle Public Cloud Infrastructure.

For reference on sizing your Event Hub Cluster refer to : https://medium.com/oracledevs/sizingeventhubcluster-dbb639e42094

Summary

An increasing interest in the connected car space has made Oracle Event Hub Cloud Service as the choice for high performance data ingestion service for reasons below:

• Event Hub Cloud Service is based on Open Source Kafka that provides a plethora of reusable tools sets built by a very active development community
• High Availability with built in Kafka HA options with additional layer of high availability made possible by Oracle Event Hub Cloud Service’s Stretch Clustering and Mirroring options for AD/Regional data center failures.
• Highly performant — achieved by use of Kafka Native APIs on top of the best in class Oracle Cloud Infrastructure
• Integrated with Oracle IAAS and PAAS Services for an end to end solution experience