Delivering one lakh ride stats everyday!
In our day-to-day life, most of us spend a minimum of 2–3 hours on the roads — to get to work, getting groceries and what not. The scooter has undeniably become one of the most popular modes of transport for such everyday use. At Ather, we believe that a vehicle, especially a connected EV, can be more than just a mode of commute. With Ride Statistics, the Ather 450 and Ather 450X can help you celebrate your tiny moments of joy, be it clocking your longest ride on a Sunday morning, or helping you affirm your decision of switching to an electric alternative.
Over these years, the tens of scooters have become tens of thousands. Today our scooters clock in around 1.5L trips per day across the country, and over 90% of these trips have been available to view on the Ather app near real-time (within 15 minutes).
To support this massive number of rides each day, our platforms have gone through at least three iterations, most of them requiring us to go back to the drawing board and come up with scalable pipelines to give consistent outcomes.
The Ride Stats Saga: The Tech Stack and its Nuances!
A ride on your scooter is one of the many transitions it goes through every day, and identifying each of these transitions is fairly complex. There could be tiny rides that were meant to get out of the parking lot, or a long ride for a weekend getaway. There could also be rides navigating through the city’s traffic and stops at signals stretching longer than a few minutes.
The ride engine should identify these nuances to accurately categorize the rides to be ignored and the rides that need to be stitched together.
The stream processing engine: computing the ride stats real-time
Catering to the different scenarios seen above is more complicated than it seems. Try relating it with the way each person rides the two-wheeler. The high frequency data generated by the scooter hits the cloud IOT broker, which is constantly tackling various scenarios (such as network challenges, reconnection issues, and of course, the asynchronous nature of data).
The IoT broker pushes data via topics using a Messaging Queue (MQ) — Kafka in this case. MQs are set up with different topics, each topic playing publishers and subscribers among a group of microservices and each playing an independent but pivotal role in this “Session” enrichment.
Every ride is a session, and sessions can be different transitional states of the vehicle like ride or charging. Defining a session is done through multiple stages:
- Parsing: to make sense of the data in the message queue
- Timesorting: Microbatch to account for robust session definition and taking care of the real-world traffic nuances
- Enrichment/Enhancement: Defining the business logic and the primary metadata for the session
- Publishing: Publish the data inferred to a data store and the customer’s mobile application
Going (growing) by the numbers
With the first generation of the Ather 450, all of this intelligence was embedded in the pipeline. The heuristics for session computation accounted for a few corner cases but as the vehicles on the road increased, the session definition got far more complex. Adding to this, is the layer of OTA orchestration for every small change with the definition. Hence, it made sense to compute the session real-time on the cloud.
The initial real-time pipeline was built with off-the-shelf Google IoT broker, cloud managed data stores such as Influx, BigQuery and a quick MVP of Node.js API server to aggregate the metrics that are today published on the mobile app. This enabled us to get this feature quickly to the market, but leveraging managed services did prove to be an expensive option for scale.
With a challenge of enabling this feature for further versions of improvement and catering to tens of thousands of bikes, our best bet was to evaluate open-source alternates and track the per-bike cloud costs closely as a driving factor. The real-time stack today (see image in previous section) leverages a custom tweaked flavor of the Artemis & Kafka integration and series of components built on Java and Node.js each performing a distinct function. We also leverage Elasticsearch as a temporary data store during the flow of data through these stages.
All of the application, infrastructure and data nodes are containerized applications running on the Kubernetes engine. The Kubernetes orchestration helps us spin up pods, scale up applications quickly depending on the volume of data, and the operation is semi-automated through Terraform.
Our dev-ops processes have evolved through tackling various scenarios that are very unique to Ather. For example, The scooters are dependant on a network provider. We’ve faced scenarios where with a drop in network connectivity and sudden re-connections, there is a significant back-pressure on the Kafka and the only option to handle such erratic load of data bursts on the platform is to manually scale the applications and data notes by 10X. Today, with processes and scale tests in place, we’re able to handle such situations with minimal or no interventions at all.
Gazing at the horizon
While Ride Stats have been one of our core features, we believe there are more than a few ways of looking at what this data and analytics could mean to customers. We are constantly innovating, and one such innovation is the Savings tracker, developed as part of our Ather Labs offering (more on that in an upcoming blog).
Psst. We have some ideas on leveraging the personalized efficiency of each rider/bike to make the most of the battery as well.
Here’s to clocking more revolutions!
Originally published at https://blog.atherenergy.com on February 11, 2022.