How Connected Built a Record-Breaking IoT Cloud Messaging Service on Kubernetes
Connected built a record-breaking IoT cloud messaging service on Kubernetes. Learn all about it during their presentation with Bose at KubeCon on Dec 12.
Intro
KubeCon is one of the most cutting-edge technology conferences around, and this year, Connected is proud to present how they built a record-breaking IoT cloud messaging service alongside the leading audio electronics brand Bose.
A Platform Like No Other
“The presentation will be a case study in how we built a high-scale messaging service and state store on Kubernetes,” says Dave Doyle, Software Engineer Manager at Connected. Doyle will be presenting alongside Bose’s Cloud Architecture Manager Dylan O’Mahony.
Built using Kubernetes, a tool for open-source container orchestration management, the messaging service can support over five million active messaging connections simultaneously. That’s 2x more concurrent connections than any off-the-shelf IoT cloud messaging service available.
“It’s precisely the kind of solution that’s been missing from the marketplace,” adds Thomas Aston, Lead Product Manager at Connected. “Built specifically to power future user experiences, this platform will support real-time product interactivity at a global, consumer scale.” Aston has been deeply involved in the project since day one, orchestrating the team’s efforts leading up to the presentation with Bose.
The messaging service can support over five million active messaging connections simultaneously. That’s 2x more concurrent connections than any off-the-shelf IoT cloud messaging service available.
Why IoT Cloud Messaging Matters
As demand for IoT devices grows — from internet-connected home appliances to self-driving cars — the rate of innovation in the underlying technologies that enable these devices is quickly following suit. Hence the rise of IoT cloud platforms from AWS, Google, and Microsoft, among others.
Yet these off-the-shelf IoT solutions have remained largely focused on commercial and industrial applications (such as capturing and analyzing sensor data to optimize manufacturing processes), while efforts to support IoT experiences at consumer scale continue to lag behind.
For leading consumer electronics producers, however, the opportunity cost of waiting around for a viable, consumer-scale IoT solution is high. They are forced to choose between adapting their product roadmaps to fit the timelines of cloud IoT service providers, or forging a path to build their own cloud platforms and related IoT services.
Off-the-shelf IoT solutions have tended to focus on commercial and industrial applications, while efforts to support consumer-scale IoT experiences lag behind.
What We Did
Of all the things a consumer-scale IoT solution must solve for, messaging is perhaps the hardest — that is, full duplex communication from client to client (devices, apps, services) as well as between clients and the cloud. To be successful, a consumer-scale messaging solution must support millions of persistent, concurrent connections; enable tens of thousands of messages per second; be globally addressable; store millions of states; and respond with near-zero latency.
In pursuit of a solution that met these requirements, here at Connected we were asked to assess the technical feasibility and cost viability of existing off-the-shelf messaging solutions as well as to prove out a scaled messaging service built with open-source components.
To be successful, a consumer-scale messaging solution must support millions of persistent, concurrent connections; enable tens of thousands of messages per second; be globally addressable; store millions of states; and respond with near-zero latency.
We deployed a load test framework against leading IoT solutions to capture a baseline of performance, capability, and reliability data across a number of representative scenarios. We identified both technical limitations and cost constraints, underscoring the need for a custom messaging solution.
We then leveraged the same test framework to iteratively build out an IoT messaging proof of concept that exceeded the state-of-the-art benchmarks across almost every technical and cost dimension — from connection count to throughput, addressability to round-trip latency.
How We Did It
We split into competitive teams of ‘makers’ and ‘breakers’ and applied lean principles to explore solution approaches. Makers developed the scaled messaging stack while breakers repurposed Locust, a high-scale load testing framework, to simulate client behaviour. Iteratively deploying the test framework against the solution stack enabled us to rapidly evolve the solution set and land on a viable outcome.
Leveraging the flexibility of Kubernetes, we scaled the stack and solved the many blockers on the path to a viable solution. These included ingress, file descriptor limits, service discovery, and resource limits.
The team built an IoT messaging proof of concept that exceeded benchmarks across almost every technical and cost dimension — from connection count to throughput, addressability to round-trip latency.
Conclusion
What’s exciting about this development is that messaging at this speed and scale represents a paradigm shift in the product experiences consumers can expect to enjoy — one where we move from devices that are merely internet-enabled to devices that are so seamlessly connected they react to the user’s world and other devices in real time.
And the applications are limitless. From IoT earbuds that allow for language translation in real time to AR glasses that enable real-time conference room meetings across the globe. From eradicating rush hour through the optimization of live traffic flows to perfectly competitive pricing at the pumps based on current, global crude output. From immersive, real-time distance education to accurate microclimate weather reporting that’s unique to the individual on their daily commute. The point is: it’s fast, it’s global, and it can support millions of devices at the same time.
Messaging at this speed and scale represents a paradigm shift in the product experiences consumers can expect to enjoy.
One final use case isn’t for consumers but for businesses. At the end of the day, most consumer electronics companies would likely be more inclined to purchase managed IoT capabilities than risk building and maintaining an IoT solution on their own. With scaled messaging at its core, the platform Connected helped build won’t just power a new line of amazing products for end-users, but represents exactly this Platform-as-a-Service opportunity for other consumer-facing companies.
We can’t wait to talk about it in more detail at KubeCon in Seattle. See you there.
Notes
KubeCon is put on by the Cloud Native Computing Foundation. Here’s how KubeCon is described on their website: The Cloud Native Computing Foundation’s flagship conference gathers adopters and technologists from leading open source and cloud native communities in Seattle, WA from December 10–13, 2018. Join Kubernetes, Prometheus, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for three days to further the education and advancement of cloud native computing.
About Dave Doyle
Dave Doyle leads a diverse team of engineers, exploring the viability of high-scale, low-latency messaging solutions for next-gen IoT applications using VerneMQ and Cassandra. He spent the last 18 years building high-performance systems across a range of technical domains. He has led the development of a high-reliability exchange rate platform, as well as a variety of experience in CMS, CRM, process automation suites, legal research systems, and parallel distributed file systems for supercomputing. Dave is the former organizer and frequent contributor to the Toronto Perl Mongers and a speaker at the Greater Toronto Linux User Group. He is a Software Engineer Manager at Connected.
About Dylan O’Mahony
Dylan has spent his entire career at Bose, focused on bringing new technology and software solutions to bear on the business. His journey has taken him from managing the bose.com web technology team, to running an IT innovation lab to, most recently, leading the architecture of a scaled IoT Platform-as-a-Service based on Kubernetes and a plethora of other cloud-native technologies.
Originally published at www.connected.io.
