Numaproj — Driving Dev Velocity with Real-time Analytics, AIOps on Kubernetes
The number one driver of business performance for digital-first businesses is investment in best-in-class development platforms, according to a recent McKinsey & Company study on developer velocity.
Over the past four years, Intuit’s investment in cloud-native technologies such as Kubernetes and Argo to provide modern platforming company-wide has resulted in a 6X increase in development velocity.*
As the global financial technology platform that makes TurboTax, Credit Karma, QuickBooks, and Mailchimp, we’re proud of our progress to date in fueling product innovation at speed and scale for 100 million consumer and small business customers worldwide.
However, we recognize that achieving the next 6X increase in development velocity here at Intuit can only happen if we take steps to catalyze the next phase of AIOps innovation. Building an even smarter platform with a simplified user experience means managing more (and more) data in an actionable way.
Collectively as an industry, we have a long way to go.
Even in today’s cloud native world, over 90% of IT, DevOps, and Site Reliability Engineering professionals reported an increase in service incidents. An even higher percentage reported that incidents take longer to resolve while working remotely.
More data, more problems.
Driving the next 6X in development velocity through AIOps
Next-generation development platforms must enable application teams to focus on specifying high-level requirements for running their applications, rather than configuring and managing the underlying infrastructure.
Using AIOps to derive real-time, actionable insights from operational data, the platform should be able to quickly detect, isolate and remediate production problems, and automatically tune many system parameters that are manually configured by developers and platform engineers today.
Introducing open source project Numaproj!
That’s why we’ve built — and open sourced — Numaproj, a Kubernetes-native collection of tools for real-time data analytics and AIOps.
Numaproj has two initial tools to help with critical areas in your AIOps journey:
- Numaflow — Massively parallel real-time data/stream processing engine with automatic failure recovery, exactly-once semantics, autoscaling, back-pressure, and watermarks. It allows for easy creation of fire-and-forget pipelines that reliably perform their assigned data processing and analytic tasks until instructed to stop.
- Numalogic — ML models and algorithms for real-time data analytics and AIOps including anomaly detection. The current version includes ML models for continuous real-time data enrichment including anomaly scoring.
To learn more about Numaproj, check out this technical blog here, head straight to the repo, and watch the demo of Argo Observability powered by Numaproj at the recent ArgoCon.
*Development velocity as measured by the number of production releases per developer per week as well as notable improvements in availability and time to detect and resolve production incidents.
