The Nexa of Application Management
This is a loose and unfinished thought, so I hope you’ll bear with me. I am honestly interested in your opinion and observations. Please comment below or find me on Twitter at @jamesurquhart.
Much of my background is in the world of managing distributed applications. I started as a C (then C++) developer on the Mac, building client-server applications against a custom server-side record store for a printed circuit board manufacturer. I followed that up with more client server apps built in SmallTalk (against a FoxPro database, of all things).
But I really cut my chops at Forte Software, a publisher of a 4GL development environment that, in many ways, covered (at a small-ish scale, by today’s standards) a wide variety of the problems and solutions we see in today’s distributed architectures.
It was service-based, with every service being given a name that could be called directly in code. It allowed for replicated scaling and recovery. It worked best when building many specialized services that were either data services that stored and retrieved specific sets of objects tied to specific databases, or compute services that were stateless and easily replicated.
Forte also had functionality to support the deployment of these services (and the applications UIs), as well as the monitoring and operational control of the services once they were deployed. All in all, it was a pretty amazing operational environment for its time, as well as being a decent development environment.
The Nature of Distributed Application Operations
My experience with Forte left me fascinated with the concept of operating distributed applications, and that has been a focus of my career off and on ever since. Specifically, there were two things that were made exceedingly clear from those early days:
Distributed Application Operations requires automation. It is nearly impossible to sanely control a multiple tier, replicated distributed application without some automation. At the very least, installation and configuration should be automated in some way, with clear ways to distribute config information to each node as required. However, automation around adding (or removing) additional nodes, replacing a lost node, capturing and/or destroying logs, and so on, add sanity to a complex and often chaotic situation.
Distributed Application Operations requires monitoring. Visibility into what is actually happening across the entire application — and, often, external dependencies — is critical for both humans and automation software to do their jobs well. Monitoring can include infrastructure monitoring, process monitoring, logs, network traffic, and a myriad of other things that are produced or affected by application operations. Furthermore, monitoring can (and, I would argue, should) happen at the systems level, driving decisions across all applications participating in an interconnected system of applications.
These two elements, automation and monitoring, are the key functions of application management, and I believe we are seeing that in how the market is evolving today.
The Two Nexa of Application Management
Increasingly, all players focusing on helping humans operate distributed applications are falling into one of these buckets. Only we tend to know of them by different names. Application monitoring is known as Application Performance Monitoring, or APM. Application automation is increasingly referred to as PaaS (or, at least, orchestration).
On the one hand, companies that started out focusing on network monitoring (e.g. Boundary) or log management (e.g. Splunk) are positioning themselves as application performance monitoring competitors to leaders such as NewRelic or AppDynamics. Dell’s monitoring tool, Foglight, is increasingly gaining APM features, as are tools from IBM and HP.
On the other hand, application packaging and operations automation is increasingly becoming tied with the term “platform” — explicitly, in the case of Cloud Foundry, or implicitly in the case of packaging and orchestration vendors like HashiCorp or Docker. Even collections of open source tools for the build, deploy and operate lifecycle stages are described in terms that much resemble those applied to PaaS and orchestration.
What is really interesting is how infrastructure centric tools, such as most early “multi-cloud management” tools (such as Dell Cloud Manager) are becoming application focused, and thus joining the ranks of “platform” components. While none are even arguably “Platform as a Service” today, they are increasingly playing a role in getting PaaS or other orchestration tools deployed and configured.
Can Application Operations Be Both?
This leads to the question as to why APM tools don’t increasingly add features to execute on the events they detect through monitoring, or why platforms don’t increasingly add advanced application monitoring to their own feature sets. I think the answer is pretty simple, though, and I hinted at it earlier.
While operations platforms (PaaS or orchestration) focus on managing whatever the unit of delivery is for software components (servers or containers, for the most part), and are thus operated much more on a “per application” basis, as far as I can tell, the APM world is increasingly focused on how you monitor and measure across software components.
In fact, as I’ll write about shortly, it seems much of the monitoring data of the world (including performance data, availability data, etc) can be usefully correlated with business metrics, (such as conversion rates, etc.) to help identify ways in which applications can be optimized for the business.
Thus, I am learning about more and more digital businesses (or digital business units) that are referencing application data in the context of business processes, which is absolutely fascinating to me, and opens up a world of opportunity.
Thus, I think APM (or APM-like) services will remain decoupled from orchestration and platform services, though the two will often share data and events as needed. The power of correlating IT operations contexts from business contexts without interfering with application architectures is too great.
What do you think? Again, if you think I’m wrong, I’d love to hear from you. Even if you think I’m right, but have additional thoughts, please comment below or reach out to me at @jamesurquhart on Twitter.