Different Way to Unbundle Enterprise Capabilities
How NYC lets customers build what they really want
This is part 3 in a series about enterprise CodeBuilder technology. In this installment, I am going to discuss a popular use case.
Here is the setup: your enterprise has amassed a set of capabilities, but are unsure how to unlock and leverage them to your internal and external customers.
Is there a way for your customers to discover and experiment?
Silicon Valley versus Excel
For decades, the finance world struggled with this dilemma. Banking is messy stuff and raw data is too granular. NYC found the venerable Excel spreadsheet indispensable. Users love the ability to ingest and play with data — but decade after decade the west coast has yet to offer up much of an answer (although the reactive community has noticed an odd parallel to spreadsheets and the Python community loves Jupyter). But why aren’t the billion dollar vendors unable to compete with such a banal app?
Square Peg Meets Round Hole
Excel fills an interesting feature gap. We all know that inside every enterprise system lives the real system… trying to break out. I’ve done countless migrations and I am amazed how much data is no longer needed, how many tables and columns were never used, and conversely how much “custom” functionality was crammed into various nooks and corners. Because so much functionality must be “baked” up front, vendors provide massive “one size fits all” packages that hopefully cover all the bases. I’m not sure CIOs and CFOs realize how often they are basically overpaying to inherit someone else’s messes. For example Oracle Utilities was originally built for PacifiCorp and Oracle EAM was built for Duke Energy etc. Having a large vendor standing behind these systems almost works against you — because they have little incentive to adjust it once it is working.
But even if you build the perfect system, along comes M&A and something else gets dumped in the mix. This usually means constant user retraining and retooling. At best, your monolithic (SAP?) installation has all your business units lock step with the slowest denominator, unable to react quickly to changing business conditions. Sound familiar?
For the finance world, this lockstep approach fell apart during the crash of 2008. But Goldman Sachs suddenly rose from the ashes and retooled in hours instead of months. Word got out that Goldman had either made a pact with the devil or figured out some sort of equivalent to the ‘software spreadsheet’. As I discussed earlier SecDB allowed Goldman to rewire itself in Agile ways not previously thought possible anywhere.
Could this architecture be formalized? Could customers do the ‘wiring’ instead?
The Platform Approach
Classic thinking first suggests making an API. Basically take whatever existing functionality and slap a RESTful lightweight surface. There is certainly value in this approach — and led to a proliferation of fintech startups claiming to exploit unbundling of finance by offering ‘platforms’ that look a bit like point solutions. Meanwhile, Visa, MasterCard etc. are still taking their cut — after all, someone must still program and run those massive IBM mainframes to process the world’s transactions each night. Sounds like fintech failed to penetrate the real “platform”.
Beyond Platforms: Virtual Layer
I would suggest the API itself is not enough. Imagine Visa as a dynamic layer of software ‘wiring’ inserted between the consumers of the API (e.g. merchant services) and providers (member banks). Visa is constantly inventing new products and services, despite relying on mainframes that can take a year or more to safely tinker with. But Visa would be hard-pressed to describe their business model — there are so many moving parts. Visa is a giant financial switchboard.
CodeBuilders as the Unifying Force
CodeBuilders essentially buy future flexibility by unbundling existing capability into a homogenous medium. We always start with whatever is currently in place:
Note the colored objects are simply whatever legacy capabilities the organization wants to offer— existing apps, databases, services, tools etc. First thing we do is bring the heterogenous mix to a lightweight common layer:
But we don’t stop with just an API — that simply creates more chaos. A CodeBuilder provides a layer of virtual programming (think non von Neumann) that wires things up. A platform:
Keep in mind these are not hardwired connections (and interestingly enough can be arranged to mimic how the human brain works).
Note that unbundling could also be handled by a third-party — which may help resolve competitive pressures between certain providers and consumers (e.g. the consumer may want to wire up to several providers and/or keep wiring confidential). But in the end, many of the legacy systems are simply absorbed by the platform and phased out:
Hot Loading, State and Code Maintainability
Hot loading is a core feature of modern CodeBuilders (and we will explore this later) but is not well known how to program in these sorts of environments outside NYC, particularly for large enterprise systems. Some of you may be familiar with tools such as JRebel (or the old LiveRebel) which attempt to shorten the edit-run-compile cycle in limited situations. The JVM is modified to manage pointers so that Java code changes can be patched at runtime. The CodeBuilder approach is to simply use a late-binding language in the virtual layer (in the case of Java it might be Scala) that points to the Java code. However it does require structuring things up front in a functional manner to get away from control-flow issues when hot loading. While this happens naturally if you use a CodeBuilder, it is much harder to envision using flat files. Indeed, I suspect LiveRebel struggled because it is somewhat incompatible with von-Neumann thinking. State is a major issue when hot loading and why functional (state-free) approaches are so compelling. But what you are really doing is designing for maintainability (DFM) — a key design principle found in other engineering disciplines. There is nothing more frustrating than bringing your (unnamed) luxury car into the shop and finding out they need to pull the entire engine just to replace a minor pump. That may have been passable practice 20 years ago but no longer.
Function vs Configuration
Stepping back from the weeds even further, you are actually separating function from configuration. Functional programming is great but what about all that dirty stuff leftover? One could argue that a CodeBuilder is the natural complement to functional programming. Yes data belongs in a database but what about all that funky non-functional “glue” code that ties the whole system together?
Every time a modern developer ‘writes a program’, they are mostly selecting from existing libraries. This unto itself is a configuration-related activity; the developer will prefer not modify the library code. But once that usage is codified, it is difficult to untangle.
From my experience, this is a core reason why so many enterprise projects have struggled to get anywhere. Despite beautiful architecture diagrams, some basic questions of computer science have been left unanswered. How do you really architect a system for maintainability? How do you integrate systems without the ‘function call’ concept introduced by Alonzo Church? Clearly something major was missing.
Meanwhile, some NYC financial institutions took a radical approach to their enterprise capabilities — that would be considered heresy to California. Of course, this was undiscovered country — and NYC had to go it alone with considerable pain. Goldman was so far ahead of their time (1992) they had to invent their own programming language. Bank of America found virtual layering technology so disturbingly flexible they initially decided to “lock things down” old-school style with their Quartz platform and ended up sacrificing a lot of the Agile flexibility they originally sought. JP Morgan realized Athena required a high level of organizational trust — in an age when no one had even heard of scaling Agile across entire divisions, particularly where M&A was involved (e.g. absorbing Chase).
Part 4 is here.