That thing about core banking platforms

I recently had a discussion on Twitter about core banking platforms, and whether they’ll be (a) an important means how challenger banks can leapfrog the established players by providing better services, and (b) whether those established players will need to implement new systems in order to remain competitive.

What is a core banking platform?

The pre-history of core banking platforms

Core banking platforms are as old as banking itself, if one counts the ledgers, coded letter, or whatever other system bankers used to record and transact their business. I am not privy to the exact details of banks adapting computer systems instead of paper based ledgers, but I have not doubt that when IBM came out with their 700 series of mainframes at the end of the 1950’s many banks were seriously thinking about using them, or did actually use them. In parallel, the development of the COBOL programming language started in 1959 and it was adapted in 1962 as IBM’s main programming language. In 1964 IBM came out with its System/360 series of mainframes, and at this point all but the tiniest banks moved to mainframe based back-office processing — in other words, the electronic core banking platform was born, even though noone called it like that.

IBM System/360

As an aside, I was mentioning COBOL, but of course in the beginning the real programming language to use was assembler. Firstly, it compiled much more quickly and cheaply, because (CPU) time was literally money at that point — computers were leased, and the number of active CPU cycles was a key driver of the billed amount. Secondly, assembler allow bit-level access to data structures — why waste a whole byte on a male/female flag for example, if one bit is enough? Anyway, at one point computing costs came down whilst personnel costs did not, so replacing all those assembler programs with COBOL programs was the first new overhaul of the core banking platforms, and it went on until at least the late 80s.

Initially mainframes did not have persistent state: data (and program) storage would be initially on punch cards and later on magnetic tapes. The major human-centered output device was the printer. In order to execute a program, the operator would first put in the tape (or stack of punch cards) containing the program, load it into memory, and then put in the right tapes (or card stacks) for data in- and output. Then he’d run the program which would somehow transform the input data on the tape (or punch cards) into output data on the tape (or punch cards) and the printer. So in a way, computing at the time was pure functional programming, no side effects whatsoever.

Later mainframes acquired hard drives, like that 5MB model shown in the picture below. Those drives were extremely expensive, so one would not necessarily use them for programs, but they had one very important application: they were able to hold the databases that all (or most) programs needed to access and possibly change, like the general ledger with the account balances and recent transactions, and possibly some basic customer data.

Interestingly mainframe architecture was closer to the current cloud architecture than to the workstation and PC architecture in one important aspect: user would connect remotely — via a dial-up line, initially at a speed of a blazing 300baud — and typically work their way through an elaborate structure of menus and input masks (today’s web forms) to do whatever they had to do, eg inputing or changing user data, recording money transfers, or whatever task they had to do on that terminal.

There was some kind of local area networking within the data-center, even though networking might be too big a name for a system that essentially connects peripheral devices with CPUs, and possibly CPUs with other CPUs in multi-processor systems. Real LAN networking was only developed in the 1970s, and became mainstream with IBM’s token ring and the ethernet standard at the beginning of the 80s.

Other than dial-up lines there was no wide-area data transfer, and certainly no wide-area networking. Mainframes could in principle connect to each other using modem banks via the fixed line network. However, the connection speeds of modems at this time were usually too low for this to be useful for any serious data transfer, so networking consisted of physically moving tapes from one data center to another one.

Manual batch processing

An important consequence of that set-up was that mainframes would execute all transaction in batches. Let’s for example look at what would happen with bank transfers:

1. A customer would come to bank A, requesting a transfer. For this, the customer would fill in a transfer form, filling in his bank details, the recipient details, and the transfer amount
2. A bank clerk situated in the branch would at one point process this transaction, ie sitting in front of the dial-up terminal he’d navigate to the ‘Enter new transfer’ mask, input the details, and submit them when ready
3. The mainframe would at that point only record the transaction, ie it would have a tape where everything that inputted in the branches would be recorded, typically in a flat text file
4. Periodically this text file would be moved to processing (possibly by physically the tape first); the processing might be a multi-step process, eg let’s assume the first step simply extracts all transfers from the activity tape and copies them to two transaction tapes, one for internal transfers, and one for external ones
5. In the next step, the internal transfers can be directly executed: essentially general ledger data entries for both affected accounts are updated, which finalises the processing
6. For external transfers the system has to first decide where to send the funds, and then generate the relevant records on tape (one that says that a certain amount needs to be transferred from bank A to bank B, and one that will later be sent to bank B informing it of the transaction details); it also needs to update the general ledger account, removing the funds from the client account
7. All transfer records will be periodically aggregated (running yet another batch program), and ultimately forwarded to the central bank that then will run its own batch program on it that moves the funds between the banks’ accounts at the central bank
8. At the same time, bank A will forward the message tapes to banks B, C, D… to inform them what to do with the funds they received. Those banks will in turn run batch files over the respective data tapes they received, and credit the appropriate accounts with the funds they received in step 7

This was just a simple example for a core banking process to be executed by the system. It is pretty easy to put together a process as a sequence of programs, where the programs are designed using the following building blocks

• read an input data file
• create one or more output data files
• update the database

The complexity here is n0t that much in the programs but in the scheduling, ie running the programs in the required sequence, ensuring they have access to the data files that they need to run.

Automated batch processing

There were three important developments starting in the 1970s and going on through the 80s and 90s that transformed the way the core banking systems (that were still not called that) were working. Those developments were

• random access persistent memory (aka hard drives) got dramatically larger and cheaper, allowing for more and more data to be permanently accessible, eventually leading to the development of file and database servers
• dial-up networking got significantly faster, and was eventually replaced with fast digital subscriber lines and/or Internet connection
• the rise of workstations and later PCs increased the variety of computing platforms that could be used for user interfacing, file storage, and data processing

What this meant was at first that there was less and less need for manual intervention in the compute process: programs and data were permanently available on file and database servers, so they could be launched periodically and automatically as cron jobs. Output might still be produced as computer tapes and physically moved, but as remote connections got faster, eventually the data could be transferred electronically. The rise of workstations and PC also allowed the rise of intelligent terminals that could hold a certain amount of data themselves, and also allowed for calculations and processes pushed to the edge — eg mortgage calculators could be run locally to analyse and compare different scenarios whilst the consumer was present. Also it allowed to replace text terminals with (graphical) X terminals and later web-browsers, ie where clerks would have previously entered their data into a terminal remotely connected to an IBM mainframe they’d now use an X windows application running on a Sun workstation, or a web form hosted by a http server located in the data center.

The core banking system at the end of the process was essentially a collection of compute and storage resources, partially based on mainframes, partially based on Unix/Linux based workstations (with some VMS thrown in for good measure), and partially based on Windows systems. This complex network of compute resources was sewn together via file sharing, ie programs would not talk to each other, but they’d deposit their output in a file on a central server that a subsequent program would use for further processing. Standard was to run a lot of the processing once a day at the end of the day to capture everything that had happened during that business day, and to run additional jobs on say a weekly and monthly basis that would produce more detailed reports, eg for management reporting purposes.

Real time processing

Having now worked our way to the end of 90s / early 00s, we now come to the holy grail of core banking systems: real time processing. The two big drivers here were (a) because-we-can and (b) Internet banking.

The driver (a) is about technology finally getting to a place where real time access is possible: there are now numerous tools to handle concurrency, eg daemon processes can simply sit on a server to do stuff once they are summoned, the summoning happening either via Unix-style socket connections (in development since the middle of the 80s), or via the slower http connections if the processes do not reside on the same host (they might in fact even reside in different data centers). So that’s the technology push: finally real time systems can actually be built at an acceptable cost.

On the other side there is a customer pull, ie driver (b): with Internet banking (and slightly earlier telephone banking) customers can directly interact with the banking system, eg to arrange transfers, or to check their account balances. Naturally, customers expect consistency here, ie they expect to see the transfer that they just entered to appear in their statement right away, not a few hours later, or the next day. They’d also like the money to appear in the recipient’s bank account the moment it leaves theirs, which — assuming multiple banks are involved — is a slightly harder problem to solve.

Historically most banks have attacked this in three stages: first the real-time update of account balances, then real time transfer to accounts held with the same bank, and then as-close-to-real-time-as-possible transfer of funds between accounts at different banks. The reason for the caveat with this last point is that interbank transfers depend on whatever interbank clearing system is being used. For example, in the UK retail payments used to be processed via the BACS system that would take three business days to clear, and it is only since the launch of the FPS system in 2009 that immediate crediting of the recipient’s account is possible.

What is clear is that customers come to expect real time in more and more occasions. For example, when making purchases, it is important that payments go through immediately — which is a service card payment services can offer but banks currently can’t. Card payments however are pull payments, so there is a certain fraud risk, which in turn affects the premium charged. If banks could offer guaranteed immediate settlement push payments, operated using a card-like UX, fraud potential could be significantly reduced, especially for card-not-present style payments. On other occasions immediacy is just a nice to have: for example, if you get immediate confirmations of all card purchases you make then account reconciliation is much easier.

So in conclusion, if as a bank you are not able to process most of the transactions in real time — and at volume — then you are already in a bad place, and it’ll get worse very soon.

Core banking platforms vs legacy banking platforms

In a slight change of terminology I want to from now on refer to banking platforms that use components developed before say 2000 as legacy banking platforms whilst I want to call every system that has been completely and newly developed since then as a core banking platform.

The difference between the typical core banking platform and the typical legacy platform is that the former is a coherent new system, built on (reasonably) modern technology whilst the latter is an assembly of various systems held together with duct tape and prayer. Also the legacy platform is probably slightly better adapted to the exact processes of the bank that is using it, and it works, there and then.

Also — and this is important — a core banking platform is built on the basis that it should be possible to adapt it to be used in different banks, like a platform in the car industry. Note that this of course is not entirely true: many core banking platforms currently on the market have been salvaged out of the wreckage of challenger banks that did not make it, so adaptability might not have been a concern right from the start and has only been added later, more or less successfully.

This time it’s different

From the mid 90s there was a big push by many banks to reengineer their legacy COBOL and assembler based mainframe platforms and rebuild them from scratch in the more exciting brave new world of Java running on workstation clusters. Those initiatives had mixed success, and I don’t think any of them were delivered on time or on budget — and some were never delivered at all, after having sunk a significant amount of money into the project. Having said this: a banking platform is a large scale, complex IT project, so this result is not entirely unusual.

It is however a warning bell: on the assumption that not everyone programming at that time was stupid, what has changed now? Why can we suddenly build core banking systems that we could not build before? There are multiple answers to this question:

Experience, tool chains and frameworks. Especially when we compare it to the days Java was just starting out, we have many more frameworks and tool chains that allow us to build cleaner code — through experience we have a better understanding for what works and what does not. A lot of that understanding is embedded into the tool chains and frameworks so it is applied automatically, and there are now numerous people in the market who understand the possible pitfalls when building banking platforms and who’ll be able to ensure that we don’ run into the same trap twice.

Cost of compute resources and interprocess communication. When projects failed they did not usually fail because the system could not do what it was supposed to do, but because it was too slow. People were using all those neat object-oriented techniques that allowed to separate the concerns but the overhead and duplication of tasks that this introduced was much bigger than expected, and the compute and communications resources available at that time were simply too slow (or too expensive) run those systems at the speeds required.

The culprit there are often risk and reporting systems that need to aggregate data over all products and customers of the bank. If they query the relevant single product and customer objects, duplication and overhead means that those jobs are very slow (or rather, expensive) to run. The only solution then is to either open the APIs and allow the risk and reporting systems access to the internals of the customer and product objects, or to move a lot of the aggregation functions into them. In either case, separation of concerns is broken, either because the objects are de facto fused (case of too wide API) or because risk and reporting code is now part of the customer and product objects where it does not belong.

Nowadays compute and communications resources are cheaper, so it is economically more viable to throw a lot of computing and communications power at a problem. This means APIs can remain narrower without having to move code, and whilst this still incurs a performance penalty this can be economically viably solved by throwing more resources at it.

We should however not wear our rose tinted glasses: building a core banking platform is still an awfully hard problem. A lot of the development effort is O(n x m), say where n is the number of customer types (eg countries), and m is the number of products. This means that development starts out deceptively simple (small m and n) but it gets harder and harder over time once the quadratic term starts to kick in.

Also the general nature of the systems makes the problem much harder: they are meant to adapt to a large variety of products and business processes, meaning that simplifying choices that can be taken in a bespoke system to cut down the development effort are not possible when widest possible applicability is to be maintained. In the practice we’ll most like have what could be called the SAP effect: whilst banks will be able to adapt the system to their needs they’ll also have to adapt themselves to the needs of the system, ie they’ll have to adapt their business processes to be in line with the system that they are using.

Core banking platforms and challenger banks

Challenger banks are in the interesting position of having nothing but an idea how to make banking better, in particular they do not yet have a legacy system or legacy processes to deal with.

In this environment, purchasing an off-the-shelf core banking platform is often a very good idea, even though the buy-vs-make decision is still one that should be debated, given that moving platforms is a pretty serious undertaking, and it is the more serious the more business has already been established on that platform (the aforementioned O(n x m) law applies here as well). So let’s look at a few points:

Cost. Cost is probably the most ambiguous point of all. Core banking platforms do not come cheap, and neither does configuring them. This is where O(n x m) complexity hits again: a start-up with a very simple product offering and a very narrow customer focus will probably find it cheaper to build a platform from scratch than to adopt an existing one, especially if the product it offers has a slight twist towards what is usually offered by a bank. A bona fide challenger bank however whose main goal is to offer plain vanilla banking services just better might find it cheaper to go with an off-the-shelf solution.

Time to market. If you are a start-up, almost certainly time-to-market is better with off-the-shelf systems. Whilst there might be a fair amount of configuration to do, this is a process that the vendor or its implementation partners have gone through multiple times, so lest something unexpected happens (which is a possibility of course) time-to-market is predictable and quick.

Flexibility. Purchased core banking systems are often flexible, but only to a point, so any unusual product offerings might be impossible to implement using the standard configuration methods. Most systems do allow for custom plug-ins that extend the core functionality, but using those comes with a risk: core systems change over time, and at one point plug-ins might break. At best, fixing the plug-in is a costly exercise, and at worst it is impossible to do. And in some cases the lack of flexibility is surprising: for example I have personally come across a major system whose underwriting module only allowed credit decision based on an additive score-card of the rating criteria, and it was impossibly to implement any non-linear interaction amongst the evaluation parameters. Especially in the period of hyper targeting and AI supported rating approaches this is actually a serious constraint.

Suitability. The suitability point is closely related to the flexibility point: a company with a very standard business model might find the core banking system suits its needs perfectly well, whilst a company competing on the basis of an unusual product offering or customer segmentation might find it more difficult to do so. For example, imagine you want to set up an equipment leasing company where the leasing rate is driven by actual usage, as determined by telemetry. Depending on the design assumptions underlying the core banking system in question this product might be very difficult implement.

Competitive advantage. Last but not least there is a question whether a core banking platform is a source of competitive advantage or not. Again, it depends on business strategy: if your core USP is quality of service, friendliness of staff, and that you provide food and water for the dogs of the people that drop by at the bank and your product offering is bulk standard then a core system probably does not add much. If however your goal is to offer features that no-one else can offer, or cover markets that no-one else can cover, then you might be better off controlling every aspect of your system, meaning that you have to build it yourself.

Core banking platforms and established banks

We have just seen that for a challenger bank starting up, purchasing a core banking platform is not quite a no-brainer, but certainly something that they should give serious thought to. For established banks however in my view the picture is different.

Technological needs and capabilities. Arguably the most important point to consider is what a bank’s technological capabilities and needs are. If a bank has already tried to reimplement a new system and failed, and/or it is still on batch processing for key process steps, then it should seriously consider whether moving to a purchasing a core banking system might be the right thing to do. If on the other hand a bank’s systems can now mostly do real time, and it is able to transition them piece by piece away from outdated technologies then probably their own systems are not too bad, and moving to a completely different system might not be worth the risk.

Complexity. The complexity point is similar to the one for start-ups: a simple bank with a standard customer base doing standard products might do very well with standard core banking system. A complex bank engaging in many different business lines, with a multitude of customer groups in a variety of jurisdictions might find that off-the-shelf systems don’t fit, and that adapting them is prohibitively expensive.

Size. Size of a bank is a key factor in the make vs buy decision because the cost of developing a system scales very slowly with the size of the institution at the lower end of the scale (at the upper end there is the O(n x m) issue again, and in some cases using different systems for different lines or business might be advantageous). Having said this, at least in Europe most traditional banks have already addressed this scale issue, in that the large majority of smaller banks are part of banking associations (eg Volksbanken, Sparkassen in Germany) and the IT functions are located in association-level service companies that do have sufficient scale to operate efficiently.

Having said this, if banks are big enough there can be a good reason to actually run different systems for different business lines. The arguments here are very similar to the ones discussed under the challenger banks heading: if a bank is starting a new and independent business line then under some circumstances it might want to go for an external system, even though this will invariably lead to problems, for example for group level reporting. This can also be a good trial ballon, and if the purchased system happens to work better than the self-built one this might be an opportunity to move more or even all business over..

Conclusion

Core banking platforms is a hot topic right now, and there are a number of very good platforms available. For challenger banks (and for established players wanting to extend into adjacent lines of business, or launching a distinct new offering) those platform can greatly simplify the task of setting up a banking operation from scratch, and the buy-or-make decision should be on the top of mind of every challenger bank CEO, with a strong bias towards buy.

For established players thinking about their core systems the picture is more mixed. There is a lot of complexity in banking processes, and this is forcibly mirrored in the systems. Best case is that the transition to a new core banking platform goes like the introduction of SAP: painful, with teething difficulties and a lot of business process adjustment, but ultimately successful. However, depending on the complexity of the products offered and customer segments covered, off-the-shelf core banking platforms might not be flexible enough to adequately cover a company’s needs, or rather, require so much manual adjustments and extensions that running an in-house system might be the better and cheaper solution.

UPDATE: I misdated the Java revolution in banks which started mid-90s; it was rightly pointed out to me that Java only was released in ’95, so my memories on this were wrong