Resiliency: Defense is the Best Offense

Pete Cherecwich
Face Value
Published in
5 min readApr 16, 2024

And it’s everybody’s job to stay prepared…

Who remembers Y2K? Stay with me if you weren’t in the workforce at the start of the century!

Y2K — shorthand for Year 2000 — was an expected disaster waiting to happen when the year changed from 1999 to 2000. Computer systems and infrastructures were predicted to crash worldwide, your home VHS player would self-destruct (I know, “what’s a VHS player…”), and so on.

What actually happened when the clock ticked over at midnight on December 31, 1999, was minimal disruption. Which is probably just as well.

At the time, I was living in Tokyo. We went out as a family to the Hard Rock Café for dinner, and then I scooted home in anticipation of getting a call (on the land line of course!) about a problem. The phone never rang and I fell asleep on the couch — that’s how I saw in the new millennium! Very rock ‘n roll!

Some argue it went so smoothly because the vast majority of problems were corrected ahead of time with Y2K fixes embedded in operating systems and software updates. The opposing view is that there were very few critical problems to begin with and the potential problem was over-hyped. Personally, I’m glad enough people took it seriously that we’ll never know for sure.

Either way, my point is this: the industry knew it was coming. Task Forces were stood up long in advance of the anticipated problem, codes were written, solutions were developed. Today, we don’t have the luxury of a long runway.

So how do you plan for the unknown?

We’ve managed to weather some major crises in recent years: a pandemic forced businesses to set up remote operations on the fly; geopolitical conflicts upended supply chains; and a banking crisis was set into motion by a digital run on deposits, with everything unfolding in real time on social media. What might have been historically considered tail risk seems to be happening all too often!

As a result, building resiliency, or the ability to continue to operate under stressed conditions, has become increasingly important, as well as a critical focus area for our clients.

In our industry, resiliency has always been important. At our core, we are safekeeping and valuing assets, making sure pensioners get paid their retirement income on time, so questions regarding business continuity have been an ever-present in every RFP (Request For Proposal) that I can ever remember.

So, apart from the apparent increase in frequency of major crisis, why the heightened attention?

Well, while it has always been a volatile world, we are now more reliant than ever on technology to sustain both individual operations, and broader financial ecosystems at large. Technology has propelled us into a world where information is readily and instantaneously available at our fingertips, but it also raises the bar on how much we need to prepare for if and when the lights go out.

Off the top, here are three obvious vulnerabilities:

We can “just do it manually” is an option that gets harder and harder to execute. In our business, humans no longer have to spend countless hours manually typing and processing individual trades thanks to automation. Transactions are processed in milliseconds. If systems go down, trades wouldn’t settle (complete), and cash would not move. The result could be a crisis of confidence in the financial markets.

We all know why the cloud is a good thing — better data security, improved channeling of resources, and more flexibility to build and improve software. The benefits are undeniable. That means for most businesses — especially those in the financial services sector — the resiliency of your cloud strategy is another dimension that needs to be factored into continuity planning. This is something that did not need to be considered 20 years ago.

Not only that, but those risks may be less obvious — for example extreme heat or other climate events could knock out a cloud data center. Depending on your resiliency needs, that may mean ensuring you have a “back up cloud” ready to kick-in. (Think of a hot-hot traditional data center strategy but implemented in a cloud environment).

Additionally, you need to truly understand all interdependencies, and ensure you have a backup plan for those too. You could easily find that the evolution to a hybrid environment makes you super reliant on the cloud, even for your on-prem components and processes — for example, your software, applications and other parts of your infrastructure.

And of course, the cyber threat landscape is as prevalent as ever, with attacks becoming more sophisticated by the day. A recent Financial Times article quoted a Bank of England survey of UK market participants, stating that the risk of cyber-attacks is now deemed the number one systemic risk to the financial system; and, a warning from Lloyd’s of London, the world renowned insurance market, that a significant cyber-attack on a global payments system could cost the world economy $3.5tn.

And those are just three headlines. There are a multitude of other unforeseen crises that could take shape.

To help prepare for this, it’s critical we shift our collective mindset from one of defined emergency preparedness to one of continuous resilience planning.

Every business will be different, but this is how we are thinking about it:

Understand your critical services and their value chain. Since technology is the backbone of how we do our work, we need to understand every critical platform used to deliver the service and understand what to do if one goes down. The explosion of Software as a Service (SaaS) combined with increasing cyber threats makes this much more complicated. Could you operate if a vendor was out of service for two weeks because of a ransomware attack. Could the vendor operate if one of their vendors were out?

You must continuously plan and scope out scenarios that would “stress” the environment. There is no such thing as a single business continuity plan anymore. What happens if volumes quadrupled because of a market shock while at the same time a required system went down? Looking at idiosyncratic events and ensuring that the plans are still sufficient to provide service continuity is vitally important.

You must understand the second and third order impacts. A diverse set of people need to brainstorm in order to properly consider the ripple effects of a particular crisis. During a liquidity crisis, an operations expert would tend to think about the ability to continue processing transactions. A portfolio manager might consider the viability of a fund if there were too many redemptions, while a capital markets expert might consider the availability of collateral. All of these would need to be understood and reviewed to determine what other issues might arise as the event plays out.

You need to test the plans. All good plans must be tested of course but how you approach this is also important. Box ticking adds no value. Consider multiple tests throughout the year, call an emergency with no warning for your teams. In short, try and make the test as realistic as possible. You will learn from this and practice what needs to happen during a real crisis.

It may be impossible to know what next crisis might upend the way we work. But if we adopt a resiliency mindset — the act of constantly thinking, evaluating, and planning for change — we’ll be in a much better place to tackle the next transformative event.

--

--