Why Doesn’t Agile Work at Banks?

9 min readJan 22, 2024

Companies with a technology organization often look at the success of modern tech giants (Google, Meta, Microsoft, etc) and aim to emulate the successful software processes employed by these firms. One industry which is extremely reliant on technology and employs large software engineering and technology divisions is financial services. A few examples of firms that I’m talking about are:

Bulge Bracket Investment Banks (Goldman Sachs, Bank of America, etc)
Credit Card Payment Networks (Visa, Mastercard etc)
Hedge Funds (Citadel, Bridgewater Associates, etc)
Trading Exchanges (NY Stock Exchange, Chicago Mercantile Exchange, etc)

In this short read, I’m going to explore reasons why Agile Methodology may not always be the best choice for a software development process at a financial institution, despite being almost always prescribed.

What is Agile?

Agile is a project management and development approach which allows teams to respond efficiently to changing requirements. It emphasizes continuous delivery, flexibility, accountability, and the ability to pivot quickly. Some common ideas in Agile/Scrum which are easily implemented into teams at banks include:

Sprints- period of time (often 2 weeks) which represents a block of time to deliver a set of user stories
Common Meetings- Daily Standup, Sprint Planning, Sprint Review, Sprint Retrospective
Story Points- as an indicator of how much effort each user story represents, the idea of pointing stories to guage when features can be delivered, and to determine what items to put into sprints.

The methodology and intricacies of Agile have been talked about countless times and debated, although the above points are consistent for the most part and are usually the first changes when a team ‘goes Agile’. Here are some useful references on the topic which do a much more thorough job explaining Agile, if interested:

Why do Banks Turn to Agile?

The simple answer for why banks turn to Agile is because Agile works at the world’s most valuable companies, and helped them to develop innovative products that changed the world. However, there are other, more complex reasons which can help us understand the thought process:

Hiring Talent from Tech Giants - during my time as a software engineer at an investment bank, my team was actively trying to recruit engineers from the technology firms mentioned. Not only did our team gain talented engineers, but we also gained a new team member who had just come from what we saw as a much more modern, innovative company. Engineers with Google on their resume are often asked to talk about how they would approach a problem based off their experiences at Google, whether it be a technical problem, or a process improvement. As a result, the Agile methodology was often brought to financial firms by way of a new hire.
Agile Coaches and Consultants- The financial services industry is consistently one of the largest users of management consulting, which has created a culture of seeking help outside the firm from ‘experts’ to help improve a bottom line. Additionally, in recent years, many consultancy firms specializing in Agile, Technology Scaling, Scrum, and similar lines of business have spun up and seek to take advantage of the growing demand for their services. This doesn’t mean the prerogative of these consultants/coaches is to improve a specific team, but rather to migrate that team to an Agile way of working.
Agile and Scrum Certifications- In order to prove to clients that their team is qualified, or in an effort to prove to managers that an individual is qualified, many employees at banks are pursuing certifications within the Agile space. An example of this is the Certified Scrum Master (CSM) which gives a certificate to those wishing to fill the role of Scrum Master on a team, or just be a member of a scrum based team. While these certificates are great for understanding what Agile is, they tend to not address the question of whether or not Agile works for a specific team.
Achievement for Executives- In the world of financial services, climbing the management ladder is often a cutthroat game, with promotions for one coming at the expense of another. As a result, managers will look to highlight their impact to an organization at every opportunity. One way to do that is by migrating a team from the Waterfall approach to the Agile approach, which usually involves bringing a team of coaches in, getting team members certified, and then sharing a sprint board (usually Jira) with executives. This ‘going Agile’ item on a resume is not seen as a small one. Becoming Agile is often a lengthy and cumbersome process for software teams which requires strong buy-in and rollout; in other words, executives see the ability to convert a team to Agile as good leadership.

Notice that none of these reasons involve a process to determine if Agile is the right structure for a given team; it is the wonder-drug of software engineering apparently, which is prescribed as a cure-all to enable all teams to operate at Google levels of performance.

How are Projects at Banks Different than Products at Tech Companies?

In order to understand why Agile works at Google but not at Goldman, it is important to acknowledge the large differences in technology products at the companies.

Value to Company- At modern technology companies, the core of their business is often building and profiting off of software applications and tools. Comparatively, at investment banks, software teams often are considered back-office which means they will handle operational and admin functions. Some examples of this are: processing of transactions, data management, risk visualization, generate reports, etc. These actions are integral to the success and scaling of modern banks, however, these back-office teams are not the seen as rainmakers that their investment banking/trader front-office colleagues are; at an investment bank, a majority of revenue is generated through advisory, trading, fees and other non-application driven services. The takeaway here is that at tech companies, the engineers seen as revenue generating and key to the success of the company, while at banks the engineers are seen as a cost center.
Differences in End User Relationship- A majority of products that a tech company builds will eventually be used by customers outside of the company; for example, Microsoft Teams, Google Search, Instagram are all examples of B2B or B2C products. At banks, a majority of software projects will service internal users; an example here would be building a tool that consumes data from an internal post-trade processing system and outputs dashboards where a trader can view his portfolio’s risk metrics. One type of project/product is not always more or less complex, but the relationship of the engineering team to the end user is vastly different at banks than tech firms.
Project Objectives and Success- In tech companies, the success of a product is tied with user adoption, customer satisfaction, and the impact on a company’s position in the market. In contrast, projects at banks are judged on regulatory compliance, operational efficiency, risk management, and metrics centered around internal processes rather than user experience. The idea of ‘delighting the user’ is front of mind at any innovative technology company; this same idea is often an afterthought on back-office teams at an investment bank.
Regulatory Environment- Financial institutions are under the microscope more than any other institution, and as such the risk appetite for developing innovative tools and products is low. Conversely, tech firms are much less regulated, and often operate in areas that governments are not experts in (see here for Zuckerberg explaining the internet to Congress). This allows them to be more experimental and focus on creating value; conversely banks will focus on rigorous testing, stability and regulatory approvals.
Project Lifecycle- Technology companies tend to follow an iterative approach where they will deliver value to users, get feedback, and repeat. This feedback can come from surveys, metrics, or conversations with end users, and can be extremely fast paced (weekly, monthly, hourly). On the other hand, financial institutions will have a more structured roadmap approach, where a team will focus on delivering a large list of features over a longer period of time in order to give value to their business unit. This roadmap is seen as a bible, with all features on it needing to be delivered at the end; this makes the role of product/project managers to sequence features, rather than to prioritize them. There are less feedback sessions and conversations with users, and more rigorous testing and risk analyses.

Hopefully you are starting to see some of the reasons why Agile is not always the best prescribed method of delivering software, when the very nature of Agile is irrelevant on certain projects!

Why Doesn’t Agile Work?

In order for Agile to work, a product must have a need for the benefits of agile. These benefits at a high level include:

Flexibility and Adaptability.
Faster Time-to-Market.
Constant Customer Feedback Loops.

These are all things that are rarely a priority for the technology teams at financial institutions! Yes, the teams want to make sure that they are building software that matches the end users’ needs. Yes, the stakeholders want it to get done quickly. Yes, it is a positive to have the ability to be flexible as a product team. However, none of these things are a priority for technology teams at a bank. There is not a culture of prototyping and getting continuous integration, banks are usually not racing to get software out ahead of their competitors (its normally just internal), and the requirements are typically given at the beginning of a project, rather than being driven by feedback and metrics.

In fact, not only are these concerns not relevant for tech teams at banks, but by forcing Agile onto a team, it artificially creates the concerns. In a logical setting, the team would be worried about the things that their stakeholders should actually care about: is it being done right, is it being done in a risk averse method, is it compliant with regulation, and is the team working to get it done in a reasonable manner. By working in the Agile way, teams are accounting for problems that they will never encounter!

Why is this Bad?

Trying to force Agile on teams where it doesn’t work is like trying to fit a square peg into a round hole. Some of the negatives that end up happening as a result of these teams trying to force Agile are:

Time spent transitioning to Agile could be considered wasted time not building a product.
Limits a team’s effectiveness by putting too much emphasis on being Agile rather than other performance based metrics.
Time spent in Agile Meetings (refinements, retros, plannings) might better be spent developing if the roadmap is extremely clear.

While Agile is great for building a product that has dynamically changing needs and priorities, if the roadmap is set in stone, one could argue that it doesn’t make sense to do many things the Agile way. A team could simply work on features, still in a collaborative way, without the extra headache of pointing tickets, extra meetings, and the pressure of committing to a set amount of work over a sprint.

Another impact is that the Agile coaches, certifications, and often executives hired from tech firms make the assumption that a team is operating in a product driven model. Instead of accounting for the fact that a team might be roadmap driven, the Agile method is prescribed as if a product, team, and goals are that of a big tech organization. “If we try to play like the Yankees in here, we will lose to the Yankees out there” is a quote from Moneyball, where Billy Bean/Brad Pitt acknowledges that the constraints and challenges that his organization faces are different than than the challenges that the Yankees face. It may be time for banks to admit that they are NOT the Yankees of technology products, and must adapt and apply different strategies than Big Tech in order to have impact in the technology world.

What Usually Ends Up Happening?

The most common approach to a pure Agile team structure is a Waterfall approach, which is an approach to software where each phase must be completed before moving onto the next. Typically there will be a fixed scope, extremely clear requirement specifications and documentation, and a set roadmap, all of which are defined and agreed upon at the start of the project. User involvement is limited, which leads to higher chance for surprises during a release or demo; customer feedback is primarily only gathered at the start and end of Waterfall projects.

Another alternative, which is the most common outcome, is for teams to accept a modified version of Agile. What this looks like varies on a team-by-team basis, but typically involves incorporating elements from Agile + Waterfall and looks something like:

Fixed Roadmap defined at start of project
Classic Agile meetings + sprint structure
Demoes internally to project managers/business analysts on a frequent basis
Demoes to business stakeholders (end users) on an infrequent basis
Limited pivoting mid-project, testing done at the end of project

Have I just described the way that your team operates? If so, it may be worth asking if more value could be delivered through going all-in on either the Agile or Waterfall method. Or the question that I’m very curious about….is there another alternative method yet to be discovered which is designed with these teams in mind?