How Do I Choose the Right Programming Language for My Team?
Lessons from the leadership trenches
This series will allow me to share some history and experiences with you. I’ve been in and out of the development trenches for over 20 years. In my current role as a Distinguished Engineer at Capital One, I get to move up and down the gradients between hands-on tactical development and strategy work. My knowledge is fresh and pretty wide and I want to share with you some thoughts and observations to help you make decisions from a well informed platform.
Today, I wanted to do a writeup on tech leadership and helping your teams choose the right programming language for a given problem. In an imaginary scenario, let’s say we have to build a new RESTful API to manage access to a relational database. We have the ability to write this application in almost any modern programming language. Some teams may pick Java, some may choose Go, others may like Python and some diehards may want to do it in C (for the performance!). Let’s walk through how we can rationalize a decision on which to choose.
Full Freedom vs Prescribed Usage
A lot has changed in the past 20 years of software development; the ease of software development is not one of them. Today, we have a huge array of tools available in our tool belts. However…
“With great power, comes great responsibility.” — Uncle Ben
So where does one begin when picking the right programming language for a new web app? Just do a Google search for “open source web application framework” and you’ll see dozens, in every language. So how do you decide which to use? Especially when the decision between using one you know and one you can learn can have such high stake impacts?
Let’s say you go with a Python Flask application for an API, great! Did you consider that now your team will need Python backend developers and JavaScript frontend developers? This isn’t terrible since it’s a pretty traditional team setup, but would you consider building a backend in JavaScript / Node.JS?
“Back in my day, I had one viable language per layer of my stack… and I was happy with it!” — a sentiment shared by many a wizened old developer
I’ve worked in a range of organizations. Each of which has a different approach to choosing which programming language to use for projects. My goal with this article is to help you, a leader in your organization, to take a stance when you’re presented with the opportunity to guide which language your teams use. Let’s work through some examples on choosing the right programming language. We’ll work through the spectrum of high learning curve and efficient languages (like C) and compare it to a lower learning curve yet potentially inefficient language (like Python).
As we consider this spectrum, leaders should consider striving for balance, taking all aspects of the job into consideration.
Standard Development Language
Let’s start with languages that your programmers might write in. If you’re a small shop, you’ll most likely have a single developer or maybe a team made up of specialized individuals who are building your application(s). The larger you get, the more likely you are to have a large number of development teams solving a plethora of problems.
“ Jefe, what is a plethora? “ — El Guapo, Three Amigos
The rest of the article, let’s talk from the standpoint of enterprise development or a large software shop. My general process will be assuming your corporate environment looks like most large companies I’ve worked at in my 20 years in tech — there are a bunch of development teams, teams have tech leads, and teams are assigned a feature or two to work on.
Let me use a real example — The group I support at Capital One sends messages across digital channels for our customers. We send billions of messages every year. As you can imagine, it’s a pretty large pipeline of data processing microservices. We have regulations to abide by, customer preferences to respect, and other hygiene-oriented activities to account for in order to send a message. A simplified version of the pipeline looks something like this:
With this pipeline, whether it’s a series of microservices being served across Kafka/RabbitMQ ( Stateless and Dumb Microservices on a Message Bus) or a chained series of APIs, there are a bunch of components in this process that have to execute in order to deliver a message.
In typical enterprise fashion, we have lots of teams working on this process, as well as other teams that our platform supports. Naturally, teams will ‘own’ certain components, but we fully support and encourage an open-source / innersourcing model for development. Also, anyone can work on anything with the proper code reviews and pull requests to track/isolate changes.
BYOL! (bring your own language)
Now, let me use an imaginary example. My teams have built a new platform that follows a similar data pipeline but operates with some developer freedom around programming languages and frameworks. In this platform, the engineers can choose Scala over Java, or Go over Python. Each team works with their strengths and decides which language or frameworks to write their applications in. It looks something like the below.
Look at that! It’s a beautiful mixture of specialized tools and code that makes the magic happen. This is great!
… until the day it isn’t.
Like the day that Team Brothers, the team cleansing data in the ‘hygiene’ step in our imaginary scenario, all win the lottery, put in their notices, and retire early. No biggy. It’s just another component to the pipeline, right? They built their step using Scala, Spark, and Kafka but I’m sure the Java engineers on the other four teams can pick it up overnight. Heck, Scala can’t be that different, right? A JVM language is a JVM language. They’ll both compile!
Wrong. In this scenario we’ll start to see productivity take a nosedive. The performance of the Scala application will begin to suffer. We won’t be able to integrate new features as quickly as we used to. And eventually, unless we hire some Scala engineers, we’ll need to rewrite it completely.
In this imaginary example, was Scala the right language for the job?
… it was.. until it wasn’t.
The data pipeline design was good, Scala was perfect for the workload and Spark Streaming was able to handle the throughput with huge reference files. The implementation was spot on but the issues with specialized languages shows up as it affects maintainability.
In our imaginary example, the data cleansing team won the lottery and quit and no one on the remaining teams knew Scala. If there were any issues with the function, someone would have to learn enough Scala to be able to update the application. If that wasn’t feasible — since larger updates are riskier if you aren’t familiar with the way Scala/Spark work — then you’d be stuck rewriting it entirely. You might think you can just hire more engineers to cover the niche languages you have to support, but as I’ll show below, that may be harder than you expect if you’re not using a mainstream/standard development language.
So How Do We Pick the Right Language?
Look across your organization, ask yourself some questions:
“If you’re part of a team, can the whole team support the language?”
- If not, it may not be the right answer. I always worry about the team winning the lottery and deciding not to show up tomorrow.
“If you’re part of a large program, can other teams in the program support the language?”
- If it’s the right language for the job, what kind of investment is needed to train other teams on how to maintain the application.
“Why are you using the ‘new’ language?”
- If you’re using it to learn, awesome! But find the right sized task to try a new language and forego using it for mission critical components until the teams are fully up to speed.
“Does the use case lean towards a specific language construct (functional, procedural, OO)?”
- Streaming data may lean towards procedural while ‘business’ applications such as a web app may benefit from an object oriented language.
“How easy will it be to hire new developers to work on this language?”
- When your star developer wins the lottery will it be easy to replace her with someone in the open market?
Look at the chart below. TIOBE puts together a list of the most popular programming languages each year.
For a language to make it to the top of this list it’s probably fair to assume there are lots of developers using the language. This would make it easy to flex developers across tasks, as well as allow you to hire new developers when your teams grow or replace folks. Popularity is one metric that can be used to pick a programming language, but which ones are best suited for which kinds of projects? The below is by no means prescripted to everyone, but it’s a place to start when thinking about your language choices.
Web Applications
Java — Spring Boot
Python — Django
Javascript — Node.js
REST APIs
Java — Spring Boot
Python — Flask
Javascript — Node.js
Streaming Data Microservices
Python — pySpark
Scala — Spark
Data Analysis
Python — SciPy, Pandas
Java — MLlib, H20
Desktop Applications
C# — UWP, WPF
Swift — Cocoa
Mobile
Swift — iOS
Kotlin — Android
Conclusion
Modern software development is complex, especially in the era of open source development and developer freedom. Software developers tend to suffer from shiny object syndrome, where we want to try something new and cutting edge when we’re building a new project. Sometimes those motivations are good (right tool, right job) and sometimes they may be driven by more short term perceived benefits. It’s up to us, the leaders of our organizations, to help our teams balance out their decisions.
By no means am I advocating that you force a one-size-fits-all technical solution or programming language on your team. Instead, I hope this article helps you realize that there are pros and cons that need to be considered when we’re building software; especially software that will hopefully have a very long, productive life cycle.
DISCLOSURE STATEMENT: © 2020 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.
