Picking Technology Stacks
What should we consider when choosing tools?
When developers are free to choose a technology, they have two or three choices over their career in an organization.
They can favor their own interests, prioritize the company’s interests, or choose a combination of both. Needless to say, the third option is ideal — but let’s unpack this slowly.
It’s a problem common in micro-service based architectures, which require constant re-evaluation of the technology stack. In monolithic architectures, there are fewer choices for developers to make.
Needless complexity
The first option is acting in their own interest. Developers may be somewhat egoistic in choosing technology stacks (languages, libraries, frameworks) based on theoretical use-cases that are more complicated than necessary.
They may do this for the benefit of career growth, learning, and a broader sense of cognitive challenge. After the project is complete, they might switch jobs with a realistic possibility of negotiating a raise, as their freshly gathered knowledge may be valuable for other organizations that are moving to new technologies in which they lack expertise.
In these cases, developers might consciously or subconsciously think of exotic technological choices as some sort of job protection. If they create a series of applications in exotic languages and frameworks, the management team may feel obligated to keep them.
Developers of this sort might try to renegotiate the terms of their employment and even threaten to leave if they don’t get a raise. Morally this is very dubious, but not everyone is as loyal to the companies they work for as they are to themselves. They may doubt whether the company will be loyal to them should hard times arrive. From an organizational point of view, such needless complexity can be very expensive. Why should technical leadership hire expensive experts that are hard to find, when simpler technologies could be maintained for much less?
Keep it simple stupid
The second option is acting in the company’s interest. Responsible developers will choose technologies that are just enough to meet the requirements of a particular use case. This option is better for the organization, as the resulting technology stack is easier to maintain and can be tended to by less senior members. (Seniority aside, some languages are also easier to read than others).
Giving complex technology stacks to junior developers (or those who are inexperienced in that technology) may be viewed as a management or organizational flaw rather than any particular developer’s responsibility — especially when new team members have to implement features, or refactor the code without the support of the person who introduced the complicated stack in the first place.
While some developers may welcome the challenge of learning new technologies, it can also slow them down. There needs to be a shared understanding that developers who are inexperienced in a certain technology will be slower to contribute to a project as they have to learn it first. This approach offers growth opportunities at the expense of project velocity. On a positive note, once developers learn the technology stack better, they may not only like it but also apply it in other contexts as part of new projects. If this technology is part of a shared architectural goal, the investment may well be worth the initial ramp-up time.
People are basically good
Here at eBay, we believe people are basically good. Developers may observe that certain technologies are limiting progress and strive to gradually replace them. In choosing new tools, they seek to position the organization towards the future, allowing teams to find the most driven and passionate developers that will help them ship state-of-the-art applications, and deliver value to the business.
When developers begin experimenting with new technologies, they may not yet realize that the new approach could lead to a paradigm shift. However, when they recognize that potential after initial experiments, it’s important to communicate this to leaders before introducing any fundamental changes to production. If this communication is missing, it becomes somebody else’s problem when teams and people are reshuffled for new strategic initiatives.
Scrutiny is painful
If leaders are unsure of developers’ intentions, it may be difficult for them to evaluate the decisions developers make and decipher the real motivations for technology choices. Are developers acting in the interest of the organisation or pursuing their own interests?
What often tends to happen is that the arrival of new technologies brings fundamental shifts of paradigms. It is not always the case that developers choose new technology for fun (to learn another programming language, for example, or because everybody’s been talking about it at conferences). In many cases, the current solutions that an organization has may be superseded by equivalents that are genuinely superior.
In this context, superior may mean solving problems cheaper, with better external support, better performance and/or a reduction of time required to develop a feature.
Recent paradigm shifts
The recent past offers countless examples of new technologies that brought paradigm shifts to application development, including Node.js, React, Git and GitHub, Docker, Kubernetes and the Helm package manager, and more recently Kafka.
Reorgs and their hidden costs
Companies are commonly reorganized to align with new business objectives. In these cases, new teams may be formed and others dissolved. People will move around. If a tech organization has too many technologies and stacks to maintain, it becomes difficult to reorganize everyone while taking both business constraints and technical constraints into account.
Everything is about the business
For micro-service organizations, reorgs can hit technology people hard. After a few rounds of restructuring, what used to be a responsibility of one team — “you built it you run it” — tends to become someone else’s problem.
Developers unhappy with earlier technology stack decisions may ask team leads for time to rewrite the use-case to a technology stack which they are familiar with. This can put team leads in a hard position as they may not be able to pitch for time for such rewrites. This sort of technical debt is part of the hidden cost of business reorganizations.
While different technology stacks tend to increase the probability of dissatisfaction when teams inherit tools they wouldn’t have chosen themselves, it can also happen that a team may not be happy with the quality of a product even in the same technology stack.
Unit or functional tests may be missing, code style can be far from ideal, etc. In many cases, the business domain of newly inherited applications may take time to learn. Something as trivial as fixing a production bug may take weeks due to missing knowledge.
You wanna’ be a leader? Go ahead but wait…
I don’t envy leaders that have to evaluate intentions and assess their options. Leaders aren’t fortune tellers — how would they know when the long term direction in which the business is moving and the next reorg will completely dissolve teams and create new ones? Which new technologies are really necessary for the organization? Making decisions with missing information can be very hard and usually leaders should strive to minimize human dissatisfaction.
But a “free for all” won’t work either.
In some companies, developers are allowed to pick any technology or programming language they want to use. But after time it becomes apparent that this doesn’t work very well and tech leadership needs to limit the number of languages employees use. If each developer picks their own stack, it becomes increasingly difficult to support projects when people leave — and sooner or later, people will leave.
Companies like Google have recognized this challenge, and taken a much more conserve approach to technology proliferation. They have only a handful of languages — and the process of introducing a new language is not easy. Even the Scala programming language was not accepted at Google early on. Clearly they have processes for deciding when to introduce a new programming language.
It’s hard to identify paradigm shifts
I have to confess that despite being in IT for many years and doing my best to stay up-to-date, I was not always able to correctly identify certain paradigm shifts. I am not alone in this regard. Many of us will fail to recognize the fundamental changes in the technology around us the first time we see it.
At first, some new technologies that other developers were introducing in other parts of the organization (or even in our own team) seemed like unnecessary complication. But after a few months — or even sometimes years — I could see that these changes were among the necessary and liberating technological paradigm shifts that took us to a new level. Each new technology fixed some problems but brought also some limitations or even problems.
In several cases, I found myself a “convert” when I “saw the light” and started to promote what I had initially seen as a needless complexity.
I have also experienced cases where unnecessary complexity was introduced where the technology stack or code could have been kept simpler. I saw many cases in which other developers struggled to understand technology that was foreign to them and way too complicated for the use case at hand.
Seemingly innocent stack choices by previous contributors took them way longer to deliver any tangible business value than would have been otherwise necessary. What I consider to be success is when teams pick technology that is so understandable, well-documented, easy to start locally and debuggable that other teams can easily contribute by opening pull requests for review.
Complicated technologies hinder external contributions.
It’s not obvious — an open end
What do you think? How much freedom should leaders give to developers? Should they limit the technologies developers can introduce? Where should they draw the line? How can leaders identify whether a new technology that developers want to introduce is worth it for an organization?
My opinion is that an organization should develop a clear process on how to introduce new technologies. But the devil is in the details — it’s important to be aware of the pros & cons before allowing or blocking any new technology.
