Make Technology Decisions in 5 Easy Steps

Making technology decisions can be difficult.

If you’re a software company today, you have seemingly endless options for every decision you make:

  • Programming languages
  • Server-side framework
  • Front-end framework
  • Database
  • Infrastructure provider
  • Code repository host
  • Continuous integration host
  • Bug tracking software
  • DevOps tools

How can you be sure you’re making the best decisions? Also, how can you avoid endless internal debates for each topic?

Below are the five steps I use when making an important technology decision.

1. Evaluate Comprehensively

A technology department doesn’t exist in a vacuum. It exists to deliver a great product that can make a profit today and in the future. That means it’s required to take a holistic view of how that technology decision will impact hiring, systems administration, and more.

The best way to ensure you are thinking comprehensively is to use a decision matrix.

Here are the steps to follow when creating a decision matrix:

  1. List everything that can factor into the decision.
  2. Weight each factor on a scale from 1 to 5 (1 = not important, 5 = very important)
  3. Add a column for every option being evaluated.
  4. For each option, rate it on a scale from 1 to 5 on how well it performs for that factor (1 = not well, 5 = excellent).
  5. Calculate the sum-product for each option (SUM(weight * performance)) to generate a score for each option.
  6. Compare the scores for each option. The highest score wins.

The beauty is that it takes one gigantic decision and breaks it into many smaller evaluations that are easier to decide and discuss rationally as a team.

Recently at RVshare, we had to decide what language and framework to use for our new API. We were at a turning point where the MVP product had been built in PHP with some Ruby scripts used for background processing. We were open to any option for the new code base going forward.

We considered many options and narrowed it down to Ruby on Rails and Python/Django. Below is the output of our decision matrix comparing Ruby on Rails and Django in detail:

First, I can guarantee that you will dispute at least a few of these numbers. That’s fine. This table represents our own engineering team’s collective evaluation for our specific use case. Also, individual opinions within the team varied.

At the end of the day, we had to make a decision based on many factors. So while it may not have been perfect, at least we were honest and transparent with ourselves with our evaluation.

The entire discussion took the better part of one day. There were strong debates at times, but we walked away in total agreement with our decision. We also had the feeling that we left no stone unturned.

So that’s the framework and the results. But where do we get those numbers?

2. Quantify

It’s easy for technology discussions to stay in the realm of opinion. However, whenever possible, actual data should be used as a guide. Things like language performance and framework popularity can be quantified.

Below are a few resources that you can use in your discussion for evaluating programming languages and frameworks:

  • State of JavaScript Survey — See which frameworks have the most momentum today.
  • Meetup Membership Numbers — In your local area, compare the number of people that have joined each language Meetup group.
  • Hacker News Who’s Hiring thread — Do a quick Ctl+F on the latest Hacker News who’s hiring thread for that month. It can serve as a rough estimate of hiring trends for companies.
  • Google Trends — See how many people are searching for a given language / framework.
  • Github — Gauge popularity by the number of stars, forks, unresolved tickets, and last commit date for a project.
  • Web Framework Shootout — Actual performance tests for many languages and frameworks.

3. Develop a Taste for Technology

When there is no way to gather data on a subject, you will need to rely on personal opinions. Here is where having good taste is important. The same way someone can have good taste in design, art, or food, a person can develop good taste in technologies.

The way to develop a taste for technology is to use a lot of different ones. If you are a one-language developer, you haven’t had an opportunity to develop a good taste in technology.

Technology leaders need to stay hands-on and actually use the technologies they are evaluating, even for a small test project. Don’t commit your company to a technology that you haven’t used yourself.

4. Discuss as a Group

It’s critical that the decision matrix be used as a template to guide the group decision. Timebox each discussion cell, but the key stakeholders should have a say in each step of the process. The group should discuss both the weighting factor and the performance of each option.

Not everyone has to agree, but they need to have their voice heard. By using a decision matrix, the discussion can stay focused and avoid bikeshedding.

5. Document the Decision

When you’re done, put the decision result into your internal company wiki. This is important because you should occasionally go back and evaluate your own decision-making process. Also, new hires will want to know why decisions were made. It helps solidify and communicate your company values.

Conclusion

There are many decisions to make in the lifecycle of a software engineering department. I hope that this guide will help you with your next major decision.