Building Your Testing Skills Makes You A Better Software Developer

I’ve been a software engineer for over 15 years and I’ve spent 10 of those years working at the Internet Corporation for Assigned Names and Numbers (ICANN) where I am Vice President of Software Engineering. Over the course of my career, key experiences have influenced and grounded my approach to software quality assurance and testing. This approach has formed my core principles and we act on these principles daily at ICANN. Given the criticality of web applications in our everyday lives, this blog outlines the rationale for why test automation is a vital part of software development and why software developers should develop their testing skills.

My Early Influences

One of my first jobs as a software developer was with a startup. We were tasked with developing a large, enterprise application to manage IP address networks. Despite the fact that we meticulously reviewed the code and spent hours analyzing each and every line, we constantly worried about making changes to the code for fear of breaking the code in another part of the application.

So, how do you enable developers to have the creativity and freedom to improve their code without unwittingly unleashing more bugs elsewhere? Two words — testing strategy.

While I was still in the early stages of my career, the CTO at another company I worked for was a big proponent of Extreme Programming (XP). In XP, you write your acceptance tests (aka, Test Driven Development or TDD) before you code the application. TDD helps ensure that you are coding the application in a way that will meet the acceptance criteria.

Testing as a Safety Net

Developing a good testing strategy is vital. If you don’t have a good testing strategy, you won’t know the impact your changes will have in other parts of the code. Having a good testing and quality assurance strategy brings far more benefits than the time you’ll spend creating and implementing that strategy. Spending the time at the start of the project will reduce the time (and company money) you will need to spend fixing bugs later on.

While time and cost savings are tangible benefits, there are also the intangible benefits that you and your teams will earn. With a robust testing mechanism, you’ll give yourself and your team the confidence, freedom, and room for experimentation to improve their code.

Think of testing as a safety net. When a trapeze artist is learning how to perform acrobatic aerial routines, there is a soft net between the artist and the pavement. As with the net for the trapeze artist, a test-net gives developers the courage to consistently improve the software and push the limits. This confidence also extends beyond initial releases as it gives developers the ability to refactor the code to make it more efficient, or experiment with new features.

While there is the often-used phrase, if it ain’t broke, don’t fix it, I don’t subscribe to that particular maxim. It’s not a good way to manage your software.

Thinking about the test before you begin coding helps you design what you want to build and improves the implementation of your code. The test helps you understand the problem you are trying to solve in a better way. You’re not just writing tests for the sake of writing test procedures, you’re integrating the test into your development activity — a test-driven approach to development. And code written by a developer using test-driven development is much cleaner and more readable.

This focus on automated testing opened my eyes to the benefits of testing early and often, and testing has always been part of my process. I write code and unit tests to validate. Writing tests doesn’t make you any less of a software developer — it makes you a better programmer.

About ICANN

As I mentioned earlier, I work at ICANN. For those of you unfamiliar with ICANN, I’ll give you a brief background. ICANN’s mission is to help ensure a stable, secure, and unified global Internet. To reach another person on the Internet, you need to type an address — a name or a number — into your computer or other device. That address must be unique so computers know where to find each other. ICANN helps coordinate and support these unique identifiers across the world. ICANN was formed in 1998 as a nonprofit public benefit corporation with a community of participants from all over the world.

To support ICANN’s mission, the Engineering & IT team, wherein I work, develops, deploys, and maintains custom applications and software development infrastructure needed to meet the unique and global needs of the organization.

Our Approach to Testing — Testing Pyramid

At ICANN, we follow a testing pyramid — prioritizing unit tests early, adding integration tests second, and finally a smaller set of critical UI and end-to-end (E2E) tests at the top of the pyramid. Unit tests provide the fastest feedback and comprise the bulk of our tests. The higher you go up in the pyramid, the longer the tests take to execute because these tests involve more of the system you are coding for.

If you are implementing a story or fixing a bug, and run an automated test that takes two hours, you will move on to something else. If the build fails, you waste more time because you have to switch context, get back into the previous story to debug it, wasting even more time.

Fast feedback (enabled with unit tests) is excellent for catching issues early, while slow feedback introduces inefficiencies due to context switching. As mentioned, unit testing is the fastest test because you are testing a smaller part of your code. This way, you receive immediate feedback. If you break things down into smaller components, it is easier to test and subsequently easier to fix, as opposed to integration and E2E testing. The more unit testing you perform, the less testing you’ll need to perform further up the pyramid. As I mentioned, you want to reduce the number of tests you need to perform further up the pyramid, as these take more time and are more complex. Increasing unit testing allows you to do that.

That said, E2E tests are critical to application quality. If a user can’t access the feature, they aren’t much use. However, E2E automation can be more trouble than it’s worth. At ICANN, we are pragmatic about our use of automation. If the tests take too long to write or run, they aren’t serving the goal of fast feedback. Neither are flaky, failing E2E tests that require troubleshooting, context switching, and rerunning. E2E tests should provide a line of defense against bug escapes, which boosts the confidence that regressions will be caught.

Quality Assurance (QA)

We also have a specific philosophy about QA. QA shouldn’t have a goal of preventing the release of software that has bugs. The goal should be to ensure that the team has a good understanding of the quality of the software and the acceptance criteria. We ensure that the development and QA teams work together to improve the quality of the code and release faster. Similar to Toyota’s manufacturing process, you want everyone to have the ability to stop the production line if they catch a problem. You don’t want to go back to the beginning to get things fixed. You want quality built-in from the start. Quality is not something that is added at the end.

The Transition from Selenium to Testim

At ICANN, we used Selenium for our functional UI and E2E tests, and we wrote a lot of wrapper code to improve it. Tests took hours to write and even more effort to maintain, meaning this testing infrastructure was a considerable investment. And with browsers changing on a very regular basis, it was hard to keep up.

So, we decided to try another approach.

Testim.io is our functional UI and E2E testing mechanism. It is a low-code test automation platform that leverages AI to help with the authoring and stability of UI tests. You may be wondering why a developer would opt for a low-code test automation platform.

We have a team of developers and automation engineers who perform testing at ICANN, so we wanted a team solution. We also recognized that the faster we can create tests and the less time we spend maintaining them, the more time we have to work on our applications. Developers shouldn’t need to prove their coding skills by writing UI tests.

Why Testim?

We feel that there are several advantages to Testim over other testing platforms:

1. The process to create and organize tests, and share reusable components is straightforward. Testim organizes tests into building blocks that can be combined into groups to help you follow coding best practices. Even though you don’t write coded tests in Testim, you can use the same clean coding principles.
2. Testim has a method of identifying duplicate test steps and replacing them with shared groups (like coding methods) for lower test maintenance.
3. Testim includes a managed cross-browser grid. We are a smaller organization and didn’t want to add resources to support a grid infrastructure.
4. We wanted to ensure our tests are clean and easy to understand. If you need to run a debugger to figure out why it failed, it’s taking too much time. Testim’s failed step indicators and screenshot comparisons quickly identify the source of the problem. If needed, an additional console and network logs are available.
5. The network mocks accessibility testing and code review/pull requests are all fairly unique to Testim, giving us a lot of flexibility to improve our processes over time.

When we evaluated Testim and compared it with Selenium, the reduced level of effort (LOE) was significant. Tests that took two weeks to create in Selenium took two days with Testim. Additionally, the amount of experience we needed with Testim was considerably less. Overall, Testim makes it easier to manage your tests, you get better tests, it’s not as fragile, and is more resilient to changes.

If Industry statistics are accurate, in general, developers spend 70 percent of their time fixing existing code, not writing new code. That is not ideal. We want to increase the amount of time we dedicate to writing and improving new code and by utilizing Testim, we’re able to create good tests that we can read and understand, which minimizes the amount of time we spend on bugs.

Our Testing Implementation

Rather than a lift and shift approach to replacing all Selenium tests with Testim, at ICANN, we’re taking a targeted approach and are evaluating whether we really need to replace existing Selenium tests. Too often, we duplicate functional tests where the same component is used and tested on multiple pages. This involves a mindset shift from “I’m measured by the number of tests that I write” to “Have we adequately tested this component?”

We have a few applications that we are testing using Testim. One is built and hosted in our internal infrastructure, another we constructed on Salesforce’s service cloud. We have a set of functional UI / E2E tests written on Testim that we run in Continuous Integration (CI) on each deployment for user acceptance testing.

At first, not everyone on our team was on board with the decision to go with a low code platform. Some of our automation engineers wanted to continue using Selenium and advance their coding skills. But because of the time-savings we gained using Testim, we freed up time to focus on more difficult testing challenges. And because Testim can run JavaScript in test steps, the automation engineers could continue to write code, only now they are focused on more exciting problems.

Takeaway on Testing

You need to build a testing mindset as a developer. TDD and XP are great ways to help train your testing muscle and translate it into writing code that is easier to test and meets acceptance criteria. It also means that your test code is as critical as your application code. It should be clean and easy to debug. The same principles apply to low code UI testing. Your tests need to be understandable, debuggable, and should reuse components to minimize maintenance. We believe that in Testim, we’ve found a tool that works well for both developers and QA.

Testing is not just an activity, it’s a mindset and a process that requires continuous improvement. You want to learn from your experience, adjust, and do better the next time. Every iteration should realize value — in experience, time, and better code. That’s the journey we’re on — constantly evaluating and improving our development and testing processes.

My goal is to write stable tests faster, so I have more time to write application code. This means that, in many ways, testing code can be more important than writing application code.

Your productivity as a developer will only increase when you spend less time rewriting or debugging your application code and maintaining your test code, and when you focus on what is critical — improving your application code.