9 Things I Wish I Knew About Automated Software Testing Before I Started My Career

A significant part of workplace success is defined by the quality of work you produce. There are many ways to do this in software engineering but one easy way is to start out strong and become familiar with testing because producing quality code is much more important than producing a large quantity of unmaintainable and broken code. So, while testing itself is key to high quality code, testing automation is a revolution in rapid and efficient software development.

I’ve been working as a software engineer for over four years and have used testing automation to accompany my features’ development in various forms since day one. Along the way, I’ve picked up more than a little experience that I wish I’d known prior to working in industry, so here are my top 9 learnings I’d like to share with others who are just starting out in the technology world.

1. Practice Writing Unit Tests

First, truly, truly understand the definition of a unit test by reading blogs and looking at examples of unit tests in well-written software. For reference, a unit test describes a test for the smallest building block of software, usually a method or function. Here are some tips to get started with unit tests:

• Though often forgotten, remember that unit tests document your code. Therefore, make sure you make the methods descriptive so future engineers will understand the underlying business logic and your thought process.
• Also, make sure you’re checking expected values and expected code flows within your tests. Don’t just call a function without validating whether or not the expected behavior occurs.
• Some common examples of unit testing frameworks are JUnit for Java, Jasmine for Angular, and Jest for React.

The importance of making your test names descriptive is demonstrated in the code snippet above. The method “calculate_value” is extremely vague, but the final test gives you a better understanding of its expected behavior. If you’re ever tasked with working on a legacy codebase, you might come across this situation often.

2. Understand the Testing Pyramid

There are many ways to perform testing (through unit tests, integration tests, end-to-end tests, etc.) and each company or team will use different adjectives to describe similar concepts. You’ll save yourself a lot of confusion by getting familiar with the different types of testing strategies common in the industry and understanding where each one is needed in the software cycle. Then, learn how they are used in your company and on your team. You might be able to persuade other engineers to use a more standardized form of testing.

The image above demonstrates one version of the testing pyramid. While you’ll see many differing “testing pyramid” philosophies, the main idea is the same. For instance, the base of the pyramid represents having the most test coverage: this will always be the unit tests. These tests should be designed to run fast and cover every scenario within your functions so developers can have more confidence when they make changes or add new code. In the middle is integration testing which allows your small components to be tested when they are used together. This helps validate that your code has adhered to the proper API contract and allows you to see if you missed handling valid inputs. At the top of your pyramid, you have end-to-end testing (E2E testing). Typically for customer-facing applications, these tests are run through some GUI like a web browser or a mobile phone simulator. These tests tend to be slower and more brittle, so in this round of testing you want to avoid testing scenarios that can be easily handled in a unit test. This will ensure proper test coverage, but it will also prevent you from having to rework large portions of your framework when the GUI or user-facing application layer changes.

A concrete example of the testing period for a simple calculator might look something like the diagram above. Here, the unit tests are covering the mathematical functionality, such as the multiplication of different input values. Next, the integration tests cover the interaction between classes, assuming that there is a class that holds the memory of the previous calculation and you’re validating that the correct value is sent across to the mathematical processing class. You won’t need to re-check that two negative numbers being multiplied together equals a positive number, but you will need to validate that the correct number is being passed correctly between components. Lastly, the end-to-end tests cover the specific overarching GUI flows, such as the user entering different values on the screen and receiving a valid response styled appropriately.

3. If You Fix a Bug, Write an Automated Test to Catch it Next Time

This is key! When you fix a bug, make sure to write an automated test that will fail if the bug is ever reintroduced. This prevents the bug from ever reoccurring. Sometimes developers will fix a bug and think their work is complete, but how will the next engineer know to avoid making the same mistake? If you write a test for it, you’ll never have another engineer cry over the same issue.

In the above images, it looks like there’s a bug in our code: “calculate_value” doesn’t handle decimal values correctly (top). Let’s write a test (middle) that catches this mistake to make sure it doesn’t happen again and validate that it now runs successfully (bottom).

4. Defend Your Reasoning for Removing Tests

Refactoring tests is part of the job and a good way to drive confidence in testing quality. Many platforms end up having redundant, vague, and meaningless tests, all of which slow down development time. If you come across some of these, take the liberty to refactor and make the test suite better than you found it. However, if you remove tests, make sure to validate that those scenarios are being covered elsewhere in the testing suite. Additionally, best practice is to always go through a code review with your engineer peers to make sure you’re not missing anything.

5. Use Mocked Data to Your Advantage in Unit Tests

Mocked data within tests are designated objects set to have specific properties so you can test various scenarios that you wouldn’t be able to directly access otherwise. In fact, there’s a world of great tests you can write using mocked data, so don’t just get stuck in an “assert-happy-path-works-as-expected” route. Try other scenarios such as verifying a method is (or isn’t!) called or that a specific exception is thrown. For these situations, you’ll need to make sure to condition your mocked data properly so these various edge and exception cases are triggered.

6. Mock Integration Test Data (Sometimes)

There are times you’re going to write code when downstream systems are experiencing stability issues and aren’t available. Other times, you might be waiting for an API you plan to consume to be finished being built by another team or developer. Even more often, there may be an API response that is nearly impossible to generate but must be accounted for by your part of the system. Whatever the case may be, don’t let API unavailability hinder your progress! Use mocking frameworks to mock out API calls so you can continue your development as if the real API is in place. Wiremock is one great tool to mock out API requests that your system relies on so you can keep developing your piece in the system.

7. Don’t Mock Integration Test Data (Sometimes)

While there are times you’re going to want to use your mocked data, mocked data is never a substitute for real data! API contracts change, miscommunications happen, and real systems can provide additional details a mock might not be aware of. Don’t let that catch you off guard when you move into production-ready code, so consider spot testing the integration of systems with real data; and always, always use real data before you sign off on your initial development.

8. Automate Everything

Any piece of code that isn’t covered by an automated test is already legacy code. To properly ensure expected behavior, you’ll need someone to manually test your non-automated features. This puts a huge drain on resources, and, let’s be real: adding test coverage almost always takes a backseat to product intent or other tech initiatives, so make maintainability and scalability easier by automating functional tests upfront. Even better — automate your automatic tests!

9. Line Coverage Only Tells One Side of the Story.

There are so many ways and metrics to “test the tests.” Most engineers are familiar with line coverage. Line coverage testing ensures that the tests in the software suite cover a specific threshold of lines in the code. However, line coverage is easy to cheat! I could write a test without any form of assertion that goes through the happy path of a method and probably generates over 80% line coverage. But was that an effective test? Of course not; we weren’t validating any behavior within that method — all we know is that my input didn’t generate an exception. Additionally, by only caring about line coverage percentage, I could forget to add a critical unit test for an edge case that would’ve only improved the coverage percentage minimally. Both of these situations can result in bugs. Therefore, move beyond just leveraging line coverage, and spend extra time trying to cover the various branching scenarios within your tests — even if they only improve line coverage by several percent.

Hint, hint: read up on mutation testing and even try out PIT for Java.

To conclude, you don’t have to be a quality assurance engineer to write impactful tests. In fact, you’ll become a much better software engineer when you understand and apply testing basics to the features and systems you deliver. Try these 9 tips and see how it goes — you might end up loving testing and becoming the testing SME on your team.

DISCLOSURE STATEMENT: © 2020 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.