Test Automation Approaches for Developers

As the last days of the year slip by, I find myself in a contemplative frame of mind. What have I achieved this year? What have i learnt? How can I make myself a better developer in 2020? Do you, dear reader, find yourself in the same asking the same questions? One of the exciting, tiring, stimulating, frustrating (I could go on) parts of being a developer is that there are always so many new things we can learn: a new language, framework, should I deep dive into native apps, but what about PWAs, web components and all the stuff I haven’t even heard of yet? So what should we pick for next year? I don’t know! Pick the tech that you enjoy and you feel productive in. But if you want to end 2020 a better, more polished developer than you are now, invest some time in becoming a better tester. I’m going to spend the rest of this post looking at what automated testing options are available to individual developers and then give you a quick introduction to automated UI testing using TestProject’s C# SDK.

Testing is a part of every developers job. It doesn’t matter if you work in a firm that employs test engineers, or the system you work on is run through rigorous QA before being released, you are still going to be doing some kind of testing on the code you write before shipping it down the line. The smaller the team, the greater the responsibility you have to take as an individual for the end quality of what you produce. I’m sure we’ve all had experiences like the following.

Phil: “Hi, Al? It’s Phil from BigClient here.”
Oh, cool, he must be wanting to congratulate me on the new feature I’ve just released.
Me: “Hi Phil, have you got that new release?”
P: “Yeah, thanks we’ve just been looking at it. Umm, as soon as we click on the new option we get an error message. Did you test this before you released it?”
Me (silently): $#@%
Me: “Let me look into that for you.”

How embarrassing. And unprofessional. Let’s see what we can do to minimise these exchanges.

To ground our theory in the real world, let’s imagine an app that displays a list of books and information about each book’s author. A user can filter the list by author name.

The diagram below shows a possible architecture for building this app, using ReactJS on the client, ASP.NET Core on the server and Azure Storage or CosmosDB for storage. We might find that we develop several React components, plus an ASP.NET Core API Controller that gets the book data from a Books service and the author data from an Authors service, aggregates it and provides it to the client.

So how can we go about testing what we develop? Well, we could definitely just use the trusty debugger and manually test all the features as we build them. But, if you are like me, doing the same thing over and over becomes extremely tedious, plus we risk missing some part of the system, or assuming it still works and ignoring it, which can all too easily lead to a conversation like the one with Phil from BigClient. Finding an automated test solution that works for you and your project makes a big difference to the quality of your output.

For the purposes of these examples, an automated test is one that you write once but can execute many times, without having to perform each step yourself. Also for the purpose of our examples, I’ll be grouping the types of test by the number of technical components we are exercising with each test. A unit test test a single component in isolation. An integration test tests the behaviour of several (but not all) components together, and an end-to-end test tests the complete application, just as a user of the application would experience it. A test professional will tell you that there are many other types of test, but for our purposes the above definitions are enough to get started.

Automated Unit Tests

A unit test verifies that a single component, or unit of code, works as expected given a set of input and responses from other components. In our book app we might want to test that the BookListItem renders the correct HTML for a given set of book data. Or, at the API level, that the BookListController calls the Book and Author service appropriately and correctly combines the result of those calls into a single JSON response. To allow a single component to be tested in isolation we mock the components it interacts with, either manually or with one of the mocking libraries available for our language of choice.

When implemented across a code base, unit tests give developers a great sense of confidence in the changes they make. And since a unit is tested in isolation there are no side effects to handle (both integration and end-to-end tests can require you to setup up things like data before running a test and then making sure you clean up after yourself).

Unit tests can be difficult to introduce into an existing code base, since the system needs to be designed in a way that units can be isolated.

There are many unit testing frameworks available. In our example app we might choose Jest to unit test our React components and XUnit or NUnit to test our .NET code.

Automated Integration Tests

Sometimes it makes more sense to test that two or more components work together correctly. For example, perhaps we want to verify that given a certain response from the API, the BookListContainer and all its children are rendered correctly. And we probably also want to know that for a given request to the API we can retrieve book and author data from our Azure Storage and return the correct JSON response. At its most generic, these tests are referred to as integration tests.

Integration tests can reveal issues that unit tests miss, particularly issues around how our application interacts with real databases or storage services for example. The downside is that if they are using the real database we will need to ensure the data is setup in the way our tests expect and that we clean the data up after we have finished (whether the test succeeds or fails).

We can use the same frameworks for integration testing as we use for unit testing.

Automated End-to-End Tests

Finally, what about just performing testing exactly the way an end user would interact with the system, but automate things so we can run the tests over and over again? In our app we want to see that the list of books is displayed with all the correct info and that if text is entered into the filter input that the list is filtered appropriately. Welcome to the world of end-to-end or automated UI testing.

The advantages are obvious. You are testing as close to the experience of a real human interacting with your system as is possible with an automated test. You can run the tests on the environments your users or clients will be using, so you can have confidence that each release is working. By their very nature you are testing all the components of your system.

The disadvantages have traditionally been daunting. Commercial tools to make the process easy have been prohibitively expensive for small teams or solo developers. For those without the cash open source tools like Selenium and Appium provide the means, but the learning curve was steep and many teams (mine included) flirted with the idea and found we couldn’t spare the time to become proficient. You are also faced with the same issues of ensuring data is setup before your tests and cleaned up afterwards that you encounter with integration tests. It was a shame, because if you are looking for the best uptick in quality for the lowest outlay, end-to-end testing is a great place to start, especially in situations where you are inheriting an old, existing system or are developing a Minimum Viable Product (MVP) where you need to maximise the time spent delivering new features rather than ensuring every component approaches 100% test coverage.

Fortunately, a new breed of solution has emerged to make the experience of working with Selenium and Appium easier. I’ve written previously about how we were able to upgrade and enhance a large, old, PHP solution for a fraction of the price quoted by other companies, primarily due to investing in automated UI tests across the application to allow us to make radical changes to the code with confidence. Some of the responses to that article suggested evaluating TestProject, so I’ve been doing just that on the latest MVP we’ve delivered for a client at DataDIGEST.

Enter TestProject

TestProject is a cloud-based, end-to-end test automation platform for web, mobile and API testing. It offers everything you would expect from such a product today: a test recorder, online test editor, the ability to parameterise tests and create data driven tests. You can setup your tests to run on any number of agents (devices that you control and onto which you’ve installed the TestProject Agent application), make use of community contributed addons and see comprehensive reports and analytics on your test runs. Under the covers the tests are using Selenium for web tests and Appium for mobile tests. All of these features are free, with up to 2GB of storage available to you on the platform. Of most interest to me however, was how the experience of authoring tests using the C# SDK stacked up against other platforms built on top of Selenium.

TestProject — First Impressions

If you’ve used other Selenium-based testing tools, you will have discovered that getting setup and ready to write tests can be a bit of a chore. Happily, this is something that the TestProject team have got right. The onboarding process is absolutely first class, guiding you through creating an account and installing your first agent. I was particularly impressed at being encouraged to view an introductory video whilst the agent was downloading. I had my first recorded test running against a deployed instance of my application in little more than 15 minutes.

To be of maximum benefit to a developer however, a testing solution needs to allow for tests to be authored and run on a developers machine before code is committed and deployed to a testing environment. To be able to achieve this with TestProject you need to head over to the Integrations tab and choose one of the available integration options (currently Java and C# are available). Choosing C#, I was presented with a link to the Nuget package and the QuickStart guide to get me started.

Once again, TestProject’s documentation was right on the money. Everything you need to get started with code-authored tests is in the QuickStart guide and its accompanying examples in Github. I’m not going to repeat the getting started information you can find in that guide, but here are some more generic tips that will help you get on your way with authoring end-to-end tests alongside your application code.

1. You need to create a project to hold your test code. When you look at the QuickStart samples you will see that they have a separate project to contain the runner (the code that will kick off the test). You should follow this pattern, as it will make it easier to upload your compiled tests to the TestProject platform for sharing with your teammates later.
2. I typically have a mixture of unit, integration and end-to-end tests in my projects. XUnit is my preferred unit and integration testing framework within the .NET ecosystem. One of the advantages of the TestProject platform versus others I have tried is that I can continue to develop with the tools I am familiar with and not step outside my IDE to perform end-to-end testing. However, XUnit may not be the best choice as a runner for TestProject tests, given its lack of parameterised test fixtures.
3. I like to have a single end-to-end test for each feature in my application. These act as smoke tests to quickly let me know if something has gone wrong as the project develops. My process for authoring end-to-end tests as I develop is as follows:

• I first get the feature I am working on developed, using manual tests to see that everything looks OK;
• I ensure that each web element that I will want to reference in my tests is given an ID (this just makes it easier to locate, rather than having to come up with complicated XPath expressions).
• For each web element with an ID, I create an IWebElement in my TestProject project, using the Page Object Model approach recommended by Selenium and TestProject in their QuickStart. (As an aside, this is a great candidate for some code generation).
• With the page objects in place I can write my test. I find it is best to treat test code as a first class citizen, so if I see a chance to refactor my test code to avoid repeating myself, etc., I take it. Common actions get refactored into the Page Obejct class. See the following example from the QuickStart sample for logging in.

public void Login(string name, string password)
{
  TypeName(name);
  TypePassword(password);
  ClickLogin();
}

• The TestProject SDK makes it quite easy to add reporting steps to your tests. I only tend to add these steps if I find the test has a problem, usually it is enough to know whether the test passed or failed. But if you were going to upload your tests to the TestProject platform to execute from there, you would want to add some step reports.

public ExecutionResult Execute(WebTestHelper helper)
{
   // Get driver initialized by TestProject Agent
   // No need to specify browser type, it can be done later via UI
   var driver = helper.Driver;
   TestReporter report = helper.Reporter;
   // Navigate to TestProject Demo website
   driver.Navigate().GoToUrl(“https://example.testproject.io/web/“);
   // Initialize the properties of the LoginPage with the driver
   var loginPage = PageFactory.InitElements<LoginPage>(driver);
   report.Step(“Navigated to TestProject Demo”, loginPage.Displayed);
  // Login using provided credentials
  loginPage.Login(name, password);
  // Initialize the properties of the profilePage with the driver
  var profilePage = PageFactory.InitElements<ProfilePage>(driver);
  report.Step($”Logged in with {name}:{password}”,      profilePage.Displayed);
  // Complete profile forms and save it
  profilePage.UpdateProfile(country, address, email, phone);
  report.Step(“Profile information saved”, profilePage.Saved, TakeScreenshotConditionType.Never);
  report.Result = “Test completed successfully”;
  return ExecutionResult.Passed;
}

Summing Up

Being more structured and methodical with your testing is a great way to become a better developer. If you haven’t started exploring automated testing the options can be a little daunting. Automating a few end-to-end tests is a good first step to introducing automated testing into your development process. TestProject makes getting started with these tests easier than it has been in the past.