TFS — Setup CI/CD pipeline for automated functional tests

In this blog, I’ll provide all the information from creating your functional automated tests to setup CI/CD pipeline. This blog is especially for Microsoft Team Foundation Server (TFS).

Today’s World:

In today’s world, every check in should be considered as production ready to achieve this we write automated tests which help a lot to make sure the quality.

All in all, if you have automated tests and CI/CD pipeline ready then this combination becomes awesome because it allows you to execute your automated tests and release your solution in a very short time.

A Typical CI/CD Pipeline:

Let’s Start:

Let’s see everything gradually, we need to follow below 3 steps all in all:

1. Write your automated tests
2. Setup TFS QA build
3. Setup TFS QA Release

For the process which I’m going to explain this time is based on the following assumption:

• We have a Build Server to build the code
• We have a Test Server to execute Functional test

In case you’ve build and test server as same machine then you need to setup Machines information accordingly.

• Write your automated tests:

If you’re new to this, you can follow my blog to start with it: Selenium with C# and xUnit

• Setup TFS QA build

So, in order to achieve this first you need to commit your all of the code to a repository, so let’s say name of this repository is “Functional-Tests” for this blog.

1. Access Visual Studio Team Services (VSTS) account where you checked in your test code and mouse over on Build & Release tab then select Builds. Now click on New.

1. Once you click on New you’ll see the following screen, Create build definition by selecting template as Visual Studio and click Next

1. In Create new build definition, select Repository as your test repository say, “Functional-Tests”
2. Select Default branch as Master or you can select the branch which you actually want to build and want your tests execute on.
3. Check Continuous Integration check box.
4. Leave other settings as Default (Defaultagent queue denotes in which you have installed an agent. You can choose the Manage link to set this) then click Create.

1. After clicking Create you will land up to Build Definition page as follows:

1. Now let’s setup build steps as follows:

• NuGet restore
• Path to solution or packages.config: Select your app solution (.sln) file, for our case FunctionalTests.sln
• Build solution
• Solution: Select your QA solution (.sln) file.
• Platform: $(BuildPlatform)
• Configuration: $(BuildConfiguration)
• Visual Studio Version: Select the version of VS
• Test Assemblies
• Delete or Disable this as this is generally used to execute unit test and we’ll not need this to execute out automated functional tests.
• Publish symbols path
• Search pattern: Set it to “**\bin\**\*.pdb”

1. Leave all other build steps as Default, now click save, you’ll see a pop-up, enter a name for this build definition say “ QA Build Definition” and click OK

• Setup TFS QA release

Now we are ready with the Functional code build, in order to execute the tests we need to setup release as follows:

1. Mouse over on Build & Release tab again and select Releases. Now click on + icon to create a new release.

1. Select Empty on Create release definition window and click next

1. On this window select Source as your Functional Build, check Continuous Deployment box

1. Click Create and rename release to “ QA Functional Release” and Environment to QA and Save

1. Now we will add tasks to our release to do this click on Add tasks button and select Tests catalogue in that

1. Add following tasks to the release:
2. Visual Studio Test Agent Deployment
3. Windows Machine File Copy (From Deploy)
4. Run Functional Tests
5. We need to setup Tests agent as mention below:

• Test Machine Group
• Machines: Comma-delimited list of machine names where you want to execute the tests
• Admin Login: Username for Test server
• Admin Password: Password for Test server
• Protocol: HTTP
• Select Machines By: Machine Names
• Agent Configuration
• Username: Agent machine username.
• Password: Agent machine password.
• Interactive Process: Checked
• Advanced
• Test Agent Location: Mention the path of test agent exe

1. Now we need to copy Build files from build server to test server to do this configure Windows machine file copy as follows:
2. Source: Location of the folder which contains QA build artifacts
3. Machines: a Comma-delimited list of machine names where you want to execute the tests
4. Admin Login: Username for the Test server
5. Admin Password: Password for the Test server
6. Destination Folder: Location of the folder where we will drop tests
7. To execute automated functional tests on a machine we need to set up the Run Functional Tests task:

• Setup Options
• Machines: a Comma-delimited list of machine names where you want to execute the tests
• Test Drop Location: Same as in Destination folder in the last step
• Execution Options
• Test Selection: Test Assembly
• Test Assembly: **\*Test*.dll or you can directly mention the test dll, like FunctionalTests.dll

1. Now click on Save. Release pipeline will look like this:

This is the stage where we ready with complete CI/CD pipeline for our Functional Automation to test this you just need to commit to your Functional code and then it will automatically trigger the build if the build is successful it will automatically trigger the release which actually is executing the test on the test machine/machines.

You can actually see the tests executing on those machines after your tests are completed you can see a detailed report by clicking Tests tab on Release, like below:

I tried to cover end to end process how you can write automation tests and setup CI/CD pipeline while working with Microsoft Technologies, please comment if you have any feedback about this blog.