Live Documentation in CI/CD
Written By Lucas Souza Romao
Continuous Integration (CI) and Continuous Delivery (CD) are modern development practices that making big difference in a development cycle. By using CI/CD approach, it’s possible to achieve fast delivery of features and get early feedback from stakeholders. To get a better idea, let’s have a look at CI/CD cycle in details:
I would propose to focus on testing stage, as there are certain questions, that are worth your attention. CI/CD cycle is famous for introducing automation into the stages of software development. As a conclusion out of it, we can assume that a pipeline should not have manual effort. However, does it mean no written test cases? Does it mean that we don’t need documentation anymore? Not quite.
Even for CI/CD cycle, documentation is required. A good approach to solve this, is by using Live Documentation, also known as BDD (Behaviour Driven Development). BDD is human-readable resource that use Gherkin language, that allows better cooperation for everyone in the team, mainly between Stakeholders, PO’s and QA’s. What is the reason behind this? It is to have a good documentation of the scenarios that should be tested, we must have clear acceptance criteria’s written.
With clear acceptance criteria defined, QAs can start developing the automation tests. But first, let’s check in on one BDD Cycle how it should work:
As you may notice, we have stages 4 and 5, that are also known as TDD (Test-driven development). It’s a practice where first you write the test and after code it. BDD and TDD are generally together in development cycle.
Okay, now we have a view about CI/CD pipeline and BDD Cycle. You may still wonder, where are the test cases?? Once scenarios have been written in BDD style, then test cases are live documented. It solves problems with outdated test documentation and maintaining it via test case management tool, when after each new cycle in every deployment, we have an updated version of the scenarios tested.
How to start your tests using Live Documentation?
Currently we have supported framework to almost all the languages such as SpecFlow for C# and Cucumber for Java being those two the most popular, but of course each language can have more than one framework. To give a practical example in how to start you tests let’s use Cucumber.
1- First thing is to create a new Java project.
2- Once create you will need to add the following dependencies:
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>4.7.1</version>
<scope>test</scope>
</dependency>PS: Don’t forget also to add Selenium dependencies.
3- Create a .feature file inside your project.
4- Open the feature file. At this point in your cycle, it is already expected that you had all your clarifications or at least most of them to start write your scenarios.
5- As you may have noticed in image above all the steps are highlighted since we still don’t have a definition for it. Let’s run it.
As you can see it fails, that’s the expected, in the output we can see that we gave all the steps definition that will need to be implemented.
6 - Now we have two options to move forward with our Live Documentation, the first one is to copy the output and create a class of your step definitions and the second one, your IDE should give the option to create one step or all the steps, as you can see in the image below.
Easy way is to use your IDE to generate the class and create the definitions.
7 - Once your steps are defined you will notice that in your steps in the future file are not highlight anymore.
8- Let’s run again and this time we must get a success run.
After those steps you should have an idea how your test looks like, one the most important part is the feature file, because this file will be our documentation about scenarios always updated. Let’s say you have a new functionality to test, that will affect your existing scenarios which will require a new step, so you will need to add it in your feature file. Once you do that, you are maintaining your documentation, simple and easy.
It’s important to mentioned that in the test stage of the pipeline we can separate our set of tests, it can be divided in Smoke, Regression, Functional and more if need it, with this division we can speed up our pipeline since we create script to trigger the test in parallel. The way that will be trigger will depend of the necessity of each project.
Report and Artefacts
So far, we have seen in how to create the feature file and also understood that this file will be our test documentation. Each framework has your own built-in options of basic reports but most of them allow us to customise and include charts as well, in the example above the report generate is similar to this:
Feature files and the reports should be our output artefact in the pipeline, allowing everyone in the team to have access on it without problems.
Achieving live documentation in CI/CD development cycle, it requires teamwork and communication. It’s a responsibility of everyone in the team. Code quality is the one of key points of product quality. Remember, if one fails everyone fails.
Why use Live Documentation in your pipeline?
· Fast feedback
· Human-readable scenarios for non-technical people
· More interaction between the team
· Reports are very easy to understand
· BDD is supported in almost every language
