One year using Azure DevOps (VSTS) CI/CD with Git and good practices

Photo by Farzad Nazifi on Unsplash

When I started at my current job as a software engineer around a year and 10 months ago at the time of writing this article the process let’s say it was not ideal, they used Asana to follow up the tasks and they used to publish directly to Azure using Visual Studio itself, in addition to using TFS with poor handling of branches without any type of code review and sense of good practices, but that was not the whole problem, there were almost 20 remote developers sending code for the same branch and they all shared the same database which caused the inconvenience of having to wait to finish pulling the code while a migration was already in progress.

The majority of developers were juniors without any supervision of someone with experience and of course as expected, the code was extremely tight and fragile, any modification or bug fix triggered other bugs in an exponential process, the feeling when publishing a new version to production was always of uncertainty, in fact in the 10 months prior to migrate to Azure DevOps (VSTS) and start forcing good practices with Pull Requests (PR) and code reviews, there were at most only 8 releases to production.

After an extensive battle with the person who was supposed to guide the
development process and the developers (almost 10 months) and thanks to our project manager, a capable person who from the beginning knew that
we had to look for alternatives because that path we were on wasn’t feasible, we began to migrate everything to Azure DevOps (VSTS) with Git. We stopped using Asana, moving all the tasks for the backlog of VSTS and the TFS code was moved to Git with the idea of promoting the use of Pull Requests (PR) and code reviews.

Branching Strategy

The first goal that we had to accomplish with Git was to create a branching strategy, that would allow us to have better control of what was going to production and of course would allow us to do it more often, in the safest way and automatic. For that porpoise, two branches were created, a master and another develop (default branch) all developers had to send PRs to develop (direct commits to any of these branches was not allowed). We started performing intense code reviews based on good practices by at least two developers, with the idea of creating a common language of good practices and trying to raise the general level of the developers.

At first, it was complicated because it kept the same number of remote developers sending PR to those who could be literally several comments in the same line of code making it very difficult to approve those PRs.

Agile methodologies were adopted since the problem was not only in the code but also in the process (another big battle that we won) and we started using Sprints and tried to publish the new features every two weeks (goal not yet reached).

The process was the following:

During the sprint a the developer sent a PR to the develop branch, after an intense code review and visual (the application running) and having checked that everything was fine with the code and the functionalities, the PR gets approved, process in which a Senior Developer was set as required as part of the code review group, with this new mode of development the code review began to be part of the developers tasks. At the end of the Sprint, we created a new branch based on the develop branch with the name release, for example, if it was the 5th Sprint we created the branch release-s5. The new release branch was published on Azure and began to be intensively tested, once it seems good, it was merged with master branch and published to production. Shortly after, that release branch was merged with develop branch and finally deleted. 
If we found a critical bug in production, we created another branch based on master with the name hotfix, after the critical bug was fixed, merged to master and published to production, we merged hotfix into develop branch and deleted it.

Every time you put some code in master meant a new publication to production and a tag was added to keep track of the versions that were published.

It was previously mentioned that direct commit was not allowed to any branch of the main ones, everything was based on PRs, the developers had to start a new task, for example, if I had to work in task #454 created a branch based on develop branch with the name custom/alejandro/454 finished sending the PR to develop branch where I had to give priority to solve the comments after code and visual review.

In the end, the structure of the branches that we handled was the following.

  • develop (default branch)
  • master (production)
  • release-s{#} (QA)
  • hotfix (critical bugs)
  • custom/{developer_name}/{task #}

CI/CD

Once you had a better way to handle the code based on an organized branching strategy, the next step was to improve the way that was published on Azure without depending on someone who published using Visual Studio. For this purpose, the VSTS pipeline was adopted. Allows building the solution, run the Unit Test (CI) and if everything was good to pack and publish directly to Azure instance (CD) in its different environments.

Let’s put some example of the workflow:

After the developers send a PR to develop branch, we need a deep code review base on good practice and a visual review too, for review it visually we have two variants, push the branch of the developer custom/{developer_name}/{task #} run the project with Visual Studio and revise it visually or you could publish that branch on Azure using the VSTS pipeline and review it in the cloud, this last variant is useful when the one who tries to check visually is a QA guy or the project manager that doesn't have Visual Studio installed, we call this instance on Azure UAT (User Acceptance Testing) if all was well we approve the PR and merge with develop branch, after merge with develop branch we publish develop branch on Azure automatic with a DEV environment to allow visually review the Sprint in an early stage of development, run the integration tests and E2E (at least in concept) with the idea to detect problems before creating the release branch at the end of the Sprint.

When the Sprint is done, we create the release branch and we publish on Azure with an environment called QA. As develop branch every time a PR is approved to this branch we publish automatically on Azure the latest changes. When release is tested end-to-end and is ready it is merged with master branch where the same happens with the difference that we have to approve the deploy first and this goes to staging (slot), make the last slot review with a production environment and if everything is still good, we swap, putting the last changes in production.
The same happens with the hotfix branch, each PR approve in hotfix is publish directly in the instance destined to hotfixes on Azure.

With the exception of the master branch, if the PR is approved it doesn’t matter which branch it is, develop, release or hotfix automatically the code is published on Azure in their environments, the only drawback is if you have to run migrations in the database, the migrations are executed manually avoided running automatic migrations in production by mistake.

Good practices, design patterns, and architecture

It was already commented that the code was extremely coupled and fragile, without any use of good practices or design patterns, the worst nightmare for a programmer that unfortunately is not the exception rather it seems to be part each you see more of the generality of the cases that can be found in companies small growth without a strong technological background. But where there is chaos many times there is hope the only thing what is needed is a cold mind and a well thought out plan of how to progress to be able to get out of that.

The first thing that we added was Dependency Injection (DI) SimpleInjector and another layer of abstraction called services (two new projects one with the interfaces and another with the implementations), the idea was to move all the business logic contained in the controllers to the services and inject the interfaces instead of instantiating controllers adding the possibility to start writing Unit Test, the implementations of these interfaces were resolved using the DI allowing mock them in the Unit Test.

The other that we added was the design patterns UnitOfWork and Repository to try to avoid directly accessing to Entity Framework (EF), there is a big discussion in the community if the use of these patterns since EF itself implements them and add that layer of abstraction it’s like adding another non-profit layer since you can change the data provider without having to modify your code, but what about the dependence on EF? The question is: why does my application have to be coupled to EF?, what happens if tomorrow you want to use another provider that EF does not support or instead of using EF you want to use another Object Relational Mapping (ORM)? For that, we added that new layer of abstraction, to remove the dependence to EF, the controllers and services do not access directly to EF and that allow tomorrow if we want we can remove EF without having to modify anything in the application layout.

…, software architects working in systems written in OO languages have absolute control over the direction of all source code dependencies in the system. They are not constrained to align those dependencies with the flow of control. No matter which module does the calling and which module is called, the software architect can point the source code dependency in either direction.
That is power! That is the power that OO provides. That’s what OO is really all about — at least from the architect’s point of view.
Martin, R.C.- Clean Architecture (Uncle Bob)

Another tool that we put into the game was Automapper to avoid performing the mapping of the entities to the Data Transfer Object (DTO) manually leaving the code much cleaner also began to adopt a standard for end-points (API) that they have to return Task<IActionResult> and we started using extended async/await for everything that involved access to database looking for better use of the pool the threads.
Also work intensively in performance looking to avoid calling ToList() in a stage early consultation using IQueryable and trying to just get from the database only the columns needed instead of getting all the columns. Finally, through code review, the emphasis was put on the importance of good name conversion according to the business language so that the same code could serve as a source of documentation.
 
From the system architecture point of view, we started using WebJobs to the tasks in the background and Azure Functions for some more dynamic processes that require getting out of the publishing cycle and to provide integration with third parties.

Conclusions

Nowadays it is very common to start working in places where the focus is more to deliver a feature or fix a bug regardless of the quality of the code
that is delivered or without putting refactor time in the planning, causing an abyss between what is desired as a product and the code.
Only when it begins to make development unmanaged by creating more problems of those who resolve then is that we start talking about good practices and to think a little more in each decision making, if on top of this we don’t have modern tools like VSTS and Git to help organize integral form the work the problem becomes more serious.

It doesn’t take a lot of skill to get a program to work. The skill comes in when you have to keep it working.”
Martin, R.C. (Uncle Bob)

That’s why a well-defined branch strategy, where the code passes through a strong review using some kind of tool that helps to carry follow-up of the tasks and CI/CD (DevOps) that is a fundamental part today of any cloud-oriented software development, a goal that VSTS fulfills perfectly, in addition to a focus on the quality of the code applying good practices looking for the highest decoupling possible help to get out of situations like the one described above.

Will it be easy? Of course not! Will it be possible? Absolutely! We are achieving it.