From monolith to microservices is not just sw architecture change: code repository might change too

What happens to code repositories when you move from monolith to microservices and fragment your engineering team

As we wrote in previous articles, at THRON we are in the process of migrating our architecture from a monolith structure to a microservices-based one and in this article I’d like to write about how this change led to unexpected issues regarding our git code repositories.

Where we started from

Our source code was organised in a few big git projects and our services architecture was based on monoliths, each repository structure was based on git-flow. We had three branches with infinite lifetime: master, quality and development that represented che code deployed in the relative environment (ie. a merge in master is a release in production). Moreover, every feature was developed into a separate branch whose name corresponds to the internal ticket code (we use YouTrack), this allows to track which tickets were solved just by looking at merge commits.

Our previous workflow. You can see the three infinite lifetime branches master (purple), development (red) and quality (aqua green). The activities included can be derived by looking at merge commits.

The problem

When we decided to move from our monolith architecture to a microservice one, we started having many more small repositories and we started experiencing problems/limits regarding our current git management:

• Our git workflow was too complex to manage due to the three infinite lifetime branches we were using because we needed to keep them up-to-date. For example, when we performed an hotfix we had to do three merges: in master to actually release the hotfix and in quality and development to propagate the fix in the other environments.

In the “hotfix” case we have to perform three merges just to keep master, quality and development on sync.

• Different teams were managing different repositories and no defined standard, this was quickly leading into anarchy and confusion.

We realised that

the architecture reflects the code organization and it also affects the git workflow used in each repository

We thus understood we needed to define our new standard git workflow, and we also were aware that it had also to satisfy the following requirements:

• easy to apply: despite being a fairly small team it’s important to implement changes that are intuitive and easy to understand and perform for anyone. A complex change would increase the resistance or the inability to follow the new standard when under pressure;
• minimise low value operations: making a new release or an hotfix should require a smaller number of operations. Having many infinite lifetime branches makes the git workflow more readable but it requires a lot of merges to keep everything up to date;
• easy to read: the new workflow should allow ticket tracking with a simple glance to commits, because this really eases our ability to check what goes into the repositories and link it to the expected task list.

How we did it

The most used git workflows we see online are git-flow and one-flow but both of them failed to match our requirements:

• git-flow is too complex for us because it also has two branches with infinite lifetime,
• one-flow seemed promising and it could fit our use case, but it requires to have an advanced knowledge of git.

We decided to start with some hands-on experimentation with one-flow by applying it to a simple internal utility library. After using it for a while we liked the new workflow and we found it to be cleaner than our current one.

The final solution was to use one-flow with a single development branch called develop and to allow feature branches to start both from release and hotfix branches (therefore the feature released are still split into different branches in the history). The master branch is just used as a pointer to the last released version: we do not commit or merge anything into it, we just fast forward it to the latest release.

During the process we also created a git repository to write the new workflow as guideline, this repository allows all developers to raise issue which can be openly discussed like open source projects and it allows to version the guideline for future improvements.

Result: our new git workflow

The new git workflow leads to a much cleaner history as you can see in the screenshot below since the number of merges is reduced.

The new workflow. The merges involved are just the required ones, no “sync” overhead.

Using community development to write a company guideline (via a git repository) really helped us because the final work was the result of everyone’s effort, therefore every developer feels that he has contributed and was happy to use it. Having a company guideline helps also standardize git repositories and we plan to write guidelines also for other aspects.

Next steps

We like our new git workflow very much but there are still some open issues:

• Our applications have the version written in a file and when we create the release branch the first thing to do is to bump the development version to the next release (ie. from 1.4.0-SNAPSHOTto 1.4.1-SNAPSHOT) and when the release is ready we remove the snapshot from the version (ie. from 1.4.0-SNAPSHOT to 1.4.0) and tag the relative commit. This meas that when we close the release we have a systematic conflict when merging the release branch back to develop.

The merge from release/1.4.0 into develop will always generate a version conflict.

• One-flow expects you to have just themaster branch with infinite lifetime and the only one to be used as development branch. During our experimentations we decided to use develop for this purpose and to keep master as a pointer to the last production release since many developers felt uncomfortable keeping master reserved for production releases.

Conclusions

When we chose to migrate our infrastructure from monoliths to microservices we did not expect to change also our git workflow, however this issue gave us the opportunity to review our workflow and to better understand how this aspect affects development efficiency. We still have some issues in order to adopt one-flow but we plan to cover them in the future.

Are we the only one who had this kind of problem? Which git workflow are you using and why? Feel free to reach us out, we would love to share opinions.