From monolith to microservices is not just sw architecture change: code repository might change too
What happens to code repositories when you move from monolith to microservices and fragment your engineering team
As we wrote in previous articles, at THRON we are in the process of migrating our architecture from a monolith structure to a microservices-based one and in this article I’d like to write about how this change led to unexpected issues regarding our git code repositories.
Where we started from
Our source code was organised in a few big git projects and our services architecture was based on monoliths, each repository structure was based on git-flow. We had three branches with infinite lifetime: master
, quality
and development
that represented che code deployed in the relative environment (ie. a merge in master
is a release in production). Moreover, every feature was developed into a separate branch whose name corresponds to the internal ticket code (we use YouTrack), this allows to track which tickets were solved just by looking at merge commits.
The problem
When we decided to move from our monolith architecture to a microservice one, we started having many more small repositories and we started experiencing problems/limits regarding our current git management:
- Our git workflow was too complex to manage due to the three infinite lifetime branches we were using because we needed to keep them up-to-date. For example, when we performed an hotfix we had to do three merges: in
master
to actually release the hotfix and inquality
anddevelopment
to propagate the fix in the other environments.
- Different teams were managing different repositories and no defined standard, this was quickly leading into anarchy and confusion.
We realised that
the architecture reflects the code organization and it also affects the git workflow used in each repository
We thus understood we needed to define our new standard git workflow, and we also were aware that it had also to satisfy the following requirements:
- easy to apply: despite being a fairly small team it’s important to implement changes that are intuitive and easy to understand and perform for anyone. A complex change would increase the resistance or the inability to follow the new standard when under pressure;
- minimise low value operations: making a new release or an hotfix should require a smaller number of operations. Having many infinite lifetime branches makes the git workflow more readable but it requires a lot of merges to keep everything up to date;
- easy to read: the new workflow should allow ticket tracking with a simple glance to commits, because this really eases our ability to check what goes into the repositories and link it to the expected task list.
How we did it
The most used git workflows we see online are git-flow and one-flow but both of them failed to match our requirements:
- git-flow is too complex for us because it also has two branches with infinite lifetime,
- one-flow seemed promising and it could fit our use case, but it requires to have an advanced knowledge of git.
We decided to start with some hands-on experimentation with one-flow by applying it to a simple internal utility library. After using it for a while we liked the new workflow and we found it to be cleaner than our current one.
The final solution was to use one-flow with a single development branch called develop
and to allow feature branches to start both from release and hotfix branches (therefore the feature released are still split into different branches in the history). The master
branch is just used as a pointer to the last released version: we do not commit or merge anything into it, we just fast forward it to the latest release.
During the process we also created a git repository to write the new workflow as guideline, this repository allows all developers to raise issue which can be openly discussed like open source projects and it allows to version the guideline for future improvements.
Result: our new git workflow
The new git workflow leads to a much cleaner history as you can see in the screenshot below since the number of merges is reduced.
Using community development to write a company guideline (via a git repository) really helped us because the final work was the result of everyone’s effort, therefore every developer feels that he has contributed and was happy to use it. Having a company guideline helps also standardize git repositories and we plan to write guidelines also for other aspects.
Next steps
We like our new git workflow very much but there are still some open issues:
- Our applications have the version written in a file and when we create the release branch the first thing to do is to bump the development version to the next release (ie. from
1.4.0-SNAPSHOT
to1.4.1-SNAPSHOT
) and when the release is ready we remove the snapshot from the version (ie. from1.4.0-SNAPSHOT
to1.4.0
) and tag the relative commit. This meas that when we close the release we have a systematic conflict when merging the release branch back todevelop
.
- One-flow expects you to have just the
master
branch with infinite lifetime and the only one to be used as development branch. During our experimentations we decided to usedevelop
for this purpose and to keepmaster
as a pointer to the last production release since many developers felt uncomfortable keepingmaster
reserved for production releases.
Conclusions
When we chose to migrate our infrastructure from monoliths to microservices we did not expect to change also our git workflow, however this issue gave us the opportunity to review our workflow and to better understand how this aspect affects development efficiency. We still have some issues in order to adopt one-flow but we plan to cover them in the future.
Are we the only one who had this kind of problem? Which git workflow are you using and why? Feel free to reach us out, we would love to share opinions.