How We Tried Using Git Submodules (And Why It Did Not Work for Us)

By Lukas Varkalis, Engineering Manager at Kilo Health

You might know that feeling when you are talking with your colleague and discussing a technical problem in your project, and he says that everything is working fine on their computer. Not the greatest feeling, right?

I hope you try to understand why it is happening instead of pushing the changes to the server and testing it there.

Those issues could appear for various reasons. It can be different node or browser versions, or you just have not pulled the latest changes. Because it could also be possible.

Today, I would like to share some experiences and lessons that we learned working with Git submodules. But before that, a short introduction about what it is and how they work generally.

What are Git submodules?

Submodules are Git repositories nested inside a parent repository’s working directory. Simply put, it allows you to include repositories as sub-folders inside another repository. SHA-1 hash will link them to know which version of the package you are using now.

This approach with submodules could help you keep everything in one place and easily maintain shareable packages.

The nice thing about submodules is that Git allows you to commit and push to these repositories independently. It helps when you use merge requests, issues, and different versions to develop a library for several projects.

Everything sounds good from the first look: you could maintain packages more easily and don’t have to stick to a specific version. So we decided to use this approach with a few of our internal packages and check how it goes.

First, we tried to implement it with a small team in one of our projects. A submodule you can add very easily just by running one command:

git submodule add https://github.com/<user>/<repository-name> /path/to/directory

Then using NPM workspaces, you can install dependencies required by the added package. After that, some additional configuration for ESLint, and Typescript, and this is it. You’re ready to go! 🚀

3 issues with Git submodules

Everything worked perfectly for a small developers team, where you always have clear and effective communication. But after working with more teams and less experienced developers, we realized that there are some cons to working with Git submodules.

I will share three of them from our experience:

1. The learning curve

Running a project locally using Git submodules is not intuitive for new developers.

They have to know some additional steps to take. For example, download submodules from remote directories and install their dependencies. It could be hard to understand for backend developers.

git submodule update —init && npm install

Also, not all developers are familiar with this implementation. So, you have to write clear documentation and guidance on how to work with submodules: fetch the latest version, check out to a new branch, create a new release, and so on. It takes time and effort to do it, especially for junior developers.

2. Hard to track the version in use

Reviewing pull requests of other developers could be very annoying. You can’t see which library version they are using or switching on. You could see only the SHA-1 hash, which is hardly readable and says nothing about a specific version.

For example:

Subproject commit 33d6a12560f348c5c663c004ac13c7a124dd982c

3. Manual updates

Changes in the recorded SHA-1 hash are not automatically reflected in the submodules.

This means that if someone updates the recorded version of a submodule and you pull their latest changes in the parent repository, your submodule repository will still be pointing to the old version of the submodule.

You have to run the command manually to update it:

git submodule update

The same problem is when you solve merge conflicts at the parent repository: it still doesn’t update the submodule pointer. You also have to do it manually.

To sum up

After some discussions and unpleasant incidents, we decided to use NPM packages, the previous solution, instead of Git submodules. It is more straightforward, everyone is familiar with it, and there is no learning curve.

Also, we look forward to trying new and different approaches to improve our developers’ experience with shareable libraries. So stay tuned!

Would you like to join this journey? Check out the open positions here.