https://www.flickr.com/photos/brendanlynch/5013468408

How To: Merge a Git submodule into its main repository

Git submodules seem like a good idea at first glance. They allow you to partition your project into several smaller repositories, while keeping a well defined relationship between the main git repository and its submodules.

In practice, at least for us, they have proven to be clunky and hard to work with.

If you just want to merge a single submodule back into your main repository, or if you want to go full blown Monorepo then this is the guide for you.

The naive solution

The simplest solution for this task is to just copy the files from the submodule into the main repository and commit everything in a one big commit. This will work just fine, but you will lose your submodule’s git history. If you’re reading this you probably value the history and won’t accept this compromise. Lets remove this option from the table.

Our goals

• Have all the code of a submodule appear in the main repository
• Have the code in whichever directory or sub directory we desire
• Not lose git history at all

Example project

Lets examine a simple web app. It has an src directory with controllers, routers and views. The models directory is a git submodule. Our goal is to place the models directory into the src directory in the main repository.

├── index.js
├── src
│   ├── controllers
│   │   └── posts_controller.js
│   ├── routers
│   │   └── posts_routes.js
│   └── views
│       └── posts_views.js
└── models (submodule)
    ├── index.js
    ├── comments.js
    └── posts.js

main repository git history:

* 2144b65 - (HEAD -> master) Update models reference
* 0d1671a - Fix horrible bug 
* 52c4b1a - Update models submodule
* 773024f - Add posts feature
* 7e01883 - Add models submodle
* 10c61f0 - Initial commit

models repository git history:

* a11655d - (HEAD -> master) Fix horrible bug
* ed201c4 - Add comments model
* 3d7f342 - Add posts model
* c723ee7 - Initial commit

Stages of execution:

To achieve our goal we will follow the following steps:

1. Restructure the submodule’s file tree to fit our needs.
2. Remove the submodule from the main repository.
3. Merge the submodule master branch into the main repository.

1. Restructure the submodule file tree

Before we can merge the submodule we need to do some directory restructuring. Why? Lets look at the file structure in the models repository:

├── index.js
├── comments.js
└── posts.js

Our end goal requires the files to be under ./src/models. We need to create the ./src/models directory and move all the tracked files in there.

In the models directory:

• mkdir -p src/models
• git ls-tree master --name-only | xargs -I{} git mv {} src/models
• git commit -m 'Move all files into src/models directory'
• git push

2. Removing the submodule

Next we need to remove the submodule from the main repository. The submodule is referenced in several locations, so removing it cleanly requires several steps:

1.1 Edit .gitmodules

Git submodules metadata is stored in the .gitmodules file:

[submodule "models"]
path = models
url = git@github.com/exampleUser/models

• remove the submodule from that file.
• git add .gitmodules

1.2 Edit .git/config

.git/config has a similar entry, edit that file as well.

1.3 Remove the submodule from git tracking

git rm --cached models

1.4 Cleanup the .git/modules directory

rm -rf .git/modules/models

1.5 Commit the changes

git commit -m 'Remove models submodule'

1.6 Delete the submodule directory:

The models directory is now untracked, we can delete it.
rm -rf models

3. Merging the submodule

All the pieces are in place. The final step is to merge the models repository into the main repository.

In the main repository run the commands:

• git remote add models-origin git@github.com/exampleUser/models
• git fetch models-origin
• git merge --allow-unrelated-histories models-origin/master
• git remote remove models-origin

Success! Our git history now looks like this:

*   1a2f1aa - (HEAD -> master) Merge remote-tracking branch 'models-origin/master'
|\
| * cb445ce - Move all files into models directory
| * a11655d - Fix horrible bug
| * ed201c4 - Add comments model
| * 3d7f342 - Add posts model
| * c723ee7 - Initial commit
* 346be76 - Remove models sumbmodule
* 2144b65 - Update models reference
* 0d1671a - Fix horrible bug
* 52c4b1a - Update models submodule
* 773024f - Add posts feature
* 7e01883 - Add models submodle
* 10c61f0 - Initial commit

Lets make this official:

git push origin master

Other team members

Team members will receive the updated repository structure on git pull, If you’ve moved the submodule into a location other than its original — make sure to delete the old, now untracked submodule directory.