Mastering Git subtrees

Subtree fundamentals

Three approaches: pick one!

The manual way

The git subtree contrib script

One of the key benefits of subtrees is to be able to mix container-specific customizations with general-purpose fixes and enhancements.

git-subrepo

Check out our amazing Git-related video courses! Some of them are even free!

Subtrees, step by step

  • main acts as the container repo, local to the first collaborator,
  • plugin acts as the central maintenance repo for the module, and
  • remotes contains the filesystem remotes for the two previous repos.

Our subtree structure

.
├── README.md
├── lib
│ └── index.js
└── plugin-config.json

Adding a subtree

Manually

manually/main (master u=) $ git remote add plugin ../remotes/plugin
manually/main (master u=) $ git fetch plugin
warning: no common commits
remote: Counting objects: 11, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 11 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (11/11), done.
From ../remotes/plugin
* [new branch] master -> plugin/master
manually/main (master u=) $
manually/main (master u=) $ git read-tree \
--prefix=vendor/plugins/demo -u plugin/master
manually/main (master + u=) $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
new file: vendor/plugins/demo/README.md
new file: vendor/plugins/demo/lib/index.js
new file: vendor/plugins/demo/plugin-config.json
manually/main (master + u=) $ git commit \
-m "Added demo plugin subtree in vendor/plugins/demo"

[master 76b347a] Added demo plugin subtree in vendor/plugins/demo
3 files changed, 19 insertions(+)
create mode 100644 vendor/plugins/demo/README.md
create mode 100644 vendor/plugins/demo/lib/index.js
create mode 100644 vendor/plugins/demo/plugin-config.json
manually/main (master u+1) $

With git subtree

git-subtree/main (master u=) $ git remote add plugin \
../remotes/plugin
/
git-subtree/main (master u=) $ git subtree add \
--prefix=vendor/plugins/demo plugin master

git fetch plugin master
warning: no common commits
remote: Counting objects: 11, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 11 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (11/11), done.
From ../remotes/plugin
* branch master -> FETCH_HEAD
* [new branch] master -> plugin/master
Added dir 'vendor/plugins/demo'
git-subtree/main (master u+4) $
git-subtree/main (master u+4) $ git log --oneline --graph \
--decorat
e
* 32e539d (HEAD, master) Add 'vendor/plugins/demo/' from…
|\
| * fe64799 (plugin/master) Fix repo name for main project…
| * 89d24ad Main files (incl. subdir) for plugin, to populate its…
| * cc88751 Initial commit
* b90985a (origin/master) Main files for the project, to populate…
* e052943 Initial import
git-subtree/main (master u+4) $ git reset --hard @{u}
HEAD is now at b90985a Main files for the project, to populate its…
git-subtree/main (master u=) $ git subtree add \
--prefix=vendor/plugins/demo --squash plugin master

git fetch plugin master
From ../remotes/plugin
* branch master -> FETCH_HEAD
Added dir 'vendor/plugins/demo'
git-subtree/main (master u+2) $
git-subtree/main (master u+2) $ git log --oneline --graph \
--decorate

* 352af7a (HEAD, master) Merge commit '03e04026fdba2ff1200a226c3…
|\
| * 03e0402 Squashed 'vendor/plugins/demo/' content from commit…
* b90985a (origin/master) Main files for the project, to populate…
* e052943 Initial import

Grabbing/updating a repo that uses subtrees

With subtrees, cloning/pulling just works.

git-subtree/main (master u+2) $ git push

manually/main (master u+1) $ git push
manually/main (master u=) $ cd ..
manually $ git clone remotes/main colleague
Cloning into 'colleague'...
done.
manually $ cd colleague
manually/colleague (master u=) $ tree vendor
vendor
└── plugins
└── demo
├── README.md
├── lib
│ └── index.js
└── plugin-config.json
3 directories, 3 files

Getting an update from the subtree’s remote

manually/colleague (master u=) $ cd ../plugin
manually/plugin (master u=) $ git log --oneline
fe64799 Fix repo name for main project companion demo repo
89d24ad Main files (incl. subdir) for plugin, to populate its tree.
cc88751 Initial commit
manually/plugin (master u=) $ date > fake-work
manually/plugin (master % u=) $ git add fake-work
manually/plugin (master + u=) $ git commit -m "Pseudo-commit #1"
[master 5048a7d] Pseudo-commit #1
1 file changed, 1 insertion(+)
create mode 100644 fake-work
manually/plugin (master u+1) $ date >> fake-work
manually/plugin (master * u+1) $ git commit -am "Pseudo-commit #2"

manually/plugin (master u+2) $ git push
manually/plugin (master u=) $ cd ../main
manually/main (master u=) $

Manually

manually/main (master u=) $ git fetch plugin
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From ../remotes/plugin
fe64799..dc995bf master -> plugin/master
manually/main (master u=) $ git merge -s subtree --squash \
plugin/maste
r
Squash commit -- not updating HEAD
Automatic merge went well; stopped before committing as requested
manually/main (master + u=) $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
new file: vendor/plugins/demo/fake-workmanually/main (master + u=) $ git commit -m "Updated the plugin"
[master 4f9a839] Updated the plugin
1 file changed, 2 insertions(+)
create mode 100644 vendor/plugins/demo/fake-work
manually/main (master u+1) $
git merge -X subtree=vendor/plugins/demo --squash plugin/master

With git subtree

git-subtree/main (master u=) $ git subtree pull \
--prefix=vendor/plugins/demo --squash plugin master

remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From ../remotes/plugin
* branch master -> FETCH_HEAD
fe64799..2872e5d master -> plugin/master
Merge made by the 'recursive' strategy.
vendor/plugins/demo/fake-work | 2 ++
1 file changed, 2 insertions(+)
create mode 100644 vendor/plugins/demo/fake-work
git-subtree/main (master u+2) $
git-subtree/main (master u+2) $ git log --oneline --graph \
--decorate

* 1782c08 (HEAD, master) Merge commit '636facf64d416210e90bb8b83…
|\
| * 636facf Squashed 'vendor/plugins/demo/' changes from fe64799..…
* | 352af7a (origin/master) Merge commit '03e04026fdba2ff1200a22…
|\ \
| |/
| * 03e0402 Squashed 'vendor/plugins/demo/' content from commit fe…
* b90985a Main files for the project, to populate its tree a bit.
* e052943 Initial import

Updating a subtree in-place in the container

Subtree updates can be freely performed within the container codebase.

  • Commits touching only the subtree, intended for backport (e.g. fixes);
  • Commits touching only container code;
  • Commits touching both container and subtree code, the latter part being intended for backport;
  • Commits touching only the subtree, in a container-specific way that is not to be backported.
git push
echo '// Now super fast' >> vendor/plugins/demo/lib/index.js
git ci -am "[To backport] Faster plugin"
date >> main-file-1
git ci -am "Container-only work"
date >> vendor/plugins/demo/fake-work
date >> main-file-2
git ci -am "[To backport] Timestamping (requires container tweaks)"
echo '// Container-specific' >> vendor/plugins/demo/lib/index.js
git ci -am "Container-specific plugin update"

Backporting to the subtree’s remote

manually/main (master u+4) $ git log --oneline --decorate --stat -5
28e310b (master) Container-specific plugin update
vendor/plugins/demo/lib/index.js | 1 +
1 file changed, 1 insertion(+)
71d2d12 [To backport] Timestamping (requires container tweaks)
main-file-2 | 1 +
vendor/plugins/demo/fake-work | 1 +
2 files changed, 2 insertions(+)
c693673 Container-only work
main-file-1 | 1 +
1 file changed, 1 insertion(+)
92bc02d [To backport] Faster plugin
vendor/plugins/demo/lib/index.js | 1 +
1 file changed, 1 insertion(+)
4f758af (origin/master) Updated the plugin
vendor/plugins/demo/fake-work | 2 ++
1 file changed, 2 insertions(+)

Manually

manually/main (master u+4) $ git checkout -b backport-plugin \
plugin/master

manually/main (backport-plugin u=) $
manually/main (backport-plugin u=) $ git cherry-pick -x master~3
[backport-plugin 953ec4d] [To backport] Faster plugin
Date: Thu Jan 29 21:54:45 2015 +0100
1 file changed, 1 insertion(+)
manually/main (backport-plugin u+1) $ git cherry-pick -x \
--strategy=subtree master^

[backport-plugin 34f50a4] [To backport] Timestamping (requires con…
Date: Thu Jan 29 21:55:00 2015 +0100
1 file changed, 1 insertion(+)
manually/main (backport-plugin u+1) $ git log --oneline \
--decorate --stat -2

34f50a4 (HEAD, backport-plugin) [To backport] Timestamping (requir…
fake-work | 1 +
1 file changed, 1 insertion(+)
953ec4d [To backport] Faster plugin
lib/index.js | 1 +
1 file changed, 1 insertion(+)
manually/main (backport-plugin u+2) $ git push
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (7/7), 877 bytes | 0 bytes/s, done.
Total 7 (delta 2), reused 0 (delta 0)
To ../remotes/plugin
dc995bf..34f50a4 backport-plugin -> master

With git subtree

# BEWARE: NOT WHAT WE WANT. Backports everything, no choice.
git-subtree/main (master u+4) $ git subtree push \
-P vendor/plugins/demo plugin master

git push using: plugin master
-n 1/ 10 (0)
-n 2/ 10 (1)
-n 3/ 10 (2)
-n 4/ 10 (2)
-n 5/ 10 (3)
-n 6/ 10 (3)
-n 7/ 10 (4)
-n 8/ 10 (5)
-n 9/ 10 (6)
-n 10/ 10 (7)
Counting objects: 11, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (11/11), 1.11 KiB | 0 bytes/s, done.
Total 11 (delta 4), reused 0 (delta 0)
To ../remotes/plugin/
2872e5d..e857a74 e857a74119c3e1c1b237b367c4a6c8f79deca1a7 -> m…
git-subtree/main (master u+4) $ git log --oneline --decorate -4 \
plugin/master

e857a74 (plugin/master) Container-specific plugin update
ddabc13 [To backport] Timestamping (requires container tweaks)
73a22ea [To backport] Faster plugin
2872e5d Pseudo-commit #2

Removing a subtree

main (master u=) $ git rm -r vendor/plugins/demo
rm 'vendor/plugins/demo/README.md'
rm 'vendor/plugins/demo/fake-work'
rm 'vendor/plugins/demo/lib/index.js'
rm 'vendor/plugins/demo/plugin-config.json'
main (master + u=) $ git commit -m "Removing demo subtree"
[master 3893865] Removing demo subtree
4 files changed, 24 deletions(-)
delete mode 100644 vendor/plugins/demo/README.md
delete mode 100644 vendor/plugins/demo/fake-work
delete mode 100644 vendor/plugins/demo/lib/index.js
delete mode 100644 vendor/plugins/demo/plugin-config.json
main (master u+1) $

Turning a directory into a subtree

cd ..
mkdir remotes/myown
cd remotes/myown
git init --bare
cd ../../main
mkdir -p lib/plugins/myown/lib
echo '// Yo!' > lib/plugins/myown/lib/index.js
git add lib/plugins/myown
git ci -m "Plugin sez: Yo, dawg."
date >> main-file-1
git ci -am "Container-only work"
echo '// Now super fast' > lib/plugins/myown/lib/index.js
date >> main-file-2
git ci -am "Faster plugin (requires container tweaks)"
git push

Manually

manually/main (master u=) $ git checkout -b split-plugin
manually/main (split-plugin) $ git filter-branch \
--subdirectory-filter lib/plugins/myown

Rewrite 973cfacecb645f66b89accedac8780c19140401b (2/2)
Ref 'refs/heads/split-plugin' was rewritten

manually/main (split-plugin) $ git log --oneline --decorate
5af0de1 (HEAD, split-plugin) Faster plugin (requires container twe…
4fc711a Plugin sez: Yo, dawg.

manually/main (split-plugin) $ tree
.
└── lib
└── index.js

1 directory, 1 file
manually/main (split-plugin) $ git remote add myown \
../remotes/myown

manually/main (split-plugin) $ git push -u myown \
split-plugin:master

Counting objects: 8, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (8/8), 617 bytes | 0 bytes/s, done.
Total 8 (delta 0), reused 0 (delta 0)
To ../remotes/myown
* [new branch] split-plugin -> master
Branch split-plugin set up to track remote branch master from myow…
manually/main (split-plugin u=) $

With git subtree

git-subtree/main (master u=) $ git subtree split \
-P lib/plugins/myown -b split-plugin

-n 1/ 14 (0)
-n 2/ 14 (1)
-n 3/ 14 (2)
-n 4/ 14 (3)
-n 5/ 14 (4)
-n 6/ 14 (5)
-n 7/ 14 (6)
-n 8/ 14 (7)
-n 9/ 14 (8)
-n 10/ 14 (9)
-n 11/ 14 (10)
-n 12/ 14 (11)
-n 13/ 14 (12)
-n 14/ 14 (13)
Created branch 'split-plugin'
a54c695c65db858a68720dd9b93061ea28d13243

git-subtree/main (master u=) $
git-subtree/main (split-plugin) $ git remote add myown \
../remotes/myown

git-subtree/main (split-plugin) $ git push -u myown \
split-plugin:master

Counting objects: 8, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (8/8), 556 bytes | 0 bytes/s, done.
Total 8 (delta 1), reused 0 (delta 0)
To ../remotes/myown
* [new branch] split-plugin -> master
Branch split-plugin set up to track remote branch master from myow…
git-subtree/main (split-plugin u=) $
git-subtree/main (split-plugin u=) $ git checkout master
git-subtree/main (master u=) $ git subtree pull --squash \
-P lib/plugins/myown myown master

From ../remotes/myown
* branch master -> FETCH_HEAD
Can't squash-merge: 'lib/plugins/myown' was never added.
git-subtree/main (master u=) $ git rm -r lib/plugins/myown
rm 'lib/plugins/myown/lib/index.js'

git-subtree/main (master + u=) $ git commit \
-m "Removing lib/plugins/myown for subtree replacement"

[master bf59e62] Removing lib/plugins/myown for subtree replacement
1 file changed, 1 deletion(-)
delete mode 100644 lib/plugins/myown/lib/index.js

git-subtree/main (master u+1) $ git subtree add \
-P lib/plugins/myown --squash myown master

git fetch myown master
From ../remotes/myown
* branch master -> FETCH_HEAD
Added dir 'lib/plugins/myown'

git-subtree/main (master u+3) $

So, which approach should I use?

Want to learn more?

--

--

--

I make cool stuff and teach others to (Git, Rails, JS/CoffeeScript/Node/etc).

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Christophe Porteneuve

Christophe Porteneuve

I make cool stuff and teach others to (Git, Rails, JS/CoffeeScript/Node/etc).

More from Medium

Understand how to use Git pull requests in 1 minute.

Version Controlling & GIT

Difference between XML and JSON (XML vs. JSON)

How to install Git