dep 101
I’ve had the pleasure of working with several other gophers the last few months on a prototype dependency management tool named dep
.
dep
is part of the project started last year and is organized by Peter Bourgon. I was asked to join the team working on this project due to my involvement in a different tool, “godep”, the OG dependency management tool for Go, (inherited from Keith Rarick) and my work at Heroku serving our customers who use Go.
The other members of the team, aside from myself and Peter, are Jessie Frazelle, Andrew Gerrand and Sam Boyer. Andrew is part of the Go team @ Google. Jessie works at Google and is involved in large Go projects such as Docker and Kubernetes. Sam wrote and maintains gps, the constraint solver powering dep
.
The team has published a bunch of info about our progress over the course of our work. To date, various other tool authors and concerned parties have also been involved in different ways.
NOTE: dep
is pre-alpha, so anything below may or may not be true moments after this blog post is published. However, I will try to keep it up-to-date as development continues.
In the beginning…
Let’s take dep
for a drive and pretend we’re writing a web application using github.com/gorilla/mux
. Here is some pretend code to get us started:
The first thing to do after writing some code, or when using dep
for the first time on an existing project is to run dep init
.
dep init
adds the current version of any of the project’s dependencies found in $GOPATH
to the manifest.json
file. Because I have github.com/gorilla/mux
in my $GOPATH
, the manifest.json
file includes it. I recently ran go get -u github.com/gorilla/mux
, so the version of github.com/gorilla/mux
in my $GOPATH
is at master
. If the version in my $GOPATH
matched a Semver compatible tag (ex: v1.2.3
), that tag’s name would have been used instead.
dep
works across architectures and go versions. Using github.com/gorilla/mux
with older versions of Go (< 1.7.0) pulls in the github.com/gorilla/context
package. When I last rango get -u github.com/gorilla/mux
I was running Go 1.7.5 so the github.com/gorilla/context
package is NOT in my $GOPATH
. Because this could be a dependency necessary to compile the project, it is included in the lock.json
file. In situations like these, if a dependent project has a semver compatible release tag, dep
chooses the latest release. In this case that is v1.1
of github.com/gorilla/context
.
Because github.com/gorilla/mux
does not include a manifest.json
file dep
don’t know if github.com/gorilla/mux
currently works with github.com/gorilla/context
@ v1.1
. Barring an override (see below), dep
honors the constraints found in dependencies’ manifest.json
files.
dep init
includes all dependencies, analyzed recursively and the exact versions being used in the lock.json
file.
For the example app, this creates the following two files:
Ensuring the project can build
After dep init
is run, dep ensure
should be run to populate the vendor/
directory with a copy of packages required to build your project. This ensures that any of your project’s dependencies are included in the lock file and vendor directory. Any time you want to ensure that you have all of your dependencies recorded, run dep ensure
.
Adding another dependency
You don’t have to do anything up front to add another dependency, just start using it in your code. When it’s time to check in your work though you need to run dep ensure
to update the lock.json
file and vendor/
. This will lock the project to the latest released version of each dependency.
In the sample app I added a sub package math
that can be used to add, subtract or retrieve a named value and a HTTPHandler that can be used in main.go
. After writing that code adep ensure
run produced these changes to the lock.json
file and the vendor/
directory.
But what if the latest doesn’t work for me?
If you need to specify a version, you can use the alternate form of the ensure
command like so: dep ensure github.com/gorilla/mux@^1.3.0
. That command modifies the manifest.json
file, constraining dep
to use ^v1.3.0
of github.com/gorilla/mux
, resolves the dependency tree and updates the dependencies in vendor/
to reflect any differences resulting from the change. On the example application, that looks like this because dep
has already chosen the latest available version, 1.3.0
. Future updates (dep ensure -update
) however will no longer track the master branch and instead use semver tagged releases ≥ 1.3.0 and < 2.0.0.
dep
mostly uses Rust’s cargo
operators for selecting versions of dependencies. These include ^
, ~
and =
. To have a more restrictive, forward compatible match than ^
use ~
. For example dep ensure github.com/com/gorilla/mux@~1.2.0
will match any version ≥ 1.2.0 and < 1.3.0. To lock to a specific version use the =
prefix (e.x. dep ensure github.com/com/gorilla/mux@=1.2.0
). In the future it’s planned that dep
will default to ^
when no prefix is specified.
Keeping current
To keep a project’s dependencies up to date use dep ensure -update
, which updates all dependencies to the latest versions allowed by the constraints in manifest.json
, ignoring the contents of lock.json
. New versions are written to vendor/
and the appropriate meta data updated in lock.json
.
In the future it should be possible to dep ensure -update
a single dependency.
Status
When a dependency is missing, dep status
tells you which project and which packages contained inside are missing. For instance, here is what dep status
shows after this commit to the example app:
When the project’s lock.json
is up-to-date, the dep status
command shows you a list of all dependencies, as projects, and for each:
- the constraint(s) applied
- the selected version
- the selected revision
- the latest version or revision available
- the number of packages used.
After adding bolt to the sample project and running dep ensure
to update lock.json
and vendor/
, dep status
looks like this:
These two modes may be combined in the future.
Removal
dep remove <dep>
removes the dependency from manifest.json
, lock.json
and vendor/
, once it’s no longer used. Using the example app this commit removes the usage of github.com/gorilla/mux
from the application and the next commit is the result of dep remove github.com/gorilla/mux
. Because github.com/gorilla/mux
is removed, github.com/gorilla/context
is no longer needed and is also removed. If a dependency is still being used when dep remove
is run, the command will fail. Removal can be forced with the -force
flag, which results in the constraint being removed from manifest.json
. However, since the dependency is still in use it's still listed in lock.json
and copied into vendor/
.
In the future….
We’re still in an experimental phase with dep
, there are many issues open and much work to do still. I hope this post clearly showed you some examples of how the tool is expected to be used. This usage may change over time via community feedback and pull requests. If you have some spare cycles and are interested in working on some Go tooling please drop by #vendor
on gopher slack and pick up an issue or two to work on.
Thanks for reading!