Using Go modules with Kubernetes API and client-go projects

Published in

Programming Kubernetes

5 min readOct 22, 2018

The world of Go and Kubernetes move really fast. Just a year ago, I published a story about using Go dep to organize package dependencies for Kubernetes API client projects. A year later, Go dependency management has taken a another major leap forward with the introduction of Go modules.

This write up explores the steps necessary to use Go’s new go mod tool, for dependency management, with Kubernetes client-go API projects.

Before continuing, I want to point out that the Go dev team has succeeded in making the transition to Go module work with little to no friction. Depending on your starting point (greenfield, existing project update), you may or may not need to make adjustments.

This writeup is not intended to be a Go module tutorial as there are plenty of materials on the subject. A good starting place is here .

Pre-requisites

Before you get started:

Install Go 1.11 or later
Create a directory outside $GOPATH/src for your project source
Or, run all Go tool commands with GO111MODULE=on env variable

For this write up, I created a directory outside of my local GOPATH. To keep things simple, I will reuse a simple PVC controller (discussed in this blog as part of my Building Stuff with Kubernetes API series) that watches for requested PVC quantity.

Starting from Dep (or other tools)

For many of you writing Kubernetes API tools with client-go, you are probably using Dep to manage your dependencies. Fortunately, the go mod tool can import dependency settings from Dep (and other tools like Godep and Glide). My sample project used dep with the following Gopkg.toml:

[[constraint]]
  name = "k8s.io/api"
  version = "kubernetes-1.9.0"[[constraint]]
  name = "k8s.io/apimachinery"
  version = "kubernetes-1.9.0"[[constraint]]
  name = "k8s.io/client-go"
  version = "6.0.0"

As you can see, the code uses an older version of client-go version 6.0.0 and Kubernetes APIs version 1.9.

The first thing to do is to initialize the project as a module. From the root directory of your code, use the following command:

$> cd ./pvcwatch
$> go mod init github.com/vladimirvivien/pvcwatch

The mod command will create a new file called go.mod and report its progress copying dependency info from Dep:

go: creating new go.mod: module github.com/vladimirvivien/pvcwatch
go: copying requirements from Gopkg.lock

Take a look at the content of go.mod generated from Gopkg.* files:

module github.com/vladimirvivien/pvcwatchrequire (
  github.com/PuerkitoBio/purell v1.1.0
... 
  k8s.io/api v0.0.0-20171214033149-af4bc157c3a2
  k8s.io/apimachinery v0.0.0-20171207040834-180eddb345a5
  k8s.io/client-go v6.0.0+incompatible
  k8s.io/kube-openapi v0.0.0-20180216212618-50ae88d24ede
)

The require section of go.modhas remained faithful to the declared version constraints from Gopkg.toml.

go mod will resort to the latest version of discovered packages that do not have any version information (from Dep or otherwise). If you don’t want that, you can update or downgrade to your preferred version (discussed later).

At this point, you can build the code as you would normally do. Except now, the build tool will show progress of package resolution:

> go build .
go: finding github.com/golang/protobuf v1.0.0
...
go: downloading github.com/modern-go/reflect2 v0.0.0-20180228065516-1df9eeb2bb81
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20180322165403-91ee8cde4354

If all packages are resolved, the build will succeed and you will have a binary that you can run.

Starting fresh

Starting with a greenfield scenario follows the same steps. First, initialize the project as a Go module:

go mod init github.com/vladimirvivien/pvcwatch
go: creating new go.mod: module github.com/vladimirvivien/pvcwatch

At this point the go.mod file is empty until the next Go tool command (build, get,test, etc) is invoked. Next, invoke the build command inside the project’s directory as before:

$> cd ./pvcwatch
$> go build .
go: finding k8s.io/apimachinery/pkg/util/runtime latest
...
go: downloading golang.org/x/sys v0.0.0-20181021155630-eda9bb28ed51
go: finding golang.org/x/text/unicode latest
go: finding golang.org/x/text/secure latest

At the end of the build process, if there are no dependency issues, you should get a built binary. In this scenario, go mod will pull the latest versions of all resolved packages automatically as seen in the updated go.mod file:

$> cat go.mod
module github.com/vladimirvivien/pvcwatchrequire (
...
    k8s.io/api v0.0.0-20181018013834-843ad2d9b9ae
    k8s.io/apimachinery v0.0.0-20181015213631-60666be32c5d
    k8s.io/client-go v9.0.0+incompatible
)

The previousgo.mod snippet shows that client-go v9.0.0, the latest tagged version, is being used (k8s.io/api and k8s.io/apimachinery are not SemVer’d yet, so the latest HEAD versions are used).

The trouble

You know it was coming.

Trouble will come when you want to adjust (upgrade/downgrade) client-go to a specific version. For instance, in the previous section, go mod selected v9.0.0 of client-go. But, let us say we want do downgrade to v7.0.0 because:

$> cd ./pvcwatch
$> go get k8s.io/client-go@v7.0.0

This, unfortunately, will create a version mismatch between client-go and its dependent packages k8s.io/api and k8s.io/apimachinery:

$> go build .
go: finding github.com/howeyc/gopass latest
go: finding k8s.io/client-go v7.0.0+incompatible
#../../pkg/apis/clientauthentication/v1alpha1/zz_generated.conversion.go:39:15: scheme.AddGeneratedConversionFuncs undefined (type *runtime.Scheme has no field or method AddGeneratedConversionFuncs)

Now, you have to manually figure out your dependency graph. Fortunately, client-go comes with a handy compatibility matrix that tells us exactly which versions we need (see following figure).

client-go compatibility matrix (source: https://github.com/kubernetes/client-go)

According to the matrix, client-go v7.0.0 is compatible with Kubernetes 1.10. So, let us downgrade the other dependent components to a matching version using go get (which supports either a branch name or a non-semver tag name).

Downgrade k8s.io/api to Kubernets-1.10:

# Downgrade with a tag name
$> go get k8s.io/api@kubernetes-1.10.9
go: finding k8s.io/api kubernetes-1.10.9
go: downloading k8s.io/api v0.0.0-20180828232432-12444147eb11
...# equivalent to using matching branch name
$> go get k8s.io/api@release-1.10
go: finding k8s.io/api release-1.10

Downgrade k8s.io/apimachinery to Kubernetes-1.10

> go get k8s.io/apimachinery@release-1.10
go: finding k8s.io/apimachinery release-1.10
go: downloading k8s.io/apimachinery v0.0.0-20180619225948-e386b2658ed2

Now, the build will work as expected. This was a simple fix since all other packages happened to work with no issues during the downgrade. However, you can imagine situations where things may not be as simple.

Conclusion

Dep took Go package management to great heights. Now, Go module and its deep integration with Go command-line tools, has catapulted Go dependency management to the stars. Doing this exercise gave me a sense of great potential for modules that will help scale the Go ecosystem.

While dependency misalignment will continue to be an issue during the early days, things will get better as popular packages are semver’d and project authors adopt Go modules. Efforts like Project Athens will provide registries that maintain module snapshots ensuring version durability even if original project goes away.

Looking forward to the day when the all Kubernetes related projects adopt fully adopts Go modules!

As always, if you find this writeup useful, please let me know by clicking on the clapping hands 👏 icon to recommend this post.