Golang Modules & Immutability #2

Yesterday, I explored some ways to take advantage of an immutable Golang package store using Docker and Cloud Build. It seems as though there may be a way to take advantage of this immutability using deconstructed multi-stage builds.

Multi-Stage Builds

This is my boilerplate for Golang multi-stage builds and Google’s distroless

FROM golang:1.12 as build
WORKDIR /app
COPY . .
RUN GO111MODULE=on GOPROXY=https://proxy.golang.org go build app
FROM gcr.io/distroless/base
COPY --from=build /go/bin/app /
ENTRYPOINT ["/app"]

Each time this process is run, the golang image’s /go/pkg will be populated by packages relevant to the build.

Hypothesis: Interim containers are anonymous but must be persisted to disk. It should be possible to discover these interim containers. But…

Hypothesis: by deconstructing a multi-stage build, it should be possible to enforce persisting the interim containers and we could use such an interim to aggregate distinct package pulls from multiple builds into a single source.

Deconstructed Multi-stage Builds

I’m envisage something like this seed step done once with your first go.mod:

FROM golang:1.12 as origin
WORKDIR /app
COPY go.mod .
RUN GO111MODULE=on GOPROXY=https://proxy.golang.org go mod download

and build the image:

docker build --tag=golang:modules --file=Dockerfile.orig .

Now we have our baseline golang:modules.

And — subsequently — for each Go build— add this build’s packages to golang:modulesrather than golang:

FROM golang:modules as modules
WORKDIR /app
COPY go.mod .
RUN GO111MODULE=on GOPROXY=https://proxy.golang.org go mod download

NB I discovered that a directory containing only go.mod failed with e.g. go get ./... However, there is a go mod download command that does exactly what we need.

and (re)build the golang:modules (packages) image:

docker build --tag=golang:modules --file=Dockerfile.mods .

NB Yes, golang:modules is potentially going to become problematic with so many layers being added. Docker has an experimental docker build … -squash ... command that may help.

And then, to build this project:

FROM golang:modules as modules
FROM golang:1.12 as build
COPY --from=modules /go/pkg /go/pkg
WORKDIR /app
COPY . .
RUN GO111MODULE=on GOPROXY=https://proxy.golang.org go build app
FROM gcr.io/distroless/base
COPY --from=build /foo /
ENTRYPOINT ["/foo"]

and build the image:

docker build --tag=thisproject --file=Dockerfile.proj .

Conclusion

Using this — admittedly more complicated approach — we (should) be able to accrue distinct packages in golang:modules so that, once pulled, a package is retained and never pulled a second time even across distinct builds.

The deconstructed multi-stage build Dockerfile may be reconstructed by binding the two build statements together.

That’s all!