Building a web app: Golang, GitLab & Docker
Part 1: Environment: CI & static analysis
Quite a big part of modern software development is knowing what tools to use and how to glue them together. The services and the apps we usually build don’t contain many novel techical problems. We just need to choose the right tools for already solved problems and to work with them.
In this blog series I will try to overview the common problems of a web app development and tools to solve them. It is obviously useful for the project at the very beginning. If your project is already up and running, the series might help to uninvent a few wheels too.
We will use git and GitLab for source control (gitlab.com). It could be installed locally at it has a CI built in.
If you plan to use GitHub and TravisCI, most of the steps still apply anyway, so keep reading! ;)
We begin with setting up a CI runner with static analysis tools. Why?
It is much easier to protect the codebase from inflow of an obviously wrong code, than to fix all the linter issues afterwards. Linters can slow down the development for a few days. However they prevent potential bugs on a very early phase, it saves a lot of debugging time.
Static analyzers can also find security related issues. This feature is very useful, if you don’t have a person who can pen-test your app.
Also, static analyzers can be set up at any phase of the project, even before any code is written.
For Golang, we will use 4 static analysis tools that complement each other.
gofmt & goimports (golang.org/x/tools/cmd/goimports)
These tools help with the file formatting and correct imports. I strongly recommend to hook running
goimports when you save the file, but having an additional check when the merge request is submitted is very useful.
Golint will catch all the style errors that aren’t caught with
go fmt, such as variable naming, comments, etc. It finds more issues like ignored errors and many more.
goconst tries to save you from hardcoding the strings over and over again. It finds the repetitions and asks you to extract it to constants.
This one is a very powerful and flexible scanner. Among other things, it reports security issues. It is very important to catch these issues on a very early phase.
Huh, that’s a lot of linters!
Luckily, there is one tool run all of these tools together. It is called gometalinter (github.com/alecthomas/gometalinter). We will use it in our article.
Step 1: Create a docker image with linters
Create the file called
Dockerfile (no extension) in the root directory of your code.
# Inherit the container from a pre-built `golang` image.
# Install the static analysis tool(s)…
RUN go get -u -v github.com/GoASTScanner/gas
RUN go get -u -v golang.org/x/tools/cmd/goimports
RUN go get -u -v github.com/golang/lint/golint
# ...and one tool to rule them all!
RUN go get -u -v github.com/alecthomas/gometalinter
COPY . .
This creates a directory
/go/src/<your-gitlab-host>/<path-to-the-repo> inside the container, makes it current and copies all the current folder contents there. So we will end up with an image with the golang environment, our linters and our code.
Step 2: Make a shell script to build and run tests in the container
Now we will make a shell script which will run the linters (
run-static-analysis.sh). Put it in the same directory where the
docker build -t my-project-linters .
docker run my-project-linters gometalinter \
--exclude=".+should have comment or be unexported \(golint\)" \
Let’s dive into the contents a little bit.
docker build -t my-project-linters — build a Docker image from the Dockerfile in current directory and tags it as
my-project-linters (needed for the next step).
docker run my-project-linters <command> — runs
<command> inside a container tagged as
gometalinter --exclude=".+should have comment or be unexported \(golint\)" ./... --disable-all -enable=… - this excludes not very useful warning from
golint that every exported method should have a comment. It also specifies that only
go vet and
goimports should be launched. Otherwise there will be an error that the certain linter is not installed.
Try running it locally:
chmod a+x run-static-analysis.sh
You should see the output like this:
Sending build context to Docker daemon 7.378MB
Step 1/7 : FROM golang:latest
latest: Pulling from library/golang
Status: Downloaded newer image for golang:latest
Step 2/7 : RUN go get -u -v github.com/GoASTScanner/gas
---> Running in c336ef1584e1
Removing intermediate container c336ef1584e1
Step 3/7 : RUN go get -u -v golang.org/x/tools/cmd/goimports
---> Running in 884c9ee8f6b0
Removing intermediate container 884c9ee8f6b0
Step 4/7 : RUN go get -u -v github.com/golang/lint/golint
---> Running in 9fb39dc6568e
Removing intermediate container 9fb39dc6568e
Step 5/7 : RUN go get -u -v github.com/alecthomas/gometalinter
---> Running in 140a331c5c94
Removing intermediate container 140a331c5c94
Step 6/7 : WORKDIR /go/src/github.com/mandrigin/kbk2/backend
Removing intermediate container 2f47d3b6cdaf
Step 7/7 : COPY . .
Removing intermediate container e62e68a9f1cd
Successfully built 24558cb5a437
Successfully tagged my-project-linters:latest
…and the errors actually found in the code you are checking:
common/gdrive.go:23::error: no formatting directive in Printf call (vet)
main.go:32:1:warning: comment on exported type Session should be of the form "Session ..." (with optional leading article) (golint)
api/comments.go:12:2:warning: struct field Id should be ID (golint)
api/comments.go:20:2:warning: don't use ALL_CAPS in Go names; use CamelCase (golint)
api/comments.go:20:2:warning: exported const COMMENT_TYPE_POSITIVE should have comment (or a comment on this block) or be unexported (golint)
Now, we are ready to set it up for the CI environment.
Step 3: Run static analysis on every Merge Request
We need to install and register a CI runner. The instructions can be found in the GitLab documentation.
Keep in mind that the runner executor (step 8) should be set up to
docker and share the docker socket (see more here). We need share the socket because we will need to start a new container (aka our
my-project-linters) from another container (a CI runner container). Sounds complicated, but it is very simple to do. Just run this code on the CI machine to register the runner:
sudo gitlab-ci-multi-runner register -n \
--url https://gitlab.com/ \
--registration-token REGISTRATION_TOKEN \
--executor docker \
--description "My Docker Runner" \
--docker-image "docker:latest" \
More information about why we need to use the socket sharing is discussed in this article.
Now go back to your repository and create a file named
.gitlab-ci.yml at the root directory of the repo.
Now commit your branch and make a merge request. You should see that it got a nice blue progress indicator.
Step 4: Don’t allow to merge requests that fail static analysis
Go to the GitLab settings and check the checkbox titled “Only allow merge requests to be merged if the pipeline succeeds”.
Now you are set up.
In the future posts, I will show how to add unit and functional tests to our CI environment and solve some typical security-related issues.
Stay tuned and thank you for reading this!