Geek Culture
Published in

Geek Culture

Visual representation of git log of simple mono-repo with multiple go modules

Go modules in mono-repo

Balancing development flexibility with structured release process

Go modules have come a long way since the August 2018 when the first module capability was launched as part of version 2.11. Traditionally, the recommendation has been one repo = one module. But, I think, over the past year, the Go module tooling along with GOPROXY protocol and infrastructure have matured to enable good support for multiple modules. This is an important implication for developing solutions in a mono-repo. This article is not an introduction to go module or advantage or disadvantage of mono-repo.

TL;DR

The following are the takeaways

  1. Go tools use version control features like tags, branch and revisions to identify module version and generate psuedo-versions.
  2. Use module specific branch to create and manage module development along with release specific branches (e.g. alpha, beta) for integration testing. This may require splitting code base across multiple branches at the cost of readability (which can be addressed by approaches identified below)
  3. Given the out-of-box support for Version Control System within the tools, consider restricted access to your mono-repo and use of GOPROXY protocol to serve artifact for a formal release cycle of packages for external/public consumption. This approach is currently limited by lack of general tooling (e.g. github does not support GOPROXY protocol for package registry).

If this sounds interesting, let’s get started. All the flows shown below are available in git repository go-demo-module and go-demo-module-client.

Modules

Modules are how Go manages dependencies.

A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.

A simple go.mod file

A simple module represented by go.mod file in root directory of module defines the module that is being exported, dependency on the version of go compiler along with other modules that it is dependent on and corresponding version.

Version to tag mapping

Directory structure of a repo with parent and child modules

Go uses branch, revisions and tags to identify the required version. In the scenario shown here, the following modules and corresponding versions can be defined
1. A go-demo-module v0.1.0 in go.mod file will translate to code being pulled corresponding to tag v0.1.0 but this code will not contain mod1 since that is identified as a different module.
2. An entry go-demo-module/mod1 v0.1.0 in go.mod file will translate to the code being pulled corresponding to tag mod1/v0.1.0.

This mechanism ensures that each module can be used independently within a repository without any ambiguity.

Git flow

Establishing a standard process for working in a mono-repo helps reduce friction within the team and improve productivity of team. This is one such flow that can be used to streamline working on mono-repo across multiple go modules

Initial setup

It is recommended that a branch (similar to “new-module” in the flow above) be created.

Base directory for new-modules

This branch will contain basic files like .gitignore and other template files that would be helpful for defining new modules.

In case an existing repository is being enabled, this can be achieved by creating a new branch from first commit and then reverting the commit.

Initial set of branches defined in mono-repo

The new-module can be used to create the various release branches like alpha and beta shown here. These release branches will be used later for defining and building packages for release by defining go.mod files with specific version of modules defined in child directory as described below.

Please ensure that the branch naming convention does not follow a standard that conflicts with go versioning mechanism.

New Module

Any new module can be created at any time by branching from new-module. The module would be created in the corresponding directory. In case of standard project layout this may translate to pkg/<module>, cmd/<module> and so on.

mod1 v0.2.0 with dependency in mod2 v0.1.0

After the development is complete the branch can be tagged accordingly that aligns with directory structure (e.g. mod1/v0.1.0, mod2/v0.2.0, pkg/mod1/v0.1.0, and so on).

Once tagged, these modules can then be used by other modules either within or outside the repo as shown here. In case of external dependency, a more standardized release process may be followed to ensure that all the packages being released have been adequately tested.

Release

The release process ensures that entire development package is well integrated and tested prior to packaging and release.

This, in case of go, can be achieved by defining a virtual module with dependency on other internal modules that should be part of release and performing system integration tests. In go-demo-module, the alpha branch shows one such approach for packaging and testing. This branch contains go.mod which defines the module (i.e. go-demo-module)and associated dependencies (i.e. mod1 v0.10 and mod2 v0.1.0), along with system integration test in alpha_test.go file. The alpha.go has been added to ensure that test can be run.

An alternate way is show in dev branch which establishes a process for module developers to submit a pull request to dev branch. As part of merging pull request, the corresponding module and associated version is added/updated to go.mod as dependency. This provides a more formal release process while also increasing the overall readability of project (since the module code is available as part of branch).

In case a more formal release process (such as above) is implemented, it is important to ensure that external project don’t form dependency on modules directly. This can be achieved by making the project private and releasing tested package as zip file and serve through GOPROXY protocol.

Go modules provide a very flexible way to decompose system developed in go and simplify the development by allowing developers to develop the component independently and then performing integration testing once ready.

--

--

--

A new tech publication by Start it up (https://medium.com/swlh).

Recommended from Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Shekhar Jha

Shekhar Jha

My current focus is cloud security. I have worked in identity and access domain and have experience running infrastructure for a small company.

More from Medium