A memoir on the crusade to write our own Go framework
Doubling Down on Go
To go[0], or not to go? — A not-so-long time ago, I took part in the most exciting, and impactful, developer experience project I’d ever been involved in: Devbox. We set out to provide a remote development solution compelling enough to convince each and every one of our three hundred developers to move over to. Setting the record straight, it was a wild success, with the average amount of hours a developer wasted per week on their development environment reduced by more than 60%. You can read more about it here[1], straight from a developer’s bone box.
With little regard for the side effects of unbounded honesty, the most enjoyable part of that project, and its subsequent maintenance duties, personally, has been in learning and dealing with the quirks of the language it was written in, Go.
Having had experimented with virtually all kinds of languages, there’s just something special about it, in that I am able to just jump in any project and almost immediately be productive.
In Pipedrive, Go is the second language with the most repositories, nonetheless, it still remains significantly behind Javascript/Typescript in terms of adoption, being mostly relegated to either CLIs, such as the aforementioned Devbox, or services that supposedly have high compute needs.
A good question arises. Why is Javascript the king, even in my tribe, DevOps Tooling? Why don’t we just choose Go, after all, the entirety of the cloud software ecosystem is written in it, and that’s exactly what we are doing as well, tooling to aid our developers to develop on our microservices architecture. The answer is very straightforward, poor developer experience.
Pipedrive is an amalgamation of over 750 microservices; There have been multiple in-house frameworks written that alleviate the amount of boilerplate one needs to write, in order to seamlessly create, or update, a new microservice that adheres to already established standards and protocols, such as API response formats, logging, authentication, and others.
Writing a new service without all that cushioning, irrespective of language, is a pain. However, as glanced over, it did not stop hundreds of projects from using it, since the use-case is clear: services where high-performance and concurrency are paramount. But then, is that mutually exclusive with the level of developer productivity seen on our Javascript services? Absolutely not.
Sometime in the beginning of 2022, I managed to bring that topic up to our developer experience stakeholders, and found much support for the until then unthinkable: writing our own Go framework, in order to standardize our projects, attempting to tackle all of the developer productivity issues, that stem from the lack of tooling, at the same time:
- Significant wheel reinvention for routine boilerplate code
- Little cross-team knowledge sharing
- Uneven code quality across different repositories
A couple months later, this came to fruition in a spectacular way, with the most seamless mission I’d ever taken part in. Imagine working in a team where everybody is not only passionate about the subject at hand, but skilled too. I would liken the making of our framework to having had as much love poured into it as your grandmother’s cookies.
The Underlying Framework — Uber’s Fx
The namesake of our framework, fx[2], is one of the most divisive Go projects out there. It is a heavily opinionated take on Rob Pike’s options paradigm[3], whose gist is to use functions to customize struct’s attributes, that expands it to being a general solution to dependency injection.
Take the following struct:
Notice how both fields are private. In order to set them, one has to create a function type, with its sole argument being a pointer to the struct, and then two functions, used to set their respective attributes:
From that, we can create a nice constructor function:
This approach not only makes it easy to deprecate an attribute, just by removing the option setter, but also provides a convenient way to set defaults to structs.
What fx provides, is a set of abstractions to compose your application as an acyclic directed graph; Each node is a constructor function, with edges being the necessary inputs for it to be instanced. For instance, in order for `New` to provide `*Obj`, it requires, if any, `Option`s, the options themselves, are constructors too!
This composition is done automagically by fx in itself. That’s right, you do not build any sort of graph on your own, in fact, everything is quite simple, relying only on two primordial functions, and a canonical run call:
- `fx.Provide(…)` — Takes as an input any constructor function, and adds it as a node to the graph
- `fx.Invoke(…)` — Receives a function that emits a side effect, possibly having dependencies, and runs it after the application is composed
- `fx.Run(…)` — Accepts primordial functions, assembles the correct dependency graph in runtime, and then invokes all side effects.
In practice, a very simple `foobar` example would be as follows:
Executing this prints “obj: 2 two”. Once again, `fx.Run` assembles the dependency graph, which in this case is: `Foo` or `Bar` (both of them have the same precedence), `New`, since its dependency matches the signature of both `Foo` and `Bar`, therefore can only be instanced after, and then, at last, the invocation, that depends on the struct `Obj` having had been instanced.
Our Framework — Pdfx
The Boilerplate issue
Alright, but how can what has been introduced significantly help, aside from eliminating the very messy usual `main.go` files? In order to see that, we have to defy a common piece of wisdom that is often parroted by the go community: don’t return interfaces.
Allow me to tell you the opposite, do return interfaces. In order to grasp why, let’s jump right on to porting an actual tiny service running in production to pdfx, introducing the `pd` in `pdfx` along the way.
The service is called `schemer`, it is a simple registry for JSON schemas, with one single endpoint. You send a request, with the name of a schema, and it will fetch the most updated version from the github repository it is at, and return it to you.
In order to give some magnitude to the amount of non-code boilerplate that is necessary to bootstrap a service, I counted over twenty configuration and ignore files in the repository. Given that, back when that service was written, there was no standardized go service template, as there is for Javascript, each of these files would have to either be created from scratch, or copy pasted from other repositories.
The most immediate benefit of `pdfx` is how by ensuring that services use the same framework, therefore sharing as much code as it is reasonable, quite accurate assumptions about what configuration files are needed, alongside reasonable defaults, can be made.
Moving on to the code itself, `schemer` was built entirely by snagging as much code as possible from other high-profile pipedrive go services. Here is how its `main.go` looked like, with certain parts trimmed:
Notice the ungodly amount of boilerplate code: graceful shutdown, the server object, logger, and our own service discovery mechanism, that has been omitted. Every single service has these snippets, that is a wasteful amount of duplication. Furthermore, how could this be tested? Spinning the server up, then doing it black-box? That’s a possibility, but there could certainly be far more convenient ways to get that coverage up.
When testing for the functionality being provided, in isolation, it is not necessary to have the whole networking stack up, and that transitively implies that both service discovery and logger are unimportant as well. Integration testing for such a small and specialized service with no non-human consumers is a waste of time, it would be optimal for my development feedback cycle if I could quickly iterate over the core functionality locally.
Streamlining Boilerplate
Now, let’s take a look at how does that `main.go` file look like, ported over to `fx`
That is right, it’s only that. And no, we did not just move all contents to a function called `Module` under `internal` :).
Here’s `internal` in its entirety:
Using `fx`’s building blocks, `fx.Provide` and `fx.Invoke`, which are both of type `fx.Option`, we provide wrapped versions of Pipedrive boilerplate, removing all static copy-pasted code, as it can be seen from the imports section, under `pdfx`. In turn, this means that the semantics of calling `pdfx.Run` entail running the outlined graph with the actual live dependencies, remaining unchanged from the previous `main.go` file.
Boosting Productivity and Isolation in Testing
The biggest benefit of `fx` is that it allows for easily swapping out any nodes in the graph, as long as it remains an isomorphism. In human lingo, as long as the outputs of the constructors are interfaces, we can easily swap them out for mocks. Remember that talk about indeed returning interfaces from constructors?
The following snippet showcases a functional test that can be run entirely locally, as a regular go unit test, demonstrating just that:
There’s quite a lot going on there. Let’s start off from top to bottom, we are heavy users of gomock[4], that uses `codegen` to create mocks. There are two very important interfaces that are mocked, `gitRepository` and `schemataRepository`, the former provides the behavior `Clone`, that returns a repository worktree, and the latter returns a list of schema objects, which contain the necessary information to make the clone call, and then fetch the actual JSON schema file.
The function `pdtesting.Run` takes care of providing the `logger` mock, and many others, that can be reached for with the utility function `fx.Populate()`. The mocks swap out the original implementations with the `fx.Decorate` function, as it can be seen from `gitRepository` and `schemaRepository`, by intercepting the current node in the graph that has the same edges, same function signature, and swaps it out for the output of the given function.
The last piece of the puzzle is `request`, which is merely a handy way, inspired by Javascript’s fastify, to easily “inject” a http request.
As we can see, now Go developers in Pipedrive can be much more productive, by having much of the groundwork done for them, not needing to write as much boilerplate, and in writing tests with as much isolation, through mocking the dependency graph, as needed.
Summary
In this half-memoir-half-technical article, Pipedrive’s Golang framework is introduced. The main takeaways are:
- Developers who do not work with the main stack often have much poorer developer tooling, a fact which is often cluelessly used as an argument by main stack developers as an excuse to keep using the same hammer for any and all situations.
- The developed framework, pdfx, standing for pipedrive fx, is built upon uber’s easy to use high-level reflection-based dependency injection library, fx.
- Given constructors and functions that require these constructors, fx automatically assembles the dependency graph, and correctly resolves it during runtime.
- pdfx is a collection of boilerplate modules and their respective mocks, such as standard web framework, echo in our case, logger, service discovery, mysql utilities, and else, that are specifically tailored to be comfortably used with fx.
- Due to the fact that fx characterises a node in the graph exclusively by function signature, entire parts of the application can be easily mocked, simply by swapping them out for a mock. For instance, given that a mock implements the same interface as an actual dependency, a constructor that returns a dependency could be swapped out for one that returns a mock.
- The combination of seamless dependency injection, a repository for all common boilerplate, and a standard repository structure, brought the developer experience of writing Go in pipedrive to parity with Javascript, at least on paper.
