When Java needs to Go (or at least step aside)

This blog post was written under the Wandera brand prior to its acquisition by Jamf. That’s history, we are now #OneJamf”

Background

At Wandera, microservices are our daily bread and butter — microservices in Java to be more specific. Most of our services currently running in production clusters are based either on Dropwizard framework or Spring Boot. We are trying to continuously innovate and look for technologies and frameworks that can make our lives easier, but most of the time we tend to stay in a comfortable and (rather) safe Java ecosystem. This has changed significantly with a new project on the horizon called Wandera Private Access (WPA).

Wandera Private Access

(Picture: How WPA enables its users to control the access to their cloud-based or on-prem apps)

I wouldn’t want to put you off by product talk, but it can shed some light on our motivation to move from the Java ecosystem towards Go. WPA aims at replacing traditional VPNs with a cloud-based software defined perimeter providing a so-called “Zero Trust Network Access”. This allows its users to control the access to their cloud “Software as a Service” (SaaS) applications or on-prem applications in a more granular and dynamic way than when working with traditional VPNs. WPA relies on a global mesh network of interconnected nodes to provide this functionality, which brings a set of new requirements to the microservices used underneath this overlay network.

Motivation behind the shift

As some of you might have already guessed, this set of requirements is tightly coupled with features we want to be able to provide for the WPA customers. Probably the most important one is the ability to run our microservices in multiple instances spread across data centers around the world to provide as low latency as possible to our end-users. Being able to run our services in dozens of replicas implies great emphasis on resource consumption and cost effectiveness of the deployed services as well as on their performance. These requirements naturally point towards some lightweight language designed with performance and scalability in mind.

There are a few more reasons why Go quickly became our number one candidate for the switch.

The first reason is that we have chosen WireGuard — a modern, secure and state of the art VPN protocol for implementation of our overlay network. It’s originally written in C as a Linux kernel module but its Go version (WireGuard-Go) is heavily used across multiple platforms. As for the second reason, we already knew we would need to build our own DNS server incorporating some enhanced features. (Some of our previous POCs already leveraged the Coredns DNS resolver which is also written in Go.) This was yet another argument for working in Go in this project. Last but not least — our internal platform team already has a lot of experience with Go since it is the “lingua franca” of the Kubernetes world.

How it started

Even though this all seems like a bit of a no-brainer in retrospect, the steps leading towards this decision were rather small and gradual.

Our journey started with a comparison of multiple possible options spanning across the use of the original kernel version of WireGuard written in C, the Rust version of WireGuard library and the aforementioned WireGuard-Go. Our initial tests showed that WireGuard-Go performance is not comparable with the kernel version (for reasons I don’t want to dive into) but it was (performance-wise) on par with the Rust version. As there is almost no history of Rust in our company, we slowly started to gravitate towards Go implementation of the WireGuard protocol.

Go guild and initial Go standards

We’ve all probably been there. New sexy technology or language comes around and we immediately think it’s the one that will definitely make our dev lifes easier. However, there are some concerns too. Won’t we just end up with an immature technology with no community, standards and no senior developers to guide our steps through the new environment?

It’s relatively easy to find an expert for some Java ecosystem technology (or even hire one), with Go it’s a different story. We quickly realized that in order to move fast enough with our product, we would have to find a way to gain as much knowledge as possible from our midst (namely from our internal platform/tools teams) and secure prompt product development by sharing this knowledge among other devs as effectively as possible. Essentially, we needed to learn and deliver at the same time!

We came up with three “tools” which allowed us to do that:

• We became actively involved in our Go guild — we have already used guilds for other areas of expertise. They serve as a platform to share knowledge about technologies that are not necessarily limited to a single team or a product area. We applied the same idea with Go and our team members quickly became the most active guild contributors, driving discussions about and making decisions on topics that affected our development the most.
• We involved all team members into code reviews — especially from the very beginning of the project we wanted to make sure that nobody missed out on any important decision made during the discussions about the newly created code. That’s why each team member was invited to review every newly created pull request. While this might be seen as an unnecessary overhead, it helped us spread knowledge about new components, coding standards and strategic decisions quite effectively.
• We embedded our internal tools team Go expert from the beginning of the project. He helped us with all the initial design decisions when it comes to Go services, their architecture and code reviews. He provided a ton of materials for self-study, organised knowledge sharing sessions and fought hard against our Java-world paradigm which wasn’t useful in the Go universe.

As a result, a set of Go coding standards was created. We came to an agreement that helped us a lot during the development of new services (not only in our team). We created a set of guidelines that should help others decide whether Go is the language to go for when creating a new service (as this is definitely not a silver bullet that would make every problem you’re facing in your company “Go” away).

Thorough consideration of pros and cons and discussion with a broader audience showed that Go is beneficial/useful in some situations while in others it’s not. This turned Go into one extra tool in our development toolbox which can be used if a project’s requirements match identified advantages of Go language.

Hiring

Moving forward we found that another process which needs to be tweaked is that of hiring new devs. While in some cases mentioning Go during an interview can be seen as a huge incentive, in others it can scare a candidate away or suddenly downgrade a “Java Jedi master” to a “Go apprentice” (with implications on a pay grade etc.).

Our hiring process had to be adjusted so that we could communicate the requirements more clearly. The necessity to create a more appropriate set of interview questions for Go developer candidates came hand in hand with that. This forced us to think more about and identify the skills we are looking for in our new-joiners into Go-centric teams.

Lessons learnt

To summarize, these are some of the lessons we have learnt along the way:

• Go has proved to be the right tool for the job for this use case because it is naturally suited to building high performance, scalable network services.
• To know the syntax and to be an actual developer is a big difference — think twice before transitioning to a different language, as it might mean that the quality of the output will suffer and will not (at least at the beginning)follow your company’s code quality standards. It also means that the code is very likely to be heavily refactored after some time as the developers start having an itchy feeling every time they look at the code they wrote a month ago. Try to come up with a healthy balance between delivery speed and code quality.
• Be aware that some of your design decisions might not stand the test of time and will need to be reconsidered — try to count with this in your high-level estimates and planning sessions.
• Expect a fast and lively development process — some of the guidelines (e.g. the package structure of our microservices) will still be a point for discussion in near future and continue to change relatively dynamically (in comparison to our Java coding standards which are pretty stable).
• Be sure to communicate the differences between the Java and Go code styles to the involved developers — some code style standards were quite a shock for Java experienced developers.
• Expect a lot of decisions about the coding standards to be done at the beginning of the project if you already have a set of standards your Java (or other) services have to comply with. This can slow the development down significantly and can be frustrating or painful but has to be done.
• Intentionally become aware of the workings of your existing frameworks, tooling and features your microservices support — you might find that things you work with daily and take for granted in the Java ecosystem need to be rethought in the Go world. How do you approach health checks of your services, which solution do you use to build your HTTP clients, how do you collect metrics, logs, etc. You will have to find an appropriate alternative for all these use-cases in Go.

Summary

All in all, after a year and a half of development and almost 6 months after WPA became generally available to our beta customers, the decision to use Go seems to have been the right one.

Gopher image credits: Renee French

Experimenting with language change might not work for every company, but I would strongly encourage you to at least consider it at the beginning of new projects instead of just throwing it off the table and staying in the warm and comfy world of Java :)