From GopherCon Europe 2023 to Your Codebase

In the dynamic world of IT, staying updated with the latest trends and technologies is not just an option, it’s a necessity. This year, I had the unique opportunity to attend my first ever international conference, GopherCon Europe 2023. This conference, held in the vibrant city of Berlin, brought together some of the brightest minds in the Go programming community and was a melting pot of ideas, insights, and innovations.

Over the course of two days, I attended numerous talks, engaged in enlightening discussions, and even got my hands on some code. The experience of being in a foreign country, surrounded by a diverse group of tech enthusiasts, was as enriching personally as it was professionally.

In this blog post, I’ll share my key takeaways from the conference, focusing on practical strategies and examples that can help enhance your existing codebase. So, whether you’re a fellow Go programmer or just someone interested in IT, read on to discover how the insights from GopherCon Europe 2023 can be translated into actionable improvements for your code.

Golden hour in Berlin, Source: lostluck at Twitter

Keynote: State of the Go Nation (Cameron Balahan)

The conference kicked off with a keynote address by Cameron Balahan that set the tone for the rest of the event. Balahan emphasized that there would be no breaking changes in Go 2.0, a relief for many developers who have built extensive projects in Go. This commitment to stability is one of the reasons why Go has become a favorite among many programmers.

Keynote: State of the Go Nation, Source: fmt_sprintf at Twitter

Balahan also highlighted a significant focus on security and performance in the future development of Go. These are two areas that are critical in today’s fast-paced, data-driven world, and it’s reassuring to see that the Go team is prioritizing them.

One of the most heartening takeaways from Balahan’s keynote was the mention of the high satisfaction rate among Go programmers. As a Go developer myself, I can attest to the joy of working with a language that is efficient, powerful, and constantly improving.

Towards Modern Development of Cloud Applications With Service Weaver (Robert Grandl)

Robert Grandl’s talk was a deep dive into the world of microservices and distributed apps. He introduced Service Weaver, a framework designed to simplify the process of writing microservice applications. The beauty of Service Weaver lies in its ability to deploy modular binaries as a set of microservices, offering a new level of flexibility in application development.

One of the key insights from Grandl’s talk was the idea of delaying the decision between monolith and microservices architecture as long as possible. This approach allows developers to avoid the trade-offs associated with each architecture until they have a clearer understanding of their needs.

Service Weaver also stands out for its ability to reduce the amount of code needed in a project. It replaces configuration files with code generation, leading to a more streamlined development process. This approach is not only more efficient but also results in applications that are extremely resource-friendly, with lower CPU and memory usage.

In addition, Service Weaver comes with built-in logging, metrics, and telemetry/tracing via OpenTelemetry, further enhancing its appeal as a comprehensive solution for modern cloud application development.

You can learn more about Service Weaver on their website.

API Optimization Tale: Monitor, Fix and Deploy (Maciej Rzasa)

In a captivating talk by Maciej Rzasa, three key principles stood out: simplicity, learning, and tradeoff. These principles, as Rzasa demonstrated, are crucial in the realm of API optimization.

Simplicity was a recurring theme in Rzasa’s talk. He suggested starting with REST and only moving to GraphQL when REST no longer suffices. This approach emphasizes the value of simplicity in technology choices. It’s a reminder that the simplest solution is often the best starting point, and more complex solutions should only be adopted when necessary.

Learning was another key principle. Rzasa stressed the importance of testing in a real environment, on production, to gain genuine insights. He reminded us that while automated tests are crucial, they cannot replace the human touch entirely. This point underscores the need for a balanced testing strategy that combines the best of both manual and automated testing, and the importance of continuous learning in a rapidly evolving field like software development.

Finally, Rzasa discussed the concept of tradeoffs. Using the example of sending all notifications exactly at 17:00, he illustrated how an ideal scenario from a business perspective could overload the system. Recognizing such trade-offs is crucial in software development, as it allows us to balance different needs and make informed decisions.

In addition to these key principles, Rzasa also emphasized the need for quick response in a fast-paced environment, even if it means bringing down production temporarily. This approach prioritizes rapid reaction, including fixing the master, over maintaining a potentially flawed system.

Swag was of course available!, Source: gopherconeu at Twitter

Go Right Ahead! Simple Hacks To Cut Memory Usage by 80% (Yiscah Levy Silas)

Yiscah Levy Silas’s session was a treasure trove of practical tips and strategies for optimizing memory usage in Go. She shared a range of techniques, from field alignment to the judicious use of pointers, all aimed at making your Go programs more efficient and resource-friendly.

When it comes to memory optimization in Go, one of the first things you’ll encounter are edge cases and unusual input data. These can be challenging to test locally, but they’re a crucial part of the optimization process. It’s also important to remember that there’s almost always a trade-off involved. Improving memory usage can often come at the cost of code readability and understandability.

Reducing memory usage isn’t a one-size-fits-all solution. In some cases, it can lead to other performance issues, such as an overloaded disk due to read cache. And when you’re working with third-party libraries, optimization can become even more challenging.

Understanding the Go memory model is key to effective memory optimization. This includes getting to grips with the hybrid stack-heap memory management and the role of the garbage collector. Knowing how these work behind the scenes can give you a significant advantage.

With this foundation in mind, let’s dive into some practical techniques I learned from Yiscah Levy Silas’s talk at GopherCon Europe 2023. These strategies, from field alignment to the judicious use of pointers, are all aimed at making your Go programs more efficient and resource-friendly.

Field Alignment

Levy Silas explained how the arrangement of parameters in a struct can affect memory usage. This is due to the way memory is allocated in Go. When you declare a bool (which takes 1 bit) followed by a float64 (which takes 8 bytes), Go will allocate 8 bytes for the float64 and 1 byte for the bool (because 1 byte is the smallest addressable unit of memory). However, if you alternate between bool and float64, you end up using more memory than necessary.

Here’s an example of inefficient field alignment:

type InefficientStruct struct {
    a bool     // 1 byte
    b float64  // 8 bytes
    c bool     // 1 byte
    d float64  // 8 bytes
}

In this case, Go will allocate 1 byte for each bool and 8 bytes for each float64, resulting in a total of 18 bytes.

A more efficient arrangement would be to group similar types together, like placing all float64 parameters first, followed by bool parameters. This way, the bools can share the same byte, leading to memory savings.

type EfficientStruct struct {
    b float64  // 8 bytes
    d float64  // 8 bytes
    a bool     // 1 byte
    c bool     // 1 byte
}

In this case, Go will allocate 8 bytes for each float64 and only 1 byte for both bools, resulting in a total of 17 bytes.

To make this process easier, you can use a field alignment fix tool.

Size Declarations

Another tip was to be explicit about the size of the data structures you’re creating. If you know that a data structure will reach a certain size, it’s more efficient to initialize it with that size using the make function. This approach can prevent Go from allocating unnecessary memory. Built-in benchmarking can help determine the correct size.

Pointers

When working with pointers in Go, it’s important to remember that they consume memory. While pointers can be incredibly useful, there are times when it might be more efficient to pass a copy of an object to a function rather than a reference. This approach can be beneficial because the function can quickly dispose of the copied object once it’s done, saving memory.

There’s also a handy tool in Go called sync.Pool that can help manage the creation of new pointers. It's designed to cache allocated but unused items for later reuse, reducing the pressure on the garbage collector.

As with many aspects of programming, deciding when to use these strategies can be a bit of a balancing act. Benchmarking can be a great help here, providing concrete data to inform your decisions.

Here’s a simple example to illustrate the difference between passing by value (copying) and passing by reference (using pointers):

// Passing by value (copying)
func processCopy(obj MyObject) {
    // Function body
}

// Passing by reference (using pointers)
func processPointer(obj *MyObject) {
    // Function body
}

In the first function, processCopy, a copy of obj is created and used within the function. Once the function finishes, the copy is discarded, freeing up memory. This can be more memory-efficient, especially for small objects.

In the second function, processPointer, a pointer to obj is used. This means that the function is working with the original obj, not a copy. While this can be more performance-efficient when working with large objects, it can be less memory-efficient because the memory for the object needs to be kept alive as long as the pointer exists.

Regex on Repeat

Regular expressions can be quite memory-intensive. This is especially true if you’re compiling a new regular expression each time it’s used. A more efficient approach is to compile the regular expression once and reuse it. This way, you’re not constantly creating new objects, which can put a strain on memory.

// Inefficient: Compiling the regex each time it's used
func processInefficient(input string) {
    re := regexp.compile("pattern")
    // Use re
}

// Efficient: Compiling the regex once and reusing it
var re = regexp.compile("pattern")

func processEfficient(input string) {
    // Use re
}

Overuse of GoRoutines

While goroutines are a powerful feature of Go, they can lead to increased resource usage. Too many goroutines can exhaust the heap, leading to out-of-memory exceptions. Levy Silas recommended limiting the number of goroutines, implementing a timeout or cancellation mechanism, and monitoring and profiling goroutines using tools like pprof.

Go Sync or Go Home: Advanced Concurrency Techniques for Better Go Programs (Yarden Laifenfeld)

Concurrency is a cornerstone of Go, and mastering its effective use can significantly enhance the performance of your programs. In Yarden Laifenfeld’s talk, I delved into several advanced concurrency techniques and their application in Go.

WaitGroup and ErrGroup: These are two useful tools for managing groups of goroutines. WaitGroup allows you to wait until a group of goroutines has finished executing. ErrGroup does the same, but it also handles errors, returning the first error that occurs in the group of goroutines.

One variant of ErrGroup that stood out to me was errgroup.WithContext. This function not only waits for all goroutines to finish and handles errors, but it also cancels all running goroutines if one of them returns an error. This can save resources by preventing unnecessary work.

Another useful function is SetLimit, which limits the number of goroutines that can run in parallel. It has two methods: Go, which waits until there's room to start a new goroutine, and TryGo, which immediately returns false if there's no room to start a new goroutine.

SingleFlight: This is a powerful tool for preventing unnecessary work. It ensures that a function is only called once for each unique key. If the same function is called multiple times with the same key, SingleFlight will only execute it once and share the result with all callers. This can be a big performance boost if the function is expensive to run.

Cond: This is a more advanced tool for synchronizing goroutines. It allows one goroutine to wait for a condition while another goroutine signals when that condition is met. However, it’s worth noting that Cond can't target a specific goroutine - it can either signal one random waiting goroutine or broadcast to all waiting goroutines.

Brief Summary of Other Noteworthy Talks

GopherCon Europe 2023 was packed with insightful presentations, and while I won’t delve into each one, several deserve a mention.

During the Panel Discussion, a wealth of package recommendations emerged. Standouts included runtime/metrics, testing/fstest, testing/iotest, testing/quick, and stretchr/testify. An intriguing point was the ability to retroactively add interfaces to third-party structs.

In a session on Vulnerability Management for Go, the spotlight was on govulncheck, a tool for vulnerability checking in Go. Although not yet part of the core, it's a tool to watch.

Julie Qiu on Go Vulnerability Management, Source: DomeMastrangelo at Twitter

The upcoming Go 1.21 release will introduce slog, a new logging library. Highlighted in the A Fast Structured Logging Package session, slog is a fast, structured logger (key-value), with logger.LogAttrs promising to boost logging speed.

For game enthusiasts, the Go Beyond the Console: Developing 2D Games With Go session introduced several libraries for game development in Go, including yohamta/donburi, Ebitengine, and faiface/pixel.

The Useful Functional-Options Tricks For Better Libraries session explored the use of functional options for better library design and introduced the concept of multi-errors, a new feature in Go 1.20.

Finally, the Lightning Talks were a rapid-fire round of insights and recommendations. Two highlights for me were fyne, a GUI library for Go, and the upcoming Go 1.21 release, which promises improved generics.

Wrapping Up

GopherCon Europe 2023 was a testament to the vibrancy and dynamism of the Go community. The conference brought together Go enthusiasts from around the world, offering a platform for sharing knowledge, presenting innovative ideas, and fostering connections.

The event provided a comprehensive exploration of Go, from advanced concurrency techniques to practical hacks for reducing memory usage. It highlighted the power and flexibility of Go, and its continuous evolution to meet the demands of modern programming.

The insights and techniques shared at the conference are valuable for all Go developers, whether seasoned experts or newcomers to the language. They offer practical ways to enhance the efficiency, readability, and maintainability of Go programs.

As we look forward to GopherCon Europe 2024, it’s exciting to imagine what new developments and insights the next year will bring!