Cross compiling go and cgo targeting armv7l & musl libc

One of the greatest things about writing go code is the ability to build and deploy small and fast binaries on a wide range of target systems from Mac to Linux to that W-word to <insert whatever else here>.

Recently, I acquired a Raspberry Pi 3 Model B (Rpi) and have been playing around with writing some basic software to run on the board. Getting go to run was easy. Installing musl libc and compiling C code against it on the Rpi was also a breeze. But, when it came to bringing the two together with go calling C and building the binary against musl instead of glibc, well, there was some complexity.

As it turns out, making this all work was fairly straight-forward, but there were a few gotchas. I’ve put together a really simple example to show how to make it all work, but much more complexity could be included. In the case I was fiddling with, I compiled a gRPC server as a single binary to run on the Rpi; the server then listened for calls from external machines, processing arguments received in a protobuf payload, and then calling C to do some other operations before returning results. This ended up working like a charm, communicating flawlessly between client and server.

Let’s start with a basic main.go, which should be familiar:

package main
import "fmt"
func main() {
     fmt.Println("Hello from Go!")
}

Run this and you will get what you expect… a nice little message in your terminal saying hello. Now, let us add some more logic with a simple, and contrived example:

main.go

package main
import (
"fmt"
"github.com/bsandusky/xcompile-rpi-cgo-musl/funcs"
)
func main() {
fmt.Printf("main():\tHello from Go!\n")
defer fmt.Printf("main():\tI am done now! Bye-bye.\n")
funcs.CallGoCode()
funcs.CallCCode()
}

funcs/gocode.go

package funcs
// #include "../ccode/implementation.c"
import "C"
import "fmt"
// CallGoCode is your typical go func that will be called from another package
func CallGoCode() {
fmt.Println("CallGoCode():\tThis is go code that lives in
another package called by main.")
}
// CallCCode is a wrapper function for a C call
func CallCCode() {
fmt.Printf("CallCCode():\tThis is a go wrapper function for a call to C code that is defined elsewhere.\n")
     C.c_function_call() // Call C function
     fmt.Printf("CallCCode():\tThis is Go, again.\n")
}

ccode/header.h

#ifndef HEADER_H
#define HEADER_H
void c_function_call(void);
#endif

ccode/implementation.c

#include <stdio.h>
#include "header.h"
void c_function_call(void)
{
     printf("c_function_call():\tHello from C! This is C running on RPi 3 Model B using musl libc!\n");
}

The basic ideas is that main is calling a go function, then a go wrapper for a C function, which is declared in a header and implemented in a c source code file. The output of all of this rigamarole is the following:

main(): Hello from Go!
CallGoCode(): This is go code that lives in another package called by main.
CallCCode(): This is a go wrapper function for a call to C code that is defined elsewhere.
c_function_call(): Hello from C! This is C running on RPi 3 Model B using musl libc!
CallCCode(): This is Go, again.
main(): I am done now! Bye-bye.

As you can see, the calls go between go and C as expected. Now, about the compilation bit. I used the following the compile the binary for Rpi:

env CC=arm-linux-musleabihf-gcc GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 vgo build -o xcomp --ldflags '-linkmode external -extldflags "-static"' .

Whoa boy! Let’s break this down:

  1. First, I am setting the compiler (arm-linux-musleabihf-gcc) which I installed via Homebrew (instructions below).
  2. Then the typical GOOS and GOARCH flags for cross-compilation. I added GOARM in for good measure.
  3. The next bit, CGO_ENABLED is super important. Without this the cross-compilation fails.
  4. I am using vgo build, but go build would work, as well. -o xcomp specifies the name of the output file that I am generating.
  5. Finally, I pass the static link flags and the current directory .

All of this and the brew command to get the toolset are available in the readme of the sample repo here.

Once the binary was compiled, it was as simple as moving it to the RPi using samba or scp or whatever other file transfer tool. A quick ./xcomp and voilà!

Hope this was helpful, if you’re looking to do the same.