Calling Go Functions from Other Languages

Vladimir Vivien
Feb 24, 2017 · 8 min read

Starting with version 1.5, the Go compiler introduced support for several build modes via the -buildmode flag. Known as the Go Execution Modes, these build modes extended the go tool to compile Go packages into several formats including Go archives, Go shared libraries, C archives, C shared libraries, and (introduced in 1.8) Go dynamic plugins.

This post is about compiling Go packages into C shared libraries. With this build mode, the compiler outputs a standard shared object binary file (.so) exposing Go functions as a C-style APIs. Here we discuss how to create Go libraries that can be called from C, Python, Ruby, Node, and Java.

All source code for this writeup is available on GitHub.

The Go Code

  • The package must be amain package. The compiler will build the package and all of its dependencies into a single shared object binary.
  • The source must import the pseudo-package “C”.
  • Use the //export comment to annotate functions you wish to make accessible to other languages.
  • An empty main function must be declared.

The following Go source exports functions Add, Cosine, Sort, and Log. Admittedly, the awesome package is not that impressive as you will see. However, its diverse function signatures will help us explore type mapping implications.

package mainimport "C"import (
"fmt"
"math"
"sort"
"sync"
)
var count int
var mtx sync.Mutex
//export Add
func Add(a, b int) int { return a + b }
//export Cosine
func Cosine(x float64) float64 { return math.Cos(x) }
//export Sort
func Sort(vals []int) { sort.Ints(vals) }
//export Log
func Log(msg string) int {
mtx.Lock()
defer mtx.Unlock()
fmt.Println(msg)
count++
return count
}
func main() {}// See awesome.go GitHub

The package is compiled using the -buildmode=c-shared build flag to create the shared object binary:

go build -o awesome.so -buildmode=c-shared awesome.go

Upon completion, the compiler outputs two files:awesome.h, a C header file and awesome.so, the shared object file, shown below:

-rw-rw-r —    1362 Feb 11 07:59 awesome.h
-rw-rw-r — 1997880 Feb 11 07:59 awesome.so

Notice that the .so file is around 2 Mb, relatively large for such a small library. This is because the entire Go runtime machinery goodness and dependent packages are crammed into a single shared object binary (similar to compiling a single static executable).

The header file

/* Created by “go tool cgo” — DO NOT EDIT. */
...
typedef long long GoInt64;
typedef GoInt64 GoInt;
...
typedef struct { const char *p; GoInt n; } GoString;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;
...
extern GoInt Add(GoInt p0, GoInt p1);
extern GoFloat64 Cosine(GoFloat64 p0);
...

The shared object file

$> file awesome.so
awesome.so: ELF 64-bit LSB shared object, x86–64, version 1 (SYSV), dynamically linked, BuildID[sha1]=1fcf29a2779a335371f17219fffbdc47b2ed378a, not stripped

Using the nm and grep commands, we can inspect that our Go functions got exported.

$> nm awesome.so | grep -e "T Add" -e "T Cosine" -e "T Sort" -e "T Log"
00000000000d0db0 T Add
00000000000d0e30 T Cosine
00000000000d0f30 T Log
00000000000d0eb0 T Sort

Next, we will explore several examples showing how to invoke the exported functions from other languages.

From C

Dynamically linked

#include <stdio.h>
#include "awesome.h"
int main() {
GoInt a = 12;
GoInt b = 99;
printf("awesome.Add(12,99) = %d\n", Add(a, b));
printf("awesome.Cosine(1) = %f\n", (float)(Cosine(1.0)));
GoInt data[6] = {77, 12, 5, 99, 28, 23};
GoSlice nums = {data, 6, 6};
Sort(nums);
for (int i = 0; i < 6; i++){
printf("%d,", ((GoInt *)nums.data)[i]);
}
GoString msg = {"Hello from C!", 13};
Log(msg);
}
// See client1.c on GitHub

Next the code is compiled, specifying the shared object library:

$> gcc -o client client1.c ./awesome.so

When executed, the binary links to the awesome.so library producing the output below.

$> ./client
awesome.Add(12,99) = 111
awesome.Cosine(1) = 0.540302
awesome.Sort(77,12,5,99,28,23): 5,12,23,28,77,99,
Hello from C!

Dynamically Loaded

As highlighted below, this C version is lengthier, but does the same thing as before.

#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>
typedef long long go_int;
typedef double go_float64;
typedef struct{void *arr; go_int len; go_int cap} go_slice;
typedef struct{const char *p; go_int len;} go_str;
int main(int argc, char **argv) {
void *handle;
char *error;
handle = dlopen ("./awesome.so", RTLD_LAZY);
if (!handle) {
fputs (dlerror(), stderr);
exit(1);
}
go_int (*add)(go_int, go_int) = dlsym(handle, "Add");
if ((error = dlerror()) != NULL) {
fputs(error, stderr);
exit(1);
}
go_int sum = (*add)(12, 99);
printf("awesome.Add(12, 99) = %d\n", sum);

go_float64 (*cosine)(go_float64) = dlsym(handle, "Cosine");
go_float64 cos = (*cosine)(1.0);

void (*sort)(go_slice) = dlsym(handle, "Sort");
go_int data[5] = {44,23,7,66,2};
go_slice nums = {data, 5, 5};
sort(nums);

go_int (*log)(go_str) = dlsym(handle, "Log");
go_str msg = {"Hello from C!", 13};
log(msg);

dlclose(handle);
}
// See client2.c on GitHub

In this version, the code uses its own defined C types: go_int, go_float, go_slice, and go_str (for illustrative purpose, awesome.h could have been used). Function dlsym loads the function symbols and assign them to their respective function pointers. Next, the code can be compiled, linking it with the dl library (not the awesome.so) as follows:

$> gcc -o client client2.c -ldl

When the code is executed, the C binary loads and links to shared library awesome.so producing the following output:

$> ./client
awesome.Add(12, 99) = 111
awesome.Cosine(1) = 0.540302
awesome.Sort(44,23,7,66,2): 2,7,23,44,66,
Hello from C!

From Python

from ctypes import *lib = cdll.LoadLibrary("./awesome.so")lib.Add.argtypes = [c_longlong, c_longlong]
print "awesome.Add(12,99) = %d" % lib.Add(12,99)
lib.Cosine.argtypes = [c_double]
lib.Cosine.restype = c_double
cos = lib.Cosine(1)
print "awesome.Cosine(1) = %f" % cos
class GoSlice(Structure):
_fields_ = [("data", POINTER(c_void_p)),
("len", c_longlong), ("cap", c_longlong)]
nums = GoSlice((c_void_p * 5)(74, 4, 122, 9, 12), 5, 5)
lib.Sort.argtypes = [GoSlice]
lib.Sort.restype = None
lib.Sort(nums)
class GoString(Structure):
_fields_ = [("p", c_char_p), ("n", c_longlong)]
lib.Log.argtypes = [GoString]
msg = GoString(b"Hello Python!", 13)
lib.Log(msg)
# See client.py on GitHub

The lib variable represents the loaded symbols from the shared object file. Python classes GoString and GoSlice map to their respective C struct types. When the Python code is executed, it calls the Go functions in the shared object producing the following output:

$> python client.py
awesome.Add(12,99) = 111
awesome.Cosine(1) = 0.540302
awesome.Sort(74,4,122,9,12) = [ 4 9 12 74 122 ]
Hello Python!

From Ruby

require 'ffi'module Awesome
extend FFI::Library
ffi_lib './awesome.so' class GoSlice < FFI::Struct
layout :data, :pointer,
:len, :long_long,
:cap, :long_long
end
class GoString < FFI::Struct
layout :p, :pointer,
:len, :long_long
end
attach_function :Add, [:long_long, :long_long], :long_long
attach_function :Cosine, [:double], :double
attach_function :Sort, [GoSlice.by_value], :void
attach_function :Log, [GoString.by_value], :int
end

print "awesome.Add(12, 99) = ", Awesome.Add(12, 99), "\n"
print "awesome.Cosine(1) = ", Awesome.Cosine(1), "\n"
nums = [92,101,3,44,7]
ptr = FFI::MemoryPointer.new :long_long, nums.size
ptr.write_array_of_long_long nums
slice = Awesome::GoSlice.new
slice[:data] = ptr
slice[:len] = nums.size
slice[:cap] = nums.size
Awesome.Sort(slice)
msg = "Hello Ruby!"
gostr = Awesome::GoString.new
gostr[:p] = FFI::MemoryPointer.from_string(msg)
gostr[:len] = msg.size
Awesome.Log(gostr)
# See client.rb on GitHub

In Ruby, the FFI::library class is extended to declare a class that loads the exported symbols. Classes GoSlice and GoString map to their respective C structs. When we run the code it calls the exported Go functions as shown below:

$> ruby client.rb
awesome.Add(12, 99) = 111
awesome.Cosine(1) = 0.5403023058681398
awesome.Sort([92, 101, 3, 44, 7]) = [3, 7, 44, 92, 101]
Hello Ruby!

From Node

var ref = require("ref");
var ffi = require("ffi");
var Struct = require("ref-struct")
var ArrayType = require("ref-array")
var LongArray = ArrayType(ref.types.longlong);
var GoSlice = Struct({
data: LongArray,
len: "longlong",
cap: "longlong"
});
var GoString = Struct({
p: "string",
n: "longlong"
});
var awesome = ffi.Library("./awesome.so", {
Add: ["longlong", ["longlong", "longlong"]],
Cosine: ["double", ["double"]],
Sort: ["void", [GoSlice]],
Log: ["longlong", [GoString]]
});
console.log("awesome.Add(12, 99) = ", awesome.Add(12, 99));
console.log("awesome.Cosine(1) = ", awesome.Cosine(1));
nums = LongArray([12,54,0,423,9]);
var slice = new GoSlice();
slice["data"] = nums;
slice["len"] = 5;
slice["cap"] = 5;
awesome.Sort(slice);
str = new GoString();
str["p"] = "Hello Node!";
str["n"] = 11;
awesome.Log(str);
// See client.js on GitHub

The ffi object manages the loaded symbols from the shared library . The Node Sturct object is used to create GoSlice and GoString to map to their respective C structs. When we run the code it calls the exported Go functions as shown below:

awesome.Add(12, 99) =  111
awesome.Cosine(1) = 0.5403023058681398
awesome.Sort([12,54,9,423,9] = [ 0, 9, 12, 54, 423 ]
Hello Node!

From Java

import com.sun.jna.*;public class Client {
public interface Awesome extends Library {
public class GoSlice extends Structure {
...
public Pointer data;
public long len;
public long cap;
}

public class GoString extends Structure {
...
public String p;
public long n;
}
public long Add(long a, long b);
public double Cosine(double val);
public void Sort(GoSlice.ByValue vals);
public long Log(GoString.ByValue str);
}
static public void main(String argv[]) {
Awesome awesome = (Awesome) Native.loadLibrary(
"./awesome.so", Awesome.class);
System.out.printf(... awesome.Add(12, 99));
System.out.printf(... awesome.Cosine(1.0));
long[] nums = new long[]{53,11,5,2,88};
Memory arr = new Memory(... Native.getNativeSize(Long.TYPE));
Awesome.GoSlice.ByValue slice = new Awesome.GoSlice.ByValue();
slice.data = arr;
slice.len = nums.length;
slice.cap = nums.length;
awesome.Sort(slice);
Awesome.GoString.ByValue str = new Awesome.GoString.ByValue();
str.p = "Hello Java!";
str.n = str.p.length();
System.out.printf(... awesome.Log(str));
}
}
// See Client.java on GitHub

The Java interface Awesome represents the symbols loaded from the awesome.so shared library file. Classes GoSlice and GoString map to their respective C struct representations. When the code is compiled and run, it calls the exported Go functions as shown below:

$> javac -cp jna.jar Client.java
$> java -cp .:jna.jar Client
awesome.Add(12, 99) = 111
awesome.Cosine(1.0) = 0.5403023058681398
awesome.Sort(53,11,5,2,88) = [2 5 11 53 88 ]
Hello Java!

Conclusion

As always, if you find this writeup useful, please let me know by clicking on the icon to recommend this post.

Also, don’t forget to checkout my book on Go, titled Learning Go Programming from Packt Publishing.

Learning the Go Programming Language

Short and insightful posts for newcomers learning the Go…