Sending any.Any thing in Golang with Protobuf 3

Protobuf is a great message format because it’s strict and allows you to enforce schemas while reducing overhead. But sometimes, we need the flexibility to send dynamic message types. Luckily, protobuf has the any.Any type to send any protobuf message type as a message property.

In order to make use of any.Any, we need a message type that uses that type as a property:

Compile using the proto compiler: protoc -go_out=. anything.proto

This will generate the following Go code:

In order to convert a message type to an any type, we need to provide two things:

• TypeURL: a string that references the location of the definition
• Value: a byte slice representation of the message instance

Determining the TypeURL is not hard. The protobuf definition specifies:

* The last segment of the URL’s path must represent the fully

qualified name of the type (as in `path/google.protobuf.Duration`).

The name should be in a canonical form (e.g., leading “.” is

not accepted).

Since we aren’t requesting the protobuf definition over the network, the hostname and full path don’t really matter. The only important part is the last path component which represents the fully qualified name of the type. This name is registered during initialization by the generated Go code (anything.pb.go):

func init() {
proto.RegisterType((*AnythingForYou)(nil), "anything.AnythingForYou")
}

We are able to retrieve this fully qualified name programmatically by using this helper function:

import (
    "log"
    "github.com/golang/protobuf/proto"
    "github.com/golang/protobuf/ptypes/timestamp"
)
func demo() {
    name, _ := proto.MessageName(&timestamp.Timestamp{})
    log.Printf("Message name of timestamp: %s", name)
}

In our example, the fully qualified name will be: example.com/yaddayaddayadda/anything.AnythingForYou

The any.Any.Value value must be a byte slice representation of the protobuf message we wish to serialize. This can be serialized using the following function:

import (
    "log"
    "github.com/golang/protobuf/proto"
    "github.com/golang/protobuf/ptypes/timestamp"
)
func demo() {
    serializedBytes, _ := proto.Marshal(&timestamp.Timestamp{})
    log.Printf("Serialized timestamp: %s", string(serializedBytes))
}

We then need to reference the generated code to marshal and un-marshal correctly:

There are some helper functions in the protobuf Go library that make some assumptions for you in exchange for convenience (such as setting the hostname for the TypeURL to a default value). You can check it out here: https://github.com/golang/protobuf/blob/master/ptypes/any.go

And that’s it! If you want to toy with the code above, check out the full project here: https://github.com/pokstad/anything