Namespaces in Go - reexec

In the previous article we learnt how to apply a UID/GID mapping to ns-process such that we are now running as the root user once inside the namespaced shell.

The purpose of this article is to provide an understanding of the reexec package. reexec is part of the Docker codebase and provides a convenient way for an executable to “re-exec” itself. In all honesty reexec is a bit of a hack, but it’s a really useful one that is required to circumvent a limitation in how Go handles process forking. Before going into too much more detail, let’s take a look at the problem reexec helps to solve.

It’s probably best to demonstrate the problem by way of an example. Consider the following - we want to update ns-process such that a randomly-generated hostname is set inside the new UTS namespace we’ve cloned. For security reasons, it’s essential that the hostname has been set before the namespaced /bin/sh process starts running. After all, we don’t want programs running inside ns-process to be able to discover the Host’s hostname.

As far as I’m aware, Go doesn’t provide a built-in way to allow us to do this. Namespaces are created by setting attributes on an *exec.Cmd, which is also where we specify the process we'd like to run. For example:

cmd := exec.Command("/bin/echo", "Process already running")
cmd.SysProcAttr = &syscall.SysProcAttr{
 Cloneflags: syscall.CLONE_NEWUTS,
}
cmd.Run()

Once cmd.Run() is called, the namespaces get cloned and then the process gets started straight away. There’s no hook or anything here that allows us to run code after the namespace creation but before the process starts. This is where reexec comes in.

🎤 reexec yourself before you wreck yourself

Let’s open up the reexec package and take a look at what’s inside (I won’t paste full code snippets here for sake of simplicity, but I advise you read along with the full implementations of the methods).

// Register adds an initialization func under the specified name
func Register(name string, initializer func()) {
 # ...
}

First up we have Register, which exposes a way for us to register arbitrary functions by some name and to store them in memory. We will use this to register some sort of “Initialise Namespace” function when ns-process first starts up.

// Init is called as the first part of the exec process
// and returns true if an initialization function was called.
func Init() bool {
 # ...
}

Next up we have Init, which gives us a mechanism for determining whether or not the process is running after having been reexeced, and for running one of the registered functions if we have. It does this by checking os.Args[0] for the name of one of the previously-registered functions.

// Command returns *exec.Cmd which have Path as current binary.
// ...
func Command(args ...string) *exec.Cmd {
 return &exec.Cmd{
  Path: Self(),
  Args: args,
  SysProcAttr: &syscall.SysProcAttr{
   Pdeathsig: syscall.SIGTERM,
  },
 }
}

Command ties it all together by creating an *exec.Cmd with Path set to Self(), which evaluates to /proc/self/exe on Linux machines. We can choose which of the registered functions we’d like to invoke upon reexec by providing the registered name of the function in args[0].

💁 /proc/self/exe is a symlink file that points to the path of the currently-running executable

Now that we have an understanding of how reexec works, it’s time to wire it up inside ns-process.

👉 Let’s Go

The first thing we need to do is to create a function and register it using reexec.

# Git repo: https://github.com/teddyking/ns-process
# Git tag: 3.0
# Filename: ns_process.go
# ...
func init() {
	reexec.Register("nsInitialisation", nsInitialisation)
	if reexec.Init() {
		os.Exit(0)
	}
}
# ...

There are two important things happening here. First, we register a function nsInitialisation under the name “nsInitialisation”. We'll add that function in a moment. Secondly, we call reexec.Init() and os.Exit(0) the program if it returns true. This is vitally important to prevent an infinite loop situation whereby the program gets stuck reexecing itself forever! Let’s add nsInitialisation next.

# Git repo: https://github.com/teddyking/ns-process
# Git tag: 3.0
# Filename: ns_process.go
# ...
func nsInitialisation() {
	fmt.Printf("\n>> namespace setup code goes here <<\n\n")
	nsRun()
}
func nsRun() {
	cmd := exec.Command("/bin/sh")

	cmd.Stdin = os.Stdin
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr

	cmd.Env = []string{"PS1=-[ns-process]- # "}

	if err := cmd.Run(); err != nil {
		fmt.Printf("Error running the /bin/sh command - %s\n", err)
		os.Exit(1)
	}
}

Here we’ve added nsInitialisation() simply as a placeholder function. It will become much more important in future articles when we actually need to start configuring the namespaces. For now, it simply passes through to nsRun(), which runs the /bin/sh process.

All that’s left to do now is modify main() such that it runs the /bin/sh process via reexec and nsInitialisation rather than calling it directly.

# Git repo: https://github.com/teddyking/ns-process
# Git tag: 3.0
# Filename: ns_process.go
func main() {
    cmd := reexec.Command("nsInitialisation")
    # ...
}

By specifying nsInitialisation as the first arg to Command, we're essentially telling reexec to run /proc/self/exe with os.Args[0] set to nsInitialisation. Finally, once the program has been reexeced, Init will detected the registered function and then actually Run it. Let’s give it a whirl.

💁 The following has been tested on Ubuntu 16.04 Xenial with Go 1.7.1

$ go build
$ ./ns-process

>> namespace setup code goes here <<
-[ns-process]- #

And there we have it. We now have nsInitialisation available in which to run any namespace setup we need, including the ability, as discussed earlier, to set the hostname in the new UTS namespace if we so desire.

📺 On the next…

We’re now in a position to configure our namespaces, but what configuration remains to be done? The answer to this and plenty more coming up, stay tuned…

Update: Part 5, “Namespaces in Go - Mount” has been published and is available here.