Intro: Slave Nodes and Remote Code Loading

An introduction to slave nodes and remote code loading with the Elixir language.

Read the informational guide below, or jump straight to a distributed walkthrough in part 2.

Slave Nodes

Slave nodes are a powerful component of Erlang/OTP which enable distributed computing without the burden of orchestrating code deployments across multiple servers or developing custom network communication protocols. Whether you’re leveraging a cluster of Raspberry Pis or scaling compute-intensive algorithms on EC2, slave nodes provide a straightforward architecture to scale your workload out to multiple machines with minimal overhead.

This guide will step through some of the fundamental building blocks surrounding slave nodes to get you started with building distributed systems in Elixir.

If availability in the presence of server failures, network partitions, or data center failures is important to your use case, the use of a decentralized, fault tolerant architecture is recommended, such as the one provided by riak_core.

Remote Code Loading

At its core, remote code loading describes the behavior of executing modules inside of an Erlang VM without the compiled module existing on the local filesystem.

A naive approach of remote code loading can be implemented in just two lines of code within Erlang:

{_Module, Binary, Filename} = code:get_object_code(Module).
rpc:call(Node, code, load_binary, [Module, Filename, Binary]).

However, this style of remote code loading places unnecessary burden on the application code to serialize, distribute, and load binary files. By using proper remote code loading techniques you can fully leverage advanced code-management features such as on-demand code loading and hot code swaps.

Understanding Erlang Code

To properly understand remote code loading, it is necessary to take a step back and look at the building blocks of Erlang.

The Module

The fundamental unit of code in Erlang (and thus Elixir) is a module. In fact, all written functions must be contained within a module. At its core, a module is simply collection of attributes and function declarations.

In Erlang, a file with the extension .erl contains a single module. Elixir files, extended with .ex, may contain multiple modules, but it is both common and convenient for a single module to be in a single file.

Elixir files may contain multiple modules solely for organizational convenience. When Elixir files are compiled, each module is separated into a distinct `.beam` file with the appropriate name. Consider a single Elixir file with two modules defined:
# test.ex defmodule ModuleA do end defmodule ModuleB do end
Upon compilation, two distinct beam files are created:
elixirc test.ex — verbose 
Compiled test.ex 
> ls *.beam 
Elixir.ModuleA.beam
Elixir.ModuleB.beam

Modules must be compiled prior to use by the runtime system. Compilation involves expanding macros, inserting include files, generating warnings or error messages for syntax errors, and many more steps beyond the scope of this document. Compiled modules have the extension .beam. When a particular .beam module is loaded into the Erlang runtime system, the associated functions are made available.

Node

A node is simply a running Erlang Virtual Machine (VM).

The only widespread implementation of the Erlang VM is what we refer to as Beam. You may think of the Erlang VM as an operating system running on top of your host operating system. It handles events, process scheduling, memory, naming services, and interprocess communication. On top of all that, a node can connect to other nodes (locally, or on remote host machines) and interact with them.

As compiled modules are loaded into the VM, their functions and attributes are made available.

Nodes must be given a name in order to communicate with each other. The format of the node name is an atom `name@host` where name is the name given by the user and host is the full host name if long names are used, or the first part of the host name if short names are used. `node()` returns the name of the node. Elixir example:
$ iex — name foo
1> node() 
 :”foo@MacBook-Pro.socal.rr.com”
> $ iex — sname foo 
1> node() 
 :”foo@MacBook-Pro”

Erlang Kernel

The Erlang Kernel application is always the first application executed inside the Erlang VM, and contains all the code necessary to run the Erlang runtime system itself. You can see the core Kernel services by viewing the registered(). process list in a fresh Erlang VM.

In Erlang terms, an “application” denotes a component that can be started and stopped as a unit, and which potentially be re-used in other systems.
$ erl
Erlang/OTP 17 [...]
Eshell V6.3  (abort with ^G)
1> registered().
[standard_error_sup,erl_prim_loader,error_logger,kernel_safe_sup,
init,user,rex,inet_db,kernel_sup,code_server,global_name_server,
application_controller,file_server_2,user_drv,standard_error,global_group]

Of particular importance to this discussion are the code_server anderl_prim_loader modules, which together locate, load, and purge compiled beam modules. Since both Erlang and Elixir code compile to the same bytecode, the process of loading modules into the runtime is identical for Elixir as it is for Erlang after the compilation phase.

Erlang Code Server

The code server’s primary task is to manage the dynamic loading and purging of modules.

However, the code server does not concern itself with where the files are stored. The code server treats a compiled module in the local file system identically to a compiled module from a remote source. This allows flexible code loading techniques, such as remote servers, databases, or any other storage mechanism imaginable. The desired code does not need to exist on the local filesystem!

Erlang Primitive Loader

The erl_prim_loader, also known as the Low Level Erlang Loader, contains details of the runtime environment regarding where to fetch modules. The Erlang runtime system ships with two loader methods:

  • efile (default) — Load modules from the local file system
  • inet — Load modules using the boot_server on another Erlang node.

To instruct an Erlang node to start up in inet mode, a command-line paramter may be passed to erl on startup. Both a cookie and the location of the desired boot server must be set. An erl_boot_server must be running and accepting incoming connections from the host IP.

erl -loader inet -hosts 12.34.56.78 -setcookie secret-cookie

Erlang Boot Server

The final Kernel module which requires an introduction is the boot server. Within this module lies the responsibility of distributing compiled modules to remote nodes. The boot server thus typically runs on a node which contains the code on the local filesystem.

The boot server is not started automatically by the Kernel application. For scenarios in which a boot server is desired, the Kernel must be instructed to start the boot server through configuration parameters or though application code.

For security, the boot server must be provided a list of IP addresses for hosts which are whitelisted to boot and receive code. Erlang nodes which specify a loader method of inet on startup will contact the corresponding boot server to receive compiled code.

Coming together

I realize we’ve (very) briefly introduced a handful of objects…. So how do they all play together?

To aid in understanding, take a minute to look at the following image which attempts to illustrate the flow of a compiled module from the local filesystem of a master node to the running VM of a remote node.

A master node should be started with the desired code on the local filesystem along with a boot server to distribute the code to other nodes. Additional nodes may be started with the proper configuration settings to instruct the VM to locate and load desired modules from the boot server of a master node.

Now that we’ve identified and laid out our building blocks for remote code loading, follow the next post to put together a working demo.

Like what you read? Give Sean Stavropoulos a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.