Build it functionally: react-native-web + OTP + Phoenix Channels + DETS*

* geared towards non-beginners in Elixir

In his keynote speech at ElixirConf 2018, Phoenix creator Chris McCord announced his team’s upcoming, groundbreaking new package: Phoenix LiveView It will provide for Phoenix.View a React-like API with some similar functionality.

LiveView will offer developers reduced complexity: no frontend, no build tool, no duplicated state or business logic. But as with most things in life, there are tradeoffs: as proposed, it won’t be capable of handling complex user interactions.

What if I told you there is a way you can get the best of both worlds: minimal duplicated business logic AND sophisticated UI?

PRESENTING Functional Web Development with Elixir, OTP, and Phoenix!

This book, by Lance Halvorsen, brings back a revolutionary concept: stateful client-server connections.

Imagine, when you visited a website, after the initial request, that the app no longer needed to use its database to fulfill the UI. That’s precisely what’s possible with react-native-web + OTP + Phoenix Channels + DETS.

react-native-web

I prefer it to React because one can easily write cross-platform code, albeit it’s not quite as seamless as I’d hope. Phoenix generates a web build, and to deploy on mobile, you run create-react-native-app, plop in your code, and follow your app store’s deployment guide.

OTP

Due to the scope of this article, I can’t go into too much depth about OTP here. 
For the complete picture, read Designing for Scalability with Erlang/OTP.

The short version is Elixir is built on top of Erlang. OTP is to Erlang what Hex is to Elixir, i.e. they are both ecosystems consisting of packages written in their respective programming languages. OTP is a set of common abstractions of Erlang design patterns. The different design patterns that are commonly used in Erlang are process-based architectures. The reason to switch to Elixir/Erlang in the first place is it gives you true concurrency (leveraging multi-core processors), yielding up to order-of-magnitude performance gains.

This true concurrency is achieved via processes. You can think of a process like a lightweight server. Synchronous operations occur on a call stack. Concurrent ones occur across a network of processes.

Processes communicate to each other using send/2 which accepts a process identifier (pid) and a message (typically a tuple). The receiving process accesses its message via a receive/1 block, which pattern-matches the message against the clauses defined in the block (like a case statement).

As a result, multiple operations can be offloaded to multiple processes (typically using the Task module in Elixir) and executed concurrently. The spawning process simply waits for each process to send back a result. This means the total time to perform a series of operations equals the slowest operation rather than the sum of all operations (assuming no operation uses data from another — otherwise, these must be executed synchronously).


A process could fail…

A common design pattern is to spawn a process to spawn and monitor child processes — a Supervisor If a child process fails, the monitoring process is sent a message. The supervisor then restarts the child process.

If you want a process that can hold state and be accessed by other processes (a GenServer), you must have that process recursively call a function infinitely. This function will listen for messages from other processes, do an operation (updating the state, and sending a reply to the other, client process), and then call itself with the new state as an argument.

Beyond that, there is the Application This is the first module a Mix project executes, meaning it is the place to access your DETS database, start supervisor processes, etc — anything that sets up your app before running any actual processes.

There are two other (less common, but still core) OTP design patterns (aka behaviours): a finite state machine, and a series of processes — some of which produce data, others of which consume this data — encapsulated by GenStage

Design Outline

Why does an application need a database? Two reasons:

In order to provide unique experiences for users, an app has them generate user-specific data.
That data needs (1) an API — a way of accessing it, and (2) persistence — a way to store it. Read & Write.

Phoenix Channels solve (1) accessing user data, without hitting the database on every request, via GenServer

For (2) persisting user data, we’ll use Disk Erlang Term Storage. DETS is simply a hash table. open_file/2 opens it in-memory, allowing our app to read from and write to it in constant time. Then, writes are recorded to the file on disk via insert/2

We are using DETS rather than ETS (similar to Redis) because were our server to crash, ETS would lose our data. We are using DETS instead of Mnesia because this is a simple app — a Battleship-esque game.

For something more sophisticated, :ecto_mnesia would be our go-to because Mnesia provides constant time reads and writes across multiple (distributed) nodes — servers running on separate machines — and this package allows us to seamlessly switch to a larger database like PostgreSQL should our data requirements exceed Mnesia’s in-memory storage capacity (e.g. blockchains).

Here’s the basic architecture:

  • You have a standard server running.
  • On load, your frontend creates a connection with it over a websocket.
A websocket has channels. A client can join one or many.
Each channel has topics. Topics are specific to a set of clients.
Put another way, each channel is a service your frontend might utilize. Topics are used when a set of clients share the resources of a service, i.e. state data.
  • On a successful, select user interaction (e.g. signing in), a client joins a channel.
On the server, joining a channel either creates a new topic or joins an existing one, which respectively starts a new GenServer process (which holds state data) or pings an existing one.
A GenServer’s initial state is either a default (generated by code) or grabbed from a datastore. (More on this later…)
  • Subsequent, select user interactions send and receive data over the channel topic.
This will be the meat of our skeleton.

There are a few other pieces, best illustrated with…

An IN-DEPTH example. (PLAY HERE! Please excuse that the project is a subdirectory.)

Booting with OTP

To run this project locally:
git clone https://github.com/English3000/Elixir.git
cd islands # root directory
mix deps.get
cd apps/islands_interface/assets && npm install
cd ../../..
mix phx.server
In your browser, go to localhost:4000

You’ll notice this project is an umbrella app, meaning there are two parts to our functional design. (Feel free to explore for yourself.)

(0) Mix

When we run mix from the root directory, it loads the projects in our :app_path This allows us to run tasks — i.e. phx.server — defined by its child projects.

In /islands/config/config.exs, both subdirectories’ config.exs are imported.

Under the hood, we Mix.Task.run("app.start") each child project, which executes each application.ex

(1) IslandsEngine

On startup, IslandsEngine.Application creates (or re-opens) an on-disk database file /islands/game

We pass a Supervisor process a list of modules to spawn and monitor as child processes.

The first child process is a Registry — a node-local data structure that stores key-value pairs of process identifiers (aka pids) and their names.

The second child process (our first file not auto-generated by Phoenix) is another supervisor, so its start_link/1 callback is invoked. This triggers init/1, calling Supervisor.init/1 which generates specs — data on how to startup child processes (in this case, just [Server]) and how the supervisor should handle crashes:

{Name, StartFunction, RestartType, ShutdownTime, ProcessType, Modules}
Cesarini, Francesco; Vinoski, Steve. Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems (Kindle Locations 4826-4827). O'Reilly Media. Kindle Edition.

(2) IslandsInterface

On startup, :islands_interface loads our :islands_engine API because this dependency is listed under :extra_applications

In IslandsInterface.Application once again we pass a list to a supervisor. But this time, we generate our specs here using Supervisor.Spec.supervisor/3

Our supervisor process starts up our Endpoint — allowing our Phoenix server to handle requests — and Phoenix Presence — which tracks events on our websocket.

Channel-based Architecture

(3) Setup

In IslandsInterfaceWeb.Endpoint we define a websocket:

socket "/socket", IslandsInterfaceWeb.UserSocket,
websocket: true,
longpoll: false

In IslandsInterfaceWeb.UserSocket we define our channel:

channel "game:*", IslandsInterfaceWeb.GameChannel
How to Code a Channel
Writing a channel involves both code on the frontend and backend.
BACKEND
In OTP terminology, a channel is just another behaviour, in this case provided by Phoenix. Using a behaviour is — in object-oriented languages — the equivalent of extending a class: You are provided with a predefined set of functions — referred to in OTP as callbacks — which you may overwrite to serve your specific use-case.
Keep in mind that a behaviour has additional generic code that is not overridable (unless you define and then use your own). This code weaves together the callbacks with the underlying OTP functionality you are using.
FRONTEND
A channel is a stateful connection between a client and a server. State is passed via events, defined on the frontend.
As this is an article geared toward an intermediate-level audience, I won’t review how an MVC framework works. Suffice to say: on a GET / request, the browser loads app.js

(4) Interface

Open socket.js You’ll notice we:

  1. Create, connect to, then export a new Socket
  2. Define and export a function which creates a new channel
  3. Define and export two functions which send game events and their payloads/state to our channel

Open IslandsInterfaceWeb.GameChannel

Ignore the defp’s.
Game.js imports from socket.js and defines a function joinGame/1
// In it, we create a new channel...
let gameChannel = channel(socket, game, player)
/* ... */
// And join it.
gameChannel.join()

Calling join/1 sends to the server’s join/3 the channel’s websocket and the payload defined in:

// socket.js
export const channel = (socket, game, player) =>
socket.channel("game:" + game, {screen_name: player})
# game_channel.ex
def join("game:" <> game, %{"screen_name" => player}, socket) do
# ...
end

Similarly, if a user were to close our app in one’s browser, that automatically disconnects the client from the channel, triggering:

# game_channel.ex
def terminate(_reason, socket), do: message(socket, "left")
defp message(socket, instruction), 
do: broadcast! socket, "message", m(instruction)
# %{instruction: instruction}

In a nutshell, if one player leaves a game, all clients on the channel are notified with a "message" event and a payload m(instruction) from broadcast!/3

Scrolling down joinGame/1 we see

// Game.js
gameChannel.on( "message",
({instruction}) => this.setState({ message: {instruction} }) )

⌘ + f <Instruction and you see we pass this message to another component, Instruction.js which matches against a one-word message to display an instruction for the user. In this way, when a player leaves the game, one’s opponent is notified with a new instruction.

There’s a lot more going on with IslandsInterfaceWeb.join/3 and after gameChannel.join/1

At a high level, join/3 does a series of checks (with error-handling) to handle different join situations:

  1. track_players/3 keeps track of which players have joined the game using Phoenix Presence It prevents a second person from joining as the same player, or a third player from joining.
  2. If the game is a new one (as opposed to a player joining, leaving, then returning to a saved game), register_player?/4 calls Supervisor.start_game/2 which calls Server.start_link/2 which starts a GenServer process with game state. 
    Then (in the third case for register_player?/4) we check whether the player joining is new.
  3. If not, we just return the game state 
    If so, we add the player to the game state, save the new state to DETS, and reply with the new state.
  4. Finally we either respond/3 :ok with the game state, or :error with a message.
  5. Now we hit the receive callback in Game.js 
    On "error" we simply display the error message.
    On "ok" we give our root component access to the game state (so two boards in Gameplay.js now display), along with setting some UI-related properties. Then, we add the query string `/?game=${game}&player=${player}` to our history

Besides GameChannel.join/3 and terminate/2, there are two other callbacks.

handle_in/3 receives an event pushed from the client, i.e. "set_islands" or "guess_coordinate" It is the channel layer — its body passes some data to another part of the backend, Server, then pushes a response back to the client.

The other gameChannel.on( handlers in Game.js then use the response to do something for the UI.

handle_info/2 handles an :after_join tuple message which, in this case, our process sends itself a few lines above. After a player connects to the game, a message is broadcast to all clients so that if there is another player, one is notified an opponent has joined the game.

One last note. Game.js defines a gameChannel.on("coordinated_guessed" handler. But so does Tile.js This is the power of Phoenix channels: not only can we handle server responses in our root component; we can also do something else further down our component hierarchy on the same event, in this case changing the color of a tile on one player’s board (for both clients).

As mentioned, we are using react-native-web In webpack.config.js, we alias it as "react-native" and change our .babelrc to

{
"presets": ["@babel/preset-env", "@babel/preset-react"]
}

In app.js at the bottom you see:

AppRegistry.runApplication( "Game", {rootTag: document.getElementById("islands")} )

AppRegistry.runApplication performs the same function as ReactDOM.render would. Then in Game.js at the bottom you see:

AppRegistry.registerComponent("Game", () => Game)

And you see similar usage of AppRegistry.registerComponent whenever we define a new component. It is an extra step required to be able to access our components from AppRegistry

Design Decisions

So now you have a good feel of the architecture. Just as we justified using DETS to eliminate latency, I am going to justify why I put some business logic either on the frontend or the backend.

(1) Placing Islands

a. Events

Open Gameplay.js When you play the game you may notice that when you are placing islands but haven’t set them, if you refresh the page and return to the game, your placements aren’t remembered.

In the Functional Web Development book, there is a "place_island" event on the backend. I disagree with this because it adds unnecessary complexity — you can’t play the game until ALL islands are set. And if you’re updating the UI on every island placement, it makes the experience bumpy waiting for the server response on each movement.

b. Validation

On the frontend in Island.js we validate whether an island is on the board. 
On the backend in island_set.ex we validate whether any islands overlap.

This arrangement makes sense because you need the island location on the frontend anyways to move it. On the other hand, checking whether islands overlapped would be very tricky. We’re only tracking the top left corner of an island, and some islands’ bounds may overlap without sharing tiles. In order to determine this, we’d need to duplicate logic from island.ex This logic must be on the backend in order to be able to store it, so if we want to avoid duplication, we should check for overlaps on the backend.

Regarding the concern that a hacker could send our server island coordinates off the board by injecting JavaScript, for now we can tolerate this. Given that there’s no prize if you win (beyond messing with your friends), it simply isn’t worth the effort.

(2) Data Transport

Let’s return to game_channel.ex

If you look at all of our responses, you’ll notice we only send the full game state on join/3 All broadcast!/3 and push/3 calls send the client only the data that has changed.

In addition, even in join/3 we intelligently delete the opponent’s islands so a tech-savvy player can’t find that data in one’s browser.

(3) Phoenix Channels vs. Redux & GraphQL

For Redux, we’d define the functions in socket.js as RESTful routes in their own file. In place of the .on( "event" listeners, we’d define actions in their own file. Then we’d need reducers and containers to get our state to update properly — adding complexity and extra logic to our frontend views.

For GraphQL, we’d define our data requirements in our JS file as queries, and also mutations to communicate with our backend. In Elixir, this would mean setting up Absinthe on top of Phoenix — adding complexity and extra layers to our backend.

We still could use DETS as our datastore in either case, replacing in our resolvers calls to an Ecto Repo with calls to :dets But we have to ask: Is the extra complexity of GraphQL and Ecto worth their power for our project?

For this particular project, the data we are keeping track of is a game with two players. We need to be able to:

  1. add players’ data (including whether one has won) to our game data structure
  2. store the location of their islands
  3. and, record their guesses

Rather than having a separate table for each data structure,we use data structures to encapsulate the logic of a game. Then we simply save to DETS our game state whenever we update it.

For data storage this simple, we simply don’t need the power or complexity of GraphQL or even Ecto. If we wanted to add more features to our game — e.g. tracking players’ win-loss records — then we’d want to move players to their own storage, introducing foreign keys. Then we might consider Mnesia and Ecto…

By contrast, using Phoenix Channels, the architecture is as simple as defining what would be called actions in Redux on the frontend, plus a channel interface (much simpler than a schema) and a business logic layer, Server (which serves a similar function to resolver functions).

We barely even use our controller layer…

Wrap Up

There are still a few details I haven’t mentioned — e.g. the finite state machine in stage.ex, and the plug block_query/2 in router.ex which protects against requests to a game’s URL.

But we’ve covered a lot of territory! If there’s anything you’d like further explained, please comment.

In all, I hope this article has introduced some new things about the Elixir ecosystem to you and gotten you excited about the possibilities for full-stack apps using Phoenix Channels!

P.S. If you haven’t heard, also check out React Hooks — which are currently in alpha for v16.7.0.