Building a REST API with Phoenix 1.3 — Part 1

It’s Become Even Easier

Overview

Early this year, I taught myself to build a REST API with Phoenix and Elixir and wrote about it. This has been by far my most popular blog post ever. Now that Phoenix 1.3 is out, that article is a pretty outdated so I thought I’d take the time to teach myself Phoenix 1.3 and share my learning here.

Like before, I’m going to build a fairly simple API that can be used to process a form that looks like this.

Getting Started with Phoenix 1.3

I found it a bit difficult to get started with this new version of Phoenix. The documentation doe a good job describing how to use individual pieces of the framework, but there is no written guide that I’ve found that clearly describes the philosophy of the changes. I’ve found that it’s important to understand this philosophy and it’s best done by watching this talk by Chris McCord. This talk is not totally up-to-date but gives a good overview of the philosophical changes. The big changes in this version of Phoenix are.

• The web directory has moved inside the lib directory
• There is now a Web namespace inside your App namespace in the Elixir module hierarchy
• There are no more models
• Uses contexts to subdivide your application
• Better generators that help guide you to make skinnier controllers
• Supports fallback controllers to more concisely handle API errors

We will walk through each of these changes as we build out our API. So let’s dive right in and walk through the process of building this simple API.

Creating our New Application

The command to create a new Phoenix application has changed slightly with Phoenix 1.3. Note that the mix command has changed from mix phoenix.new to mix phx.new (less typing, yay!).

% mix phx.new api13
% cd api13
% mix ecto.create

With this in place, we can then take a look at the generated directory structure.

lib
├── api13
│   ├── application.ex
│   └── repo.ex
├── api13.ex
├── api13_web
│   ├── channels
│   │   └── user_socket.ex
│   ├── controllers
│   │   └── page_controller.ex
│   ├── endpoint.ex
│   ├── gettext.ex
│   ├── router.ex
│   ├── templates
│   │   ├── layout
│   │   │   └── app.html.eex
│   │   └── page
│   │       └── index.html.eex
│   └── views
│       ├── error_helpers.ex
│       ├── error_view.ex
│       ├── layout_view.ex
│       └── page_view.ex
└── api13_web.ex

Here you will see the big change. The top-level lib directory now contains two main directories <app> and <app_web>. The concept here is that logic related to the web will be separated from business logic unrelated to the web. In this simple example, we will be working totally within the web directories. You’ll also note that there is no models directory any more. We will get into how that transforms our application as we proceed.

Using Generators to Create the index Action

Here we will create a route for an API endpoint and configure a controller to user data. This is done by editing the lib/api13_web/router.ex file. You’ll note that there are two pipelines set up by default: one for browser and one for an api. These pipelines describe what operations are performed as data is processed. Since we are only building an API here, we will only be using the api pipeline.

You’ll also note that there is one route setup within the “/” scope and a commented out scope for “/api”. Again since we only want to create an API, we will delete the “/” scope and uncomment and modify the “/api” scope. I want the API to respond to /api/v1/users so I created a scope that looked like this.

scope "/api/v1", ApiExample do
    pipe_through :api
    get "/users", UserController, :index
end

Here we are setting up a single get endpoint that should respond to /api/v1/users. If we try to navigate to this path, we get this error.

** (UndefinedFunctionError) function Api13Web.UserController.init/1 is undefined (module Api13Web.UserController is not available)

That’s not unexpected since we haven’t created the controller yet. I’m going to build a controller following the Phoenix 1.3 conventions. The best way to learn about these conventions is to use the generators that can be run as mix tasks. While I’ll likely never use the generators for real applications, in this case, I found them essential to learning the best practices of building a Phoenix 1.3 application. A generator for building a JSON resource is included and has this syntax

mix phx.gen.json <context> <resource> <resource plural> <fields...>

Your first question here is likely to be, “what is a context?” This version of Phoenix encourages us to divide our web application into a number of modules that describe different areas of business logic. For instance you might have modules called Accounts, Blog and Billing. each of these modules is known as a context. Given that we are creating a user, we will create a context called Accounts and use this command to generate a scaffolded application.

mix phx.gen.json Accounts User users \
    email:string \
    password:string \
    age:integer \
    stooge: string

Running this generator produces the following files.

* creating lib/api13_web/controllers/user_controller.ex
* creating lib/api13_web/views/user_view.ex
* creating test/api13_web/controllers/user_controller_test.exs
* creating lib/api13_web/views/changeset_view.ex
* creating lib/api13_web/controllers/fallback_controller.ex
* creating lib/api13/accounts/user.ex
* creating priv/repo/migrations/20171126174114_create_users.exs
* creating lib/api13/accounts/accounts.ex
* injecting lib/api13/accounts/accounts.ex
* creating test/api13/accounts/accounts_test.exs
* injecting test/api13/accounts/accounts_test.exs

This does all the heavy lifting for creating our first resource and gives us a good guide to best practices for building out an endpoint. This generator actually creates the code for all of the actions for this endpoint, but we will walk through them one-by-one to show how we’d build each endpoint from scratch. So, let’s walk through the steps for creating the :index action.

Understanding the Controller

Let’s first dive into the controller.

# lib/api13_web/controllers/user_controller.ex
defmodule Api13Web.UserController do
  use Api13Web, :controller
  alias Api13.Accounts
  alias Api13.Accounts.User
  def index(conn, _params) do
    users = Accounts.list_users()
    render(conn, "index.json", users: users)
  end
end

This looks a lot like the controller we used in previous versions of Phoenix. The major differences are.

1. The controllers now live within the Api13Web namespace instead of the Api13 namespace.
2. We no longer make Ecto calls directly from the controller. Instead of using Repo.all(User), we call a function list_users that is stored within the Accounts context.
3. We no longer call the json function to render JSON directly from the controller.

We will now look at how each of these changes affects our application.

Moving Endpoint Actions into the Context

Let’s take a look at that list_users function.

# lib/api13/accounts/accounts.ex
defmodule Api13.Accounts do
  import Ecto.Query, warn: false
  alias Api13.Repo
  alias Api13.Accounts.User
  def list_users, do: Repo.all(User)
end

There’s no magic here. Note that we have just moved the logic to get a list of users outside of the controller and into a module defined within the Accounts context. Now, there’s no reason you couldn’t use the same logic you used previously. It’s just that Phoenix 1.3 is trying to promote good controller hygiene and encourage skinny controllers with the database access logic cleanly segmented into the contexts.

Migrating and Testing the index Endpoint

With this logic in place, we can test it out, but first we need to migrate the database and insert some test logic. The generator we ran created a migration that looked like this.

defmodule Api13.Repo.Migrations.CreateUsers do
  use Ecto.Migration
  def change do
    create table(:users) do
      add :email, :string
      add :password, :string
      add :age, :integer
      add :stooge, :string
      timestamps()
    end
  end
end

We can run the migrations that will create the user table with:

mix ecto.migrate

At this point, at the Phoenix command prompt, we can manually create a few records.

iex> u1 = %User{email: "joe@example.org",
               age: 32, password: "blah", stooge: "Larry"}
iex> u2 = %User{email: "jane@example.org",
               age: 24, password: "fake", stooge: "Moe"}
iex> Repo.insert(u1)
iex> Repo.insert(u2)

At this point, if we visit the /api/v1/users endpoint we will see the data we’ve manually added. Now let’s move on to the second GET endpoint.

Implementing the /users/:id Endpoint

Now that we can get a list of all users, let’s put in place the code we need to retrieve a single user. This will add a little complexity as for the first time, we need to handle the potential user not found error.

Let’s first try to visit /api/v1/users/1. As expected, this fails as there is no router entry for this endpoint.

** (Phoenix.Router.NoRouteError) no route found for GET /api/v1/users/1 (Api13Web.Router)

We can fix this by modifying the router.ex file as follows.

# /lib/api13_web/router.ex
defmodule Api13Web.Router do
  use Api13Web, :router
  pipeline :api do
    plug :accepts, ["json"]
  end
  scope "/api/v1", Api13Web do
    pipe_through :api
    get "/users", UserController, :index
    get "/users/:id", UserController, :show
  end
end

This gives us the expected error as we have not yet defined a show function in our router.

** (UndefinedFunctionError) function Api13Web.UserController.show/2 is undefined or private

We can fix this issue by adding this new function to user_controller.ex

def show(conn, %{"id" => id}) do
  user = Accounts.get_user!(id)
  render(conn, "show.json", user: user)
end

Once again, note that we aren’t calling any Ecto functions directly in the controller. Instead, we delegate to a function in the Accounts context.

# lib/api13/accounts/accounts.ex
def get_user!(id), do: Repo.get!(User, id)

With this code in place you’ll find that the endpoint works when given a valid user ID. However if you give it a non existent ID, you get an ugly error like this.

Exception:
** (Ecto.NoResultsError) expected at least one result but got none in query:

This is because we are not handling the exception generated by the Repo.get! function call. This is an area where I actually decided to sway from the code from the generator. Instead of generating an exception, I used Repo.get which returns a nil instead of an exception in the case of a user not found. To make this happen, I modified the controller as folows:

def show(conn, %{"id" => id}) do
  user = Accounts.get_user(id)
  if user do
    render(conn, "show.json", user: user)
  else
    conn·
    |> put_status(:not_found)
    |> render("show.json", user: user)
  end
end

I also need to modify the user context function like this.

def get_user(id), do: Repo.get(User, id)

This allows me to return a 404 without returning the exception data that was previously being returned.

You’ll note that we’ve not yet discussed the render function called by the controller actions. We will examine this in the next section.

Setting up the View

The controllers we are creating have two main purposes.

1. Use parameters passed in with the conn to fetch appropriate data
2. Call a function to tender that data to JSON

You’ll note that our controller actions each have calls to a render function. The Phoenix generators create render functions for each resource as part of the view. Let’s look at the two functions created for the actions we have been working with.

defmodule Api13Web.UserView do
  use Api13Web, :view
  alias Api13Web.UserView
  def render("index.json", %{users: users}) do
    %{data: render_many(users, UserView, "user.json")}
  end
  def render("show.json", %{user: user}) do
    %{data: render_one(user, UserView, "user.json")}
  end
  def render("user.json", %{user: user}) do
    %{id: user.id,
      email: user.email,
      password: user.password,
      age: user.age,
      stooge: user.stooge}
  end
end

You’ll note that there is a render function that pattern matches each of the types of queries we are performing. They simply return a record that is shaped in the way we would like our JSON data to be shaped. You’ll also note that they use functions named render_many/3 and render_one/3. These functions are defined in the Phoenix.View module. Each of these functions users the “user.json” render definition to describe the shape of an individual rendered element.

Back in my previous articles on Phoenix and JSON, I called the json function directly. Using these helper functions makes the code much cleaner and easier to maintain. Once again, the Phoenix generators are building us code expressing a strong opinion on how our code should be structured.

Summary

This article describes the basics of building the GET endpoints for a single REST endpoint using Phoenix and Elixir. The key takeaways for me in repeating this exercise with Phoenix 1.3 are these.

1. Phoenix is very non-prescriptive in telling you how your code must be organized.
2. The new Phoenix 1.3 directory structure greatly encourages you to break up applications into smaller contexts. Rather than having one large, flat model, we are now encouraged to break our application into several contexts and store these in their own namespaces.
3. The Phoenix generators are your friend. I highly recommend using them to build your initial application structure. They build a structure that encourages you to separate the data layer from the controller and keep the controller logic very thin.
4. The view helper functions included with Phoenix make it easy to render JSON in a modular way. Using these along with custom render functions called from the controllers makes for clean code that separates concerns and allows a number of small functions to be composed in logical ways.

Next week’s post will continue these thoughts and demonstrate how to complete the remaining create, update and delete actions for this endpoint.