Building A Reading Queue in Ember and Phoenix, Part 1: Getting Started with Phoenix

Last Updated: 06/30/2016

Dr. David Bookenridge always struggled with near-sightedness. Actually, he was a book. Wait, he has a phD?

Introduction

In the last few posts on Ember and Phoenix, we started toying around with some tutorial applications and very basic Phoenix/Ember setups, which is a great way to start off and familiarize yourself with the general toolsets and get a clearer picture of how the two interact. That being said, it’s also not quite the same as actually building out a full application that we design, plan, and love!

We’re going to aim to create a new site in Phoenix and Ember, where the general goal is to allow Users to create Article Lists. These Article Lists will be able to be public or private, contain any number of Articles, and Users should be able to share out their reading lists simply!

Versions Used In This Series

Elixir: v1.2.4

Phoenix: v1.1.4

Ember: v2.4.2

Creating Our Project

We’ll start off by creating our Phoenix project first. There’s not much sense in getting our Ember site created without a backend to support it, of course! We’ll create our app that I’ve lovingly dubbed ArticleQ. We’ll skip the brunch assets for this project since we’ll be relying on Ember to handle everything UI for us! Run the following commands (and answer Y to any questions about fetching dependencies). We’ll also get the database set up and running so we can dive right into active development.

$ mix phoenix.new articleq --no-brunch
$ cd articleq
$ mix ecto.create

Including Comeonin

Since our project is going to offer user accounts, we’re going to need to include some libraries to help us with the authentication piece. The first of these is a library called Comeonin, which will provide us password encryption/passwird checking for our user accounts. We’ll start off by opening up mix.exs, and adding the following to the to application function:

:comeonin

Next, still in mix.exs, add this to the to deps function:

{:comeonin, "~> 2.3"}

Now that we’ve added the Comeonin library to our project, we need to fetch our dependencies and recompile our project. Run the following command to do so:

$ mix do deps.get, compile

Creating Users

Now that we have the library we need to be able to support users, we’ll need to actually create our User schema! We’ll use a very simple user model that relies on the username and password for login and also includes a required email address (since we’ll eventually want to provide a means for users to provide access to accounts when they lose their password). We’ll use Phoenix’s JSON generators to set up our base user scaffold:

Run the following command:

$ mix phoenix.gen.json User users username:string email:string encrypted_password:string

As per the command’s instructions, we’ll now open up web/router.ex, and we’ll add the following code:

scope "/api", Articleq do
  pipe_through :api
  resources "/users", UserController, except: [:new, :edit]
end

What we’re doing here is setting up a namespaced route that we’re expecting all of our JSON API calls to pass through. So, for our users example, the URL would be http://[domain]/api/users.

Next, we need to bring our database state up to match our code. We’ll run our migrations in our dev environment with:

$ mix ecto.migrate

Now that we have our base scaffold set up, we need to start making a few modifications to support dealing with passwords with our API. First, in web/models/user.ex, add the following line to the schema definition:

field :password, :string, virtual: true

(Remember that password was not one of the columns we set up with our model; if we want to have an attribute on our schema that won’t actually get persisted to the database, we need to declare it as a “virtual” attribute).

We’ll also need to change @required_fields and @optional_fields at the top of our model:

@required_fields ~w(username email)
@optional_fields ~w(password encrypted_password)

Also, we’ll need to change the changeset function. Add this to the bottom of the function, and we’ll explain what hash_password will do in a minute.

|> hash_password

Now, add new function hash_password. This will take in the changeset that we’re passing along and if we’re modifying the password at all (via the get_change function), it will put a hashed version of the password into as a new change onto the changeset. We’re using Comeonin’s Bcrypt implementation to handle this.

defp hash_password(model) do
  case get_change(model, :password) do
    nil -> model
    password -> put_change(model, :encrypted_password, Comeonin.Bcrypt.hashpwsalt(password))
  end
end

You’ll also need to modify the test located at test/models/user_test.exs test or it will fail on the validity test (since we need to pass in a password):

@valid_attrs %{email: "some content", password: "test", username: "some content"}

Including Guardian

Now that we have the baseline for our tests, we need to add in a library that will help us create our JSON Web Token that we’ll use for authentication with our Ember-based Single Page Application. Guardian is a great authentication library that provides a lot of nice functionality for us, so we’ll stick with using that! First, we’ll go to mix.exs and add to the deps function:

{:guardian, "~> 0.10"}

Next, open up config/config.exs, and add this to the bottom of the file:

config :guardian, Guardian,
  issuer: "ArticleQ",
  ttl: {30, :days},
  verify_issuer: true,
  secret_key: <your secret key>,
  serializer: Articleq.GuardianSerializer

“ArticleQ” is our project name, so we’ll use that as our issuer name. We’ll set the tokens to expire after 30 days, and we’ll need to supply a secret key. (If you need to generate one, just use mix phoenix.gen.secret which will give you a secret key to use). We will also tell our Phoenix project that we’ll have a special serializer module (that we still need to write) called GuardianSerializer (we’ll get to this one later!) Now run:

$ mix do deps.get, compile

Next, open up web/router.ex, and add this to the :api pipeline:

plug Guardian.Plug.VerifyHeader
plug Guardian.Plug.LoadResource

Finally, create lib/articleq/guardian_serializer.ex and populate it with the following contents:

defmodule Articleq.GuardianSerializer do
  @behaviour Guardian.Serializer
  alias Articleq.Repo
  alias Articleq.User
  def for_token(user = %User{}), do: { :ok, "User:#{user.id}" }
  def for_token(_), do: { :error, "Unknown resource type" }
  def from_token("User:" <> id), do: { :ok, Repo.get(User, String.to_integer(id)) }
  def from_token(thing), do: { :error, "Unknown resource type" }
end

Let’s talk about this for a bit. The first thing we’re doing is saying that this is adopting the Guardian.Serializer behaviour. (No, that’s not a typo: learn more about this topic here). Next, we create aliases for our Repo and User since we’ll be using both of those a bit.

Next, we need to set up both directions of serialization. The first being serializing a user for a token (so, if we pass in a User struct, we’ll get back a string telling the type is a User and the id for the User). Otherwise, we throw an error about this being an unknown resource type.

from_token is clearly just the reversed version of this. From the user token, if we were able to decode the token and saw that the token’s key was in the pattern of “User:1234”, we know it’s telling us to look for a User with an id of 1234. Otherwise, throw that same unknown resource type error!

Setting Up Guardian with Users

Now that we have Guardian configured, let’s actually use it to build something! First, create web/controllers/session_controller.ex. We’ll do all of our logging in/logging out logic through this file:

defmodule Articleq.SessionController do
  use Articleq.Web, :controller
  alias Articleq.User
  def create(conn, %{"username" => username, "password" => password}) do
    case Repo.get_by(User, %{username: username}) do
      nil ->
        Comeonin.Bcrypt.dummy_checkpw
        login_failed(conn)
      user ->
        if Comeonin.Bcrypt.checkpw(password, user.encrypted_password) do
          {:ok, token, _} = Guardian.encode_and_sign(user, :api)
          render(conn, "token.json", %{token: token})
        else
          login_failed(conn)
        end
    end
  end
  defp login_failed(conn) do
    conn
    |> render("login_failed.json", %{})
    |> halt
  end
end

This is a pretty standard API build. It uses the standard Phoenix controller structure. We’re telling the controller that our create API call is expecting two parameters: “username” and “password”, otherwise it won’t try to execute our create function.

Next, we try to find our user using Ecto’s “get_by” function, and we search by username. If we couldn’t find the user, we call Comeonin’s “dummy_checkpw” function first to simulate the password check function (to prevent timing attacks allowing someone to guess valid usernames). Then, we call out to the login_failed function (which we’ll discuss later).

If we found the user, we check the password against the user’s encrypted password. Next, we grab the token from Guardian’s encode_and_sign function and encode an API token for that user. Finally, we render out the successful result.

Finally, our login_failed function just renders out a login_failed json structure and halts all further modifications to the connection!

We still need to implement those render blocks. Create web/views/session_view.ex and give it the following contents:

defmodule Articleq.SessionView do
  use Articleq.Web, :view
  def render("token.json", %{token: token}) do
    %{token: token}
  end
  def render("login_failed.json", _) do
    %{errors: ["Invalid username/password combination!"]}
  end
end

We can see our two different render function calls. The first just returns the token as the root element, the second returns an errors array with a single error in it. Simple!

Finally, in web/router.ex, we need to expose the login and logout API calls, so add these under the “/api” scope:

post "/login", SessionController, :create, as: :login

Now, we’re ready to see how our login API works!

Simulating Login/Logout With CURL

Pre-Simulation Setup

We’ll need to start off by creating a database seed to generate a starting user for us to use. A seed file is a way to tell Phoenix/Ecto that you’d like to insert some data into the database to start off. So, we’ll open up priv/repo/seeds.exs and add the following:

alias Articleq.Repo
alias Articleq.User
User.changeset(%User{}, %{username: "test", email: "test@test.com", password: "test"})
|> Repo.insert!

What we’re doing here is just creating a new changeset for the User with a username of “test”, an email of “test@test.com”, and a password of “test” and inserting that changeset into our Repo. You now have the setup needed to be able to run the sample login CURL request! Let’s run our seed file really quick:

$ mix run priv/repo/seeds.exs

Valid Login Simulation

Run the following command (if you have CURL installed. You could also use something like Postman to simulate):

$ curl -X POST -d '{"username":"test", "password":"test"}' http://localhost:4000/api/login
{"token":"eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJVc2VyOjEiLCJleHAiOjE0NjI4NTMyMjksImlhdCI6MTQ2MDI2MTIyOSwiaXNzIjoiQXJ0aWNsZVEiLCJqdGkiOiJiMTE0ZGFiYi1hMmVkLTQ1NjAtODFiNS1mOTcwMDA0NjMyMWEiLCJwZW0iOnt9LCJzdWIiOiJVc2VyOjEiLCJ0eXAiOiJhcGkifQ.FmS-FqV-4lkNJKtAKrQUDKWS4qFDlam5tGnDeEu0FqebCnyJjY0n69VHBIcPKqImh13eVagzvBYH9QwazZzsZw"}

Notice that we specify a username and password and get back the token!

Invalid Login Simulation

Run the following command to simulate the response to a bad username/password combination:

$ curl -X POST -d '{"username":"test", "password":"fail"}' http://localhost:4000/api/login
{"errors":["Invalid username/password combination!"]}

Including Corsica

We still have more libraries we’ll need to build out our JSON API. We’ll be sending cross-origin requests, so we need to tell the API specifically to allow these types of requests (usually blocked by something called the CORS policy). We’ll use Corsica to handle the modification to allow Cross-Origin Resource Sharing:

In mix.exs, add to the deps function:

{:corsica, "~> 0.4"}

Then in lib/articleq/endpoint.ex, add this before plug Articleq.Router line:

plug Corsica, origins: "*", allow_headers: ["accept", "content-type"]

Now our API will allow requests from our local Ember server (Ember will, by default, run on port 4200). This is also a little bit of future proofing in case we need to switch the port on Ember.

Including Ja_Serializer

Finally, we need to set up our API to support JSON-API. We’ll add the ja_serializer library to our Phoenix application to accomplish this!

Again, in mix.exs, add this to deps:

{:ja_serializer, "~> 0.8.1"}

Then in config/config.exs, add to the bottom of the file:

config :phoenix, :format_encoders,
  "json-api": Poison
config :plug, :mimes, %{
  "application/vnd.api+json" => ["json-api"]
}

This sets up a new format encoder (which will just use Poison), and also says that any request coming in as the “application/vnd.api+json” mime type will use the json-api format encoder.

Now run (taken from the Corsica docs):

$ touch deps/plug/mix.exs
$ mix deps.compile plug
$ MIX_ENV=test mix deps.compile plug

Finally, in web/router.ex, modify the :api pipeline to:

pipeline :api do
  plug :accepts, ["json", "json-api"]
  plug Guardian.Plug.VerifyHeader
  plug Guardian.Plug.LoadResource
  plug JaSerializer.Deserializer
end

Getting A User List When Logged In

Now that our libraries and dependencies are set up, let’s modify our user controller to be JSON-API compliant!

First, modify web/views/user_view.ex:

defmodule Articleq.UserView do
  use Articleq.Web, :view
  use JaSerializer.PhoenixView
  attributes [:username, :email]
end

Next open up web/controllers/user_controller.ex. First, modify the scrub_params line (since we will be expecting data as the root data element inbound):

plug :scrub_params, "data" when action in [:create, :update]

Next, modify the index function (note the change of keys that we’re passing along from “users” to “data”):

def index(conn, _params) do
  users = Repo.all(User)
  render(conn, "index.json", data: users)
end

Then the create function. Note the new structure here. New data coming in will be of this format:

{
  "data": {
    "attributes": {
      "foo": ...,
      "bar": ...,
      etc
    }
  }
}

So it makes it clear why we need this modification to the pattern matching in the create function. Also, we again need to change any of our calls to render to use data instead of user(s).

def create(conn, %{"data" => %{"attributes" => user_params}}) do
  changeset = User.changeset(%User{}, user_params)
  case Repo.insert(changeset) do
    {:ok, user} ->
      conn
      |> put_status(:created)
      |> put_resp_header("location", user_path(conn, :show, user))
      |> render("show.json", data: user)
    {:error, changeset} ->
      conn
      |> put_status(:unprocessable_entity)
      |> render(Articleq.ChangesetView, "error.json", changeset: changeset)
  end
end

Then the show function. Same pattern here too:

def show(conn, %{"id" => id}) do
  user = Repo.get!(User, id)
  render(conn, "show.json", data: user)
end

Then update (same pattern as the create function):

def update(conn, %{"id" => id, "data" => %{"attributes" => user_params}}) do
  user = Repo.get!(User, id)
  changeset = User.changeset(user, user_params)
  case Repo.update(changeset) do
    {:ok, user} ->
      render(conn, "show.json", data: user)
    {:error, changeset} ->
      conn
      |> put_status(:unprocessable_entity)
      |> render(Articleq.ChangesetView, "error.json", changeset: changeset)
  end
end

Fixing Our Broken Tests

Right now, after all of these modifications without any modifications to our tests, we have lots of broken tests! Let’s start off by seeing which tests are broken. Run:

$ mix test

Like I said, too many red tests! Let’s fix these one-by-one. Open up test/controllers/user_controller_test.ex and change the module attributes to:

@valid_attrs %{email: "test@test.com", username: "test"}
@valid_create_attrs %{email: "test@test.com", username: "test", password: "test"}
@invalid_attrs %{}

Then change the setup block (remember that we’re using ja_serializer which is expecting json-api standards):

setup %{conn: conn} do
  conn = conn
    |> put_req_header("accept", "application/vnd.api+json")
    |> put_req_header("content-type", "application/vnd.api+json")
  {:ok, conn: conn}
end

Next, change the test for show (note the new structure; json-api dictates that we send back the id of the resource, the type, and then an attributes map that contains the other exposed properties):

test "shows chosen resource", %{conn: conn} do
  user = Repo.insert! %User{}
  conn = get conn, user_path(conn, :show, user)
  assert json_response(conn, 200)["data"] == %{
    "id" => "#{user.id}",
    "type" => "user",
    "attributes" => %{
      "username" => user.username,
      "email" => user.email
    }
  }
end

Next, tests for create. Note the change to the post function call.:

test "creates and renders resource when data is valid", %{conn: conn} do
  conn = post conn, user_path(conn, :create), data: %{ attributes: @valid_create_attrs }
  assert json_response(conn, 201)["data"]["id"]
  assert Repo.get_by(User, @valid_attrs)
end
test "does not create resource and renders errors when data is invalid", %{conn: conn} do
  conn = post conn, user_path(conn, :create), data: %{ attributes: @invalid_attrs }
  assert json_response(conn, 422)["errors"] != %{}
end

Finally, tests for update:

test "updates and renders chosen resource when data is valid", %{conn: conn} do
  user = Repo.insert! %User{}
  conn = put conn, user_path(conn, :update, user), data: %{ attributes: @valid_create_attrs }
  assert json_response(conn, 200)["data"]["id"]
  assert Repo.get_by(User, @valid_attrs)
end
test "does not update chosen resource and renders errors when data is invalid", %{conn: conn} do
  user = Repo.insert! %User{}
  conn = put conn, user_path(conn, :update, user), data: %{ attributes: @invalid_attrs }
  assert json_response(conn, 422)["errors"] != %{}
end

Now, run:

$ mix test

And everything is back to being all green! Fantastic!

Simulating API Interaction (Round 2)

With these changes, let’s go back and try our API again. First, we’ll run:

$ curl http://localhost:4000/api/users
{"jsonapi":{"version":"1.0"},"data":[{"type":"user","id":"1","attributes":{"username":"test","email":"test@test.com"}}]}

We get this even if we’re not specifying a token, which is no good! Anyone could get a list of users this way! Let’s specify a proper access restriction behavior with Guardian instead!

Verifying Token For Users Access

At the top of web/controllers/user_controller.ex, before the scrub_params line, add a plug call for Guardian’s “EnsureAuthenticated” plug:

plug Guardian.Plug.EnsureAuthenticated, handler: Articleq.AuthErrorHandler

This tells our app that we should delegate any error states introduced by the “EnsureAuthenticated” call go to the Articleq.AuthErrorHandler module (that we haven’t written yet). So, let’s write it! Create web/controllers/auth_error_handler.ex:

defmodule Articleq.AuthErrorHandler do
  use Articleq.Web, :controller
  def unauthenticated(conn, _params) do
    conn
    |> json(%{"errors" => ["You must be an authenticated user to view this resource!"]})
    |> halt
  end
end

It expects the existence of an unauthenticated function with an arity of 2 (it’ll send in the conn and the parameters). We’ll just render out some json saying that you must be authenticated and call it a day!

Now let’s test it all again!

Simulating Our API Interaction (Round 3)

First, verify that our original check works. Run:

$ curl http://localhost:4000/api/users
{"errors":["You must be an authenticated user to view this resource!"]}

That looks good! Getting a list of users with no authentication fails!

Next, run:

$ curl -X POST -d '{"username":"test", "password":"test"}' -H "Accept: application/vnd.api+json" -H "Content-Type: application/vnd.api+json" http://localhost:4000/api/login
{"token":"eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJVc2VyOjEiLCJleHAiOjE0NjI4NTMyMjksImlhdCI6MTQ2MDI2MTIyOSwiaXNzIjoiQXJ0aWNsZVEiLCJqdGkiOiJiMTE0ZGFiYi1hMmVkLTQ1NjAtODFiNS1mOTcwMDA0NjMyMWEiLCJwZW0iOnt9LCJzdWIiOiJVc2VyOjEiLCJ0eXAiOiJhcGkifQ.FmS-FqV-4lkNJKtAKrQUDKWS4qFDlam5tGnDeEu0FqebCnyJjY0n69VHBIcPKqImh13eVagzvBYH9QwazZzsZw"}

Okay, our login call still works. Now, let’s get a list of users with our auth token specified! Run:

$ curl -H "Authorization: eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJVc2VyOjEiLCJleHAiOjE0NjI4NTMyMjksImlhdCI6MTQ2MDI2MTIyOSwiaXNzIjoiQXJ0aWNsZVEiLCJqdGkiOiJiMTE0ZGFiYi1hMmVkLTQ1NjAtODFiNS1mOifQ.FmS-FqV-4lkNJKtAKrQUDKWS4qFDlam5tGnDeEu0FqebCnyJjY0n69VHBIcPKqImh13eVagzvBYH9QwazZzsZw" http://localhost:4000/api/users
{"jsonapi":{"version":"1.0"},"data":[{"type":"user","id":"1","attributes":{"username":"test","email":"test@test.com"}}]}

And that’s it! We were able to get some users! Now, we’re ready to move on to the next part of our project: implementing the initial Ember project structure!

Conclusion

This gives us a pretty solid start to building our Article Queue app! We have a lot of configuration that we’ve run through and a lot of initial work to make it all flow together properly, but now we should be able to repeat the last few steps over and over for each new piece of complexity we add to our API project! Even better, our application is all ready-to-go for everything that Ember is expecting of us!

If you’d just like to mess around with some of the code or dive right into the Ember side of the implementation, you can grab the repo at any time from the following link:

Diamond/articleq-phoenix

In our next post, we’ll implement the initial Ember side of this application and handle some basic authentication with ember-simple-auth!