Create an Elixir Phoenix API — Part 1 — Initial Barebones Setup

When getting started with Elixir and Phoenix there is plenty of help online with setting up a Phoenix app with the local HTML and JavaScript stack but few that focus on a pure API. I know that when I start learning a new language or environment I find it useful to have articles that directly illustrate what I’m hoping to create. Stackoverflow is great and all but it’s not the ideal platform for tutorials.

Anyways, the goal of this series is to help demonstrate how to

• start a fresh phoenix project, setup a simple RESTful JSON based API with few routes (this tutorial)
• generate a swagger specification that can be ingested by other applications like http://petstore.swagger.io (Part 2)
• simplify generating useful documentation and examples so your API is always up to date (Part 2)

Series Goal

As with all APIs we need a purpose! In this series we will create the API for a distance based activity tracker. Our users will be able to create, read, update, delete, paginate, filter and sort. I’ve chosen these actions as they are typical of a RESTful API.

Our expected payload to create an activity:

And our routes:

Routes:
GET "/"       # List all of my events recorded
GET "/:id"    # Show only a single event based on it's UUID
POST "/"      # Record a new activity
DELETE "/:id" # Delete an activity by UUID
PATCH "/:id"  # Update an activity by UUID

Filtering and sorting will be determined by parameters on the route GET "/”

So that’s the goal but we need to start somewhere!

Setup a barebones Elixir Phoenix API

This tutorial assumes a relatively beginners level with Elixir and you have at least followed the initial Phoenix Up and Running guide. It was written with Elixir 1.4.1 and Phoenix 1.2.1

As setting up a barebones API can be time consuming, in this initial tutorial we will not create user management but just track activities.

This is a Phoenix based API, we need a Phoenix app underneath, let’s create one! We will choose no-brunch as we do not need any of the javascript libraries and no-html:

mix phoenix.new --no-brunch --no-html distance_tracker

Then when prompted by Fetch and install dependencies? [Yn] choose Y to get straight into it. Once everything is setup you will be prompted to run mix ecto.create but let’s wait until we have something we want in our database.

So right now we have an almost barebones Phoenix application, just a few bits of boilerplate left to remove:

• Thanks to the no-html and no-brunch we only have one file to remove

rm web/channels/user_socket.ex

• Remove socket “/socket”, DistanceTracker.UserSocket from lib/distance_tracker/endpoint.ex as we are creating a REST API, not sockets as amazing as they are.
• Tidy up lib/distance_tracker/endpoint.ex clearing out the only: static assets but leave it there for now as we will want it in a future tutorial.

plug Plug.Static,
    at: "/", from: :distance_tracker, gzip: false,
    only: ~w()

• Annnnd that’s it! We have an empty Phoenix application to get going.

Setup our model

Quick aside: If you have Phoenix 1.2.3 we can use the task mix phoenix.gen.json to create our JSON Model/View/Controller/Migration and Test but that hides a lot of pieces of the API puzzle. Here we’ll walk through crafting our API by hand.

So we have a barebones app, now we need to actually do something with it. Let’s start with our data model as we will need something to serve.

First we need a migration to add our data:

mix ecto.gen.migration create_tracker

Open up priv/repo/migrations/<the-timestamp-when-you-created>_create_tracker.exs

There’s not much in there. Just a change function which we will update but, before we change it let’s look at our expected payload again:

We can see we need a date for completed_at, the string of an activity and a number to represent distance. We also highlighted that our key when working with our routes will be UUIDs. Lets update our migration to create a new table :distance_tracker:

For this tutorial we won’t add users with the goal to keep things simple as can be.

Great! Next we need a model to access the data.

touch web/models/tracker.ex

And we want to populate it to so we can work with the model. I’ll add a changeset function so we can easily convert our request params into a model we can work with. This will save us coming back later:

Yay! We have a model! Now let’s finally create and migrate. This step assumes that you already have postgres installed and your config/dev.exs has the correct credentials:

mix do ecto.create, ecto.migrate

So we have a model we can use with Ecto. Next let’s create our Controller and Views to use it!

MVC because Phoenix likes MVC

Even though we are not creating a client application it is still good practice to follow MVC patterns. If not for the typical separation of concerns that you get from keeping your layers independent at least you get to follow standard Phoenix conventions and no-one will be surprised where your code resides.

Create a controller!

touch web/controllers/distance_tracker_controller.ex

And put code in it!

Let’s start with just the index route so we can at least start our server and make our first request.

Views! But we deleted all of our views! Oh noes! And we have an “index.json” route to render! What will we do?!

Create a view of course online drama star:

touch web/views/distance_tracker_tracker_view.ex

We know we will need a show.json route eventually let’s just add it in now to save coming back.

Quick note around render_one and render_many here. If we named our view DistanceTracker.View the render function would need a 4th parameter of as: :tracker but since we only have .TrackerView Phoenix creates a map with tracker: tracker for us.

Now that we have our MVC configured we need a route. Open up web/router.ex and get our route in there! Your router.ex file should look like:

Whew we made it this far so let’s finally fire it up!

mix phoenix.server

And go see your new API: http://localhost:4000/

[]

BOOM! Isn’t that amazing? Just look at those brackets standing tall, taking no trouble from no-one!

Errors!

Okay, okay. Before we add the guts of the controller so we can actually use our API let’s setup our error handling.

We need to update `web/views/error_view.ex` as we are likely to use more error response codes. Lets just add the 422.json for now. For brevity I will not use the assigns provided during render but feel free to add more useful logic:

If you’re interested in testing you will also need to update the tests related to the error views. It’s our only test so it only seems right to use it. If you’ve added more informative error messages than you will need to do a little more work here.

We’ve also removed the :view helper so let’s remove more code!

rm web/views/error_helpers.ex

While we’re here let’s clear away the last of the boilerplate:

• remove import DistanceTracker.ErrorHelpers/ from web/web.ex

I think we’re ready to create our routes!

APIs do API things

Finally! Let’s add show, create, update and delete routes! So long, much code, wow.

First, lets add an alias for plug to the top of the controller to simplify status handling and add ErrorView to our list of DistanceTracker aliases:

alias DistanceTracker.{ErrorView, Repo, Tracker}
alias Plug.Conn

Let’s add a show action:

There’s a few things going on here that you might not be familiar with. We’re using a bunch of pattern matching to keep the function concise. We need a UUID from the request, which is provided by our router, so we grab that immediately. Next we want to make sure that we get an activity from our database so we use with and the struct match a result. If none is found we have an else block to pickup the not found status. Typical Elixir goodness.

API’s rely heavily on status codes so in our error state we are going to return a 404 when a page is not found. If our stars have aligned correctly we will use the previous View.render(“show.json”, %{tracker: tracker}) to display our activity.

“Okay, great, I can now list all activities and show a single one but our database is still empty! Let me create something!”

Okay geez! These conversations with myself are really … not as rewarding as I hoped. I seem kind of needy and demanding.

Here is our create function:

Working with dates is hard. Historically Elixir DateTime didn’t work well with dates but today it will convert iso8601 dates nicely. As long as our API documentation states this is the format we accept then we are golden.

I’ve also added more pattern matching to simplify our error handling.

Our delete function:

Update!

I have left out the update function as I think it is a good exercise for the reader. The required pattern matching is above and the Ecto docs are incredible

I want access!

We are now at the final step of our tutorial. Set the routes and start using our API.

Open up web/router.ex and add the final routes:

You can use your API! Open up your favourite API tool (https://www.getpostman.com/ is quite popular and free) and start exploring.

POST the initial payload we determined and see the response. Use the show and delete routes. Explore!

Testing

We have reached the end of the tutorial to create a very simple CRUD API but unit testing is a good practice to have. Let’s wrap up with some not so TDD unit tests.

First things first. As we are using UUIDs we will need to import a library that helps us generate our own UUIDs. Open up mix.exs and add {:uuid, “~> 1.1”}

Now let’s create a factory to help save Trackers to the database. The goal of a factory is to simplify creating Models for us to test with. Let’s get straight to creating our Tracker:

touch test/support/factory.ex

So here we have some very simple to use helper functions to create new Trackers. As the comments highlight we just use tracker = insert(Tracker, activity: "what-I-did") and then request or use what we’ve saved.

Next we’ll create the test and add some helpers:

touch test/controllers/distance_tracker_controller_test.exs

We use ConnCase so our tests receive a connection we can test against. This is followed by importing tracker_path so we can make our web requests. Next we have our useful aliases to make our code easier to read and lastly a helper function to allow for short hand Tracker creation.

While we’re here let’s also update our conn_case.ex and add default headers to our connection. Jump down to setup tags do and change

{:ok, conn: Phoenix.ConnTest.build_conn()}

to

conn =
  Phoenix.ConnTest.build_conn()
  |> Plug.Conn.put_req_header("content-type", "application/json")
  |> Plug.Conn.put_req_header("accepts", "application/json")
{:ok, conn: conn}

This helps ensure that all of our web requests from the test has the expected content types.

Now that we have our basics let’s test our controller!

So the :index route is a good place to start. We will use our route helper, raise a request then compare our response with the data format we expect. Let’s get into it:

Here we use our helper to create some records to query for. Then make the index request! Kind of obvious I guess… We use our conn that’s provided to the test, make a get request to the :index route, extract the body and decode it for easy assertion. Then we assert! That is actually really straight forward and self descriptive. The system works!

We have a show route, let’s test that one next. Get the easier routes our of the way first:

So easy! We insert it, request it, match the payload, match the status and get on with it. We inserted 2 records deliberately as well, that’s not a typo. We want to confirm that we are getting back one result and the controller isn’t just caching the most recent.

We haven’t tested the 404, let’s do that quickly:

Pretty simple and straight forward. Make the request, match the status and the body. Let’s move on!

Next let’s create a record. Let’s go back to our original payload:

and send that!

So this is a straight up POST where we take the expected fields from the response and compare vs our payload. Simple. Easy. Direct. Descriptive. It passes! We also want to assert the status code. We chose 201 in our controller so we ensure that is the response code we receive.

That’s it for create right? Well, obviously not or we wouldn’t be having this virtual discussion! To successfully pass that test we could just funnel back what we receive so we also need to confirm it’s saved to the database:

Now this is a more involved test. We need to get the :uuid from our initial request to query the database, then we compare with our initial payload. Now, if it wasn’t for dates being a little bit rough today, it would be a simple comparison. Here we also need to convert our initial date and compare the record we retrieved.

Let’s add the unhappy paths:

And then we run our tests!

test Requires completed_at when creating a request (DistanceTracker.TrackerControllerTest)
     test/controllers/distance_tracker_controller_test.exs:105
     ** (MatchError) no match of right hand side value: {:error, :invalid_format}

Testing works! After all this time I still find it rewarding when tests catch bugs. Let’s go and fix our controller. The :invalid_format error is an issue with the date parsing. As we haven’t provided a date of course it will error. A question which comes to mind is whether this should be a concern of the Model or the Controller. I can see both arguments but, I think a function to parse the received date in the Controller is good for now as it’s a concern of the controller to validate what we receive from our clients:

Then we need to update our create function. We’ll change

{:ok, date, _} = DateTime.from_iso8601(params["completed_at"])
params = %{params | "completed_at" => date}

to

date = parse_date(params["completed_at"])
params = Map.put(params, "completed_at", date)

Simple enough change that resolves this failed test. We are parsing the date and now adding it to the map if it wasn’t there before.

Now, let’s finish up by deleting records.

So we followed the same pattern as create by comparing the response first and then the database change afterwards. Finally a 404 for when it’s not found. Much simpler as we are just confirming it’s gone.

Again, I’ll leave it up to you to add the update tests. I believe in you!

That’s a wrap! Finally!

So that’s it for this tutorial. We’ve been through a lot together already, you and I, yet this is only the beginning. Now that we have a base to build upon we can work on the fun stuff, our Swagger implementation! It’ll be a whole lot easier to parse I promise! Code wise a whole lot easier anyways.

Until next time.

-Aaron