Create an Elixir Phoenix API — Part 2— Generate an API Swagger Specification

<< Part 1 — Initial Barebones Setup

Welcome back! This time we’ll focus on how to generate and surface our API’s documentation, adding a route to help explore, and update our tests against the specification we generate! We’re in for a ride.

Upgrade to Phoenix 1.3

So it’s been a while since we last spoke in the first article! Phoenix 1.3 has been released in that time, and an update is necessary before we can use PhoenixSwagger.

You can follow http://phoenixframework.org/blog/upgrading-from-120-to-130 to update everything, or just update the dependency. As we’re not using the scaffolding tools to build our API, a simple dependency upgrade is all we need to continue.

Open mix.exs and update phoenix:

def deps do
  [{:phoenix, "~> 1.3.0"},
   ...]
end

That’s it! Now, let’s talk about swagger, why we want it and how to make it work for us!

What is Swagger and the Open API Specification?

If you visit https://swagger.io you’ll see:

The specification creates the RESTful contract for your API, detailing all of its resources and operations in a human and machine readable format for easy development, discovery, and integration.

but what does that boil down to for us? While writing APIs I’ve always appreciated a good amount of documentation and specifications around it. For me an API isn’t truly an API until it’s documented. Knowing the internals of an API, how it works, and what it does by pure virtue of having the code is nice but we all forget what we wrote and why. Having an API documented at the time of creation gives us a point to build on and work with.

Following the Open API specification helps our APIs maintain portability. You can use the Swagger Codegen tools to help clients more easily integrate. Have a .NET customer? Done! Javascript client? There’s a generator there too.

So now with a brief intro to what Swagger and the Open API specification is, let’s start using it within our API. We won’t write our document manually though! That’ll fall out of date as soon as your first feature branch is merged so we’ll use PhoenixSwagger to generate it for us.

Getting started with PhoenixSwagger

If you’re just joining us, the latest code from this series will be hosted https://github.com/aabrook/medium. To view and follow along with this article check out this pull request https://github.com/aabrook/medium/pull/2. It is structured by commit so if reading the code is informative enough for you it’s all there.

PhoenixSwagger is an OpenAPI Specification generator that follows the Phoenix mainline pretty closely when discovering your document schemas. There are a lot of assumptions internally about your project structure and how it generates an OpenAPI Specification but, thankfully, there is an escape hatch which allow you to leverage it’s creation macros without needing to restructure your API to match the changes of Phoenix. This article will not restructure the original API to match the latest Phoenix layout as APIs evolve and restructuring large projects can be time consuming and expensive.

PhoenixSwagger’s repo: https://github.com/xerions/phoenix_swagger

Colocated code generation is pretty sweet and we really like PhoenixSwagger here at everydayhero, so, we help support it through code contributions.

Setup Swagger

So first thing’s first, add PhoenixSwagger. Open up your mix.exs and add the following dependency:

{:phoenix_swagger, "~> 0.6.2"}

That was easy enough! We will also need a generator. Typically this would be as simple as running mix phx.swagger.generate priv/static/swagger.json but this relies on you having a project structure that is updated to match Phoenix 1.3.0 and later. We haven’t made those changes so let’s use the command line arguments available. We can use --router to point at our Router, and --endpoint to point to our Endpoint.

Open your mix.exs again and add a new alias within the aliases private function:

This will inform PhoenixSwagger where it can find the swagger_info, schema and controllers within our codebase.

Add swagger_info

Swagger_info is used to specify details of our API. It is required to include the version and title if we provide a swagger_info function. Named APIs are great for discovery, as is versioning for consistency so let’s do that. Open up web/router.ex and add the following towards the end of the file:

It’s that simple. This is Version 1.0 of our activity tracker “Distance Tracker”, hosted at “where-our-app.is.hosted”. localhost:4000 is where our app is hosted by default so, in the short term, you could set host to be this and use http://petstore.swagger.io/ to explore it. Without specifying a host PhoenixSwagger will default to the host specified in “config/config.exs”

Define your Schema

Great! We now have all that boilerplate setup! So little boilerplate in PhoenixSwagger when getting started. It is time to start documenting the structure of our API.

What’s a schema? A schema in PhoenixSwagger is a map which represents your API’s structure, the structure your data is served and how it can be consumed. It is the Elixir blueprint to generating your specification.

Add payload schema

Here we will define the schema of our API. We will need to specify the structure of our requests, the responses, and our routes. We will be able to colocate the description/definitions with our routes so that when we make changes it will be easy to update our schema.

First, let’s start by specifying the structure of our Tracker model. This will be used to represent our responses and requests. In our DistanceTracker controller (web/controllers/distance_tracker_controller.ex) we need to add PhoenixSwagger so we have access to it’s macros. Simply add use PhoenixSwagger to the top of the file:

Now we add the structure of our document. Where we have used PhoenixSwagger we can add the function def swagger_definitions do. This function will return a Map which is the definitions or our schema. In the first article we determined what came up with the structure of our API requests/responses:

This is the shape of data we return to our clients and require of our clients when they create records. Within our swagger_definitions function we will add to the map a Tracker attribute representing our structure above. I’ll also add uuid, inserted_at and updated_at as they are used by our API. It’s good to document a complete picture.

Here we have used 4 macros:

• title: When viewing the swagger document this contains the Title of the model
• description: What is the purpose of this model? How will it be used? What expectations do you have on it?
• properties: This macro takes a block in the format
  attribute :type, “description of the attribute”, options (Options examples include format: “date-format”, require: true)
• example: This is a map which will be translated into JSON as an example. It helps illustrate what we need from our clients.

For our base structure that’s it! We do return other responses though. On our index view we return a list of Tracker activities and when something goes wrong we return a structured Error object. Let’s add those while we here:

Here we’re using the items macro to specify that :Trackers is a list type. Schema.ref(:Tracker) informs the generator that we are returning a known type of :Tracker. This saves code duplication so we can use Schema.ref(:Tracker) anywhere it’s needed. Error is structured the same as our Tracker above.

That should cover everything we need to represent our payload structures. Now let’s document our routes!

Documenting Routes

So what good is documenting of our request/response structure without actually having routes to use? I suspect you’ve already read the title and discovered that’s exactly what we’ll do.

swagger_path is the macro used to generate our route’s documentation. Let’s start with our index route. Add this just above your def index(conn, _params) do function:

It is pretty straight forward. The function that this route directs to is index as represented with :index, with a request method of type get and resides at / . The summary is the brief of what our route will do. In http://petstore.swagger.io/ this will be the title displayed on the collapsed tab. description is a more verbose description to inform our clients the purpose and usage of our route. Finally, response contains the types of responses this route can return. response is structured by the response code, status, and defined Schemas. Our index will display all trackers we have so we simple reference the :Trackers definition from swagger_definitions.

So, we have our schema and our first route documented! We should be able to test it now! Open up the terminal where this app is hosted, then run mix swagger. Provided you added the swagger alias in the setup above you should see Generated priv/static/swagger.json. Feel free to explore this file but I feel it’s better if we just run it.

Petstore Explorer

I’ve mentioned http://petstore.swagger.io/ a couple of times now. This is a simple, client side app that you can use to explore your swagger specs. As the client is not served from within our domain we will run into CORS problems. As we want our API accessible from other sites let’s open up! I’ll be brief here as it’s a bit of an aside but still important to share.

Setup CORS

Add the CORS Plug dependency, which will manage CORS requests for us.

In mix.exs add {:cors_plug, “~> 1.4”} to our deps function. Next add the plug to lib/distance_tracker/endpoint.ex

By default the CORS Plug will allow any origin. Ideally we want to restrict this to trusted clients but here we’re just looking to test so we’ll leave it as default.

Start our service! We’ve upgraded to phoenix 1.3.0 so we now launch with mix phx.server

Once you’ve launched you should see [info] Running DistanceTracker.Endpoint with Cowboy using http://0.0.0.0:4000

Test that we can access our swagger asset by navigating to http://localhost:4000/swagger.json. If you can see your document we’re ready to continue!

Explore with the Petstore

Open up http://petstore.swagger.io/ and add our swagger.json route to the input at the top of the page. Click “Explore”. That’s it! You should now see your index route and be able to Try it out

Here you’ll be able to see the schema, titles, descriptions, examples and anything else we configure within our application. Have a look around, play with swagger details we’ve written to get a feel for how the document can be used.

Now, let’s finish our routes.

Setup remaining routes

We’re pretty close to having our entire app documented and usable now. We just need to add the last few routes and we’re finished! Let’s go straight ahead and do just that:

Here’s show, and should live near our show function. We’ve just introduced path parameters which use the {} notation. get "/{id}" will generate our show route and expect a parameter of id to be present. We’ve also added the parameters block. Here you specify all path and body parameters your API will use:

id :path, :string, "The uuid of the activity", required: true

We start with the parameter name, followed by where it is. This can either be :path or :body, which we will see in our create block. Next is the description and whether it is required or not. Showing a record an id is pretty important, so let’s make it required.

We’ve also added the 404 response, which returns the Error schema. Feel free to generate a fresh swagger file and test in the petstore again. You will now be able to provide the ID, as well as review the response types and examples.

Now :create

The main difference between this and index is we now have a post action and a :body parameter. Here we reference the :Tracker which will provide a good example of what we will expect. Give it a go!

I think that covers all the finer points but for completeness, here are the update and delete routes:

And that’s it! We now have a fully documented, explorable API! It’s a long way to go but now we can quickly and update our APIs without having to change external sites, other files, anything else! It’s all colocated!

We’re not quite finished though. PhoenixSwagger has validators that we can add to our test suite! This will help us keep our documentation up to date with any changes we make to our API. So let’s get straight into it.

Validate APIs with our tests

Add ex_json_schema

PhoenixSwagger has an optional dependency in ex_json_schema. We haven’t had to worry about that until now as we need it to validate our APIs in testing. Simply add {:ex_json_schema, “~> 0.5.1”} to our list of dependencies in mix.exs.

Generate swagger file on test run

While we’re here in mix.exs let’s update our test alias to generate a new swagger file on each run. This will ensure that each new test run has a fresh swagger spec. Update our test alias by adding "swagger”: ”test”: [“ecto.create — quiet”, “ecto.migrate”, “swagger”, “test”].

Update tests

Like everything else with PhoenixSwagger, we have another macro to use. Add use PhoenixSwagger.SchemaTest, “priv/static/swagger.json” amongst your test dependencies. If you have saved swagger.json somewhere else then make sure that you point to that file.

SchemaTest will add swagger_schema to our test setup map, so we can validate against it. It also adds the functions validate_resp_schema and json_response which we can use in our assertions. Let’s update our first test, index. At the end of the previous article it looked like this:

I will update this test now:

We haven’t changed too much between these tests. We’ve added the swagger_schema: schema to the test parameters. This is the schema which we generate. We’ve also updated our pipeline to include validate_resp_schema . This is the magic that we need! The response from our API request get(tracker_path(conn, :index)) will be validated. We expect to see “Trackers”, which is our list of Tracker. If we ever update our index route and return something else, e.g. missing required fields, this test will fail. Nice!

We also added json_response(200). This will assert that our API returns a 200 and convert our response into a map. This tidies up the Poison decoding too!

Simple changes but this gives us a lot of power in our API testing. We don’t need to do anything more special than this and can update all our tests to suit. In my repo I have a commit that does just this: https://github.com/aabrook/medium/pull/2/commits/2665e90ce287649c2312d5b20d3283ffa6804257

Once you’ve updated the tests run mix test and see your validations.

Broken Tests!

Oh no! I’ve broken some tests with these validations! I’ve got expectations that no longer exist!

In an interest to simplify my API I’ve ignored the nested data attribute. Here we can either update the response schema to include Trackers as a property of data, or, we can update our renderers. I feel it’s best to update our renders as we should be able to assume we’re rendering … data. Funny that people want data when they request … data.

Let’s open up web/views/distance_tracker_view.ex and simply remove the %{ data …} map and just return the renders.

Rerun the tests and you should be good to go!

Soooo many changes in here. We are now in a great place with our API though. It is explorable, documented, accessible and tested! PhoenixSwagger is powerful, flexible and easy to use. It can also generate JSON API specifications (http://jsonapi.org/). In the final article, let’s check that out. We will update our current API to a JSON API spec. All the while maintaining the good stuff we’ve worked on today.

Until next time!

Bonus! Embedded Swagger UI

Bonus time! Let’s embed the petstore within our app. PhoenixSwagger already does that for us, we just need to create a new route.

Open up web/router.ex and add

Now restart the server, open up http://localhost:4000/swagger annnnd that’s it! You now have an embedded swagger explorer in any app you have that uses PhoenixSwagger.

Now, until next time!

-Aaron