Phoenix with image upload to S3 in an API: Implementation and testing

David Magalhães
May 2, 2017 · 5 min read
Image for post
Image for post

Here it is, another small article about the implementation of a new functionality in the API service we are building using Elixir and Phoenix.

Implementation

We need to implement a way to upload user images to the cloud (S3 in this case) so after some research I found this article that explains in a very good way the basic code on how to implement it. Arc is a very good dependency to handle image upload either for local or for S3 storage.

After installing all the dependencies mentioned on the article, you can have a look into a simple controller, in this case I only specify the index and create methods, but all other methods (for example update) are similar.

defmodule MyApp.Web.AvatarController do...  def index(conn) do    # We use Guardian to fetch user information     
current_user = current_resource(conn)
if current_user != nil do
image_url = Avatar.url({"image.jpg", current_user})
conn
|> Phoenix.Controller.redirect(external: image_url)
else
send_resp(conn, :not_authorized, "")
end
end
def create(conn, %{"avatar" => avatar}) do current_user = current_resource(conn) if avatar != nil do
case Avatar.store({avatar, user}) do
{:ok, _file_name} -> send_resp(conn, :ok, "")
_ -> send_resp(conn, :service_unavailable, "")
end
else
send_resp(conn, :bad_request, "")
end
end
end

In this example, we start our methods by getting the user object. In this case we use Guardian, but you can get it from the database using Ecto. In our Avatar module generated by Arc using mix arc.g avatar we decided to incorporate the user ID into the file name.

defmodule MyApp.Avatar do

...
# We use this so other users can't check other user profile images
def filename(version, {_file, scope}) do
:crypto.hash(:sha256, "a_very_long_string_#{scope.id}_#{version}")
|> Base.encode16
|> String.downcase
end
# Override the storage directory:
def storage_dir(_version, {_file, scope}) do
"upload/user/avatars/#{scope.id}"
end
...end

Note that #{scope.id} will use the user.id field, be sure you have it or change it to the field that you want to use.

After that, we check if the request has the avatar parameter, if not return a bad request HTTP status. If the parameter is present, we try to store it on S3 and we can check if it was successfully stored, checking the match pattern {:ok, filename} , in this case we could use {:ok, _} because we won’t use the filename.

To use Arc with Ecto, you also need to add arc_ecto dependency. To add the field in the user table schema you need to start by creating a new migration to insert a new field.

# new_migration.exsdefmodule MyApp.Repo.Migrations.NewMigration do
use Ecto.Migration
def change do
alter table(:user) do
add :avatar, :string
end
end
end

After that, you need to add a new field into your user model and use the cast_attachments to validate the image upload and store the necessary information into the database.

# user.exdefmodule MyApp.User do
use MyApp.Web, :model
use Arc.Ecto.Schema
schema "user" do

...
field :image, MyApp.Avatar.Type
end
def changeset(user, params \\ %{}) do
user
|> Ecto.Changeset.cast(params, ...)
|> cast_attachments(params, [:image])
end
end

Be aware that when insert a new user the user.id isn’t available until the object is inserted on the database. You can generate an UUID (for example time + random number) to be associated to the filename or you need to insert the values first and then execute an update with the image only.

To view the image URL on the JSON structure I’ve created a simple method to correctly display the URL.

defmodule MyApp.Web.UserView do
...
def render("show.json", %{user: user}) do
‰{
"id": user.id,
"username": user.username,
"avatar": render_image_url(user)
}
end
def render_image_url(user) do
if user.avatar != nil do
Avatar.url({user.avatar.file_name, user}, :original)
else
nil
end
end
end

To test if it’s working you can try it using Postman to send a POST request with form-data selected on the body and selecting a image file to upload.

Testing

In this example, I am going to write a simple upload test and check if it was successful or not.

defmodule MyApp.Web.AvatarControllerTest do...test "Uploading test", %{user: user} do  upload = %Plug.Upload{path: "test/assets/user_avatar.jpg", filename: "user_avatar.jpg"}  post_params = %{"avatar" => upload}  conn = conn_build()
|> post(avatar_path(conn_build(), :create), post_params)
# In this case we send in the response the location URL of the image
assert List.first(get_resp_header(conn, "location")) == Avatar.url({user.id, user})
assert conn.status == 302
end

In this test, we select an image from our test assets and upload it using a POST method. We get the response, in this case we check for 302 (redirect) because we return the final url into S3 storage.

In this second test I’m going to show a test using Ecto (and ExMachina to build the model).

defmodule MyApp.Web.UserTest do...  test "Image Upload with Ecto" do
{:ok, avatar_struct} = MyApp.Avatar.Type.load("x.jpg?1234567")
user = build(:user, image: avatar_struct)
# Be sure the image is available in the test folder
upload = %Plug.Upload{path: "test/assets/avatar_user_1.jpg", filename: "avatar_user_1.jpg"}

# POST parameters
post_parameters = %{ "avatar" => upload }
conn = build_conn()
|> post(user_path(build_conn(), :create), post_params)
json_response(conn, :ok) == render_json("show.json", user: user)
end
defp render_json(template, assigns) do
assigns = Map.new(assigns)
MyApp.Web.UserView.render(template, assigns)
|> Poison.encode!
|> Poison.decode!
end

In order to implement tests to assure the behaviour of the functionality to upload images to AWS S3, we decided to use Fake S3, a fake AWS S3 API that replies in the same way as the real one, so we can test it at will without incurring in extra expenses and without an internet connection.

The readme file is pretty straight forward, and only two command lines are necessary to have it installed and running in your machine. After that you can check the code samples for different languages, in this case we are interested in Elixir. You can modify it on your config/test.exs and check if the port is the same as used in the fake server.

After that ensure that the Fake S3 server is running with the following command fakes3 -r ~/.s3bucket -p 4567, before you perform the tests. Run mix test and check if everything is as expected.

To add this to your continuous delivery pipeline you can add the following lines into before_script, for example:

before_script:
- gem install fakes3
- fakes3 -r $HOME/.s3bucket -p 4567 &

To not let the service hanging we can kill it after the tests finished running.

after_script:
— kill $(pgrep -f fakes3)

Thank you for reading this, hope you liked it and learned something new!

Thank you so much for reading and if you enjoyed this article make sure to hit that 👏👏 button. It means a lot to us! Also don’t forget to follow Coletiv on Medium, Twitter, and LinkedIn as we keep posting more and more interesting articles on multiple technologies.

In case you don’t know, Coletiv is a software development studio from Porto specialised in Elixir, iOS, and Android app development. But we do all kinds of stuff. We take care of UX/UI design, web development, and even security for you.

So, let’s craft something together?

Coletiv

Thoughts, dreams and rants about technology and work life…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store