Phoenix: Understanding how to seed data.

Leverage test fixtures, Ex Machina, and Faker to create high-quality test and development data.

Photo by Binyamin Mellish from Pexels

What is seeding?

Seeding is the initial creation of data in your database typically run before your tests or to create a convenient developer or QA testing environment. Though, you can seed in a production or other environment as well.

Why should you care?

By learning how to create seed data effectively, you can dramatically improve your tests' reliability, reusability, and readability.

More importantly, you will improve the speed and quality of your development cycle to release more features and fewer bugs.

Examples using the demo blog application

All of the following concepts and examples can be applied in any phoenix application. If you like, you can see all of the code samples In the example blog application.

Global Seed Data

In any phoenix application generated using mix, you should have a seed.exs file.

You can create data in this file that you want to pre-exist for all of your tests or in your development environment.

Notice the Repo.delete_all(Post) line. Unless you want the seed file to re-create existing data over and over, you might want to consider deleting any data that already exists so that your seed data is consistent.

You can run the seed file using:

mix run priv/repo/seeds.exs

Make you are in the correct environment when running your seed data.

# on mac or linux:
MIX_ENV="test" mix run priv/repo/seeds.exs # for test env
MIX_ENV="dev" mix run priv/repo/seeds.exs # for dev env
# On windows you'll have to set the env first using either: $env:MIX_ENV="test"
$env:MIX_ENV="dev"
# then run the seed command like normal
mix run priv/repo/seeds.exs

Configure different test and dev seed files.

The seed.exs file is automatically configured to run in the mix ecto.setup alias.

You can configure different behaviors for your global seed file depending on what environment you are in using the Mix.env() variable.

If you’d like this global data in your tests, you can add the command to run your seed file to your mix test alias.

Make other seed files.

There's nothing special about the seed.exs file so that you can make other seed files. In this blog app example, you might want to have a different seed file to add five posts for you.

You would run this file using:

mix run .\priv\repo\five_blog_posts.exs

Realistic fake data using Faker.

Faker is an elixir library for creating fake data. This is useful when you want to create realistic data to test your application. For example, in the demo blogging application, It would be convenient to generate long, medium, and short content.

• Install Faker: likely you’ll have to add an up-to-date version of the following in your mix dependencies:

{:faker, "~> 0.16", only: :test}

Faker recommends you only have the faker module in the test environment. However, since we’re seeding data for dev testing, you may also want to have it for the dev environment.

The installation steps on Faker likely also say to add Faker.start() to your test_helper file.

Make sure the faker dependency is installed by running mix deps.get.

Once Faker is installed, You can use it to generate fake data conveniently. I’ll cover the most useful modules, but check out their documentation if you want to see everything Faker can do.

Faker Lorem

Generate words, sentences, and paragraphs with complete control over the size of your content. Lorem is short for “Lorem Ipsum”, which is a common way to generate fake text.

alias Faker.Lorem 
# fake word with 5 characters
Lorem.characters(5) 
# fake word with between 5 to 10 characters
Lorem.characters(5..10)
# fake sentence with 5 words
Lorem.sentence(5)
# fake sentence with between 5 to 10 words
Lorem.sentence(5..10)
# fake paragraph with 5 sentences
Lorem.paragraph(5)
# fale paragraph with 5 to 10 sentences
Lorem.paragraph(5..10)
# fake list with 5 paragraphs joined in a string
Lorem.paragraphs(5) |> Enum.join("\n")
# fake list with between 5 to 10 paragraphs joined in a string
Lorem.paragraphs(5..10) |> Enum.join("\n")

You could use this to improve the five_blog_posts.exs seed file from before.

Faker Person

Generate names for a person

alias Faker.Person
Person.first_name() "Elizabeth"
Person.last_name() "Hayes"
Person.name() "Mr. Bianka Ryan"
Faker.Person.prefix() "Mr."
Faker.Person.title() "Dynamic Identity Administrator"

Faker Address

Generate pseudo-realistic addresses

alias Faker.Address
Address.country() # "Gibraltar"
Address.state() # "Kansas"
Address.city() # "East Casey"
Address.postcode() # "84979"

Faker Internet

Generate internet-related data such as emails, images, links, usernames, and IP addresses

Faker.Internet.email() "elizabeth2056@rolfson.net"
Faker.Internet.image_url() "https://placehold.it/936x936"
Faker.Internet.url() "http://sipes.com"
Faker.Internet.user_name() "elizabeth2056"

Custom test fixtures modules and test fixture functions

When you first create a phoenix context, you are provided with test fixture functions to help you in testing.

For example, when you run:

mix phx.gen.html Posts Post posts title:string content:text

A file will be generated that looks something like this.

Test fixture functions are a great way to create some data for tests local to the same test file.

the post_fixture function creates a post so that you can test the Posts.list_posts() function. Even better, the data is only created for this test and not others. So there are no side effects on other tests causing hard to debug test interaction.

Reusing test fixture functions

This is great, but what if you have another test file that needs the post_fixture?

Rewriting this would be a lot of duplication! A simple way to reuse test fixture functions is to move them to a single test fixture module that you can use in your test file.

Then use the PostsFixture in any files where you want to access the @valid_attrs, @update_attrs, or @invalid_attrs values or the post_fixture function.

For an explanation on use and __using__ check out Elixir Schools’ lesson.

Factories using Ex Machina

If using your own custom test fixtures and Faker data isn’t enough, you may want to consider using the Ex Machina elixir library for a convenient test data infrastructure. Note that Ex Machina uses the term “factory” instead of “fixture”.

• Install Ex Machina: Follow the repo installation instructions for the most up-to-date installation steps.

Ex Machina factory module

Once you have Ex Machina installed, you’ll want to set up a factory module.
the factory module will use the ExMachina module.

defmodule Blog.Factory do
     use ExMachina.Ecto, repo: Blog.Repo
end

Ex Machina factory functions

The factory module contains factory functions which must end in _factory due to how Ex Machina works under the hood.

def post_factory do
    %Post{
      title: Example Title,
      content: "Example Content"
    }
end

Factory functions can also be derived from other factory functions using struct!

# derived factory
  def long_post_factory do
    struct!(
      post_factory(),
      %{
        content: Faker.Lorem.paragraphs(5..10) |> Enum.join("\n")
      }
    )
 end

Sequence

Ex Machina provides the sequence function to increment the string or list you provide every time you call a factory function.

sequence("Name") # Name0
sequence("Name") # Name1

sequence can also be called with an atom and a list to cycle through a list of values.

sequence(:role, ["admin", "member", "superuser"]) # admin
sequence(:role, ["admin", "member", "superuser"]) # member
sequence(:role, ["admin", "member", "superuser"]) # superuser

Insert

In your test or seed files, these factory functions can be used with the insert functions provided by Ex Machina. insert takes an atom as an argument. The atom should match the function name before _factory . For example, post_factory is used with :post.

insert(:post) #  inserts  %Blog.Posts.Post{
                __meta__: #Ecto.Schema.Metadata<:built,"posts">,
                title: "Example Title0",
                content: "Example Content0",
                inserted_at: nil,
                updated_at: nil
              } into the database

Putting it all together

Putting everything you learned about Ex Machina, test fixtures, and Faker together, here’s an example factory and usage in a test for the demo blogging app.

Defining the factory

Notice that sequence can take a list of any value you want. In this case, I’m using Faker and sequence to cycle between long, medium, and short content.

Create a test with the factory

Import the factory into your test file to gain access to the insert function provided by Ex Machina.

Conclusion

You have learned to improve your phoenix data seeding reliability, reusability, and readability by leveraging custom test fixture modules, test fixture functions, Ex Machina, and Faker.

I hope you found this article useful and are excited to apply what you learned to your next or current project.

If you have any questions or comments, please let me know by posting a reply; I’d be happy to get back to you!

Further Reading

• Ex Machina Repo and Ex Machina Docs
• Phoenix Seeding Data Docs
• Faker Docs
• Example Blogging Application using Faker and Ex Machina