Prepare and release your Elixir open source package like a pro

Published in

Fresha Engineering

10 min readJan 13, 2018

At Shedul, we strongly believe in open source and we share back to the community just as much as we build our business upon it. With Elixir in particular, we’re the proud owners of a whole bunch of packages, such as Surgex, Jabbax, Confix, PlugDeviseSession or Protein.

But every time I’m releasing an Elixir open source package, I find myself re-assembling in my head the same list of actions and dependencies between them that lead to a properly managed and (hopefully) professional looking release. This guide aims to put all these actions into approachable, ordered end-to-end list of how to get your open source up and running.

There’s one more important aspect to this. Not knowing how to setup an open source app (or having it poorly organized) results in doing that later in the project development cycle, while it’s profitable for the project right from the start— preferably just after running mix new or mix phx.new.

Benefits

Here are some basic gains that come from the above approach:

thanks to setting up the repo early, an entire project development is documented via Github PRs, including early age development
having PRs with checks for tests, code quality and coverage makes code reviews easier and vastly improves the project quality from the start
it’s all backed by documentation visible both in the code during code reviews and on Hex after every package release
semantic versioning backed by changelog serves as change documentation and update guide with severity indicator (breaking vs non-breaking)
test and coverage badges visible both on GitHub and Hex advertise the project as trustworthy, giving it an edge in open source community

I consider all the things configured in this article as mandatory pieces of a modern open source project configuration puzzle. If any is omitted, you can expect to hit various project quality and/or management problems.

Steps

You can of course start with a mature Elixir or Phoenix application, but let’s assume we’re just starting with a completely new application bootstrapped via mix new just like I recommended above. For such case, you can visit repo-example-elixir to grab the final code and preview diffs for changes that I’ve made during each step.

Step 1: Repository

GitHub is a platform of choice for most of open source projects, which it hosts for free. It has nice management features like pull requests with code reviews and it’s nicely integrated with tons of CI tools. If you don’t have account yet, create one — it’s free for unlimited number of public repositories.

Once logged in, hit + and click New repository. Fill in the Name and Description of your repo. Opt for creating Public repository. Confirm the repository creation without opting in for README etc.

Repo name doesn’t have to match Elixir app/package name. You may prefer to replace underscores with dashes to match your repo naming convention or you may add elixir suffix if your package targets multiple platforms.

Then, in terminal, create new project if you’re starting fresh (remember to replace repo-example-elixir with your GitHub repo name and repo_example with Elixir app/package name) and cd into your project directory:

mix new repo-example-elixir --app repo_example
cd repo-example-elixir

Next, init the Git repo for the project and make the first commit:

git init
git add .
git commit --message "Initial commit"

Finally, push your project to GitHub:

git remote add origin \
  git@github.com:surgeventures/repo-example-elixir.git
git push -u origin master

Refresh the GitHub page and enjoy your new Elixir repository.

Step 2: `Documentation`

ExDoc is a default Elixir solution for generating documentation. It integrates with HexDocs, a platform that hosts docs for all Elixir packages released on Hex. Since docs are considered a mandatory part of package releases, we need to configure ExDoc before we attempt to publish the package.

Let’s start by adding ex_doc package to project deps in mix exs:

defp deps do
  [
    # ...
    {:ex_doc, "~> 0.18", only: :dev, runtime: false}
  ]
end

Now, you can get the package and compile the docs:

mix deps.get
mix docs

You can open doc/index.html in order to preview the docs. Now you should add missing moduledoc and doc clauses and re-compile the docs until entire public interface of your project is properly documented. You can learn more about writing documentation in official Elixir docs.

Besides Elixir docs, you should respect the open source community conventions by filling a minimum set of Markdown files that describe your repository — readme, license and changelog.

First, fill README.md to be a little less generic than what mix new gives:

# Repo example for Elixir**An example open source Elixir application.**## InstallationAdd `repo_example` to your list of dependencies in `mix.exs`:```elixir
def deps do
  [
    {:repo_example, "~> x.y.z"}
  ]
end
```## DocumentationThe documentation can be found on [HexDocs](https://hexdocs.pm/repo_example).

Then, fill LICENSE.md with your license of choice — I’m usually using the MIT license which is a sensible default that won’t constrain the package usage.

Finally, add initial CHANGELOG.md to create a space for documenting changes in versions that should follow semantic versioning:

# Changelog## Unreleased### Added- Basic ExDoc configuration
- Markdown documentation (README, LICENSE, CHANGELOG)

You should add changes as you develop to the Unreleased section and move them to specific version’s chapter just before the release. You should also make a habit of documenting all changes in the changelog as you go, which will make it easy to bump version and release later on. Learn more on structuring and managing changelogs on the Keep a Changelog page.

You can also configure ExDoc so that readme and changelog are included in the package documentation on HexDocs just as well as on GitHub. Add the following docs key to project config in mix.exs:

def project do
  [
    # ...
    docs: [
      main: "readme",
      extras: ["README.md", "CHANGELOG.md"]
    ]
  ]
end

You can run mix docs again to see that readme has now became an entry point for the documentation and that there’s a changelog section available.

Step 3: Tests

ExUnit is the default Elixir solution for running tests. It’s already configured after running mix new and it’s a great, nicely integrated test tool to start with.

Since it’s already configured, you can start by actually running the tests:

mix test

If there are errors or failures, you should fix them now to start green & clean.

This covers running tests locally. But since your package is open source, you have a wide array of CI services that’ll happily run them for you in the cloud and present results on GitHub, completely for free. CircleCI is one of such services and the fact that it’s tightly coupled with Docker makes it particularly easy to configure for Elixir and manage it eg. when Elixir gets updated.

So, add a basic CircleCI config in .circleci/config.yml:

version: 2
jobs:
  build:
    docker:
      - image: circleci/elixir:1.5
    environment:
      - MIX_ENV: test
    working_directory: ~/repo
    steps:
      - checkout
      - run: mix local.hex --force
      - run: mix local.rebar --force
      - run: mix deps.get
      - run: mix compile --warnings-as-errors --force
      - run: mix test

Make sure to push this file to master branch of your repo before proceeding if you want to have your project debut green on the CI.

After logging in to CircleCI and giving it access to your GitHub account, you can navigate to Projects sidebar section and pick your new repository, which should be visible there. Choose the newest 2.0 configuration and click Start building button. This should result in first successful build in Builds section linked to appropriate GitHub commit and, in future, pull request.

The configuration above ensures that your code compiles without warnings before starting tests, which is an added value of Elixir compiler. We may (and will) add more checks later on.

Format check task that’ll come in Elixir 1.6 along with official Elixir formatter is a great check candidate to keep in mind for the future. Be sure to add mix format --check-formatted --dry-run as a CI run task if you’re using formatter.

As a bonus, you can integrate with CircleCI Test Results facility and get a readable test summary directly on build page. This requires outputting the test results in jUnit format. In order to do that, add the following to mix.exs:

defp deps do
  [
    # ...
    {:junit_formatter, "~> 2.1", only: :test}
  ]
end

Next, add the following on top of test/test_helper.exs:

File.mkdir_p(Path.dirname(JUnitFormatter.get_report_file_path()))
ExUnit.configure(formatters: [JUnitFormatter, ExUnit.CLIFormatter])

You also need to add the following to config/config.exs:

if Mix.env == :test do
  config :junit_formatter, 
    report_dir: "/tmp/repo-example-test-results/exunit"
end

Finally, add a new run step at the end of .circleci/config.yml:

version: 2
jobs:
  build:
    # ...
    steps:
      # ...
      - store_test_results:
          path: /tmp/repo-example-test-results

For every build on CircleCI, you’ll now get a Test Summary section with total number of tests as well as details about failed and slowest test cases. You’ll also find cross-build stats about failing and slow cases in the Insights section.

Step 4: Code quality

Credo is one of most popular and modern tools for linting the Elixir code, which means it ensures a set of preconfigured coding rules and conventions is actually being followed by developer(s).

Start by adding credo package to project deps in mix exs:

defp deps do
  [
    # ...
    {:credo, "~> 0.8", only: [:dev, :test]}
  ]
end

Now, you can get the package and do initial linting with default rules:

mix deps.get
mix credo

You can customise what Credo expects from your code by adding .credo.exs:

%{
  configs: [
    %{
      name: "default",
      files: %{
        included: ~w{config lib test}
      },
      strict: true,
      color: true,
      checks: [
        {Credo.Check.Readability.MaxLineLength, max_length: 100}
      ]
    }
  ]
}

Finally, you can enforce the code quality on CI by adding an additional run step to .circleci/config.yml:

version: 2
jobs:
  build:
    # ...
    steps:
      # ...
      - run: mix credo

You could also use CodeClimate to track quality issues separately from other CI checks, but I’d recommend against it. By plugging it to CI, you expect your linter to always be green. This enforces keeping rule config up to date with actual code conventions and making exceptions via code comments instead of cloud service, which means they’re easier to track and self-documented.

Step 5: Coverage

ExCoveralls is the most popular tool for reporting ExUnit test coverage statistics as console output, HTML report or JSON report.

Start by adding excoveralls package to project deps and patching project config in mix exs to use the new formatter:

def project do
  [
    # ...
    test_coverage: [tool: ExCoveralls],
    preferred_cli_env: [
      "coveralls": :test, 
      "coveralls.detail": :test, 
      "coveralls.post": :test, 
      "coveralls.html": :test
    ]
  ]
enddefp deps do
  [
    # ...
    {:excoveralls, "~> 0.8", only: :test}
  ]
end

Now, you can get the package and check your test coverage:

mix deps.get
mix coveralls

That’s it for doing this locally. You’ll also want to have your coverage covered by a cloud service, the same way CircleCI covers test runs. Codecov is a modern tool that does that and guess what — it’s free for open source. It also comes with baked-in support for ExCoveralls.

Once you’re logged in to Codecov, click the Add a repository button and pick the proper repo. Since it’s open source, you don’t even need an API key — all you have to do is add two more run steps at the end of .circleci/config.yml:

version: 2
jobs:
  build:
    # ...
    steps:
      # ...
      - run: mix coveralls.json
      - run: bash <(curl -s https://codecov.io/bash)

Your next push will generate a report for specific branch and your upcoming pull requests will come integrated with coverage checks.

Step 6: Package

Once you feel that your application is ready for release on Hex, it’s time to fill in some final metadata about it so that everything is ready for publication.

Here’s how the relevant part of your project config in mix.exs may look like:

@github_url "https://github.com/surgeventures/repo-example-elixir"def project do
  [
    # ...
    version: "1.0.0",
    name: "RepoExample",
    description: "An example open source Elixir application.",
    source_url: @github_url,
    homepage_url: @github_url,
    files: ~w(mix.exs lib LICENSE.md README.md CHANGELOG.md),
    package: [
      maintainers: ["Karol Słuszniak"],
      licenses: ["MIT"],
      links: %{
        "GitHub" => @github_url,
      }
    ]
  ]
end

If you copy and paste the above lines at the bottom of default project config, ensure not to duplicate the version key which is already present there.

Everything is setup in order to publish to Hex, but that doesn’t mean you’re ready. Before publishing, you should always do the following:

ensure that you’ve actually bumped the project version in mix.exs
ensure that a section for new version is filled in CHANGELOG.md
ensure that ExDoc documentation compiles and actually looks fine
ensure that you’ve already pushed the code with above tweaks to master
ensure that all checks on master are green

Once that’s done, you can publish or update the package with confidence:

mix hex.publish

Once it finishes, you’ll find your release on Hex and HexDocs.

Step 7: Badges

Since you have all those amazing services backing you up, you can rely on them to advertise your project as working, well-tested and properly licensed. Yes, I’m talking about the badges. Almost every CI service offers them but they’re often visually inconsistent. That’s why we’ll unify them via Shields.io.

Let’s add some badges just under the readme title:

# Repo example for Elixir[![Hex version badge](https://img.shields.io/hexpm/v/repo_example.svg)](https://hex.pm/packages/repo_example)
[![License badge](https://img.shields.io/hexpm/l/repo_example.svg)](https://github.com/surgeventures/repo-example-elixir/blob/master/LICENSE.md)
[![Build status badge](https://img.shields.io/circleci/project/github/surgeventures/repo-example-elixir/master.svg)](https://circleci.com/gh/surgeventures/repo-example-elixir/tree/master)
[![Code coverage badge](https://img.shields.io/codecov/c/github/surgeventures/repo-example-elixir/master.svg)](https://codecov.io/gh/surgeventures/repo-example-elixir/branch/master)

It’s quite an intensive snippet since all badges can only be separated by a single newline in order to land next to each other and each one is a combination of markdown link and image. Here’s what’s in it:

Hex version badge that links to Hex package and docs
License badge that links to LICENSE.md in repo
Build status badge that links to CircleCI builds for master branch
Code coverage badge that links to Codecov reports for master branch

Summary

That’s it! Now your package can be used by the whole world of developers as a dep in their mix.exs files. Moreover, it’s well-covered by a full fleet of modern online services including repository manager, package manager, documentation browser, test builder and code coverage browser. The code is also checked for compilation warnings and lint issues.

The whole community can easily access all this data, making them eager both to use the package and perhaps even to contribute to its further development.

It’s a great time to join the growing Elixir community and develop some useful packages in this highly productive language, while being supported by a wealth of tools that further boost the productivity.

Happy open sourcing!