Easy Logging with Timber.io & Manifold for Phoenix & Elixir
No matter what language you program in, what framework you use, or what problem your application is trying to solve, observability is critical. Without having a window into your application’s behaviour, diagnosing problems or understanding usage patterns is impossible.
This post will show you how to set up the bedrock of observability, logging, for a Phoenix API.
The Tech Snack
If you’re hungry for resiliency and soft real-time performance, Phoenix, Elixir, and Erlang layer together to form the smørrebrød that will satisfy your cravings.
Phoenix is the spicy chopped radish garnish. It sits above the other layers, providing a Rails-like model-view-controller framework for application development.
Elixir is the smoked eel; the taste and texture that you notice the most. It provides a programming language like Ruby, but taking advantage of the Erlang runtime.
Erlang is the rye bread. You don’t notice it most of the time. It gives structure and support to your application through its runtime and virtual machine.
To get started with Phoenix, you can follow their excellent installation and up and running guides. For this post, we’ll look at an existing application, introduced a bit later.
Why log and why use Timber.io?
Understanding what your application is doing is important for diagnosing problems, or understanding usage patterns. Logging is the easiest way to gain some of this understanding.
Timber.io collects all your logs into a single dashboard, stores historical logs, and provides a rich search interface. This way, you can diagnose issues after the fact, across many systems, and not have to worry about running ssh.
Using Timber.io on Manifold is the simplest way to get started for free, and keep all your services and configuration in one place.
Learn how to get started with Manifold here.
Introducing KrabbyTime
KrabbyTime is SpongeBob SquarePants Time Cards as a Service (SBSPTCaaS). This API lets you request the appropriate time card for a given number of seconds. Perfect for including in slack messages after test runs!
You can find the source code for KrabbyTime on GitHub:
KrabbyTime provides a simple API to make it suited for using in Slack, GitHub, or other places where you might want to paste in a link. You request a url with the number of seconds you want a time card for. KrabbyTime responds with an HTTP 302. The response’s Location header redirects the request to thumbnail image for the appropriate time duration.
KrabbyTime requires no persistent storage or fancy HTML interface. It’s code was generated without those, using:
mix phx.new — no-ecto — no-brunch krabbytimeLet’s Get Observable!
KrabbyTime is ready to delight users. Sadly, we won’t be able to figure out what users are doing. Not unless we connect to the production machine and tail the log files there. Let’s get Timber.io configured so we can view the logs from the comfort of our browser.
The easiest way to get started with a free Timber.io account is through Manifold. Sign up for Manifold, then create your free Timber.io resource.
Once we have a Timber.io resource provisioned, we can add it to KrabbyTime with Timber.io’s automatic installation:
In mix.exs:
def application do
[applications: [:timber]]
enddef deps do
[{:timber, “~> 2.5”}]
End
In our shell:
mix deps.get
mix timber.install $TIMBER_API_KEYFor our Timber.io configuration, we’re going to set the api_key in config/prod.secret.exs. This file isn’t committed to source control.
You can see the changes needed to add Timber.io logging to KrabbyTime here.
Production Grade Time Cards
Containers, stateless services, and immutable infrastructure are near and dear to our hearts at Manifold (check out Jelmer’s post on our Kubernetes infrastructure), and a great way to enable high uptime. But, while the rest of us were scheduling weekend long outages every quarter to deploy new code, Erlang developers were performing seamless upgrades through hot code reloading. KrabbyTime, or any Phoenix application, can run just fine under a containerized model, but we’re going to experience the raw power of Erlang to deploy it.
To be able to deploy hot upgrades for KrabbyTime, we’ll use distillery. A lot of Elixir projects have drink themed names. You don’t need to know that for this post, but it might help impress your friends at the next fullstack meetup you attend.
KrabbyTime is already configured to use distillery. You can view the required change for it here. Note that at the time of writing this post, a release candidate version of distillery was needed to work with Elixir. Usually you don’t want to depend on a -rc release!
To perform our initial deploy, we need to compile KrabbyTime. Then we must copy our build artifacts to our production system, and start it. Note that you’ll need to set PRODUCTION_MACHINE to the host name of your production machine here (and actually have a production system to deploy to 😉):
MIX_ENV=prod mix release
scp _build/prod/rel/krabbytime/releases/0.0.1/krabbytime.tar.gz \
$PRODUCTION_MACHINE:/krabbytime
ssh $PRODUCTION_MACHINE
cd /krabbytime
tar zxv krabbytime.tar.gz
PORT=8080 bin/krabbytime startRemember to compile for the right architecture!
Now KrabbyTime is running, and we can view the logs in Timber.io after SSOing from the Manifold dashboard.
Hot Upgrades For More Observability
Now that KrabbyTime has been running for a while, and is poised to win several Webbys, we should add some logging to actually tell us what time cards users are getting, so we can prioritize which ones to add next.
You can view the change to add a new log entry here.
Since KrabbyTime is so popular, we don’t want to cause any downtime to roll out this change. Fortunately, we’re prepared to perform hot upgrades with distillery.
To perform our upgrade, first we need to update our version in mix.exs. Then we can build our artifact using distillery:
MIX_ENV=prod mix release — upgradeThis creates a tarball that we can copy to our production server as before. Then we’ll need to copy the tarball into distillery’s upgrade directory structure. Finally, we can apply the hot upgrade:
scp _build/prod/rel/krabbytime/releases/0.0.2/krabbytime.tar.gz \
$PRODUCTION_MACHINE:/krabbytime
ssh $PRODUCTION_MACHINE
cd /krabbytime
mkdir releases/0.0.2
mv krabbytime.tar.gz releases/0.0.2/
bin/krabbytime upgrade 0.0.2Success! We’ve upgraded KrabbyTime in place, without any downtime.
Wrap-up
In this post we learned how to add rich logging information and aggregation to a Phoenix application with Timber.io and Manifold. Then, we added extra logging entries, and hot upgraded our application using distillery.
Where will you take KrabbyTime from here? Besides adding all 150 time cards, there’s an ocean of possibilities! You could add user preferences stored in JawsDB PostgreSQL using Ecto. Or you could deliver time cards via email with Mailgun. Or you could sit back and watch the Krabby Patties roll in.
Try Timber and use the code TIMBER to receive $25 in Manifold credit.
