Counter Caching in Phoenix

If you’ve found this article, it’s likely that you’re a Rails developer whom has at least decided to try Phoenix. Hooray! Welcome, you won’t regret it.

Now don’t get me wrong, Rails is great. If it wasn’t, it wouldn’t be where it is today and neither would my career. Are there things that could be improved on, however? Sure. Elixir and Phoenix have the luxury of being new and doing things right from day one.

That being said, Elixir and Phoenix are most definitely my new default over Ruby and Rails. For the reasons you’re most likely interested in as well. Immutability, scalability, concurrency, raw speed etc. Phoenix is heavily inspired by Rails which makes picking it up fairly easily, but there are some things (understandably) that aren’t just there. Like counter caching associations.

Ecto

Ecto is the go to database interaction library in Elixir and is included and setup in Phoenix by default. You should know already that inserting and updating records in Ecto revolves around changeset’s. If you aren’t already familiar with Ecto changeset’s, I encourage you to foray off and learn about them before reading on.

Another thing I really love about Phoenix is that the database layer, among others, is completely de-coupled. That being said, this article would be much more accurately entitled “Counter Caching in Ecto”, but I digress.

The Rails way

Rails, or more specifically, ActiveRecord, provides a counter_cache option when defining associations.

class Post < ApplicationRecord
  has_many :comments
end
class Comment < ApplicationRecord
  belongs_to :post, counter_cache: true
end

In this example you will also need to create an integer field named comments_count on the Post model. Once you’ve done that, Rails will magically return the value held therein instead of running a COUNT(*) query whenever you ask for the value of @post.comments.size

You probably know this already, and is not why you’re here.. so let’s move on to Phoenix…

The Phoenix (Ecto) way

Although Phoenix does take a lot of inspiration from Rails, there is a culture of much less magic. Therefore, there’s no such counter_cache option to just switch on in your models. What you’ll need to do is leverage the power of changeset’s and increment association counts yourself.

At face value, that sure sounds like a bad deal, right? However, less magic means more speed and reliability. Plus, there’s no objects in Elixir which means automatic incrementing of a model field in the background just isn’t possible.

So, thinking about the problem functionally, whenever a Comment is created we will want to increment the value of comment_count on the Post model (note the singular “comment_count”, this is Phoenix convention as opposed to plurals in Rails with the exception of database table names).

Here’s one way we might set out to accomplish that.

defmodule CounterCaching.Comment do
  use CounterCaching.Web, :model
  
  schema "comments" do
    field :title, :string
    field :body, :string
    field :comment_count, :integer
    
    belongs_to :post, CounterCaching.Post
    
    timestamps
  end
  
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:title, :body, :post_id])
    |> validate_required([:title, :body, :post_id])
    |> increment_post_comment_count
  end
  
  defp increment_post_comment_count(changeset) do
    CounterCaching.Post
    |> where([p], p.id == ^changeset.changes.post_id)
    |> Repo.update_all(inc: [comment_count: 1])
    
    changeset
  end
end

IMPORTANT: You should never cast a foreign key in a changeset. This should be handled in your controller. For brevity, I’ve broken that rule here.

What’s happening here? Well, as with any other Phoenix model, we have a changeset that casts all of the required fields from the supplied params. We then validate our required fields are present in the changeset and lastly we pipe the result in to increment_post_count/1

This function then looks up the Post which our new Comment belongs to and increments its comment_count.

Perfect.. right?

Alarm bells!

But wait! Something is wrong here. Just because we’ve created a changeset, that doesn’t mean any transactions have hit the database. What if this changeset is invalid? We’ll have already updated the comments count for the Post in question even though the new comment is yet to be persisted. Chaos reigns!

Ecto to the rescue

Ecto provides just the mechanism we need to fix this issue. To ensure the comment_count field on the respective Post is incremented only when the Comment is persisted to the database, we need to use Ecto.Changeset.prepare_changes/2

From the documentation

Provides a function to run before emitting changes to the repository.

Such function receives the changeset and must return a changeset, allowing developers to do final adjustments to the changeset or to issue data consistency commands.

The given function is guaranteed to run inside the same transaction as the changeset operation for databases that do support transactions.

Sounds pretty nifty! Let’s see how we might apply it

defmodule CounterCaching.Comment do
  use CounterCaching.Web, :model
  
  schema "comments" do
    field :title, :string
    field :body, :string
    field :comment_count, :integer
    
    belongs_to :post, CounterCaching.Post
    
    timestamps
  end
  
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:title, :body, :post_id])
    |> validate_required([:title, :body, :post_id])
    |> prepare_changes(fn (changeset) ->
        assoc(changeset.data, :post)
        |> Repo.update_all(inc: [comment_count: 1])
        
        changeset
      end)
  end
end

prepare_changes takes a changeset and an anonymous function. It is this anonymous function that Ecto executes during the database transaction. It must also return the changeset.

Awesome! Now the Post which belongs to our new Comment will only have its comment_count field incremented when the Comment is persisted to the database.

Lastly, given Ecto runs our update in the same transaction as inserting the Comment, that means if anything goes wrong our update will also be rolled back (assuming your database supports it). Brilliant!

What about many to many associations?

The examples listed above will only function correctly for one to many associations. I’m currently working on a separate article to address counter caching in many to many associations. It will involve a few more Ecto topics which I felt would make this article too verbose. If you’re reading this, it’s not out yet!

Conclusion

We’ve learned how to leverage counter caching in the fashion which Ecto is designed to handle, via Ecto.Changeset.prepare_changes/2. We also learned never to cast foreign keys, just in case you didn’t know already!