Five years ago, I made a gem N1Loader that helps prevent N+1 issues by providing a simple DSL for handling loading in batches.

It is trivial, works perfectly fine as a standalone, or with ActiveRecord. It even has an integration with ArLazyPreload to eliminate any need for manual preloading whatsoever. Although I don’t want to praise it anymore cause I already did it a couple of times.

Not so long ago, I received a message from an ex-colleague that they had to do a monkey-patch to fix a thread-safety issue with N1Loader. My first reaction was immediate curiosity. First, I was surprised they used the gem at all. Yes, I know it’s good, but it still makes my imposter syndrome slightly shake and fight back. But then came the real thing: a use case where the code doesn’t work. Gladly, in a good, old-fashioned way by throwing errors.

The case was related to GraphQL implemented with WebSockets, so the concurrency was happening there. But you don’t need such a specific case to meet the problem.

For a while, ActiveRecord has asynchronous loading functionality, such as:

@users = User.where(active: true).load_async

The code above uses threads underthehood, and here is where I made a mistake.

You see, the original implementation has nothing more than a simple hash that is being filled with objects (all at once), then distributed across relative objects. First of all, Hash is not a thread-safe structure in Ruby. Moreover, the loading function wasn’t good either.

def loaded
  return @loaded if @loaded

  check_arguments!

  @loaded = {}
  if respond_to?(:single) && elements.size == 1
    fulfill(elements.first, single(elements.first))
  elsif elements.any?
    perform(elements)
  end

  @loaded
end

Sharp eyes can notice that the function has one more issue, apart from missing thread safety. In case you find more, please let me know or contribute!

About thread-safety, threads can simultaneously start loading, and at least two things may happen:

• They did full multiple loading, resulting in extensive resource usage (no bug, but no performance gain).
• One set @loaded variable, so others got it with no values yet set inside.

The second case is what raised the team’s concerns, because with an empty loaded object, gem raises an error which says “Something went wrong” as preloading wasn’t executed at all. (Don’t mistreat a loaded nil object vs nothing).

There are not many options to fix it, but gladly, we don’t need many. Mutex is a known mechanism to allow concurrent code to be executed in a single thread at a time. However, in its simple form, it may cause a deadlock. Therefore, we need a so-called re-entrant locking mechanism, which is implemented by the core Ruby module: Monitor and MonitorMixin.

So the new thread-safe implementation would be the following:

def loaded
  return @loaded if @already_loaded

  # comes from MonitorMixin
  synchronize do
    non_thread_safe_loaded unless @already_loaded
  end

  @loaded
end

def non_thread_safe_loaded
  return @loaded if @already_loaded

  check_arguments!

  @loaded = {}
  if respond_to?(:single) && elements.size == 1
    fulfill(elements.first, single(elements.first))
  elsif elements.any?
    perform(elements)
  end

  @already_loaded = true
  @loaded
end

Note that we check for @already_loaded two times. The first time, we do it before init locking mechanism in case the data was already loaded. This way, we save some redundant costs. The second time, in case more than one thread starts actual loading, and other threads are waiting, we won’t load data more than once as soon as the lock is freed.

Notice also that we have now introduced a @already_loaded variable. It helps us solve the second problem I briefly mentioned: idempotency. Before, if the code during execution perform fails, the @loaded would be set to an empty hash regardless, so no more errors would be thrown on subsequent calls.

By the way, I was surprised to notice that the actual CPU-wise cost of having Monitor within this code is close to zero. Nice!

Are you thinking about thread-safety when working on your projects?

In case you need me:

• Medium
• LinkedIn
• Twitter
• GitHub

GitHub’s codeowners feature helps with better control over changes for big teams collaborating on the same projects. And with the recent growth in popularity of monolith and monorepo architectures, the use of the codeowners feature has also increased.

Not a long time ago, I wrote a tool and an article about database schema ownership in the Ruby on Rails world. Now, I want to discuss something much broader and language-agnostic.

Let’s start from the beginning.

GitHub’s CODEOWNERS feature allows you to mark files of your repository to be owned by a specific group of engineers; therefore, anytime a new PR is opened that changes the file, that group will be asked for approval.

Imagine having tens of engineers working closely on a single repository, each making numerous changes simultaneously. The feature helps split ownership in a way that only the most experienced with a particular piece of code needs to review the proposed changes.

This approach has many benefits, including, but not limited to:

• You get better feedback from the most experienced reviewer
• You don’t ping everybody, so creating fewer distractions for them
• The development cycle is generally shorter, as everybody involved knows the domain

Of course, there are also disadvantages to overusing this feature, such as poor domain knowledge sharing within the project, which can be mitigated with different practices, for example, ownership rotation.

Here, let’s focus on another problem that emerges from the feature limitation: you can only assign a whole file. Check an example below.

# Whole docs folder is owned by @doc-team
docs/ @doc-team

# File with some logic written in Ruby is owned by @core-team
app/converter.rb @core-team

# This is not possible to assign lines from 80-120 to a @support-team
app/converter.rb:80-120 @support-team

This makes sense, as otherwise, if it were on a per-line basis, you would need to update the ownership almost every time you touch the file. I doubt that would be convenient.

Now, the problem that you can guess or have already is the following:

The more collaboration you have between teams (domains), the more files have multiple owners.

# File with some logic written in Ruby is owned by @core-team, @support-team, and @testing-team
app/converter.rb @core-team @support-team @testing-team

Why would that be a problem?

Well, because in some cases, we come back to the point where we started: many engineers are notified about the change requests they shouldn’t be notified about.

This happens all the time. Let’s check out the example.

1. The core team has created a Ruby file converter.rb that does some business logic.
2. Then, the support team has decided to log specific steps of that logic into an internal logging system, which they own.
3. Later, the testing team wrapped a minor function in A/B testing for their feature based on the file.

Now, with the straightforward usage of the feature, anytime any team makes a change, regardless of how small, it will require the approval of all teams.

However, most of the time, only a specific function of the file is changed due to the way teams collaborate:

• The core team will change its business logic.
• The support team might consider adding or removing logging steps.
• And so, the testing team will either finish or adjust the A/B testing.

Now, I hope you clearly see the problem I am trying to highlight here, and you follow the solution I implicitly exposed in the previous paragraph.

There is nothing innovative or new in the principle I would like to share; however, I see many still have issues with that. So, here it is.

Don’t own the code, own the functions.

Sounds simple, but what does it mean, considering the example I shared above?

So the goal is not to assign converter.rb a file to a particular team, though we still can to some extent, but to adequately write test files that we assign to specific teams.

# Let's say a file is generally owned by @core-team
app/converter.rb @core-team

# Business logic related tests owned by @core-team
tests/converter_tests.rb @core-team

# Tests cover that expected logging steps are triggered
tests/converter_logging_tests.rb @support-team

# Tests cover A/B testing
tests/converter_ab_testing_tests.rb @testing-team

The naming pattern (splitting a single test file into multiple) can vary. It can be anything as long as everybody follows it, so it’s easy to navigate within the repository.

Now, with the functions above, only the core team will be notified of any change to converter.rb the file. Other teams will be asked for approval only if their functions were adjusted.

Yes, the approach can be riskier if tests are poor; however, with good test coverage and overall code quality, it provides better flexibility for working within large teams.

Questions to you:

• What approach do you apply?
• Would you try a different one?

Consider subscribing to me!
And don’t forget to share your thoughts on the topic.

• Medium
• LinkedIn
• Twitter
• GitHub

The donation supports me in continuing my open-source work and blogging:

• GitHub Sponsor
• Open Collective

It’s been a while since my last article. Now, feeling fresh and energetic, I want to highlight a problem that has been bothering me for quite some time. As a bonus, I have created a solution right away that you can leverage in your projects.

I have over a decade of experience working in big companies with hundreds of contributors to the same projects daily. Apart from the many consequences of such workflow, there is one that I want to discuss today.

Let’s say we have Project X with a database attached. We store information about our users in this database. From a business perspective, team A owns the code about users, while other teams have their own domains. Now, assume another team needs to extend users’ data. Usually, the team needs to contact team A and ask for their opinion on the subject because they own the user’s domain. Unfortunately, engineers can sometimes overlook such changes because of a lack of automation.

There is a nice feature from GitHub called GitHub Codeowners to manage the collaborative work of big teams. It basically helps you automatically assign particular teams to be owners of specific files in your project, so every time a file is changed, the team will be pinned to review the pull request.

Unfortunately, the same cannot be done on a per-line basis; therefore, in worlds like Ruby on Rails, where a single file represents the whole database schema (schema.rb or structure.sql), we cannot use such a feature.

But what if we split the file into multiple files per table and assign responsive teams to those files? This way, any change applied to a schema would be appropriately reflected, and the right team would be asked about leaving a review for a pull request.

This is where DatabaseSchemaOwnership comes in handy. So far, it only works with ActiveRecord, but it can be extended easily. Check the repo for more dive-in examples, and I hope you will find it helpful.

Please consider subscribing or adding me in social networks!

And don’t forget to share what you think about the topic. Just write me a message. I would be happy to discuss the topic or anything else with you!

• Medium
• LinkedIn
• Twitter
• GitHub

P.S. It isn’t the first unexpected issue I encountered during the Rails upgrade. I recommend you look at these stories:

• Bug with refinement
• Time vs. DateTime difference
• Non-trivial upgrade from Classic to Zeitwerk loader

In development and test environments, it’s a typical pattern to serve static files by Rails. In my case, with Rails 5.2, I could easily access all the images through the server. However, with Rails 6.1, I noticed that some of the files can’t be found.

I couldn’t understand why this behavior had changed. I would be okay if all of them were missing, but this wasn’t the case. Let’s get into specifics.

We have three icons served (I narrowed down the number for you to describe the problem):

• something.svg
• something-supercool.svg
• anything-supercool.svg

All of these are served fine with Rails 5.2. However, one of them is failing with Rails 6.1. Would you like to guess which one?

And the answer is: something-supercool.svg

Now, you might ask yourself, why is that? I was surprised and confused. Although I found a quick workaround by playing with names to make it work, the intense feeling of solving the puzzle was tempting.

Unfortunately, the stack trace of the error was very poor.

ActionController::RoutingError (No route matches [GET] "/something-supercool.svg") 

After a quick lookup, I saw this error is coming from ActionDispatch::DebugExceptions middleware.

def call(env)
  request = ActionDispatch::Request.new env
  _, headers, body = response = @app.call(env)

  if headers["X-Cascade"] == "pass"
    body.close if body.respond_to?(:close)
    raise ActionController::RoutingError, "No route matches [#{env['REQUEST_METHOD']}] #{env['PATH_INFO'].inspect}"
  end

  response
rescue Exception => exception
  # ...
end

So now we need to find out when we set headers["X-Cascade"] to "pass". After several debug points and a little clue where to look, we come to Sprockets.

# Returns a 404 Not Found response tuple
def not_found_response(env)
  if head_request?(env)
    [ 404, { "content-type" => "text/plain", "content-length" => "0", "x-cascade" => "pass" }, [] ]
  else
    [ 404, { "content-type" => "text/plain", "content-length" => "9", "x-cascade" => "pass" }, [ "Not found" ] ]
  end
end

Indeed, Sprockets can’t find the icon. Let’s add a few more lines from the middleware’s call method for the whole picture.

case status
# ...
when :not_found
  logger.info "#{msg} 404 Not Found (#{time_elapsed.call}ms)"
  not_found_response(env)

The status is defined here:

if asset.nil?
  status = :not_found
elsif fingerprint && asset.etag != fingerprint
  status = :not_found
# ...

And this is how we load the asset variable.

path = full_path # In our case "something-supercool.svg" 

# Strip fingerprint
if fingerprint = path_fingerprint(path)
  path = path.sub("-#{fingerprint}", '')
end

# ...

# Look up the asset.
asset = find_asset(path)

# Fallback to looking up the asset with the full path.
# This will make assets that are hashed with webpack or
# other js bundlers work consistently between production
# and development pipelines.
if asset.nil? && (asset = find_asset(full_path))
  if_match = asset.etag if fingerprint
  fingerprint = asset.etag
end

You may already find that we modify the path to eliminate fingerprint.

def path_fingerprint(path)
  path[/-([0-9a-zA-Z]{7,128})\.[^.]+\z/, 1]
end

Looking at this Regexp, we can say that something-supercool.svg path becomes just something.svg.

Okay, we understand this now, but …

Why would anything-supercool.svg work if we cut the path there too?

Let’s look through the flow together.

1. change the path from anything-supercool.svg to anything.svg
2. try to load asset and it fails as there is no such file
3. try to load asset with full original path and it succeeds

In the case of something-supercool.svg, the flow is different.

1. change the path to something.svg
2. try to load asset and it succeeds as there is a file something.svg (another one)
3. define status as :not_found because asset.etag is not equal to extracted fingerprint

Now, we have a complete picture of what is happening and why.

Last question remains, what has changed since Rails 5.2?

In Rails 5.2, Sprocket had a different Regexp to extract the fingerprint.

def path_fingerprint(path)
  path[/-([0-9a-f]{7,128})\.[^.]+\z/, 1]
end

And this breaking change was introduced by David Heinemeier Hansson (DHH) with this PR to ensure compatibility with esbuilds’ base32 digests.

Unfortunately, I couldn’t find any mentions of this breaking change in the Changelog, although I see that some people had the same experience.

• https://github.com/rails/rails/issues/9788
• https://github.com/rails/sprockets/issues/749

Conclusion

This whole story reminded me of my recent conclusion during migration from Classic to Zeitwerk loader.

Ruby on Rails is a world of convention over configuration, and apparently, prefers underscores in file names.

Thank you for your time!

Please consider subscribing or adding me in social networks!

And don’t forget to share what you think about the topic. Just write me a message. I would be happy to discuss the topic or anything else with you!

• Medium
• LinkedIn
• Twitter
• GitHub

I recently did. Surprisingly, it happened during my Rails upgrade for a giant monolith.

My first impression was confusion.

I have never seen Ruby bugs before. But Ruby is another program that great engineers built. Nevertheless, it still could have some minor issues for edge cases.

So it happened to me, and I would like to share this short story on how I found it.

P.S. If you are considering upgrading the Rails version of your application, I encourage you to read my other short article with valuable advice that could help you avoid many issues I had.

In large code bases, times when you might want to change something’s behavior, are generally speaking higher. But in some rare cases, you might even want to monkey patch something only for specific contexts rather than globally. Having these strict requirements in mind, you may recall one of the least used features of Ruby: Refinements.

We used refinements to monkey-patch Kernel#puts method for specific lexical contexts where the function has slightly different behavior.

P.S. Let’s take this case as it is.

In terms of the code, the below is a minimal example.

module Refinement
  refine Kernel do
    def puts(*args)
      # We change the behavior
      super(2)
    end
  end
end

module Environment
  using Refinement

  puts(1) # => 2
end

The code above works as expected with Rails 5.2.
However, it stops working with Rails 6.1.

How is this possible? How could Rails break Ruby functionality?

It turns out the bug is already there; Rails just exposes it. So the difference is that Rails 6 patches Kernel with two classes via prepend.

 Kernel.ancestors
=> [ActiveSupport::ForkTracker::CoreExtPrivate, ActiveSupport::ForkTracker::CoreExt, Kernel]

And this is the reason that causes this bug to be exposed. Here you have a minimal reproducible example.

module Prepender
end

module Refinement
  refine Kernel do
    def puts(*args)
      super(2)
    end
  end
end

# The line below affects the behavior of Refinement
#
Kernel.prepend(Prepender)
#
# If it is commented out, then we have an expected output of 2
# Otherwise, refinement is ignored, and output is 1

module Environment
  using Refinement

  puts(1)
end

The bug has been found in Ruby 2.7.7. However, in Ruby 3, it seems it was fixed. I didn’t try to find a release note, but I feel it was a known case.

Please consider subscribing or adding me in social networks!

And don’t forget to share what you think about the topic. Just write me a message. I would be happy to discuss the topic or anything else with you!

• Medium
• LinkedIn
• Twitter
• GitHub

I’m working on a Rails upgrade for a giant monolith application. It’s been a while already, and while I’m encountering many issues trying to jump from 5.2+ right to 6.1, I want to share some of the stories.

You may know that Rails 6 has introduced a new application loader: Zeitwerk. There are some guidelines and incredibly well-put documentation on the gem itself, so you shouldn’t be lost. But yet, for enormous projects with hundreds of engineers working together, there is something that you won’t find obvious.

I have prepared for you a minimal example to outline the problem. So, without further ado, let’s get started.

Imagine your project has two modules/domains/packages or whatever you call it that two teams/engineers work on simultaneously. In our example, their names will be Discord and Skype. And let’s say that every package has its own class to deal with API.

module Discord
  class Api
    # code here
  end
end

module Skype
  class API
    # code here
  end
end

You may notice that the Discord package has Api class while Skype has API. If you’re suspicious, that’s good; however, please consider that everything is alright so far for Rails 5.2 with classic loader. And because there is no automation to prevent such naming from happening (for pure Rails applications), teams may want to have their names even though they are inconsistent with others.

Now, you decided to upgrade your application (massive monolith, I remind you) to Rails 6.1. You are even brave enough to turn on Zeitwerk immediately because you know that classic loader will be killed anyway, so why not deal with this problem right now?

After resolving all dependency conflicts, you try to start the project console.

You see that Discord::Api can be resolved fine, but Skype::API is failing.

Discord::Api
=> Discord::Api

Skype::API
Traceback (most recent call last):
        1: from (irb):2
NameError (uninitialized constant Skype::API)
Did you mean?  Skype::Api
               Api

After looking for a few documents, which I mentioned above, and even chatting on Twitter with the Zeitwerk author, you realize it’s a good tool, and you can override an inflector to map different packages correctly.

You may end up with code like that:

# config/initializers/zeitwerk.rb
class CustomInflector < Zeitwerk::Inflector
  CUSTOM = {
    'discord/api.rb' => 'Api',
    'skype/api.rb' => 'API'
  }

  def camelize(basename, abspath)
    custom = CUSTOM.find { |(custom, _)| abspath.ends_with?(custom) }&.last

    custom || super
  end
end

Rails.autoloaders.each do |loader|
  loader.inflector = CustomInflector.new
end

This code allows us to change the inflection for file names based on their absolute path. In our example, discord/api.rb expects to have an Api constant, while slack/api.rb expects to define an API constant.

P.S. We don’t need a custom mapping for Discord, but I added it for visibility.

Let’s try it out in the console now.

Discord::Api
=> Discord::Api

Skype::API
=> Skype::API

Both constants are accessible without any issues.

Now, you only need to handle every failing case for your giant application by adding a custom mapping based on its absolute path. Unfortunately, that’s not enough; there is a surprise waiting.

One of your failing cases was a controller like the one below.

module API
  class BasesController < ApplicationController
    def show
      head :ok
    end
  end
end

And you know how to fix it already, right? So you added a custom mapping too:

class CustomInflector < Zeitwerk::Inflector
  CUSTOM = {
    'discord/api.rb' => 'Api',
    'skype/api.rb' => 'API',
    'controllers/api' => 'API', # controllers under API scope
  }

  def camelize(basename, abspath)
    custom = CUSTOM.find { |(custom, _)| abspath.ends_with?(custom) }&.last

    custom || super
  end
end

Rails.autoloaders.each do |loader|
  loader.inflector = CustomInflector.new
end

And the constant is accessible fine in the console:

API::BasesController
=> API::BasesController

But here is a twist, it won’t work still!

As soon as you would try to reach that controller action via any of its routes, it will fail with:

uninitialized constant Api Did you mean? API

This happens due to the following piece of code from ActionPack (responsible for routing/controllers).

# gems/actionpack-6.1.7.3/lib/action_dispatch/http/request.rb
def controller_class_for(name)
  if name
    controller_param = name.underscore
    const_name = controller_param.camelize << "Controller"
    begin
      ActiveSupport::Dependencies.constantize(const_name)
    rescue NameError => error
      if error.missing_name == const_name || const_name.start_with?("#{error.missing_name}::")
        raise MissingController.new(error.message, error.name)
      else
        raise
      end
    end
  else
    PASS_NOT_FOUND
  end
end

So, when a router tries to find a corresponding controller for a path, it uses the camelize method.

And as you can see, it is converted to Api, not API.

"api/bases".camelize
=> "Api::Bases"

To fix it, you have to adjust ActiveSupport::Inflector:

ActiveSupport::Inflector.inflections(:en) do |inflect|
  inflect.acronym 'API'
end

And after that, camelize will work out for you as required:

"api/bases".camelize
=> "API::Bases"

As well as the controller will respond adequately to a request.

P.S. This is true that ActiveSupport::Inflector has to have the same acronym defined for Rails 5.2. The point is that it isn’t enough to have only updated Zeitwerk inflection because it’s responsible for loading only. However, with a classic loader, you could have only one adjusted.

Now, you may wonder if we have the same “double” situation with API and Api for controllers, then, we have a problem, we would need to monkey patch ActiveSupport inflection to distinguish between different prefixes/namespaces to decide how to convert the string.

However, I don’t think we should go further than we already did.

I wanted to show you how many problems can be brought by certain decisions when the Rails convention is not honored.

All of the above can be avoided if you don’t have constant names requiring custom inflection.

Please consider subscribing or adding me in social networks!

And don’t forget to share what you think about the topic. Just write me a message. I would be happy to discuss the topic or anything else with you!

• Medium
• LinkedIn
• Twitter
• GitHub

I recently stumbled upon a perplexing discrepancy while converting Time to DateTime using the #to_datetime method. Intrigued by this unexpected behavior, I delved into an investigation to unravel its root cause. I’ll share my findings in this blog post with you, as I believe it’s worth your attention.

Introduction

I’m currently engaged in the challenging task of upgrading a massive Ruby on Rails monolith. This unique experience has allowed me to dive into unfamiliar territory, learn new things, and engage with numerous stakeholders.

Amid fixing a failing spec on the new Rails version, I stumbled upon the following behavior:

# ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [arm64-darwin22]

time = Time.new(1,1,1,0)
=> 0001-01-01 00:00:00 +0100

time.to_datetime
=> #<DateTime: 0001-01-03T00:00:00+01:00 ((1721425j,82800s,0n),+3600s,2299161j)>

I couldn’t help but wonder why there was a 2-day shift when converting Time to DateTime. Although I found a quick fix for my specs, the behavior continued to intrigue me, compelling me to dig deeper.

Investigation

Initially, I wasn’t sure where to begin. I started by refreshing my knowledge of the differences between Time and DateTime in Ruby’s documentation:

Time

Time is an abstraction of dates and times. Time is stored internally as the number of seconds with fraction since the Epoch, January 1, 1970 00:00 UTC.
…
Since Ruby 1.9.2, Time implementation uses a signed 63 bit integer, Bignum or Rational. The integer is a number of nanoseconds since the Epoch which can represent 1823-11-12 to 2116-02-20. When Bignum or Rational is used (before 1823, after 2116, under nanosecond), Time works slower as when integer is used.

DateTime

A subclass of Date that easily handles date, hour, minute, second, and offset. DateTime does not consider any leap seconds, does not track any summer time rules.

I quickly scrolled those documentations, then googled a bit, but, unfortunately, I couldn’t find anything related to the situation.

Therefore, I decided to go the “hard way” and opened a Ruby source code.

static VALUE
time_to_datetime(VALUE self)
{
    VALUE y, sf, nth, ret;
    int ry, m, d, h, min, s, of;

    y = f_year(self);
    m = FIX2INT(f_mon(self));
    d = FIX2INT(f_mday(self));

    h = FIX2INT(f_hour(self));
    min = FIX2INT(f_min(self));
    s = FIX2INT(f_sec(self));
    if (s == 60) 
        s = 59;

    sf = sec_to_ns(f_subsec(self));
    of = FIX2INT(f_utc_offset(self));

    decode_year(y, -1, &nth, &ry);

    ret = d_complex_new_internal(cDateTime,
                                 nth, 0,
                                 0, sf,
                                 of, GREGORIAN,
                                 ry, m, d,
                                 h, min, s,
                                 HAVE_CIVIL | HAVE_TIME);
    {
        get_d1(ret);
        set_sg(dat, DEFAULT_SG);
    }
    return ret;
}

From top to bottom, a few things got my attention:

• decode_year — It deals with year-related operations differently from other attributes like month and day, although our issue pertains more to the “day” aspect.
• GREGORIAN — This keyword provides a clue for the following line of investigation.
• DEFAULT_SG — It is defined as #define DEFAULT_SG ITALY.

Armed with these findings, I revisited the Time and DateTime documentation and finally uncovered an explanation that shed light on the reason behind the behavior.

An optional argument, the day of calendar reform (start), denotes a Julian day number, which should be 2298874 to 2426355 or negative/positive infinity. The default value is Date::ITALY (2299161=1582-10-15).
…
Since Ruby’s Time class implements a proleptic Gregorian calendar and has no concept of calendar reform, there’s no way to express this with Time objects.

There is also an excellent explanation of when to use what.

P.S. It’s important to say that this paragraph was just a few blocks from the beginning, meaning my eyes simply overlooked it when I was “very quickly” scrolling through the page.

Conclusion

Dealing with time, especially in the context of time zones, is always challenging and prone to bugs. This investigation has highlighted an additional consideration before claiming that our code functions as expected. We can make informed decisions and write more robust code by understanding the intricacies of Ruby’s Time and DateTime conversions.

As for the conversion, it can be done with .gregorian to have the expected value.

require 'time'

time = Time.new(1,1,1,0)
=> 0001-01-01 00:00:00 +0100

time.to_datetime
=> #<DateTime: 0001-01-03T00:00:00+01:00 ((1721425j,82800s,0n),+3600s,2299161j)>

time.to_datetime.gregorian
=> #<DateTime: 0001-01-01T00:00:00+01:00 ((1721425j,82800s,0n),+3600s,-Infj)>

Thanks for reading.

Please consider subscribing!

• Medium
• LinkedIn
• Twitter
• GitHub

The Mysterious Time-to-DateTime Conversion: Unraveling Ruby’s Unexpected Behavior was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.

While this might be a good approach in some cases, in my experience, most times, it should be done differently. Today, I would like to discuss this issue with you and what are the available solutions.

Let’s start by looking at an action you might have in your plain controller. I’ve made it based on the scaffolding template. I’ve also removed unnecessary parts to keep the focus on the subject.

class UsersController < ApplicationController
  # ...

  def update
    if @user.update(user_params)
      # render success
    else
      # render failure
    end
  end
  
  private
  
  def user_params
    params.require(:user).permit(:email, :full_name)
  end
end

As always, everything has its pros and cons. We can emphasize that this code is straightforward to read.

But what can be done better here?

Before jumping into that, let’s define a business action in software engineering.

Please take a moment to stop and guess how you would describe it before reading forward.

In software engineering, a “business action” typically refers to a unit of code that implements a specific business function or process. These actions are often designed to perform related tasks in a larger business workflow.

For example, let’s say you are building an e-commerce website. One of the business actions in your code might be to process a customer’s order. This action would involve several steps, such as checking the availability of products, calculating the total cost of the order, updating inventory levels, and generating an order confirmation for the customer.

Each step could be implemented as a separate function or method, and the overall business action would tie them together into a cohesive process. By breaking down the larger business workflow into smaller, more manageable code units, you can create more modular, reusable software that is easier to maintain and update over time.

Overall, designing business actions in software engineering aims to create code that reflects the underlying business logic and workflows, which can be easily adapted to business requirements or environment changes.

After familiarizing yourself, I believe it would be fair to say that the @user.update(...) call is an implicit business action. In this context, its purpose is to update the user’s email and full name.

Where is the promised issue, then?

I will describe several below.

Code isolation and reusability

In software development, it’s important to avoid duplicating business logic and instead extract it into reusable code units.

You probably have multiple places where you need to execute the same business logic. It could be data migrations, background jobs, or other APIs like GraphQL. While copying and pasting business logic into multiple places may seem convenient, this approach is prone to bugs. It can make it difficult to maintain and update the codebase over time.

Even if you don’t currently have other consumers for the business logic, extracting it into a separate code unit can help ensure consistency and avoid errors in the future.

Honestly, it isn’t the controller’s responsibility to handle business logic. Controllers are already having a lot of responsibilities. Extracting business logic from controllers can help improve the modularity and testability of your code by allowing you to focus on testing specific units of functionality in isolation.

Concrete business meaning

When designing software, it’s important to ensure that the codebase is organized in a way that reflects the concrete business meaning of the application.

It’s easy to lose the concrete business meaning of the action when you have generic, widely open update actions. I saw such actions growing in size to an enormous number of accepted attributes where it becomes hard to say what it does, how, and when it’s being used.

Let’s update the action to accept just eight fields.

def update_params
  params.require(:user).permit(:email, :full_name, :address, :phone,
                               :balance, :default_currency, 
                               :last_signed_at,
                               :last_order_id)  

It’s only eight fields (I have seen 30+), but it’s hard to tell what it’s going on here for me.

Don’t get me wrong, it’s obvious that it just updates the user’s fields, but can you anticipate the flows/business actions?

If you look at the code above one more time, you can see that newlines split the list of attributes in a specific way. Continuing the example, this business action has four fully isolated consumers. For me, it’s a clue that an action should be split into four pieces representing their concrete meanings.

def update_personal_information
  personal_information = 
    params.require(:user).permit(:email, :full_name, :address, :phone)
  
  if @user.update(personal_information)
    # ...
  else 
    # ...
  end
end

def update_billing
  billing = params.require(:user).permit(:balance, :default_currency)

  if @user.update(billing)
    # ..
  else
    # ..
  end
end

# and separate for :last_signed_at 
# one for :last_order_id

The concrete-focused meaning is essential for understanding how the application generally behaves. It’s also much easier to extend its functionality. We are going to elaborate on this in the next bullet point.

Business functionality

Business functionality represents a list of things that business action does. So far, every business action above has described a single function — a user update.

Usually, it’s quite the opposite. In actual live applications, business actions consist of multiple steps. For example, I saw actions doing 20+ different things.

Let’s look at the instance where a user updates personal information. Probably, you don’t want to allow this change without sending a notification letter. So the resulting business action would look something like this:

def update_personal_information
  personal_information = 
    params.require(:user).permit(:email, :full_name, :address, :phone)
  
  if @user.update(personal_information)
    # Send a notification letter to the user
    NotificationLetter.to(user).send! # pseudo code
    # ...
  else 
    # ...
  end
end

And the presented flows can grow tremendously. It’s crucial to support every step appropriately, considering its nature. They can be sync or async, revertable or not, critical or optional, etc.

I hope I provided enough reasons why putting the logic into the controller is a bad idea.

Please let me know if it didn’t convince you. I would be happy to discuss it further.

In summary, building a robust DSL for business actions can help solve the issues we’ve discussed, such as reusability, transparency, and scalability.

While several existing solutions are available, such as Granite, Operations, Trailblazer, and ActiveInteraction, you can always create your custom wrapper to control the required features.

Overall, I hope this article has emphasized treating your business logic as a valuable unit that deserves attention and care. By doing so, you’ll reap the benefits of a more efficient and maintainable codebase.

If you have any other alternatives to the ones mentioned here or would like to share your experience with these tools, please feel free to reach out.

Thank you for reading!

Please consider subscribing!

And don’t forget to share what you think about the topic. Just write me a message. I would be happy to discuss the topic or anything else with you!

• Medium
• LinkedIn
• Twitter
• GitHub

If you haven’t read the original article, please do so. However, this is not required to understand the subject.

Stop using manual preloading in your Rails application; use this instead.

Without further ado, let’s get into the topic.

Skip the next section if you are well familiar with N+1 issues.

I’m sure you’ve all heard of N+1 issues here. In case you don’t, putting it simply, I would say:

“The code executes many similar inefficient database queries/HTTP requests/complex calculations.”

Most often, in the Ruby on Rails world, it’s all about database queries so we will stop on that part. Ruby on Rails offers a built-in solution for dealing with N+1 issues regarding fetching associations.

Now, let’s look at the example.

class User < ActiveRecord::Base
  has_many :accounts
end

class Account < ActiveRecord::Base
  has_many :contacts
end

class Contact < ActiveRecord::Base
end

Somewhere in the code, you want to show your users’ accounts’ contacts.

users = User.all 

users.each do |user|
  user.accounts.each do |account|
    p account.contact
  end
end

If you didn’t spot the issue yet, please stop here for a moment, look carefully at the code above, and try to find it.

The problem is that for every user, there will be a query to a database to fetch accounts; moreover, for every account then, there will be a query to fetch its contacts.

As you can see, this chain of calls can grow as a snowball, leading to hundreds or even thousands of requests.

Rails’ solution is to use includes at the very beginning of the chain. The fixed code would look like this:

users = User.includes(accounts: :contacts).all
# ...

This code will preload all needed data in only three queries: all users, their accounts and contacts.

If you have ever worked with includes before, please pause here and remember what you didn’t like about it.

To keep the post concise and focused on the elegant solution we are looking at soon, I will shortly share mine:

• you have to accurately and manually keep your beginning point consistent with the rest during the execution. If you no longer need down-the-road contacts, you better update includes, too; otherwise, you load extra data for no reason. If you need more data, let’s say referrals, you must update includes, or you get another N+1 issue. The effort required for consistency depends on how much the execution trace is spread along the project, but it isn’t trivial.
• includes fetches all the data immediately. Sometimes, we need to show information under the conditions. Following our case, what if contacts should only be displayed for primary accounts? It’s possible to do partial includes by directly calling ActiveRecord::Associations::Preloader, but this way isn’t convenient nor recommended by the guidelines.

Now, when we recall the N+1 problem and its most-popular Rails-way solution for that, let’s look at the proposed standard.

The Ruby on Rails framework is all about a convention and fast delivery. The goal is to focus closely on the business rather than technical aspects. With this in mind, let’s look at the fix provided by includes.

When I look at it, I wonder: if the only thing needed to avoid the N+1 problem is to type includes with specified associations, why can’t Rails do that for me? Is it possible?

Gladly, it is! It’s already production-proven and awaits you at zero integration cost.

1. Add gem "ar_lazy_preload” to your Gemfile.
2. Enable auto-preloading globally ArLazyPreload.config.auto_preload = true
3. Remove redundant includes

Chain your loading with .preload_associations_lazily If you don’t want to enable auto-preloading globally. For example, User.preload_associations_lazily.all. Any consequential association loading on every user (and following loaded records) won’t create an N+1 problem.

Let’s investigate what it does to avoid the N+1 problem. I will also show you when it doesn’t work as a bonus.

Want to read more topics about #ruby and #rails? Please join my network:

• Medium
• LinkedIn
• Twitter
• GitHub

There are two main parts that make auto-preloading work:

• ArLazyPreload Context (it has two implementations: one for globally enabled preloading and another for preloading through .preload_associations_lazily)
• ArLazyPreload Relation patch

Context is an object that stores metadata in with every ActiveRecord instance. It’s stored in .lazy_preload_context instance method. The most important metadata is the list of sibling records.

Sibling records are the records of the same class fetched in the same query and conceptually treated as similar records.

users = User.preload_associations_lazily.first(5)
# Records are siblings/Share single Context
users.map(&:lazy_preload_context).uniq.count == 1

other_users = User.preload_assocations_lazily.first(5)
# Records are siblings/Share single Context
other_users.map(&:lazy_preload_context).uniq.count == 1

# But "users" aren't siblings with "other_users" 
users.first.lazy_preload_context != other_users.first.lazy_preload_context

In the code above, instances in users and other_users groups are sibling records among their groups, but the two groups are not siblings.

Context also keeps track of a tree of already preloaded associations, but this is unnecessary to understand the main point.

Relation patches ActiveRecord::Relation class to look into the Context when deciding on loading the association. It will preload the association in one query for all records in the context if it exists. New records are properly distributed per referencing instance and cached there as if they would be manually preloaded with includes. After that, it assigns the context to loaded records, keeping them as siblings so they won’t produce the N+1 problem too.

Does that mean we don’t need to think about the N+1 at all?

Well, yes and no.

The answer depends on how well you know how includes is working. For example, please, look at the code below and think if it has the N+1 issue.

users = User.includes(:accounts).all

users.each do |user|
  user.accounts.where(primary: true) do |account|
    p account
  end
end

The answer is yes.

I leave it to you to understand why because this is very important and would help you avoid many pitfalls.

There are two main quick solutions:

First way is to replace .where with in-memory Ruby filtering by select { |account| account.primary == true }. However, I don’t recommend you this way as it is inefficient.

The second way is to create a new scoped association in User model and use that instead.

class User < ActiveRecord::Base
  has_many :primary_accounts, -> { where(primary: true) }
end

users = User.includes(:primary_accounts)

users.each do |user|
  user.primary_accounts do |account|
    p account
  end
end

This approach works well. However, it is very negotiable due to software design aspects; therefore, it may or may not be accepted in your project.

The same pattern comes when using auto-preloading. You can’t chain associations without declaring a new scoped association to avoid the N+1 issue. It’s simple to do, though.

That’s about it for auto-preloading in ActiveRecord (Rails default ORM). I hope this time I did better in explaining how it works and why you should start using it.

Please consider subscribing!

And don’t forget to share what you think about the topic. Do you consider N+1 issues to be important in your projects? I would be happy to hear from you.

• Medium
• LinkedIn
• Twitter
• GitHub

Before we dive into the ultimate solution, let’s quickly explore the current approach’s downsides.

If you know what you don’t like about preloading, scroll right to the last section for a much better solution.

Imagine we have the following query somewhere in a controller or any other place according to your architecture.

User.includes(:account, :payments, referral: :user)

This will execute several SQL queries.

SELECT * FROM users
SELECT * FROM accounts WHERE user_id = ?
SELECT * FROM payments WHERE user_id = ?
SELECT * FROM referrals WHERE user_id = ?
SELECT * FROM users WHERE id = ?

What do you think is wrong with that?

Potential unnecessary loading

What if we need to show referrals only for users without an account?

- if user.acccount.nil?
  = user.referral.user.full_name

With initial includes, we will query referrals and users tables even when the block is never reached, e.g., all users are missing an account. This brings an unnecessary loading that we would like to avoid as well.

Existing approaches to resolve the issue take a lot of work to maintain.

users = User.includes(:account, :payments)

# Check if any user has an account
if users.any?(&:account)
  # Keep in mind that this class changed its API in ActiveRecord 7.
  ActiveRecord::Association::Preloader.new(records: users, associations: {referral: :user}).call
end

So now, you must keep the consistency of your condition logic when you load and present the data. Even with extracting the logic and reusing it in both places, it’s still hard to ensure consistency as more places could be involved later.

Manual maintenance of zero N+1

Assuming that no code is unchanged by fixing bugs, refactoring or developing new features, you must keep your eyes sharp on N+1.

For example, one day, you want to show users’ countries on the view. So you adjust the view, and with the help of tools like bullet, you find a new N+1 issue. You go back to the loading logic and update it.

User.includes(:account, :payments, :country)

“The job is done” — you think. No more N+1, so you are free to take on another task.

While it may look like the case, unfortunately, it isn’t always true. What if the magic tool didn’t spot N+1? What if your testing/development data wasn’t producing N+1? There are many reasons for what could go wrong.

One more frequent thing is keeping preloading when it isn’t needed anymore. Let’s say one day you decided to stop showing account information. So an engineer updates the view by removing a couple of lines.

Now again, a lot of things may happen:

• you don’t have an automatic tool to find redundant preloading
• you are afraid to remove preloading as it might be used for another place
• you were focused on business value and forget to think about the technical part

While reasoning I mentioned can be questionable by some, my main point is simple: why not make developing easier having the same quality/performance?

Do you like the topic so far? I would be delighted if you subscribe:

• Medium
• LinkedIn
• Twitter
• GitHub

It’s time for the ultimate solution I promised at the beginning of the article.

When we first query for users, we could bind them with a shareable context so that every user from the group knows the whole batch at any moment.

users = User.all

users.each { |user| user.batch_group = users }

Now, all we need to do before querying the associations is to preload them for every object in the batch.

unless user.association(:account).loaded?
  ActiveRecord::Association::Preloader.new(records: user.batch_group, associations: :account).call
end

user.account

The idea is so trivial and yet so powerful. Without a surprise, tools are available for you out of the box, so you don’t need to think about preloading anymore.

• ArLazyPreload
• Goldiloader

With them, you don’t need ever to remember about includes.

It just works!

The configuration is simple (steps for ArLazyPreload):

• add the gem to your Gemfile (gem ‘ar_lazy_preload’)
• enable it globally (ArLazyPreload.config.auto_preload = true)
• that’s it.

I encourage you to check them out and give them a try. I believe you will like it a lot!

Thank you for your time! Please subscribe for further topics about #ruby and #rails.

• Medium
• LinkedIn
• Twitter
• GitHub