A trip down memory lane with Derailed Benchmarks

In this post, I will show you how I reduced the memory consumption of a Ruby gem by 92%.

Earlier this year, I finally found the time to try Richard Schneeman’s derailed_benchmarks gem in a Rails project I was working on. The first aspect I profiled was memory used at require time:

TOP: 120.1602 MiB
phony_rails: 28.8711 MiB
iso3166: 26.6094 MiB
phony: 2.0195 MiB

The results above me told me that phony_rails was responsible for 24% of the app’s memory use, with the majority coming from iso3166. I set out to report the issue to the gem maintainers, following Richard’s suggestion:

…if you see a large memory use by a gem that you do need, please open up an issue with that library to let them know (be sure to include reproduction instructions). Hopefully as a community we can identify memory hotspots and reduce their impact. Before we can fix performance problems, we need to know where those problems exist.

I couldn’t find a dependency on iso3166 for phony_rails in my Gemfile.lock, but given that phony_rails deals with country codes and phone numbers, I deduced that the countries gem was the offender. A quick Google search for “ISO 3166”, as well as a search for “iso3166 in the countries GitHub repo provided confirmation.

While searching the countries GitHub issues, I noticed someone had already reported the large memory usage, so I added my findings to issue 230. At that point, I could have waited for the countries gem to be fixed (it eventually was about a month later), or attempted to address the memory issues myself. Instead, I took a step back to see how the countries gem was being used in phony_rails. Perhaps it would be easier to solve the memory problem by eliminating the dependency.

It turned out that the only reason the countries gem was needed was to map a country’s 2-letter ISO 3166 code to its calling code:

def self.country_number_for(country_code)
ISO3166::Country[country_code.to_s.upcase].try(:data).
try(:[], 'country_code')
end

Given that this information is static, and that new countries don’t get established every day, I submitted a pull request to replace the countries dependency with a YAML file that contained the country code to number mapping. The pull request was accepted and merged, resulting in a 92% decrease in memory usage for the phony_rails gem:

TOP: 92.332 MiB
phony_rails: 2.2227 MiB
phony: 2.1797 MiB

To produce the YAML file, I wrote a script that parsed the data in the countries gem, where each country has its own YAML file named after the country code, and inside each YAML file the numerical calling code is provided by the country_code key.

require 'yaml'

class CountryCodeToCallingCodeMapper
def self.country_codes
@codes ||= YAML.load_file('lib/data/countries.yaml')
end

def self.calling_codes
country_codes.each_with_object({}) do |code, hash|
hash[code] = {}
calling_code = YAML.load_file("lib/data/countries/#{code}.yaml")[code]['country_code']
hash[code]['country_code'] = calling_code
end
end

def self.write_codes_to_yaml
File.open('country_codes_to_calling_codes.yaml', 'w') do |f|
f.write calling_codes.to_yaml
end
end
end

CountryCodeToCallingCodeMapper.write_codes_to_yaml

I encourage you to try out derailed_benchmarks in your project. You might discover memory issues similar to the one I found, that might lead to improving an open source project you use. Another takeaway from my experience is that in some cases, it might make more sense to implement your own solution rather than adding a heavy gem to your Rails app.