ActiveRecord’s Typecasting changes

I wrote about how ActiveRecord / Rails typecasting works in version 4.0 (check my previous story) and seems that applied for 4.1 too. However, in 4.2 version this process got a serious refactor in favor of polymorphism and an API that will allow us to Typecast DB columns in different ways.

This time, We will focus on changes that affect Column object and the way this typecast the DB values and not in the whole process to parse those values.

As we know each adapter inherits from abstract or specific abstract adapter which includes a columns method. This methods is resposible for AR’s typecasting. In previous versions, it calls new_column method with the SQL field result. However, ActiveRecord 4.2 has changed and the method looks like:

     def columns(table_name)#:nodoc:
        sql = "SHOW FULL FIELDS FROM #{quote_table_name(table_name)}"
        execute_and_free(sql, 'SCHEMA') do |result|
          each_hash(result).map do |field|
            field_name = set_field_encoding(field[:Field])
            sql_type = field[:Type]
            cast_type = lookup_cast_type(sql_type)
            new_column(field_name, field[:Default], cast_type, sql_type, field[:Null] == "YES", field[:Collation], field[:Extra])
          end
        end
      end

Please noticed the lookup_cast_type and that now there is not anymore the type assignment instead there is cast_type .The lookup_cast_type is a method that lives on ActiveRecord::ConnectionAdapters::AbstractAdapter class which is the parent for all specific adapter.

def lookup_cast_type(sql_type) # :nodoc:
  type_map.lookup(sql_type)
end

New objects got involve in this process, type_map is a shiny and important object that is responsible for mapping values but, wait type_map is a method what object?, here it is:

def type_map # :nodoc:
  @type_map ||= Type::TypeMap.new.tap do |mapping|
    initialize_type_map(mapping)
  end
end

Type::TypeMap introduces an object that will perform typecasting polymorphically. In the stable version of 4.2 in lives on ActiveRecord::Type::TypeMap and with it, AR introduces Types like Type::Boolean Type::String Type::Decimal and so on. All known ruby object that we used to parse using regex and old Column mechanism now are a Type object.

The Column stores the Types in the cast_type attribute. Therefore now the Column has an object that is responsible to convert DB value. This exposes an API to create custom Types in future Rails versions (rails 5).

The type_map method shown above initialize in the tap block with all valid types but, remember this can be overridden in the models or custom adapters.

Type object responds to the following methods: type, changed_in_place?, type_cast_for_database and contains a private method cast_value . The method responsible for casting the value from database looks like:

def type_cast_for_database(value)        
   case value        
   when ::Numeric, ActiveSupport::Duration then value.to_s
   when ::String then ::String.new(value)
   when true then "t"        
   when false then "f"       
   else super        
   end
end

This is a big change because opens a window to customize this method for instance instead to return t for a true value in String object we could return true word instead.

You should be wondering how the Column object looks like. As you noticed now it knows how to deal with thecast_type attribute which means it knows how to receive and send this kind of message. This is how it looks in the abstract adapter:

def initialize(name, default, cast_type, sql_type = nil, null =true)
   @name             = name.freeze
   @cast_type        = cast_type
   @sql_type         = sql_type
   @null             = null
   @default          = default
   @default_function = nil
end

This is the Column object initialize method in abstract mysql adapter

def initialize(name, default, cast_type, sql_type = nil, null = true, collation = nil, strict = false, extra = "")
   @strict    = strict
   @collation = collation
   @extra     = extra
   super(name, default, cast_type, sql_type, null)
   assert_valid_default(default)
   extract_default
end

Noticed that in this new Column object the default attribute lost importance in favor of cast_type. However, how ActiveRecord does it loads columns and attributes in models? Well, do you remember that in the previous version exists a method prepare_default_proc which stores in hash a proc that will execute a query in the DB to get fields description? If not read (previous story).

Now ActiveRecord initializes the schema_cache with empty hashes:

def initialize(conn)        
  @connection = conn         
  @columns      = {}        
  @columns_hash = {}        
  @primary_keys = {}        
  @tables       = {}      
end

Therefore, It loads this hashes when user/system ask for it. This prevents issues in hosting servers like Heroku or docker containers where the database is not available at the time when app loads and also give the possibility to open and closes DB connections and continue getting the correct columns or attributes.

# Get the columns for a table      
def columns(table_name)        
  @columns[table_name] ||= connection.columns(table_name)
end

Take a look to the new columns method, it calls columns in the connection object which is the method that executes the SQL query and get the list of the fields and cast to Types awesome, right?

But there is another new object the helps ActiveRecord to typecast DB value. How many times have you used, see, find the columns method? Mmm, not too many times, right? It is because Rails introduces another object Attribute to handle the typecasting dance.

The Attribute object as the great teaser of commit mentioned handles the typecasting and remove logic from Column object. Therefore, Column does not know about the Type methods like type_cast_for_database and the Attribute object encapsulates this into proper objects.

module ActiveRecord
  class Attribute # :nodoc:
    class << self
      def from_database(value, type)
        FromDatabase.new(value, type)
      end

      def from_user(value, type)
        FromUser.new(value, type)
      end
    end

    attr_reader :value_before_type_cast, :type

    # This method should not be called directly.
    # Use #from_database or #from_user
    def initialize(value_before_type_cast, type)
      @value_before_type_cast = value_before_type_cast
      @type = type
    end

    def value
      # `defined?` is cheaper than `||=` when we get back falsy values
      @value = type_cast(value_before_type_cast) unless defined?(@value)
      @value
    end

    def value_for_database
      type.type_cast_for_database(value)
    end

    def type_cast
      raise NotImplementedError
    end

    protected

    def initialize_dup(other)
      if defined?(@value) && @value.duplicable?
        @value = @value.dup
      end
    end

    class FromDatabase < Attribute
      def type_cast(value)
        type.type_cast_from_database(value)
      end
    end

    class FromUser < Attribute
      def type_cast(value)
        type.type_cast_from_user(value)
      end
    end
  end
end

Attribute class exposes class methods from_database, from_user like “facade class” for FromDatabase that implements the type_cast method of Attribute object because it inherited from Attribute as well. The current implementation of the Attribute in 4.0 version recommends to use these methods instead of initializing an Attribute

Why is the Attribute important?, because ActiveRecord defines the attributes using a Builder object, yes a new one. The builder is responsible to load attributes but, it builds a lazy hash LazyAttibutesHash where it stores the types before cast and values then AR can have a restore point and allow the user to create custom Types and don’t lose the DB references.

These changes allow us today to customize the typecasting of an attribute and create Type Classes that can manage an attribute to present in different ways it gets stored. This was the beta version of this type class that is possible in rails 5.

class StepType < ActiveRecord::Type::String
  def cast(value)
    if value.respond_to?(:map)
      value&.map{|step| step.join(',')
    else
      super(value)
    end
  end
end

#rails #activerecord