Go Language, Relational Databases and ORMs

Fernando Eugenio Correa

Published in

Mercado Libre Tech

8 min readJul 12, 2022

This is the first story of the series “Golang: ORM or not ORM at Mercado Libre”. Enjoy it!

When I started at Mercado Libre, I was introduced to an incredible culture and also to new professional challenges.

In the first month in the new team (Code Ecosystem), every new person faces a ramp-up challenge. This activity consists in implementing a simple application that allows us to be connected with most of the experiences that we find in our day-to-day work. It allows:

interaction with our peers;
information exchange through different channels and across different teams and people;
knowledge of the development environment and tools, methodologies and standards followed.

Implementing this challenge is pretty straightforward until we have to interact with a database from a GoLang project. I and those that came before me used to say: “Oh, I’ve joined a large company. It’s all done”; But it doesn’t work that way for us. As we keep moving and iterating our products, so happens with our technology. That’s why we wanted to answer a big question:

How should we work with databases in Mercado Libre using the Go language?

Before going deeper, I want all of you, my awesome readers, to be on our same page. Let’s take a look at some must-know concepts together. 👇

Language & Key concepts

About Golang

Golang is basically a procedural (and structured) programming language, like C, Fortran and Pascal. Go is essentially an updated version of C designed to be more accessible to developers and to address some modern computing priorities, namely concurrency and network systems architecture although the solution to the latter is mostly addressed through the concurrency solution.

Some important characteristics are (which you can consult in this article or in this documentation):

simplicity (readability and maintainability are the focus here);
standardization (its powerful standard library has all the essential stuff);
speed of compilation;
performance;
ease of learning and use (it’s user friendly with clearly identifiable advantages);
a C-like design for system-level programming;
testing support;
concurrency;
a succinct syntax (like Python).

It’s weird to think that Golang is in between a system language (original purpose) and a scripting language (as it is actually used by developers in many companies around the world).

Relational Database Management System (RDBMS)

A relational database management system is a program that allows you to create, update, and administer a relational database. Most RDBMs use the SQL language (and variations of SQL) to access the database.

Relational Database

A relational database is a type of database which uses a structure that allows us to identify and access data related to another piece of data in the database. It is a collection of data items with pre-defined relationships among them.

These items are organized as a set of tables with columns and rows (as you can see below on Image 1). Tables are used to hold information about the objects to be represented in the database. Each column in a table holds a certain kind of data and a field stores the actual value of an attribute. The rows in the table represent a collection of related values of one object or entity. Each row in a table could be marked with a unique identifier called a primary key, and rows among multiple tables can be related using foreign keys. These data can be accessed in many different ways without reorganizing the database tables themselves.

Image 1: simple relational database organization — ***Image 1:*** *simple relational database organization*

Structured Query Language (SQL)

SQL is a programming language used to communicate with data stored in a relational database management system. SQL syntax is similar to the English language, which makes it relatively easy to write, read, and interpret.

SQL became a standard of the American National Standards Institute (ANSI) in 1986. The standard ANSI SQL is supported by all popular relational database engines, and some of these engines also have extensions to ANSI SQL to support functionality which is specific to that engine. SQL is used to add, update or delete rows of data, retrieving subsets of data for transaction processing and analytics applications, and to manage all aspects of the database.

ORM (Object-Relational Mapping)

An ORM is a technique or — to put it simply — a library that automates the transfer of data stored in relational database tables into objects — and is extremely helpful for Object Oriented Programming languages. ORM is a layer of abstraction that allows to communicate relational data with object-oriented paradigms. The object/relational mapping problem is hard, and in OOP (Object Oriented Programming), the main points of reference are the objects. The abstraction presented for ORMs can solve this issue and the paradigm problem between relational data and OOP languages.

Database and Golang

Now it’s time to delve into some ORM (and not ORM) packages that will be helpful for comparison and maybe for future studies. Only some packages are mentioned below but will help us to get a more complete search or initiate a new research.

It’s possible to work with a large variety of solutions. As a developer, you can choose to manipulate your own raw SQL queries, only generate your structs, attribute total management to the ORM, maintain some focus on performance, whatever.

database/sql: it is an SQL package that provides a generic interface around SQL (or SQL-like) databases. The sql package must be used in conjunction with a database driver. To use it, you’ll need the package itself, as well as a driver for the specific database you want. It is recommended that you do not use the driver package directly. Instead, your code should only refer to types defined in database/sql, if possible. This will prevent your code from being dependent on the driver so if in the future you want to change to a different driver, you can do it with very minimal code change. This is just one of the many benefits that you’ll find when doing interface-based programming (perhaps another day we can go deeper into this topic).
To have a clear overview, some of the low level complexities in connecting to the database and managing the connection pool, the exposed APIs seem to do a bit more than you actually expect them to do, which may create a bit of confusion around how to use these APIs.
dbr: the package dbr provides additions to Go’s database/sql for “super” fast performance and convenience but the documentation is scarce and poor.
GOQU: it is not an ORM but an SQL builder and executor. Goqu does not provide common ORM features like associations and hooks. The best part of its proposal is a lightweight middle term solution.
sqlx: This ORM package makes it possible to simplify the implementation of the database layer while still maintaining two important points (compared with database/sql). This package expresses SQL semantics clearly for later optimization, especially in data analyses and other complex SQL, and reduces some “useless” encapsulations improving program performance.
dbq: This package aims to eradicate boilerplate code in database-related operations. Recently (June 2020), the creator did a major overhaul of the entire library to drastically improve its performance. Some benchmark results presented after these changes show a negligible advantage over using sqlx (the best ORM for Go considering performance).
rel: It is a modern Database Access Layer for Golang — Testable, Extendable and Crafted Into a Clean and Elegant API. It provides a testable repository with a built-in reltest package. The framework is great but too new to allow developers to rely on a project that may not have constant updates, community participation, that doesn’t support high traffic, or simply isn’t demonstrably capable of supporting yet.
ent: It is a graph-based ORM by Facebook (open source project). Providing ORM for modeling and querying data, the ent framework renders an API for modeling any database schema as Go objects. Users can run queries and aggregations and traverse graph structures. The ent project was inspired by Ent, an entity framework used internally on Facebook. The Roadmap for a V1 official version may be visited on https://github.com/ent/ent/issues/46 .
GORM: This is a developer-friendly ORM that has great community support and easy to follow documentation. GORM provides CRUD operations and can also be used for the initial migration and creation of the database schema. A few more things that GORM does well include its extensibility with native plugin support, its reliance on testing, and one-to-one and one-to-many group of associations. GORM also supports sqlite, mysql, postgres, and mssql.

Now we’re all set, let’s go to our main discussion point 😬.

How we should work with databases in Mercado Libre using the Go language

Probably you’ve searched for this answer before (if you haven’t yet, maybe you’ve done a quick search now). The answer is always obtained, not as a standard or usage recommendation that considers technical factors but as a particular positioning of preference or personal ease of use (and there’s lovers and haters everywhere). This makes it clear that there is no one silver bullet for every scenario.

Searching for possible answers on the internet, we usually have the same bias, presenting one side as the correct one without presenting an analysis or more technical information.

Personal experiences aren’t true or false, good or bad; they are simply different experiences with a lot of things to learn. Yet, sometimes they may lack the arguments and technical examples needed for a large company to base its decisions on them. And this is exactly what we do in the Code Ecosystem team: we provide the arguments and technical background that supports our experiences, in order to build a more mature, sustainable, resilient and healthy technology development ecosystem.

Read more in the second part of this series called “Using Go at Meli: ORM or !ORM” where we’ll also present technical data about the existing possibilities we currently use in our solutions. What motivated us to share this information with developers is that it’s usually hard to define whether to seek ORM or !ORM and what “filters” would be ideal to reach the objective.