Using Go at Meli: ORM or !ORM

This is the second story of the series “Golang: ORM or not ORM at Mercado Libre”. Enjoy it!

In the first article of this series, “Go Language, Relational Databases and ORM”, we made a brief introduction to the main concepts of the Go language, relational databases, ORMs and also packages that allow working with data with the Go Language. These concepts will help us in this second article of the series, allowing us to focus on sharing our experience with evaluating the use of these different packages and observations regarding our environment in Mercado Libre.

But what if “we begin from the beginning”… Ramp-ups are tasks a new developer at Mercado Libre will face that consist in implementing a simple application that lets them have contact with most of the experiences in a day-to-day work, as we’ve detailed here. During my personal ramp-up, I analyzed some alternatives and PoCs to answer the question: ORM or not ORM. I focused on 3 simple concepts to guide this work:

1. the resolution of the task,
2. the team’s knowledge for implementation,
3. the result obtained.

Based on my experience, I ended up implementing it using the database/sql (raw SQL), since I’ve got knowledge of SQL and it’s quite simple to use it in Go. However, considering that there are other ways of dealing with the same problem and that many people are going to face this problem again after me, I want to share the research with you.

Analyzing all our production GitHub repositories written in Go, we’ve found the most widely used packages that solve our problem. Let’s check the graph:

Graph 1: top 3 on 20th October, 2021

We cannot limit ourselves to the use and knowledge of a certain set of tools, but rather, be willing and able to understand the existing solution, supporting it or even enabling spot improvements and optimizations if necessary.

Demonstrating the importance of having an agnostic view of implementation, during the period of conception and review of this text, we had direct and indirect contact with different solution approaches:

• Projects using raw SQL in their implementation, which gave us advantages in interpreting, understanding and analyzing the relationship of tables due to direct access to the SQL codes present;
• Meetings to understand the project of another team that used GORM, migrating an existing module.

Analysis

There are important factors to consider before making some decisions about a project, such as:

• the depth in either POO or SQL (remember earlier we mentioned the preferences of each team?),
• the lead time of an MVP,
• the priorities over: 1. availability or uptime; 2. integrity or consistency; 3. delivery warranty; 4. throughput / latency / response time;

ORMs can help during the development and there are a lot of different possibilities, some of which are easy and others are not. Some of them work for specific use cases and others are more flexible. And finally, there are also some hybrid forms where you can make it work with SQL queries and a little magic on code.

The important thing here is that in all these scenarios you’re gonna need great knowledge of the tool and every feature it has, and why not, hybrid forms with some SQL queries and a little magic too. But you always need thorough knowledge of the tool and its every features.

Below, we present two lists with pros ( ✅) and cons ( ❌) when adopting raw SQL or an ORM.

raw SQL
✅ It’s more performant in runtime
✅ SQL is a standard language which is very useful to know
✅ It’s a standard in the Go community
❌ It presents the challenge to manage a lot of queries and increases maintenance efforts
❌ It takes time to write simple SQL statements to do basic CRUD operations for each data model

ORM
✅ It’s improving the speed of development
❌ It’s less performative (read it yourself or see this little benchmark)
❌ It’s too invasive and “magical” (Golang community doesn’t like it)

Conclusion

Considering all the points we have discussed, it would be a little inconvenient to say that using or not an ORM for a given solution is always the correct way. As developers, we always need to evaluate, verify and update our concepts and knowledge, sharing it with the other teams and (why not?) the community.

Take a few moments to answer some questions about whether it would be convenient for you to adopt straight SQL or any of the possible ORMs:

• How many times did the development team have contact with a given tool?
• Will it clearly provide a great advantage to develop the solution and obtain better business results?
• What trade offs would you do? Runtime performance, 5-star quality or low-effort maintenance?
• How will it affect user experience? After all, we aim to build great user experiences.

Remember that if we only consider the characteristics of the Go language described in the first story of this series, we would have as a natural adoption the use of straight SQL code (without using any ORMs) because it’s what it says in the docs and it pretty straightforward, but, we as software engineers must take decisions considering many more factors, and each person, each project or company has different constraints.

Before finishing this episode of the series “Golang: ORM or not ORM at Mercado Libre”, I would like to invite you to share your experience by leaving a comment below. This enriches the community as a whole. Let’s share knowledge and good vibes.

Eager about the next story? Start reading “Code Ecosystem — Improving the development experience”!

References

sql — database/sql: https://pkg.go.dev/database/sql

Go — Accessing relational databases: https://go.dev/doc/database/

ksql : https://github.com/VinGarcia/ksql

Comparing SQL, Query Builders and ORMs : https://www.prisma.io/dataguide/types/relational/comparing-sql-query-builders-and-orms

Go Proverbs : https://go-proverbs.github.io/