Take Ownership of your Data- Introduction
This blog post is the introduction of a 5 part series.
— — — — — — — — — — — —
As developers, the first thing we wonder about when hearing about a new software project is the programming language being used. This is no surprise mainly due to the fact that it’s the primary tool used to solve a problem computationally. It’s our communication line to the machine world. So the first and most basic skill of a developer is to use a programming language to solve a problem.
Upon getting a good grasp on this step, we find ourselves taking the next step, usually by necessity, and start modeling systems. We then start to understand higher level concepts with multiple steps and combinations of the former. This is no easy task and, with time, it gets messy and ugly and constant reorganization and tweaking is necessary to keep sanity in check. With time and maturity, some patterns get recognized and reused constantly, giving birth to design patterns, frameworks, libraries and other abstractions, serving as a conceptual grouping of previously sparse abstract concepts.
But when only transforming data is not enough, we need to think about persistence. That’s when a previously foreign concept is introduced to our thinking: data modeling. When we are faced with the challenge of putting the data we transform into something permanent (relatively), we have to think of data structures outside of our programming languages. How to model something that doesn’t get too attached to the programming language itself and can serve as a persistent format?
Over the years, those questions were answered with the birth of databases and particularly the success of relational databases. Data was modeled thinking about tables and relationships and interfaced with SQL queries. Programming languages served as a wrapping tool for the database, now evolved to RDBMSs, acquiring more power and responsibility.
Everything was delegated to databases. Data modeling through tables and relationships, concurrency through locks, background jobs through triggers, consistency with constraints and performance with stored procedures. The database was slowly becoming the entire system as if nothing else was necessary anymore.
What’s wrong with that? You should ask. It’s been working for decades and should work for more, so what’s the fuss about it?
If you are like me and values freedom to be an essential part of being a developer, it should’ve rung a bell a long time ago. You feel that scratch on the back of your neck that something is wrong and shouldn’t be like this. It numbs your mind and keeps you from thinking alternative ways of doing things the way you wish. It takes out your freedom to think differently. It limits your imagination.
Now it’s not always that we need total freedom of thought and limitless possibility. We would go crazy if we don’t have a focus point and goals to achieve, even if it’s something we made up ourselves. But that’s something we should embrace as a tool, not as a shackle. Every tool or framework that we choose to use that limit ourselves has a trade-off and we should always be conscious of that. RDBMSs and similar all-in-one tools limits our conscious and damages our critical thinking in the long term.
Putting aside the obviously overinflated drama, we can agree on a certain level that we want to always be able to choose our own tools. If not, the trade-off should be clear and agreed upon. Given that, what I propose here is a treatment to detach from the database and use it for persistence only. Our sessions will occur within a series of posts regarding how we should take back what belongs to the application from the database to be able to use whatever tool you’d like to solve your problems, and we’ll be using mainly Elixir to show how powerful it could be embracing statefulness and making parallel comparison to other languages and runtimes as well.
Part 1 will talk about the differences between data modeling and database modeling. They’re not and shouldn’t be treated as synonyms and you should model your data similar to the domain it belongs instead of a reflection of your database’s capabilities.
Part 2 will talk about how addicted we are to relational databases and SQLs and how we can detox.
Part 3 will talk about the implications of the programming languages paradigms and what contributions it had in the adoption of relational databases.
Part 4 will present you abstraction versus implementation: the difference between thinking about it and doing it.
Part 5 will suggest a risky transition from stateless to stateful: you can and you should.
That’s that for now, I’ll be back with Part1 soon. So stay tuned