Writing a Search Engine in Golang — The Beginning

I've started a project to create a search engine for quite some time now.

My goal with this project is to understand the challenges and drawbacks of how search engines and databases like elasticsearch and mysql store and search data.

This will be a series of posts since I won't write the whole thing in a slam, I'd rather take my time and understand the concepts and challenges of the project.

Why Golang?

The choice for using go, it's controversial for me, I have some reservations with the language. I don't enjoy much of it's super simplistic approach, were you don't have quality of life constructs, it's extremelly crude error handling patterns, a module system that's way too feature lacking, besides I enjoy a lot of the functional paradigm, being working with Javascript, Scala and Ruby for quite a while, and having to go back to a purely structured language seems like a step back for me.

However, even though all of the drawbacks, the language has unpair performance characteristics, being very close to bare metal while being confortable to write code on it, have awesome parallelism constructs, and also it's another part of the challenge, I want to write most of the things from scratch, so having a crude language can be a good thing in this regard.

Challenges

Since I have a work from 9 to 5 and I don’t want to came back home and do “productive” work, my will is to rather attend to classes, cook something, play video games, and, most commonly, grab a drink with my friends.

Given that, I’ll do most of my job only once a week, trying to use commit messages as a way to create a sort of a report, in which I’ll be getting the material for this series.

One other challenge is to write most of the algorithms and datastructures by hand, keeping most of the code using golang's standard libraries, the only external libs would be the http lib, namely the Gin framework.


The repo is here and that's mostly it, let the posts begin!