The benefits and drawbacks of decentralised Geo-Scaling (Thinking of 2019 and beyond: Infinite Scaling without bottlenecks)

The idea is to launch an app in a decentralised way to achieve infinite scalability without any bottlenecks. This is possible if your users only require a certain chunk of the database, for example if you have an app that creates closed networks for users in a location-based fashion. When I tried to build Communify, this was the case. This will be my second step of scaling.

A decentralised system that automatically pulls, builds, and restarts pm2 on multiple servers that are completely independent from each other. Every sever can be created using a docker image on Linode with automatic commands using the api of Linode and some bash scripts. Every single server is automatically up-to-date with the git repo (possibly on a specific branch, where master is the ‘beta’ branch and pushed is the production branch). Then, there should be one pointer server that directs you to the right server, based on your location. This is great because it reflects the vision haha. And it may be simpler to maintain. I’m taking advantage of the fact that communities are isolated. One disadvantage of this approach would be that you cannot easily search through all users globally. You can still have a look in another country as you can spoof location and connect to another server (or just fill in location yourself). Maybe this is a good second step. First create a server that is global, then split it up later based on location. Just clone the server. Non-used communities will automatically be deleted. Just make sure those ‘dead users’ will not receive emails.

The overlapping global db is only needed for the website, and can easily be created using an API on all servers and aggregating this data once per time period. The website is mostly read-only.

The only reason why this wouldn’t work is when a community doesn’t fit on one server, but I think this will never happen. A community can be maximum a few thousand members. After that the community is just too big.

Another (more extreme) option would be to just create one server per community and spin up servers on the fly. This would also reduce code complexity by a lot but I don’t know if it’s smart as you may want nearby communities to easily interact and exchange users.

Biggest Drawback: you’d have to login every time you enter a new location that is on a different server, and set up your profile again. However, this may be solved. Every time you log in, just go through all servers and apply the login. Then copy the record of the most recently updated user…

A drawback to vertical scaling is cost (more expensive than horizontal scaling) and downtime (15 minutes per change)

A big advantage of this decentralised fashion of everything (including database) is that you if a server goes down, it just goes down in one location. ← this is the biggest advantage. Everything can be done with an api, which means that it can be automated based on serverStats. All I need to know is the moment when I will split up a server, and I can just create a node app for doing all of this (cloning, splitting, and changing the geoproxy). This ‘master server’ keeps track of all independent servers, proxies all new traffic to the right IP address based on location, and clones and splits up servers once needed.

Hahah. I’m looking forward to it, although it will probably not be needed for another year or so. But it’s good to have a plan that scales well! And this single-cell structure scales to infinity without any bottlenecks. It keeps code simple.

— — -

I’m available for contracting work in the summer of 2018 onwards. Let me know if you’re interested by sending an email to me at karsens AT outlook DOT com. My biggest skills are: React Native, Node JS, GraphQL (Apollo), and MySQL. My expertise is creating MVP’s quickly.