Scalable Architecture For Startups

It’s 2016. Startup is everywhere. Though, it’s rare to see technology startups coming up with an engineering stack that is designed for scalability right at the beginning. Many would argue that it’s over-engineering if we think of scalability at the beginning. Therefore, frameworks like Ruby On Rails(Ruby), Play(Java), Spring(Java), Symfony(PHP), Laravel(PHP) and etc. have become the best option due to the facts below:

1. Development is faster as there is a lot of libraries available.
2. Community support is better as there is a lot of issues that you’ll bump into already have solutions.
3. Hiring is relatively easier as most engineers are leaning towards frameworks.

But my argument to the over-engineering context would be: How can one guarantee if there is time to work on the scalability requirement while having to serve the rapid change on the market needs? Also bear in mind not all startups get tons of money to hire as many engineers as they can to scale the platform and develop new features in parallel, left alone the efforts to be spent on communications between teams.

So, what am I really proposing? Here I present you the Appist framework(it is OPINIONATED), like Rails as a monolith repository, but web apps/services/api gateway can be deployed to different machines on the same cluster by using Kubernetes, along with Continuous Deployment on GitLab CI. And the monolith folder structure is as below:

|-- db (Managed by `standalone-migrations`)
|    |-- migrate
|    |     |-- ...
|    |-- seed
|    |     |-- ...
|    |-- config.yml
|-- gateway (API gateway with GraphQL)
|    |-- gen_codes (GRPC client code generated by `protoc`)
|    |-- tests
|    |-- .babelrc
|    |-- index.js
|    |-- ...
|-- mobile (Using React Native)
|    |-- ...
|-- protos (Protocol Buffer files)
|    |-- auth.proto (defines auth services and messages)
|    |-- user.proto (defines user services and messages)
|    |-- ...
|-- scripts
|    |-- ... (Python scripts for development/deployment)
|-- services
|    |-- auth
|    |     |-- main.go
|    |     |-- ...
|    |-- gen_codes (GRPC server code generated by `protoc`)
|    |-- user
|    |     |-- main.go
|    |     |-- ...
|    |-- ...
|-- web (Using React with Relay)
|    |-- ...
|-- ...
|-- .gitlab-ci.yml
|-- cli
|-- ...

In short, below is the list of libraries/frameworks/tools that we’re gonna be using:

1. standalone-migrations
   Manage migrations for databases.
2. create-react-app
   Create new web project with no build configuration.
3. GRPC
   A high performance, open-source universal RPC framework to reduce the HTTP latency between API gateway and services.
4. Protocol Buffer
   A language-neutral, platform-neutral extensible mechanism for serializing structured data between API gateway and services.
5. GraphQL
   A more efficient query layer for API for web/mobile apps.
6. Relay
   A Javascript framework for building data-driven react applications.
7. Gitlab
   A hosted version control service that provides unlimited private repositories, powerful & flexible CI, registry to keep docker images and build artifacts. All for free!
8. Kubernetes
   A very powerful tool for automating deployment, scaling, and management of containerized applications.
9. Google Cloud Platform
   Infrastructure provided by Google that works very well with Kubernetes and provides quite an amount of tools for production monitoring.

The framework is currently under active development and being used to experiment with some pet projects. Hopefully I will get to release it to the open source community soon once more things are sorted out.