System Design Principles
The goal of the article is to explain the basic principles of system design which should be carried out after a business idea is floated out. Entrepreneurial ideas often come with “a” great idea and it’s us, architects, make it real and beneficial to the customers. End product not only will have the core idea but enhanced with awesome UX designs, Fault Tolerant, Scalable, Resilient systems.
Who can design an application ?
A person,
- who can explain a business concept in the simplest way.
- who can sense the bottlenecks.
- who can look for improvements even marginal.
- who can learn from their failures and others as well.
Key aspects of application design to consider
- Requirements
- Storage which can scale and perform
- Processing architecture which can scale
- Security
- Proper Data structures & algorithms
- KPIs on limitations
Fundamentals of Requirements
- Be Clear — If we don’t, the end product will be costly.
- Be Consistent — If we don’t, we will be in rabbithole.
- Identify Importance — If we can’t, agility will be hard and will be late to market.
- Make it Testable — If we can’t, customers quit easily from buggy products.
Why is Storage important ?
Till Google, Netflix, AWS & Facebook found a way to use the data better, the enterprise organization’s most important focus was purging. Matured IT strategy at least had analytical processing for reporting but they never thought about the data mine they owned.
Transaction database needed to be small to index and work faster.
Now storage is cheaper, access pattern is more matured by open source contributions which helps architects to design data storage not only as in RDBMS format but now noSQL, Columnar, Key-Value, Graph based.
Architecture improvements over the years
Distributed computing was a nightmare decade ago but now cloud computing works on that. CAP Theorem is now understandable by many architects as more and more proven systems appear.
Now architects introduce CAP Theorem ( Consistency, Availability & Partition Tolerance ) in all layers by introducing load balancers, replications, partition techniques, Quorum techniques, Queues.
Security
Everything needs to be secured, no questions or exceptions. Infra, data in rest/transit, access points all should be secured with better encryption keys, certificates, less number of ports open, less number of peering & latest great patches.
Now architects are more aware of DevSecOps, shift-left thinking & everything as code for easier and quicker audits.
Data Structures & Algorithms
Not everything is a hashtable or list. Not everything is 2 loops. Not everything needs to be in memory.
Better designers insist not only on SLA but cost of the program; they always insist on to program for 5–10x of data. Can this be a heap based ? Can this be a Trie based ? Can it be a bloom filter ? Insist on Big(0) notation.
KPI on limitations
Architects won’t be given time and energy to look for their architecture limitations because of delivery and cost of the solution, everyone wants to market faster.
SRE principles from google, well architected framework tool from aws, chaos engineering from netflix are so open now its easy for Architects to create a strategy and readiness for day 2 to know their system limitations.
What would make a better system ?
I think, not only all the above principles need to be followed. You need to have a good feedback process to cross check the system you desired to build is actually running as expected.
