There are two fundamental things to get right when scaling engineering and engineering teams — design and code. Bad design is responsible for most of your technical challenges such as scaling, consistency and costs. Bad code is responsible for most of your people problems such as productivity, credit, ownership and motivation.
When I manage my engineering team, the most useful skill which I want to develop in my team and which has the highest return in the long run is the art of writing code.
Most teams under fast growth face the problem of not just what to do, but also how to do. They are unable to lay down the path between where they are and what they keep hearing as the ideal without impacting their deliverables. They need specific directions on how to move forward, because if they knew they would have done so. You must have good mentors who can enforce standards. Because they would know it does not take extra time or effort to get the code right. Certainly you will need to continuously refactor, but that would be because you learnt something new and not because you messed up previously.
What should code be like?
Code is not just for machines. Today’s devices can run badly written code thrown at them with great speed for most applications. Code is written for humans. Code is not just an algorithm or a flow chart. A code is a story. Any good literate can read a good piece of literature and hold on to its joy. Similarly as engineers we can aim to not just appreciate the beauty and precision behind this machines but also produce such work which creates delight in people who later visit it.
Who is your audience?
The code you write is going to be consumed by
Your successors — who will take over your code and will update and upgrade it. They will need to understand, debug and improve the systems you wrote. The improvements they make will be constrained by how extensible your code is. Whether your system grows into an orchard or a jungle of weeds will depend on how your design and philosophy is integrated into the code structure. Process documentations and guidelines are useless sermons when the code is a mess as nobody follows them.
You will be judged based on how smoothly they could do this. You will also be known for the systems you built which have lasted and grown. Unless you write such code which is easy to hand over, you will take longer to be substituted and to move on.
Your collaborators — when working on a large system it is fairly common to simultaneously develop several interacting components from ground up. If you are in a startup, chances are that you spent some time discussing an outline of contracts between sub systems and then dived into building it. Sometimes, more than one developer are simultaneously working on the same piece of code, such as an app screen. When everything is evolving, your code should be such that it is easy to understand the endpoints, requests, responses, errors and functionality without having to delve deeper.
Your superiors — they might be mentors or architects or your leads or managers or reviewers who might want to give you feedback on high level design and structure. For them, it is important that your code is readable like a story. Subjecting them to a folder of hundreds of files with single functionality or flow scattered everywhere is like asking them to read a book by the index. It should be like browsing a tree breadth-first, starting at a definite root and letting the reader dive in as much detail as he wants and not more, and all siblings under a parent tied together as a coherent chapter of a story.
Testers — Writing testable code is an entire discussion in itself. Read more about test driven development. This helps you think about the contracts your functions expose and divide your code into individually and clearly testable units.
Support — they would likely never directly consume your code, but they would look at the exceptions encountered by your users, which is a by product of your code. The cleaner errors your programs produce, and the better is the separation of concerns, the more easily support would be able to route tickets. If there are support engineers who also fix bugs, they will need to clearly understand the impact of any change they make.
Community — If you aim to ever open source your code, your code must be easily generalisable, and reproducible. If you want the community to contribute, then it becomes even more important to help them understand as fast and only as much as required. Also it must be neat, linted, documented and commented.
Case for Functional vs Imperative
So whats the hype? Every language has functions. You divide code into functions, in several classes, in several files, with encapsulation, and inheritance, etcetera, etcetera.
Practical functional programming is about using stateless functions — functions which have no side effects. And using functions of functions — higher order functions to create a tiered, layered, nested code structure. This is different that having objects which contain properties and functions which mutate those properties. Why is this important?
When you read code, you want to know what it does. Some times you want to verify what it does. And rarely you want to sit like a turing machine and keep track of what is happening with each variable as you read the code line by line. Humans are fast at interpreting and logically connecting blocks which we know semantically what they ought to be doing. We are not good at keeping track of variables and working out flow charts. Thats what we have computers for. Hence using stateless functions improves the readability of the code. They are self contained and the reader only needs to know what the function does. He does not need to keep track of any state when understanding your code.
When you read code, you may not want the detail to the last order. You simply want to understand what operation is being performed on what logical unit. Either you want to understand the unit without understanding the operation or the operation without understanding the underlying logical unit. Hence using higher order functions helps you separate the operation from the logical unit thereby improving readability.
There are other similar concepts such as immutability, optional types, sealed traits, etc. which improve the readability of your code. Why are they needed? You as the author might know that a variable is not modified once set, but one who reads your code does not. Similarly you might know which values might be null and which not, but some one reading your code does not if you are passing null values for optional types. And most probably you will also forget a few months down the line.
Somethings to remember
Understand the end goal —Why and how will your reader start off reading your code? Is he searching for an endpoint, or from where a particular endpoint is called, or how a data is transformed, or a configuration, or a list of constants.
Whats vs Hows — When thinking on how to layout your code think in terms of what is being done instead of how the machine must do.
Too many fragments are not good — Don’t focus on every case because it will unnecessary fragment your code. Readability and fragmentation are inversely proportional. Understand which ones are difficult to understand and trace.
Use your IDE smartly — Especially in strongly typed languages, your IDE can help you find usages, etc very accurately. Use it cut down some verbosity and fragmentation.
Don’t use unnecessary abstractions — like defining useless interfaces, layers of abstraction, one function simply passing parameters to another function, getters, setters, etc, unless necessary. You can always complicate things later, it is simplification which rarely happens. Encapsulate layers in one another rather than independent layers making calls to each other.
Write concise code — Long verbose code takes longer time to read. Don’t write five line classes in individual files and have hundreds of such files. Keep it DRY. Don’t eat up exceptions. Throw early catch late.
Use a modern language — With immutables, type inference, optional types, lambdas and monads and such.
And finally, refactor. No code is written perfect the first time. And refactor continuously without breaking anything. Don’t turn it into another quarterly project.