The two things in startup software architecture
You may have come around the fun game of the two things, organised and compiled by Glen Whitman:
For every subject, there are really only two things you really need to know. Everything else is the application of those two things, or just not important.
He has collected a nice list of those two things, such as buy low and sell high on trading. There are already a lot on computer science but I want to add my own on software architecture:
- Features shape context.
- Context limits the abstraction level.
Okay, my two things are orbiting around abstraction, using abstract terms. If you start following me (yay!), you will read about abstraction a lot, alongside its dark antagonist, the context.
It makes intuitive sense from a conversational view: Your hand is an abstract tool. It can do a lot of things. Playing piano, write, draw and so on.
It can as well tighten a screw to some degree, but it’s not really optimized for a do-it-yourself-context. Screwdrivers are highly specialised for this task and really excel at it — but are not really useful for anything else. The more you shape the things your hand can do, the more you limit its flexibility. Flexibility comes with high abstraction.
Abstraction is as well prominent in software engineering. Most developers know the mantra to program against an interface not an implementation.
An interface is essentials extracted out of an implementation and abstracting it into a generalisation — save(), delete() and getPath() are essentials extracted out of a File Implementation , preferably into an interface named Storage, opening the door for implementations like Dropbox, S3 and InMemory.
The normal development flow starts with implementing a requirement and open it for change — hence abstract (as an interface). Context — an interface implementation — is not harming your application as you can exchange it at will.
Anyway, today’s topic is the big picture: your stack in its entire scope. The abstraction game inverses and context becomes a strong counterforce that puts your application in danger if not handled wisely.
We want to understand these dynamics in details. Therefore we’re going to examine the nature of abstraction and context a bit more, well, abstract.
Let’s start seeing an abstraction as the boundaries of a context. If a Storage Interface does not support a copy() method, it’ll be hacky and wrong to support it in the implementation context, say a S3 Storage backend.
This clear line between abstraction and context is not limited to computer science, but is true for science in general. A theory for instance is describing something in the real world, abstracting some observations and measurements into a formula.
Like Newtons Law of universal gravitation in physics. It’s abstracting gravitational force that exists in our daily lives into some equations.
It does no longer apply for bigger distances, though. Einsteins General Relativity does:
That’s interesting, because it does not disprove Newton. It’s generalizing it onto a broader context. Newton is still fine for our daily life, in fact it is a whole lot easier than General Relativity, but it can’t do as many things as Einstein’s theory.
The more substantiated the context gets, the lower the generalization or abstraction level.
That raises a fundamental question in our search for the truth. If there’d be a comprehensive, ultimate theory in physics — what would be its context?
By definition, the theory can’t define the context, because it must be everything — including the abstraction itself.
We often encounter self-referencing paradoxes like this when we go deep into the roots of scientific questions. We can’t understand the world from within.
This is where software architecture begins.
The outer abstraction level
Let’s imagine a new, shiny the-next-facebook project we work on and there is nothing.
Getting back to our definition of an abstraction being the boundaries of our context, this means we are as abstracted as we can get — no boundaries. We could fire up the editor and build everything.
Unfortunately, this ultimate abstraction isn’t quite as useful as in physics. While it can theoretically support everything it practically can do nothing, as there is no context. We’re doomed to add features to shape our context, and take away possibilities. We need to limit our abstraction level.
We choose the language PHP or the framework Rails and therefore remove the chance to realistically build a desktop application. That’s not a big problem, our desktop app could be a different codebase, using the REST API from our framework — we’re aware & don’t care. Let’s move on.
A framework is an application with the application abstracted away. Quite literally, rails is essentially a few abstraction layers above basecamp, with basecamp abstracted into generic things one can use in the web. All the things are there: HTTP cycles, ORM, Templating. It can be anything web-based (maybe not twitter).
So having a web framework is awesome, but things get complicated here.
Features need to be added in order to make the app useful. This shapes the context of our application. Once we add a ShoppingCart, Order and Checkout we’re very deep in an E-Commerce context. It’s no longer the ideal starting point for a Content Management System. We have limited the level of abstraction.
Feature is a quite blurry term, often mixed up with words like use-case or specs. If we want to understand exactly how a feature is affecting abstraction and context of our application, we need to determine the forces that work on a feature!
A feature should have essentially four building blocks, if it has less, your product may not tap it’s full potential.
- Implementation is done by a software engineer and when she’s done, so is the iteration of a feature sprint. It’s relying mainly on the other blocks. Everyone in IT has a friend saying that she needs a website and asking you to do it. If you are, chances are that there is only this block.
- A Use Case is done by an UX Expert, Product Manager, Requirements Engineer or sometimes by clients. It’s focussing on getting one user story or detail right. A lot of in-house software or software build by an agency is build with just Use Case and Implementation. If you’ve ever done something in Lotus Notes beyond sending mails, chances are you’ve witnessed an example in the wild.
- The Product Experience is the abstraction layer of the product team. Most startups are aware that a feature must fit, being a part of bigger picture. It’s the CPO’s job to take care of this. Dropbox, Digital Ocean and Medium are examples of mastery in product experience, the AWS Console, office.com and the website of your cellphone service provider are examples of problematic approaches towards the challenge.
- While the implementation looks at a feature from the context where it applies (e.g. on the Checkout of a shop), the abstraction is looking at it from the top level. Startups want to be disruptive and conquer markets. It’s the job of the Software Architect to ensure that a feature implementation is reflecting the top-level tier of a startups strategic vision.
How does that apply in practise?
When we started out at elefunds we wanted to enable online shops to add us to their checkout so you can round-up your 19$ shirt to 20$ and that one dollar will save a chunk of the rain forest. That’s a use-case.
When I entered early stage, there was a rough MVP implementing that use-case. It had naturally a Shop Model and a Charity Model and a Donation Model.
Software that is limiting itself to a use-case is meant to be thrown away. Call it proof-of-concepts or shit-we-show-investors.
I have a different definition of what an MVP should be. MVPs should build a sound foundation and concentrate on getting the core right and abstracted. Iterating from there, selectively adding context. It needs to take the abstraction layer into account.
The first thing we did was rewriting the core actors to be Clients that send Donations to Receivers , shaping a vision of a platform where donations can be digitalised, processed and help.
elefunds is nowadays able to hook donations into any transactional process (checkouts, run miles on a marathon, an interface for managing donations and still shops) and from everyone to everyone who may be profiting from donations (like hospitals, soccer clubs or charities). It’s a platform where others build business models upon within the fundraising market.
A generalised definition of this example:
Context should not be solely added based on a use-case, it should focus first on the top-level market, where your startup is living. The use-case should derive from there.
Let’s go a bit into detail about this.
A feature shapes the context of your application, and context by definition limits your application. That means we want to implement any feature as abstract as possible.
The outer abstraction level is the market where our startup is operating. We need to keep that in mind, when we implement the underlying use-case of the feature. If we are focussing on the use-case, we loose the strategic focus. The abstraction level of our application becomes limited.
Once a sound core is established, things will start to accelerate later on. When Rocket Internet entered the market, people assumed that the incredible speed is based on interns peaking 24/7 at an insane capacity for suffering.
The truth, though, is that they invested deeply in getting the first context level right. Committing into e-commerce, they build a sound cross-department system and process model, ultimately modelled in a gigantic PHP-framework called Alice & Bob. I’ve had my share of hate with it during my rocket days, witnessing its downfall, but however: When they founded a new venture back in the days, they where already miles ahead of the competition.
Conversely, competitors were forced to add use-cases at fast pace to keep up with the joneses; routinely repeating the same mistake over and over again: Using Magento (the shop system of which Alice & Bob has become the successor).
There’s nothing wrong with Magento used in the correct context, like a LAMP stack somewhere with a small to medium business. A scaling startup with changing requirements that has a hockey curve somewhere in its slide is not the correct context.
Yes, it has a lot of use-cases, but those come with a lot of context that you do not want to have and that do not fit your strategic vision.
Magento is monolithic, widely undocumented, bloated and unsupported. It has a database model that is build to support third-party plugins not performance and it was acquired by eBay with the relatively obvious intention to let it die — Magento was back then the go-to tool to build a clone that eBay had to buy prior to enter a market.
Summary
Startups are fast-moving businesses, so is their software. Standing tall in the challenge has multiple facets, but at it’s core is the duality of concrete stuff and abstract strategic visions.
Even great startups sometimes ship a founders feature that feels like foreign matter — here’s where you can identify that the CPO got trumped over by the CEO or investor; and great startups have code fragments that have grown beyond beauty. It happens, and it’s okay as long as you still stand up and fight it.
However, getting the fundamentals right and having a formulated, shaped vision of the strategic, abstract level of your software is crucial. Here are a few questions I’m asking myself regarding my software:
- Is the core structure (database, models, services) reflecting the market we want to conquer?
- Am I able to identify the main abstraction layers by browsing the repos— is it logically unfolding into core engines, services, modules and so on?
- Are we separating services and modules by concerns and roles they play in the to-be-disrupted market?
The driving tension in every startup is the tension between the Product Team and the IT .
Product wants to push features in as fast as possible, IT wants to keep the abstraction level high.
Both push into different directions of the driving forces that define our two things in startup software architecture.
Healthy startups create a balance.
