Abstraction in software engineering — Architecture
Source: Wikiart.org, except 3. abcgallery.com & 4. paintingdb.com
We don’t know what we don’t know, yet in Software engineering systems constantly mutate throughout their lifecycle as new requirements emerge. How can systems be built at speed and at the same time be ready for unknown changes? Abstraction is a fundamental pattern that must be used in every system so that it can quickly adapt to unforeseen modifications. In this article, I’ll present a three-tier application from a high-level view to its low-level implementation and explain how convertible it can become once we start applying abstraction concepts to it. By the end of this article, you should have a clear view of how abstraction and isolation can add flexibility to your designs and implementations.
What’s abstraction in software engineering and how to use it?
In one word, abstraction means generalisation! Say you want a dog, it does not matter if you’re after a Dingo, a Husky, or a Dachshund, regardless of its breed, it’s still a DOG; software engineering follows the same concepts. Therefore, you’d never have one system per dog breed. Instead, the same system would accommodate all dog breeds there are. So, how do we make sure to accomplish this generalisation requirement from a high-level view? It’s simple. We have to design our systems based on strong interfaces, which each component of our architecture will utilise to communicate with each other. In other words, we’ll be following a well-known pattern called Service Oriented Architecture by implementing the below.
1 — The front-end web-page application that receives the dog information required and sends it to our engine for processing by utilising its HTTP Rest interfaces.
2 — An engine that receives HTTP requests, computes and validates them, and pushes to or retrieves the dog data from the database.
3 — A database that saves data regarding dogs.
Some might wonder “why don’t we have a monolith that talks to our database directly”? It’s simple, by putting the front-end and the engine together, we will sooner or later face problems, such as integration with different front-end systems, scalability issues, overcomplicated logic located in one single component, and more. Nevertheless, let’s not forget that if we put them together, the deployment complexity will increase significantly. This would mean that if we needed to change a single letter in one of our front-end pages, we’d have to deploy the whole engine altogether — Yuck!
Working with interfaces
To understand what interfaces really are, the difference between declarative and imperative programming must be crystal clear. As a quick reminder, declarative programming is “what to do”, whereas imperative is “how to do”. If this isn’t yet very obvious, let’s have a look at the below examples.
You, Josh, want to ask your friend, James, to take your dog, Skip, to the dog park. In which way would you ask him?
Declarative: Hey James, can you take Skip to the dog park that’s located at 240 Macaulay Rd?
Imperative: Hey James, can you take Skip to the dog park that’s located at 240 Macaulay Rd by following these instructions:
- Go on foot.
- Stop three times until you get there.
- Don’t take Skip before 10 AM.
- Don’t bring Skip back after 1 PM.
- Don’t go under the tunnel.
- Take the bridge.
Hopefully, it’s now clear what are the differences between “What”, declarative, and “How”, imperative; however, how do put this paradigm into software engineering designs?
Declarative:
Given the http endpoint below
myapp.com.au/dog/takeQuery params
personsName, dogsName, dogParkAddress
I would be calling it as
# take = abstraction / generalisation. Can take any dog
myapp.com.au/dog/take?personsName=“James”&dogsName=Skip&dogParkAddress="240 Macaulay Rd”Imperative:
Given the same function as above, we would would first have to create the steps that I want to be followed in that function/interface and then call the function.
# takeInJoshWay = Specific to Josh. Cannot be reused
myapp.com.au/dog/takeInJoshWay?personsName=“James”&dogsName=Skip&dogParkAddress="240 Macaulay Rd”What’s the problem with the imperative option? Let’s say that three other people want to ask James to take their dogs to a dog park; we’d have to have a different endpoint for each of them because their implementation would completely differ from one another. Something like below
myapp.com.au/dog/takeInPerson1Way?personsName=“James”&dogsName=Skip&dogParkAddress="240 Macaulay Rd”myapp.com.au/dog/takeInPerson2Way?personsName=“James”&dogsName=Skip&dogParkAddress="240 Macaulay Rd”myapp.com.au/dog/takeInPerson3Way?personsName=“James”&dogsName=Skip&dogParkAddress="240 Macaulay Rd”
We don’t want that to happen to our system. Otherwise, our abstraction would be worthless and the number of endpoints would be the same or higher than the number of implementations. So, what do we do? We create the interface by saying “I’ll take your dog to the dog park as far as you tell me the attributes personsName, dogsName, DogParkAddress, but I won’t tell you how I’ll do it”. What this means is that more people can easily get their dogs effortlessly taken to the park. In software engineering terms, more systems can integrate with our components without knowing extra details, yet take full advantage of our engine and database layers, such as the following.
Isolating and decoupling systems
What are the attained benefits when we decouple systems into isolated modules? This allows us to change different parts of our architecture without interrupting or worrying about the end-to-end flow of our platform. It also gives us the possibility to mitigate and isolate problems for better troubleshooting. Additionally, by modularising our architecture, we can scale our modules in and out independently, which we’ll cover soon. What’s the drawback? A questionable increase in complexity in the overall architecture, which many times pay itself off in the medium/long term.
To put into perspective, imagine the following scenario: We now decided to expand the capabilities of our engine, we not only want to take dogs to parks, but we also desire to save the Dogs’ and Owners’ information so that we can let the owners know when they should take their dogs to the vet. However, we don’t want to break our current interface with our existing customers that are utilising our systems. What do we do? Create a new interface that stipulates a new contract to save dogs and expose them to our customers!
Our new endpoint looks like
myapp.com.au/dog/saveQuery params
name, breed, ownerEmail
And we’ll call it like
myapp.com.au/dog/save?name=“Skip”&breed=Lab&ownerEmail="email@email.comOur implementation would look like below. Note that nothing other than the engine changed. In fact, our customers didn’t even need to know about such change.
We can now publish this new interface which some of our customers might be interested while others may decide to implement it in a future stage, which does not make any technical difference to our system at this stage because the abstraction gives this flexibility.
Our customers that have integrated with our second interface can now “save” their dog’s information in our database and will be notified whenever their dog needs to go to the vet.
We’ve just been through a very simple, but essential pattern that allowed us to change a single component of our architecture without worrying about the infinite number of integrations we could have. This isolation isn’t only beneficial during developing and troubleshooting; it also allows us to scale the components that need more resources precisely.
Scaling becoming extremely trivial
Before we dive into it, let’s get some terminology straight
- Scale up = Give more resources to a single node
- Scale down = Take resources from a single node
- Scale out = Add a node to the cluster and start the processes on it.
- Scale in = Remove one node from the cluster of nodes Node N-1
Given our abstracted and decoupled architecture, we can now scale any part of our application independently. Let’s say we now have too many customers, and our single-engine node is struggling with the number of requests. As a result, our clients are complaining that our response time is intermittently slow. Something like this
We can immediately assume that the first layer, the front-end, is sending a lot more messages than we originally thought we’d receive; however, we backed ourselves up when designing our architecture and can tackle this problem easily by scaling out our downstream layers.
Our second layer, the engine, is now compatible with our first layer, but it’s likely that the third layer won’t cope well with that, so what do we do? Correct, we scale out!
Depending on where the second and third layer of our platform have been deployed, controlling this scalability would be a lot easier. For example, if we’re in the cloud, it would be a matter of defining metrics and scalability parameters. Say, every time the response time from our engine is above 3 seconds for 2 minutes, add two pods. If it’s under 1 second, delete2 pods. Keep a minimum of 2 pods continuously running.
Applying reverse engineering concepts
Architecture isn’t trivial, and decisions made during the assembly of the foundation of our platform have direct impact throughout the lifecycle of our systems. Let’s understand the dependencies as bullet points in a reverse engineering fashion.
- We wouldn’t have been able to scale if we didn’t have created independent modules.
- We wouldn’t have been able to create independent modules without interfaces.
- We wouldn’t have created interfaces without contracts.
- Contracts wouldn’t exist without abstraction.
Moral of the story
Understand what abstraction can give you and use it. In the worst-case scenario, you’ll spend the same amount of time to design and develop systems that will be ready for future unknown changes.
Next steps
We’ve covered the high-level view of how abstraction can be applied to end-to-end architectures. In my next post, we’ll have a look at how to accomplish the same but from a low-level perspective. In other words, we’ll write some code and get our hands dirty!
