Abstraction in software engineering — Application

Published in

The Startup

12 min readMay 20, 2020

Evolution of Mondrian Paintings between 1908–1921. From Top Left — 1. The Red Tree (1908–1910) 2. The Grey Tree (1911) 3. Flowering Apple Tree (1912) 4. Composition in Blue-Grey-Pink (1913) 5. Composition with Gray, White and Brown (1918), 6. Composition with Large Red Plane, Yellow, Black, Gray and Blue, 1921.
Source: Wikiart.org, except 3. abcgallery.com & 4. paintingdb.com

Continuing the Abstraction in software engineering — Architecture publication, it’s time to have a look at the low-level side of software engineering and how to implement abstraction and other handy patterns when designing and developing systems. In this post, we’ll go through the design and implementation of dog use case publish previously. Similarly to its predecessor counterpart, the solution developed here aims to be flexible enough to rapidly adapt to unforeseen changes in use cases that would reflect in changes in our application and in the code itself.

Layered application architecture

There is an overabundance of variations on how you can design your application out there, from very simplistic to their extremely elaborated designs. Let’s be honest, there’s no such thing as a perfect “Silver bullet” application design that will fit all scenarios, but for most of the cases, the standardised layered architecture can give you a significant head-start in terms of flexibility, encapsulation, reusability, and maintainability since the first lines of code.

This is how this layered architecture looks like and how each layer interacts with each other:

Before going into each layer in detail, it’s essential to understand why we’re segregating our application in these four pieces. This separation comes from decades of experience of well-known software engineers, such as James Gosling, Martin Fowler, Eric Evans, C. Martin, Robert, and others. If you don’t know any of them, you might have heard about their publications, for example, the clean code series by Robert, or the Domain Driven Design book by Eric, or even Head First Design Patterns by Eric et al Freeman. If you’re a software engineer and don’t know what I’m referring to, I’d strongly recommend a read on these materials so that you can understand the fundamentals behind them.

Let’s quickly define our layers and lay down a few bullet points with them.

Controller/Endpoint

Presents all endpoint/interfaces for incoming requests that our system offers, save and take, in our case.
Orchestrates the other layers of the system that will be called (Define and invoke the sequence of functions to be executed).
Does not have any business validation or complex logic.

Application/Business

Defines all the heavily business validation functions.
Makes sure that atomic requests are met (everything or nothing).
Does not have endpoints.
Does not interact with third-party systems

Infrastructure/Repository

Calls and interacts with any outgoing request towards any external applications outside our system, be it the filesystem or third-party APIs.
Does not allow incoming endpoint calls like the controller layer

Utilities

Utilised by all other layers, it contains any generic code that is used by all other layers, such as input/text validations, exceptions, output message responses, HTTP code responses, etc.

Now that we’ve defined each layer’s function, let’s see how the request lifecycle would work in our four-layered application.

Given our endpoints:

myapp.com.au/dog/takeQuery params
personsName, dogsName, dogParkAddressmyapp.com.au/dog/saveQuery params
name, breed, ownerEmail

The request flows would look like

This sequence flow highlights a few important points:

Everything else that is not our application is considered an external integration (Incoming and outgoing requests).
Each layer of our application is abstracted from each other. For example, the Controller layer does not know how to validate input parameters nor how to validate an email or whether a certain dog park exists; however, it knows who does hence it passes the responsibility forward through their interface (methods/functions).
All closely related logic is stored in a specific layer and reused when invoked. For example, Both flows, Save and Take, utilise the validate function in the Utilities layer to verify the parameters inputted.

Finding, updating and troubleshooting code much faster

This segregation of duties that we’ve been talking about gives us the possibility to know exactly where to look at when updating or troubleshooting our system. Say, we’ve never seen the implementation of our code, but we know it follows the layered pattern like the one aforementioned, and we have the following improvement to make in our system:

Change API calls to the email validator from vendor A to vendor B.

The first question to be asked is: Where do I have to look to find the code that I have to change? That’s simple! It’s the repository layer! In other words, we’ve disregarded around 75% of the codebase without even looking at it.

We can go from this much information to ingest

To this much

Additionally, you know that once you change the call in the repository layer, all upstream layers will automatically capture this change, because we’re changing the implementation, not the interface and utilising encapsulation throughout our code. Remember, imperative VS declarative programming.

Reusability and encapsulation

Everyone says, “I want to build systems that are reusable so that I can add features in a copy & paste” fashion. Unfortunately, only a few know what’s really required to get such flexibility in place. In order to reuse, you need to encapsulate. Similarly, to encapsulate, one must understand the single-responsibility principle in software engineering, which emphasises that two problems cannot be put together in a function/API call because they require two different solutions / implementations.

Let’s exercise our single-responsibility principle based on our application. As discussed, we have two endpoints, which each inputs three parameters. All these parameters are validated by the validation function(s) in the Utilities layer. How are we going to implement this function? Let’s have a look at our options, shall we?

Our variables to be validated are:

personsName, dogsName, dogParkAddress name, breed, ownerEmail

Our first requirement is to make sure that our input parameters have at least three characters and does not contain invalid characters. So, let’s create a generic function for that.

//Constants
MINIMUM_INPUT_LENGTH = 3
CHARACTERS_NOT_ACCEPTED = "!#$%^&*()"MINIMUM_INPUT_LENGTH_ERROR_MESSAGE = "error message" 
CHARACTERS_NOT_ACCEPTED_ERROR_MESSAGE = "error message" validate(value){  if (value == null || value.length <= MINIMUM_INPUT_LENGTH){
    throw Exception(MINIMUM_INPUT_LENGTH_ERROR_MESSAGE)  } 
  if (value.contains( CHARACTERS_NOT_ACCEPTED){
    throw Exception(CHARACTERS_NOT_ACCEPTED_ERROR_MESSAGE)
  }
}

Now, all inputs can be validated in a generic way across our endpoints, and future endpoints can follow the same. Note that we have a single function “validate”, which validates all input text values as below.

We SHOULD NEVER create a validation function per attribute, validatePersonsName, validateDogsName, as well as a function that validates many attributes, such as validate(personsName, dogsName, dogParkAddress name, breed, ownerEmail). The reasons for this are obvious, we’d be polluting our implementation, not reusing our code, exposing our implementation to the caller (declarative programming), and the worst part of it all, we’d have to change our implementation every time a new attribute is added. In fact, we’d be moving backwards in terms of generalisation and abstraction. So, back to the single-responsibility principle. If I ask you: What does our function do? The answer should be, it validates something. What does it validate? Whatever you ask it to. How does it validate? It does not matter!

Okay, so far, so good. It’s time to take our validations to the next level. What about if I want to validate something differently and more complex? Say, the email needs to have an “@” in its value, and the breed must be Labrador or Dingo. How am I going to hide it from my upstream layers without overcomplicating and breaking my current interface? There are two possible ways to go about it, which will depend on the programming language you’re using.

Strongly typed Object-oriented programming languages

Languages such as Java that does not allow us to omit parameters in a function, so in this case, our option is to overload our method. In other words have two versions of it, which will look like below.

validate(value);
validate(value, specificFunctionValidation);

Loosely typed OOPL or others

On other languages, such as Python, we can simply omit any parameter and don’t have to overload the methods; however, the implementation has to have an additional condition as we’ll see soon. So, in this case, we only need to add the “specificFunctionValidation” as a parameter to the existing function as its initial value is None (Null/Not existent).

validate(value, specificFunctionValidation=None);

Regardless of the constraints of the programming language we’re using, our goal is to do not break the existing interface and reuse as much as possible of what we already have. So, in this case, our two possible implementations will be as below.

Strong-typed OOPL

validate(value, specificFunctionValidation){
  validate(value); // Reusing the validate function
  specificFunctionValidation.execute(value) // Calling new function
}

whereas for loosely-typed OOPL or others

# Same function as before, we're not creating a new function
validate(value, specificFunctionValidation=None){if value == null || value.length <= MINIMUM_INPUT_LENGTH:
    throw Exception(MINIMUM_INPUT_LENGTH_ERROR_MESSAGE)
  
  if value.contains( CHARACTERS_NOT_ACCEPTED):
    throw Exception(CHARACTERS_NOT_ACCEPTED_ERROR_MESSAGE)
  
  # New specific validation
  if specificFunctionValidation is not None:
     specificFunctionValidation(Value)
}

Right, so, if we’re passing this extra function via parameter, who or which layer knows which functions need to be passed? Certainly not the controller. Remember, the controller should NEVER know how things are done, only what’s done. Therefore, we declare this function in the Utilities itself and invoke from the controller. Something like the below

# All methods without most constants for simplificationLIST_OF_ACCEPTABLE_BREEDS = ["Labrador","Dingo"]
validate(value){if (value == null || value.length <= MINIMUM_INPUT_LENGTH){
    throw Exception(MINIMUM_INPUT_LENGTH_ERROR_MESSAGE)} 
  if (value.contains( CHARACTERS_NOT_ACCEPTED){
    throw Exception(CHARACTERS_NOT_ACCEPTED_ERROR_MESSAGE)
  }
}validate(value, specificFunctionValidation){
  validate(value);
  specificFunctionValidation.apply(value)
}# New functionsmustContainAt(value){
  if (!value.contains("@")){
    Exception(DOES_NOT_CONTAIN_AT_ERROR_MESSAGE)
  }
}mustBeAcceptableBreed(value){
  if (!LIST_OF_ACCEPTABLE_BREEDS.contains(value)){
    Exception(DOG_BREED_INVALID_ERROR_MESSAGE)
  }
}

Here’s how we’ll call these new validation functions:

So, What are we doing here?

Reusing our existing code
Abstracting the logic from external layers (Imperative programming)
Turning our system extra flexible for future changes and validations
Encapsulating our code and following the single-responsibility principle pattern
Not breaking existing implementations

What about if tomorrow we receive the requirement that the ownerEmail field needs to comply with three other validations? Thanks to our flexible and reusable pattern, we can do the following by changing the specificFunctionValidation parameter into a list. That’s about it! Observe how we’re still using the same interface without breaking existing implementations.

Naming conventions — Modules, Functions, Layers, Classes, Interfaces

Be mindful of how you name everything in your project; use common sense. It’s a terrible idea to give specific use case names to units in and outside projects. Let’s take a look at the methods we created to understand what I’m talking about. We created two methods mustContainAt and mustBeAcceptableBreed. Why I didn’t name the former “ownerEmailMustContainAt” and the latter “mustBeAcceptable”. Why the former is more generalised, whereas the latter is more specific? It’s simple! What’s the likelihood of us having any other field that requires the “@” validation? Very High! Any email requires that. So why would I be specific and link that function name to the ownerEmail? This can be used for any future email field, such as secondOwnerEmail, vetClinicEmail, who knows. The point is, it’s a generic function, and it should be treated as such. On the other hand, what are the other things that have breed? Only dogs have them. So, why would I make it more generic? If it gets to a point where we have to generalise that function, even more, we might as well create a whole new function.

Last but not least, on naming convention principles, don’t be afraid of naming your variables and functions in a meaningful way just because it’d be a long name. If one needs to go through the function to understand what it does, you’ve poorly named your function and are wasting people’s time! Additionally, use the parameters to give sense to the functions and variables created. For example, our method validate(ownersName), it’s clear to English speakers to understand that we’re validating the ownersName value. That’s how it should be! Imagine if we had the function “val” and someone new joins the team and look at that function. This person would have to go into the function to understand what it does because after all, what does val stand for? Variable? Vanilla, Value? This is terrible! Don’t assume other people know what goes through your mind while coding and designing — Be simple yet objective.

Strategy & Polymorphism— Dealing with complex implementations

How can we apply the concepts aforementioned, have a flexible and pluggable code and still maintain our interfaces when sophisticated logics that require many variances need to be added to our code? For example, let’s have a look at our new requirement:

We now have to apply complex business logic when taking dogs to parks based on the initial letter of their name. These validations change from letter to letter; some are more complex and some less complex.

So, we implement this new change in our design like below.

How does the implementation of the design above would reflect in our code? It’s simple. It means that we’ll have to have 26 different conditions based on the dog name and the dog park that dogs being taken to. How do we go about it? Firstly, we already know that we’ll have to do it in the Application layer because it’s a business logic decision, but how do we do it? Do we create 26 Ifs and else ifs like below? Yuck!

canTakeDog(dogsName, dogParkAddress){
   if dogsName.startsWith("A"){
     .. 40 lines of code to validate A
   }
   if dogsName.startsWith("B"){
     .. 80 lines of code to validate B
   }
   if dogsName.startsWith("C"){
     .. 25 lines of code to validate C
   }
   ..
   ..
   ..}

Definitely not! This is terrible. We don’t want to have a method that contains 500+ lines of code. This goes against as many patterns as you can think of. What do we do instead? We use Polymorphism! We segregate these logics not only into different functions, but also into different classes/files and link them through a strong contract as below.

And how are we going to be able to delegate which canTakeDog implementation will be invoked? We use another design pattern called Facade. Think of it as a Load balancer in front of the various implementations we might have. So, it will look like this.

What are we gaining in adding all of this?

Abstraction
Polymorphism
Single-responsibility principle
Decoupling

Back to our interfaces, with this implementation, our Controller does not know which class is responsible for resolving this logic, it only knows “canTakeDog(dogsName, dogParkAddress)” and expects a boolean, true or false, back. Good, we have isolation going on. Also, we’ve separated very complex logic pieces into small chunks, but what are we gaining? Imagine that after implementing and deploying this logic, the requirements for letters A, E, and Z change. Do we have to look at the other 23 implementations? Not at all! We skip 90% of the codebase and focus on what matters most.

So, again, we go from this much that we have to understand

To this much understand

We’re not only saving time in changing our code, but we’re also mitigating risks when deploying to production. Imagine that our change is wrong. What would be the result? Only the three implementations above would be impacted. Now, let’s imagine if it was a pile of code in a single class and our change was wrong, we could be bringing the whole “take” interface API down because of that instead of a fraction of its usefulness.

Applying reverse engineering concepts

Designing and developing systems isn’t trivial; however, once you have the pieces in place, it becomes effortless to plug in new requirements to our codebase. Let’s understand the concepts we applied here as bullet points in a reverse engineering way so that we can visualise the patterns applied.

We wouldn’t be able to segregate and abstract our code and responsibilities in our application without layers.
We wouldn’t be able to find, change and update our code at speed without segregation and abstraction and patterns, such as the single-responsibility principle.
Functions wouldn’t have been precise if we hadn’t named them in a plain English format.
We wouldn’t be able to reuse and encapsulate our code without abstraction.
Polymorphism assisted us to mitigate risk during deployment and saving time during development, which wouldn’t be possible without the implementation of the Strategy and Facade patterns.

Moral of the story

Understand what abstraction, mitigation, isolation, and polymorphism can give you and use it! In the worst-case scenario, you’ll spend the same amount of time to design and develop systems with a great deal of readiness to future unknown changes.

Next steps

We’ve already covered the high-level, Abstraction in software engineering — Architecture, and low-level view of how abstraction and many patterns can be applied to end-to-end architectures and systems. In my next post, we’ll have a look at how to accomplish the same results from a testing perspective. In other words, we’ll write some flexible and adaptable tests!