KISS Principle on Action

A shred of a brief example of why we keep it simple

KISS is the one of principles we are embracing while we are implementing features within Trendyol. Many of us have encountered some complex implementations during the investigation of code blocks; hard to read and understand which cost time to adapt or maybe solve the production issue.
Let's dive into the KISS principle, then a brief example.

What is KISS?

KISS is the abbreviation of “Keep It Simple, Stupid” (most common). But there are lots of variations “keep it super simple”, “keep it short and simple”, “keep it simple and straightforward” or “keep it small and simple”… Besides the variations, it is mainly a minimalist approach to designing anything (not just programming).

With just one look, it can be seen as easy; but actually, it is hard to accomplish KISS, and also easy to miss, even by seniors.
Some quotations from the geniuses of humankind:

“Simplicity is the ultimate sophistication” — Leonardo Da Vinci
“Less is more” — Ludwig Mies van der Rohe
“It seems that perfection is not reached when there is nothing left to add but when there is nothing left to take away” — Antoine de Saint-Exupery

Also, you can check Occam’s razor.

Benefits of embracing KISS in Software Design

• KISS leads to better usability: Simple and intuitive software is easier for users to understand and use, which can improve their experience and increase the adoption of the software.
• KISS leads to better maintainability: Software with a simple design is easier for developers to understand and modify, which can reduce the time and effort required to maintain the software over its lifespan.
• KISS leads to better reliability: Software with a simple design is less prone to errors and bugs, which can improve its reliability and reduce the risk of costly downtime.

Keep an eye on it while we embrace KISS

Even the benefits, when embracing KISS, we can end up with:

• Limited functionality: By focusing on simplicity, it is possible to limit the functionality of the software and make it less capable of solving complex problems.
• Reduced flexibility: Simple software may be less flexible and adaptable to changing requirements or new use cases.
• Increased complexity in the long term: In some cases, software that is initially designed to be simple may become more complex over time as new features are added or the software is modified to meet changing needs.

It is better to examine whether we avoid these.

An Example

Problem

Let's think of a hypothetical but common problem; there is many of data, let's say 100 million tuples with an equally distributed, random, unique integer key. We need to consume and calculate something from them and store the result of this calculation, and this process needs to be done at most every two hours, and assume that if we consume that whole data, it is exceeding memory and CPU limits, calculation logic is complex but necessary and optimized, and increasing resource limits is not an option due to physical limits of clusters we have, but we can have 5 distinct pods which have minor CPU and memory. In the end, the good news is we do not need the result immediately, it can be eventually, and every calculation generates the most recent result with no need for the previous results.

Way to solution

As you may think, this problem can be solved by splitting data and batching; but how. Thanks to every tuple's unique integer keys, we can use it as a pivot. Many of us used or heard about cron jobs which help to run codes within a given time or time period, we can use it. The only thing left is how we can divide data into smaller data portions.

First Proposal

In order to divide data into chunks (smaller portions) we can use such a design; every time a scheduled cron job is fired; we can consume data those keys’ last digit is a specific digit. For this specific digit, we can take 10 mods of the current minutes, and the job is fired within every 7 minutes. For example; if the job is fired at 12:14, the data that of key ended with 4 will be consumed, and every 7 minutes other ones will be consumed.

Pseudo-code

@CronJob("every 7 minutes")
@CronLock("until execution done")
function processData() {
   try {
     const digitToBeProcessed = new Date().getMinutes() % 10;
     processDataWithLastDigit(digitToBeProcessed);
   }
   catch(e) {
     logger.error("there is an error while processing data with last keys:".digitToBeProcessed, e.stack);
   }
   releaseCronLock();
}

According to this algorithm, if the job will be fired at:

12:00, 12:07, 12:14, 12:21, 12:28, 12:35, 12:42, 12:49, 12:56, 13:03, 13:10, ...

Data will be consumed for those keys ending with:

0, 7, 4, 1, 8, 5, 2, 9, 6, 3, 0 ...

actually, whole digits get hit within almost 1 hour, therefore whole data will be processed. It is really easy and fun to implement. Most probably it will run for a long time without any problem, but is it a valid design according to KISS?

Evaluation of the First Proposal

At first look at the code, we can get confused about the minute modulation. Because of such dependency, such a non-controlled but crucial parameter, it is ambiguous, especially without any documentation or comment. While going deep, for sure, we will figure out this implementation is for splitting data; but hard to be convinced that every digit will get hit without writing firing minutes as I did before. After that, some questions will raise; What if the batch is exceeding 7 minutes? It is highly possible with the data increase, it will skip the next digit. It is hard to see when this missed digit will get hit. What about changing the cron period 7 parameter to 9 minutes, can affect anything else?

To sum up, according to the key benefits of KISS as I mentioned before, we can miss better maintainability and reliability, Also, it is hard to understand and has some non-clear outcomes.

Another Proposal

We can establish more control over the last digit parameter we will consume. If we put a counter digit within a distributed cache, every pod can know which digit will be consumed next. When a pod’s job starts, it will get that digit, increase it with 1 (if reaches 9, then reset it to 0) then consume data with the keys’ last digit equal to this counter.

Pseudo-code

  @CronJob("every minute") 
  @CronLock("until execution done")
  function processData() {
      const digitToBeProcessed = distributedCacheClient.getDigitAndIncreaseIfExceed9SetAs0();
      try {
        processDataWithLastDigit(digitToBeProcessed);
      }
      catch(e){
          logger.error("there is an error while processing data with last keys:".digitToBeProcessed, e.stack);
      }
      releaseCronLock();
  }

For example, assume that we are running 5 pods and data that has keys’ last digits are 0,1,2 consumed and finished; 3,4,5,6 and 7 are processing at the moment. The counter within the distributed cache will be 8.

{ "digit": 8 }

By doing this,

• We can increase pod efficiency because every data will be consumed and every pod run within the full load with ordered digits. If one digit’s data will be processed sooner than other digits, the pod will not wait, just will take the next one.
• We can omit the ambiguity of minute modulation, processing will be more controlled by the counter, and it is sure that every digit will get hit
• If we want to re-calculation a chunk, updating the cache with the last digit of that chunk is enough.
• The equal distribution of data keys will be still considered but non-major disturbances will be doable. Due to an increase in pod efficiency, if one-digit processing lasts longer, other pods can process other digits.

To summarize, even the second proposal has more benefits than the first one, we can generate more reliable and maintainable design proposals (still data will be split into 10 chunks, elimination of race conditions needed, can be added status for monitoring and auto re-run ability, or event-based firing design can be considered … etc). There is no end to it, but the ultimate goal is to get simpler (more reliable, usable, and maintainable) with every interaction and every refactoring. Furthermore, for readability, creating a simpler design is one of the keys but also we can improve it with clean coding, it is another story.

Thanks to Tugba Bakirdoken for support with proposal evaluations and pseudo codes.

Thanks for reading. Hope it was worth your time.