Episode 1: The one with a service for sum of prime numbers.
Concurrency and Distributed Systems: A guide with Kubernetes
I am going to present a simple math problem which we use throughout some episodes. The problem might seem a little fabricated, but the point is about learning concurrent and distributed systems concepts and not solving a specific use-case, so bear with me.
Problem: Find the sum of prime numbers between two positive numbers a(inclusive) and b(exclusive).
Example 1:
For a = 1 and b = 10, the sum of primes is 2 + 3 + 5 + 7 = 17.
Example 2:
For a = 1 and b = 60000000, the sum of primes is 103468900363400.
This is a fairly easy problem to solve, but it is computationally expensive on CPU which can be equivalent to a real-world CPU intensive server operation.
For large enough ranges of a and b values, this computation could actually take a long time. On a system with the following CPU spec:
Intel(R) Core(TM) i7–10750H CPU @ 2.60GHz
the sumPrime(1, 60_000_000), takes 17 seconds to return. I know what you might be thinking. We can split the range into smaller ranges and use multi-threading to compute the result faster. That is a totally valid point and approach, but that is not what we are after. Our workload is different.
Requirements
Imagine we are offering an API which does the above computation for our users. We have already started our journey into the distributed systems world :).
There are many questions we need to address:
- How many users do we have?
- What kind of query per second(QPS) we are expecting for this API?
- Do the queries come uniformly over time or in high QPS bursts?
- What type of queries we are expecting to receive? Small, medium, big ranges and how many of each?
- How much money we have to spend on hardware?
- Do users have priorities in running queries? Do we want to impose quota on them?
And perhaps many other questions that I have missed here. Like any good software engineer we are going to scope the problem and divide and conquer and iterate and test.
We go and gather some requirements from customers and try to come up with a minimum viable product(MVP).
Here is a list of requirements:
- Most of the queries are in range 1 to 5_000_000.
- We could expect a QPS of maximum 20 for the start.
- We need a rest API which should be synchronous.
- We have little money to start with. We can possibly get more funding if the MVP goes well.
- Users are OK with waiting 2–5 seconds to get the result. So an SLA of 2–5 seconds for now is reasonable.
Just like that, we have scoped the problem and we have something almost clearly defined and solid to work on. We will work on it for a quarter, get our promotion and bye! No! I am just kidding :) Fun is just about to begin.
Design
Our API spec is fairly simple:
POST /sumPrime
BODY:
{
a: Positive integer
b: Positive integer
}
RESPONSE:
{
sum: Long
processingTimeMs: Long
}We will avoid premature optimization and would avoid caching for now. For deployment we know we are going with K8s. For implementation, we will go with Java and Spring framework.
Implementation
Let’s put everything into code!
And the main code for the server:
In the above code the sumPrime function has an async result. All that means is that the computation will happen asynchronously in a pool of threads created by Spring framework. That means that the main server thread would not be blocked and the server can keep receiving new requests. It doesn’t mean that our API is asynchronous though. We will cover asynchronous APIs later.
What else do we want? A docker file to package the server so that K8s can run it:
(Ignore the java command line options if they look strange. They are just recommended flags for running a Java server).
At this point and before we get into the K8s part, we want to test our little system in docker first to make sure it actually runs. To do this we will run:
If everything goes well, we should see an output like:
{
"sum": 12272577818052,
"processingTimeMs": 3459
}Now, we are almost ready to take this to the next level. In the next episode, we will see how we can deploy our system using K8s and how we can scale our application to be able to satisfy the requirements for our MVP.
Links
