CPU latency refers to the time delay or the amount of time it takes for a central processing unit (CPU) to complete a specific operation or task. It is a measure of the time it takes for the CPU to retrieve data from memory, perform calculations, and generate the desired output. CPU latency is typically measured in terms of clock cycles or time units such as nanoseconds.
Minimize CPU latency — Approaches
Efficient Algorithms
- One the cause for CPU latency is inefficient algorithm. Always write efficient absolute algorithm.
Efficient Queries
- Always write efficient queries because that translates into algorithms for databases by which they fetch our data or they write our data.
Minimize Context Switching
- Batch/Async IO:
- Whenever there is possible to batch multiple calls into single call do that it save a lot on CPU latency.
- Whenever we write log or read log it will part of IO and that result in context switching if we want to avoid that right way to do this is Async IO in a separate thread so that our main thread will keep running.
2. Single Threaded Model: it can be seen in technologies like JavaScript engine, node.js, databases like vaultDB, For that matter there is a reverse proxy or load balancer like NginX that also uses single threaded model.
So whenever there is a large flow of request and as part of serving this request we need to do lot of IO, then it makes sense to use single threaded model.
3. Thread Pool Size: We should not have large thread pool.
4. Multi-Process in Virtual Environment: When we run multiple processes in single machine we should run that in their environment. For example we are running 4 processes on machine and there is particular process which hog CPU, which is written very efficiently and it continues to occupy CPU for long time so other processes which are running on that machine will not get enough CPU time. So to overcome this we create virtual environment for each process that will allow running these processes with their dedicated quota of CPU and memory, so that they will not interfere with each other
