Getting Hands Dirty with Intel Cache Allocation Technology

222mzb
3 min readApr 26, 2022

--

Intel has introduced cache allocation technology to ensure controlled allocation of last level cache among all the applications which allows better quality of service. Suppose and application is highly memory intensive but there are other applications more important than that. The important application will get less priority in LLC as LLC is shared among all cores mostly.

Intel has come up with a very interesting idea. They have assigned bitmask to represent the cache and divided the LLC into certain number of Class of Service (COS). The #COS is CPU specific and you can divide the cache ways into separate COS. In each COS, you can assign app(using PID), VM, thread, containers etc. Thus, you can assign more amount of cache for a specific application or a group of applications.

Figure 1: A sample bitmask

In Figure 1 we see CLOS[3] (i.e. COS 3) has most amount of cache ways (or amount of cache) assigned. Let’s jump to the examples and installation.

Installation

Installation is pretty easy and straight forward. For easiest installation you can run

On Ubuntu: sudo apt-get install intel-cmt-cat

On CentOS 7: sudo yum install intel-cmt-cat

Sample Operations

The first we need to check is CAT capability. Here is a screenshot of my setup when I run “sudo pqos -s -V”

Figure 2: Sample output of pqos -s -V

Here we see it is shown that CAT capability detected. There are several ways to detect CAT capability:

  1. Brand String
  2. CPUID
  3. MSR (Model Specific Register)

First it checks CPU brand string, if it does not match with the cpu names that are already proven to have CAT, it shows CAT is not detected. However, MSR may show CAT is detected. As the image below on a intel corei5 7 gen processor. So, if brand string fails it doesn’t mean the CPU does not support CAT at all!

Figure 3: Although Brand String may show CAT incapability, it may be supported by MSR!

You can access CAT feature using MSR or kernel interface. To assign COS, you must know the number of ways because, it will describe the bitmask. For example in Figure 3, #COS=4 and #ways=12. Than means LLC has 12 ways to be assigned and you can divide them into 4 partitions. Suppose you want to set COS0 to last 3 ways, then you should use:

pqos -e “llc:0=0x0007;”

As 7 = 111 in binary.

You can assign multiple COS by just adding same command after semicolone.

To associate a Core with a specific COS, we can use following command

pqos -a “llc:0=0;llc:1=1–5;”

It states, COS 0 will be associated with core 0 and core 1–5 will take place in COS 1. the syntax is “llc:[COS no.] = [core no]

Besides this MSR method, we can do all the task using OS/kernel interface and even more!

For allocating COS using kernel interface:

pqos -I -e “llc:1=0x0007;

And most of the commands are similar.

Besides the regular task, we can assign PID to specific COS using kernel interface.

pqos -I -a “pid:1=1234;pid:2=5678;”

The above command will assign PID 1234 to COS 1 and PID 5678 to COS 2.

Besides assuring quality of service, CAT also provides this partition and control from kernel side which is also used as a defense mechanism against side channel attack.

References:

  1. https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-cache-allocation-technology.html
  2. https://github.com/intel/intel-cmt-cat/wiki/Usage-Examples

--

--