Kubernetes resource limits and kernel cgroups

Kubernetes resource limit

apiVersion: v1
kind: Pod
metadata:
name: busybox1
labels:
app: busybox1
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
resources:
limits:
memory: "20Mi"

restartPolicy: Always
CreateContainer(podSandboxID string, 
config *runtimeapi.ContainerConfig,
sandboxConfig *runtimeapi.PodSandboxConfig)
(string, error)
type LinuxContainerResources struct {
<<..>>
// Memory limit in bytes. Default: 0 (not specified).
MemoryLimitInBytes int64
<<…>>
}

Container runtime

% cat busybox-resources.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox0
labels:
app: busybox0
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
resources:
requests:
memory: "10Mi"
cpu: "250m"
limits:
memory: "64Mi"
cpu: "500m"
restartPolicy: Always
% pwd
/sys/fs/cgroup/memory/kubepods/burstable/pod2d42976a-6d2c-4d1e-aa52-c0fc8e3964a5/9aa8bbbe72708633daf2ee74246be0d3b965a3a2878e46e12fe0b29df34fb3db
% cat memory.limit_in_bytes
67108864

Cgroups memory limits and Linux kernel

#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#define PAGE (4*1024)
int get_key() {
printf(“\nPress any key to continue…\n”);
return getc(stdin);
}
int main(int argc, char **argv) {
char c __attribute__((unused));
unsigned char *p;
int count=0;
if (argc < 2) {
printf(“Usage: mem <pages to allocate>\n”);
return -1;
}
int alloc_mem = atoi(argv[1])*PAGE;
printf(“Pid: %d. \n”, getpid());
printf(“Page allocation requested: %d.\n”, alloc_mem);
printf(“Yet to call malloc.\n”);
c = get_key();
p = malloc(alloc_mem);
printf(“Malloc called. No writes yet.”);
c = get_key();
for (int i=0; i<alloc_mem ; i++) {
p[i] = 1;
}
for (int i=0; i<alloc_mem ; i++) {
if (p[i] == 1) count++;
}
printf(“Alloc in bytes: %d\n”, alloc_mem);
printf(“Page count: %d\n”, count/PAGE);
c = get_key();
}
% ./mem 21
Pid: 3088.
Page allocation requested: 86016.
Yet to call malloc.
Press any key to continue…
% bpftrace -e ‘kretprobe:try_charge /pid == 3088/ { @ret[retval] = count(); @[kstack]=count(); }’
Attaching 1 probe…
% mkdir cgroup-mem-demo
% cd cgroup-mem-demo/
% echo 3088 > cgroup.procs
% echo 81920 > memory.limit_in_bytes
Malloc called. No writes yet.
Press any key to continue…
Killed
@[
kretprobe_trampoline+0
__handle_mm_fault+2270
handle_mm_fault+177
__do_page_fault+641
do_page_fault+46
do_async_page_fault+81
async_page_fault+69
]: 21
@ret[4294967284]: 1
@ret[0]: 20

Summary

Acknowledgements

https://twitter.com/kkwriting

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Firestore: Copy Data from Prod to Dev or Emulator

Sprint 2 — Acebook, JIMJI team

Preparing PluFin for Launch: Sprint 3

</ OverTheWire > Bandit Level 12 → Level 13

DNS resolution works in host but not from Kubernetes Pod

Webinar Recording: Start to modernise decades of complex legacy systems

The OSI Model

Sanitizing your SNS Subscriptions for cross-region traffic

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Krishnakumar R

Krishnakumar R

https://twitter.com/kkwriting

More from Medium

Creating HTTPS/TLS Route by HELM inOpenshift/OCP

Zero Downtime Deployment of BusinessWorks Container Edition with Kubernetes — Part 4

Stateful app DR with Portworx and Rancher

Istio Service Mesh in Kubernetes and Auto Injecting It in the Kubernetes Pod as Side Car Container…