Step-by-Step Guide to Understanding and Using eBPF || write your first Linux kernel program (part 1)

In this tutorial , we’ll create an eBPF program that detects and mitigates a Distributed Denial of Service (DDoS) attack by monitoring incoming network traffic and blocking IP addresses that exceed a certain threshold of requests per second , you will also will learn how to use eBPF to monitor and analyze TCP connections .

https://ebpf.io/

Throughout my career, I’ve always been driven by a thirst for diving deep into the unknown, hunting down the odds, and embracing every challenge that comes my way. One of the most exhilarating challenges I encountered back in 2010 was grappling with security and DDoS attacks. Building infrastructure for massive applications like VoD (Video on Demand) services or IPTV was a monumental task that pushed me to master every nuance of backend development and security.

But today, I’m taking things to a whole new level. Forget the usual suspects — no iptables, no fil2bin, no snow, no honeypots. We’re going straight to the heart of the system, writing direct applications to the kernel. This means stronger security, greater control, and, of course, more responsibility. Let’s dive in and explore this powerful approach together!

Photo by Lukas on Unsplash

“ eBPF is a new type of software that provides superpower capabilities, birthing an industry of networking, performance, and security technologies. Netflix has pioneered uses of eBPF for observability, providing insight into countless areas that were previously difficult or prohibitively expensive to instrument. eBPF has helped us lower application latency and find cost savings “ .

Brendan Gregg — Senior performance engineer at Netflix, 2021

what you need to know :

1 — Basic knowledge of Linux and C programming.
2 — A Linux system with kernel version 4.1 or higher (for best results, use kernel 5.x or later).
3 — Installed tools: clang, llvm, bpftool, and bcc (BPF Compiler Collection).

what is eBPF ?

eBPF (extended Berkeley Packet Filter) is a powerful technology in the Linux kernel that allows for the efficient and safe execution of custom programs in response to various events, such as network packets, system calls, or tracepoints. Originally designed for filtering network packets, eBPF has evolved to support a wide range of applications, from network monitoring and security to performance profiling and observability.

Key Concepts of eBPF:

1. Bytecode Programs: eBPF programs are written in a restricted subset of C (or in other languages that compile to eBPF bytecode) and are compiled into bytecode, which is then loaded into the kernel. This bytecode is verified by the kernel to ensure safety, preventing the eBPF program from crashing the system or accessing unauthorized memory.
2. BPF Verifier: The BPF verifier is a component of the Linux kernel that ensures the safety and correctness of eBPF programs before they are loaded and executed. It checks for potential issues like infinite loops, invalid memory access, and other conditions that could compromise the stability of the system.
3. BPF Maps: BPF maps are a key-value store used to share data between eBPF programs and user-space applications. They provide a way to store and retrieve data efficiently and are used in various use cases, such as counting events, storing statistics, or passing configuration data.
4. eBPF Hooks: eBPF programs can be attached to various hooks in the kernel, such as:

• Network hooks: Monitor, filter, or manipulate network packets.
• Tracing hooks: Trace and profile kernel or user-space function calls.
• XDP (Express Data Path): High-performance packet processing at the network driver level.
• cgroup hooks: Apply policies to control resource usage or security in containers.

Use Cases:

• Networking: eBPF is widely used in networking for load balancing, DDoS protection, packet filtering, and network observability.
• Security: Tools like Cilium use eBPF to enforce fine-grained security policies in Kubernetes environments.
• Performance Monitoring: Tools like bpftrace and perf use eBPF for real-time system performance monitoring and tracing.
• Observability: eBPF is used for gathering detailed metrics and traces without significant overhead, making it ideal for production environments.

Advantages of eBPF:

• Efficiency: eBPF programs run in the kernel, minimizing the overhead associated with context switching between user space and kernel space.
• Safety: The BPF verifier ensures that eBPF programs are safe to run in the kernel, preventing potential crashes or security vulnerabilities.
• Flexibility: eBPF’s ability to attach to various kernel hooks and process data in real-time enables a wide range of applications, from networking to security to performance monitoring.

Photo by Nayam on Unsplash

what can we do with eBPF ?

eBPF is a versatile technology that can be applied across various domains, particularly in networking, security, performance monitoring, and system observability. Here are some ideas for projects or use cases that can be done using eBPF:

1. Network Security and Monitoring

• DDoS Protection: Implement real-time detection and mitigation of Distributed Denial of Service (DDoS) attacks by analyzing network traffic patterns and dropping suspicious packets.
• Intrusion Detection System (IDS): Develop an IDS that monitors for unusual network activity, such as port scanning or anomalous traffic patterns, and raises alerts or blocks the connections.
• Traffic Shaping: Implement traffic shaping and rate-limiting directly in the kernel to manage bandwidth usage for different services or users.
• Packet Filtering: Create advanced packet filtering rules that can be dynamically adjusted based on real-time conditions, such as load or time of day.
• VPN or Firewall Enhancements: Integrate eBPF into VPNs or firewalls to add custom filtering, logging, or processing rules.

2. System Performance Monitoring

• Latency Tracking: Track and visualize latency across various system components, such as disk I/O, network latency, or system call execution times.
• Real-Time Performance Profiling: Profile CPU, memory, and I/O usage in real-time, pinpointing bottlenecks or inefficient processes.
• Resource Contention Detection: Monitor for resource contention (e.g., CPU, memory) and automatically adjust resource allocation or notify administrators.
• Kernel Debugging and Tracing: Use eBPF to trace and debug specific kernel functions, helping diagnose performance issues or bugs.
• Dynamic Instrumentation: Implement dynamic tracing of user-space applications to gather performance metrics without modifying the application’s source code.

3. Security and Compliance

• Process Monitoring and Enforcement: Monitor process activities (e.g., file access, network connections) and enforce security policies, such as blocking unauthorized access attempts.
• Container Security: Implement security measures tailored for container environments, such as monitoring system calls or network traffic within containers.
• Data Leak Prevention: Monitor and block unauthorized data exfiltration attempts by tracking sensitive data flows across the system.
• Syscall Filtering and Enforcement: Implement syscall filtering for specific applications, reducing the attack surface by allowing only necessary syscalls.

4. Application-Level Monitoring

• Custom Metrics Collection: Collect custom metrics from applications (e.g., request latency, error rates) by hooking into application events or system calls.
• Database Query Monitoring: Monitor and analyze database queries in real-time to detect slow queries or abnormal patterns.
• User Behavior Analytics: Track user behavior across applications and detect anomalies, such as unusual login patterns or access to sensitive resources.

5. Networking Enhancements

• Load Balancing: Implement custom load balancing logic that routes traffic based on real-time metrics such as server load or network latency.
• Advanced NAT (Network Address Translation): Implement NAT with custom logic, such as IP-based routing or dynamic port allocation.
• Network Topology Discovery: Automatically discover network topology by monitoring traffic patterns and connections between devices.
• Service Mesh Enhancements: Integrate eBPF into a service mesh to add features like zero-trust security, traffic shaping, or custom routing policies.

6. Observability and Logging

• Enhanced Logging: Create detailed logs of system and application events, capturing additional context like stack traces or variable values.
• Trace Aggregation and Correlation: Aggregate traces from different sources (e.g., system calls, network traffic) and correlate them to identify root causes of issues.
• Custom Audit Logs: Generate custom audit logs that capture specific events of interest, such as access to sensitive files or changes in system configuration.

7. Automation and Orchestration

• Automated Remediation: Automatically trigger remediation actions based on detected anomalies or performance issues, such as restarting services or reallocating resources.
• Event-Driven Configuration: Adjust system configurations in real-time based on events, such as scaling resources up or down based on load.
• Proactive Resource Management: Implement proactive resource management strategies, such as migrating workloads before resource exhaustion occurs.

8. Development and Testing Tools

• Dynamic Testing and Fault Injection: Implement tools for dynamic testing, such as fault injection frameworks that can simulate failures in specific system components.
• Code Coverage Analysis: Use eBPF to monitor code execution paths and gather coverage data without modifying the application source code.
• Real-Time Debugging: Develop real-time debugging tools that can attach to running processes and capture debug information without stopping the process.

9. IoT and Edge Computing

• Lightweight Security Enforcement: Implement lightweight security measures for IoT devices, such as filtering unauthorized network traffic or monitoring critical processes.
• Edge Data Processing: Perform data processing at the edge, such as filtering, aggregation, or compression, before sending data to the cloud.
• Resource-Constrained Monitoring: Implement resource-efficient monitoring solutions for IoT devices, focusing on minimal CPU and memory usage.

10. Advanced Network Virtualization

• Network Function Virtualization (NFV): Implement virtualized network functions, such as firewalls, load balancers, or intrusion prevention systems, using eBPF.
• Custom Virtual Networks: Create custom virtual networks with specific routing, filtering, or QoS (Quality of Service) policies enforced by eBPF programs.

Photo by Lewis Kang'ethe Ngugi on Unsplash

“Hello world eBPF “ program

1 . Setting up the environment : check your kernel i use kernel 6.5
flow installation reference that match your Linux kernel .

sudo apt-get update
sudo apt install linux-headers-$(uname -r)
sudo apt install libbpfcc-dev
sudo apt install libbpf-dev
sudo apt install llvm
sudo apt install clang
sudo apt install gcc-multilib
sudo apt install build-essential
sudo apt install linux-tools-$(uname -r)

sudo apt install linux-tools-common \
                 linux-tools-generic \
                 linux-tools-$(uname -r) 
sudo apt-get install bpftrace

Verify installation

sudo bpftool version

Writing Your First eBPF Program

Let’s start by writing a simple eBPF program that logs network packets.

Create a C file named my_packet_filter.c :

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

SEC("xdp")
int packet_filter(struct xdp_md *ctx) {
    bpf_printk("Packet received!\n");
    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

now Compile your program for bpf :

change file name to your project file name

 clang -O2 -target bpf -c my_packet_filter.c -o my_packet_filter.o

The -target bpf flag tells clang to compile the program for the eBPF virtual machine.

Mount BPF Filesystem (if not already mounted):

sudo mount -t bpf bpf /sys/fs/bpf

Load and Attach Using bpftool:

in this step we will load our program to kernel and attach it to network interface using “bpftool”

sudo bpftool prog load hello_world.o /sys/fs/bpf/hello_world
sudo bpftool prog attach /sys/fs/bpf/hello_world tracepoint syscalls/sys_enter_execve

Replace <network-interface> with the name of your network interface, such as eth0. or eno1 or any interface you have you can use “ifconfig” to list all network interfaces

verify that your program is running:

sudo bpftool prog show

View the kernel trace messages:

sudo cat /sys/kernel/debug/tracing/trace_pipe

you will see some thing like that

ksoftirqd/1-23      [001] ..s11 111732.603555: bpf_trace_printk: Packet received!

 Isolated Web Co-115970  [001] ..s11 111732.605179: bpf_trace_printk: Packet received!

          <idle>-0       [001] ..s21 111732.606038: bpf_trace_printk: Packet received!

      Compositor-137109  [001] ..s21 111732.608672: bpf_trace_printk: Packet received!

          <idle>-0       [001] ..s21 111732.609666: bpf_trace_printk: Packet received!

          <idle>-0       [001] ..s21 111732.610795: bpf_trace_printk: Packet received!

to Detach Existing XDP Program !

sudo bpftool net detach xdp dev <network-interface>

On the next part we will create a more advanced, real-world example using eBPF to monitor and analyze TCP connections. I will demonstrate how to track TCP connection durations and sizes, which can be useful for network performance analysis and troubleshooting.

happay coding and have a nice day .