TCP SYN Flooding Attack

5 min readAug 27, 2023

TCP Three-Way Handshaking:

TCP handshaking is a three-way process between the client and server when they want to communicate with each other they follow the below process:

First, the client requests for connection by sending the SYN (synchronize) message to the server.
Secondly, the server acknowledges the request by sending a SYN-ACK (synchronize acknowledge) message back to the client.
The last client responds with an ACK (acknowledge) message and the connection is established between them

SYN Flooding Attack:

SYN flood is a form of DoS attack in which attackers send many SYN requests to a victim’s TCP port, but the attackers have no intention to finish the 3-way handshake procedure. Attackers either use spoofed IP addresses or do not continue the procedure. Through this attack, attackers can flood the victim’s queue that is used for half-opened connections, i.e. the connections that have finished SYN, and SYN-ACK, but have not yet got a final ACK back. When this queue is full, the victim cannot make any more connections.

The size of the queue has a system-wide setting. In Linux, we can check the system queue size setting using the following command:

sysctl -q net.ipv4.tcp_max_syn_backlog

We can use the command “netstat -tna” to check the usage of the queue, i.e., the number of half-opened connections associated with a listening port. The state for such connections is SYN-RECV. If the 3-way handshake is finished, the state of the connections will be ESTABLISHED.

SYN Cookie Countermeasure:

SYN cookie is a defense mechanism to counter the SYN flooding attack. Using SYN cookies allows a server to avoid dropping connections when the SYN queue fills up.

Instead, the server behaves as if the SYN queue had been enlarged.
The server sends back the appropriate SYN+ACK response to the client but discards the SYN queue entry.

If the server then receives a subsequent ACK response from the client, the server is able to reconstruct the SYN queue entry using information encoded in the TCP sequence number.

You can use the sysctl command to turn on/off the SYN cookie mechanism:

sysctl -a | grep cookie (Display the SYN cookie flag)
sysctl -w net.ipv4.tcp_syncookies = 0 (turn off SYN cookie)
sysctl -w net.ipv4.tcp_syncookies = 1 (turn on SYN cookie)

When the SYN cookie value becomes zero then issue the following command from the attacker machine to start SYN flooding:

sudo netwox 76 -- dst-ip <ipaddress> -- dst-port <portnumber>

Task: SYN Flooding Attack

Environment Setup:

SEED Ubuntu 16.04 pre-built VM will be used for this lab, which can be downloaded from the SEED website. Three Machines VM1 (Attacker), VM2 (Server), and VM3 (Client) are used in this lab.

Attacker’s IP: 10.0.2.15

Victim’s IP: 10.0.2.4

Client’s IP: 10.0.2.5

Establishing a telnet connecting from Victim to Client:

Establishing a telnet connection from the Victim’s machine to the client’s machine using the following command:

telnet 10.0.2.5

The credentials are as follows:

VM Login: seed

Password: dees

The screenshot attached below shows that the login was successful.

Turning OFF SYN Cookie:

SYN cookies are a technical attack mitigation technique whereby the server replies to TCP SYN requests with crafted SYN-ACKs, without inserting a new record into its SYN Queue. By default, the syn cookies protection is enabled in the SEED Ubuntu. For this attack, we have to disable this using the following commands and check whether it is set to 0.

sudo sysctl -a | grep cookie
sudo sysctl -w net.ipv4.tcp_syncookies=0

We have to set this to 0 in all the VMs.

Attacker’s machine:

Victim’s machine:

Client’s Machine:

Netstat command:

Execute the following command to check the state of the available ports on the client machine.

netstat -tna

If the ports are in a listening state, then they haven’t established a connection yet. We have to convert the LISTEN state to the HALF-CONNECTION or SYN-RECV state of about 90% of the ports present in the machine.