From Sockets to NIC: A Big Picture
Studying network programming, I understood how to use Linux sockets in order to communicate with other systems/computers. I also studied a lot of higher level technologies built on top of sockets such as http, ssh, ftp.
I wrote this blog post while improving my understanding of Linux kernel networking features, from a socket to a NIC.
Plan
I) The Network Interface Controller (NIC)
II) The Network Interface
III) From socket to NIC
IV) Network interface functions
I) The Network Interface Controller (NIC)
According to Wikipedia; a NIC is a hardware component connecting a computer to a computer network.
If your computer doesn’t have a NIC, you cannot communicate with other computers.
Indeed you are isolated from the rest of the world (private and public networks).
So the first thing to do is to plug a NIC on the motherboard.
(If you want to connect to the internet thanks to wifi, you’ll need to plug a second NIC: a wifi NIC).
NICs are connected to the computer thanks to the PCI ports of the motherboard.
On this Linux computer I only have 1 NIC plugged (wireless):
The command above shows us that a NIC device (also called adapter) is plugged on the motherboard.
It is not possible to use it directly for the moment. A network driver must be installed and activated on our computer.
II) The Network Interface
“Linux Network Interface” is a category of drivers made for networking communications.
The Linux network interface defines a set of functions which must be implemented by the driver in order to be used by the kernel as a network driver.
This concept of interface allows the Linux kernel to use networking functions without knowing much of the real implementation made by the driver.
Among every existing drivers, most implement either:
- The “char device interface” (e.g: mouse, keyboard).
A driver implementing this interface is available as a special file located somewhere in the “/dev” folder. - The “block device interface” (e.g: disks).
A driver implementing this interface is available as a special file located somewhere in the “/dev” folder. - The “network device interface” (the interesting one in our case).
A driver implementing this interface is different from the previous ones as it doesn’t have a “file” entry in our filesystem.
Network drivers have to implement the Network device interface to be considered as “network interfaces” by the OS.
The “ifconfig” linux command shows our current activated network interfaces.
As you can see I’ve got 2 network interfaces.
- ‘lo’: this is a virtual network interface. The word virtual highlights the fact it is not attached to any physical device. This network driver is also called ‘loopback’ and is used to provide localhost functionality.
- ‘wlp1s0’: this is the network interface of my physical wireless NIC.
Interesting point to highlight
A virtual network interface implementation can rely on another network interface (a physical one for instance).
Great networking features are built on top of that: VPN virtual network interfaces, VM virtual network interfaces …
III) From socket to NIC
A Network interface implements multiple functions which are used by the OS kernel to send packets through the network.
These functions are not directly called by the user from the user space but by the kernel as the last step to send the packets to the destination.
Here is global and simplified “socket-to-nic” communication scheme:
If you only have 2 linux low-level struct to remember regarding networking it would be:
- “sk_buff” : represents a “network packet” of the different layers of the OSI model. Indeed, sk_buff struct includes transport header (layer 4), network header (layer 3), mac header (layer 2).
- “net_device”: it is the network interface abstraction used by the kernel. Every network interface should implement its methods. Lot of interesting attributes are embedded in it such as the ip address(es) of the card (‘ip_ptr’), the interface name, hardware information, the assigned interrupt number, the dma zone, interface flags, MTU, …
IV) Network interface functions
A network interface should implement several functions for the kernel.
Most important ones are listed below (parameters are simplified).
open(…)
Called to enable the interface (‘ifconfig [interface_name] up’ in a terminal).
This call also:
- set up interrupt handlers and interruption request numbers (IRQ)
- set up DMA where packets will be received.
stop(…)
Called to disable the interface (‘ifconfig [interface_name] down”)
get_stats(…)
Called to fetch back statistics regarding this specific network interface (used by “ifconfig”)
hard_start_xmit(sk_buff packet)
Transmission function used to send packets to the device.
Network packets to be sent will be passed to this function thanks to the universal linux kernel packet structure “sk_buff”.
Some sanity checks will be performed on the packet before sending it out.
NIC stats will then be updated.
Once this function has been used, the packets have left the machine.
“dev_rx(…)”
Reception function called by the NIC interrupt handler (after a hardware interrupt has occurred) in order to ‘serialize’ data into a sk_buff structure. This function will end with a call to “netif_rx(skb)”. This last call aims to transmit the data (sk_buffer skb) to the upper layer in the kernel which will then handle it and route it to the proper socket.
Conclusion
The idea of writing this post came while reading the O’reilly Linux Device Driver book.
Indeed I was a bit frustrated and not satisfied by my understanding of network programming which stopped at C sockets. By doing some research for this post I learnt a lot of things and it kind of “demystified” how the linux kernel handles the implementation of the OSI model in relation with the Network interface.
I’ll use this little post (and the scheme) to keep a trace of what I learnt but also to share it with anyone who may be interested :)
BTW if you have anything to add or any suggestion don’t hesitate to let me know !
Follow me on twitter :-)
References
- O’Reilly Linux Device Driver 3rd edition (Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman)
- Multiple sections (socket, drivers, networking) of www.kernel.org
- Path of a packet in the Linux kernel stack
