A network stack mediates data flow between applications and a physical network. Its main parts are the network interface card (NIC) device driver, protocol implementations, and application-visible socket API. Because packet processing happens within a node, the stack can be a significant source of intranode latency. section 3.3.1 section 3.3
A message crosses several protocol layers. Hardware commonly handles Ethernet, the OS kernel handles TCP/IP, and the application handles a higher-level protocol such as HTTP. Moving between those layers is work; on a multicore system, colocating protocol handlers can reduce latency in some cases. section 3.3.1
Trace a message in both directions before treating a socket call as the whole network operation:
- On receipt, packets enter a NIC receive queue. The kernel driver manages the NIC and notifies the OS of available data; Linux polls for packets rather than raising an interrupt for every arrival. section 3.3.1
- On sending, an application creates a socket, establishes a TCP connection with connect() when needed, and passes its message to the OS with a call such as sendmsg(). The OS may queue it and transmit it asynchronously; Nagle’s algorithm may also defer sending when enabled. section 7.5.3
- On receiving, recvmsg() or a similar call checks kernel data structures for arrived messages. OS background processing handles packets that can arrive at any time. section 7.5.3
Sending an HTTP request
The application produces the HTTP request in userspace and hands it to the OS through a socket call. The OS performs lower-level TCP processing. A return from sendmsg() therefore does not mean that all network-stack transmission work is finished; some of that work may still interfere with the application thread. section 7.5.3
Figure 3.3 Packets traversing the network stack
Follow the packet path from the NIC through the stack toward the application to locate where receive-side processing occurs. section 3.3.1
