Networking: multicast market data, kernel bypass and gap recovery

SYS · Chapter 412 min readAsked at Hudson River Trading, Jump, IMC, Optiver

Assumes Concurrency: atomics, memory ordering and lock-free queues.

After this lesson you should be able to

  • Say why market data uses UDP and order entry uses TCP.
  • Describe how a receiver detects and recovers a gap.
  • Explain what kernel bypass removes.

The network is where a trading system spends most of its latency, and the design choices are all consequences of one asymmetry: market data goes to everyone and must never stall, while order entry goes to one place and must never be lost.

AspectTCPUDP multicast
DeliveryOrdered and guaranteedBest effort
Fan-outOne connection per subscriberOne stream, replicated by the network
On packet lossRetransmits and blocks everything behind itYou detect the gap and continue
Head-of-line blockingYesNo
Used forOrder entry, recovery channelsMarket data feeds
Table 4.1 · TCP against UDP multicast. Head-of-line blocking is the decisive row. A TCP feed that loses a packet stops delivering *newer* data until the old one is retransmitted — so a subscriber who would rather have fresh prices with a hole in them than stale prices in order cannot have them.

Proposition 4.2

Sequence numbers and gap recovery

Every message on a feed carries a sequence number, so a receiver detects a gap immediately by arithmetic. Recovery then happens out of band: request the missing range from a replay service, or fail over to the secondary feed that carries the same data on a different path.

Holds when

  • Exchanges publish A and B feeds over separate infrastructure; arbitrating between them fills most gaps with no request at all.
  • A snapshot channel periodically publishes the full book state, so a receiver can recover without replaying everything.
  • Mark the book stale while recovering rather than quoting off incomplete data.

What kernel bypass actually removes. A normal receive path copies the packet from the network card into kernel memory, wakes your thread, context-switches into user space and copies again. That is several microseconds, and — worse — it is *variable*, because the wake-up depends on what else the scheduler is doing. Kernel bypass maps the card’s buffers directly into the application and polls them, so the packet is read where it landed with no copy, no syscall and no scheduler involvement. The gain is a few microseconds of mean latency and a much larger reduction in jitter, which is the part that actually matters.

Kernel network stack10Kernel bypass1FPGA on the NIC0.1
Figure 4.3 · Where the microseconds go, wire to application. The same packet, three paths. Bypassing the kernel removes a copy, a context switch and the scheduler’s opinion about when you should run; putting the logic on the card removes the host entirely. Each step costs an order of magnitude more engineering than the last.

The rest of this lesson is in Premium

You have read the opening. 11 more sections follow, including 5 worked examples and 3 quick checks.

Start the free 7-day trialSign in

Nothing is charged for 7 days, and you can cancel before then. Or read Complexity: reading it off, and deriving it in full, free.