Systems and low latency

6 lessons · 1 equations. Each lesson below gives its formulas and key rules; open the lesson for the full explanation.

Architecture: caches, branch prediction and SIMD

Average memory access time

\text{AMAT} = t_{\text{hit}} + m \times t_{\text{miss}}

The time of a cache hit plus the miss rate times the penalty of going further out. Because the penalty is a hundred times the hit time, a small miss rate dominates.

Remember

Main memory is roughly a hundred times slower than L1.

C++ for trading: RAII, moves and what belongs on the hot path

Key rules

RAII ties resources to scope, and destruction is deterministic.
A move transfers ownership in $O(1)$ ; std::move is a cast, not an action.
No allocation on the hot path — its worst case is unbounded.

Concurrency: atomics, memory ordering and lock-free queues

Key rules

A data race is undefined behaviour, not merely an unpredictable value.
Locks are fine uncontended and fatal for tail latency when contended.
Release–acquire pairing publishes data along with the index that points at it.

Networking: multicast market data, kernel bypass and gap recovery

Key rules

Market data is UDP multicast: one stream, no head-of-line blocking.
Order entry is TCP: guaranteed delivery to a single destination.
Sequence numbers make gaps detectable; recovery is out of band.

Latency, the memory hierarchy and why the tail is the number

Key rules

Preallocate. No allocation, no resizing, no locks on the path.
Keep data contiguous and hot: arrays of structs the loop actually reads, not pointer chases.
Avoid false sharing — two threads writing different variables on one 64-byte cache line serialise.

Order book engineering: data structures and the operations they serve

Key rules

Cancels dominate, so cancellation by id must be $O(1)$ .
Array of price levels, intrusive FIFO list per level, hash map from id to node.
A dense bounded tick grid makes an array beat a tree.