Concurrency Patterns in Modern C++ for High-Frequency Trading Engines

Pipelines of single-writer stages

The dominant structure is a pipeline: a feed handler thread, a strategy thread, and an order-gateway thread, each owning its state exclusively and communicating through single-producer, single-consumer ring buffers. No stage mutates another's data, so most reasoning about races disappears by construction.

SPSC buffers with power-of-two capacity, cache-line-separated head and tail indices, and acquire/release ordering give sub-hundred-nanosecond handoffs. Multi-producer variants exist but introduce contention that is rarely worth the architectural convenience.

• One owner per data structure; share by message passing, not by lock.
• Pad head and tail indices to separate cache lines.
• Prefer SPSC over MPMC unless the topology genuinely requires it.

Memory ordering used deliberately

Sequential consistency is the safe default and the wrong default on the hot path. Release on publish and acquire on consume is sufficient for handoff through a ring buffer, and relaxed ordering is appropriate for counters that no other state depends on.

The rule that keeps teams safe: any weakening below sequential consistency requires a comment naming the invariant it preserves, and a test under a thread sanitizer. Undocumented relaxed atomics are how a system develops bugs that reproduce once a quarter.

Busy-wait, affinity, and the operating system

Condition variables trade tail latency for CPU efficiency, which is the wrong trade in this domain. Hot threads spin, with a pause instruction in the loop, and are pinned to isolated cores with interrupts routed elsewhere.

Everything that can block belongs off the critical path: logging writes into a ring buffer drained by a background thread, metrics accumulate in thread-local counters, and nothing on the hot path calls into the allocator, the kernel, or a library whose implementation you have not read.

• Spin with pause on hot threads; never block on the critical path.
• Pin threads, isolate cores, and move interrupts away.
• Defer logging and metrics aggregation to background consumers.

What modern C++ contributes

Recent standards help mostly through expressiveness and safety rather than new concurrency primitives: constexpr computation moves work to compile time, span and string_view eliminate copies, and jthread with stop tokens simplifies shutdown paths that used to leak.

Coroutines are attractive for gateway and session code where readability matters and latency budgets are wider. Keeping them out of the matching path preserves the predictability that makes the engine measurable.