Every time someone taps “Buy” on a stock trading app, a chain of events fires that most people never think about. Somewhere, a computer system has to look at that order, compare it against every other order waiting to be matched, decide instantly whether a trade can happen, and record it forever in a way nobody can dispute later. This entire chain has to finish in a fraction of a millisecond, thousands of times a second, without ever losing an order or matching things in the wrong sequence.

The system that does this job is called a matching engine, and the platform built around it is what we call a stock trading platform. This is one of the hardest problems in software architecture because it combines three things that don’t usually go together: extreme speed, perfect correctness, and zero tolerance for data loss.

A Short History

Stock exchanges started centuries ago as physical rooms where traders shouted prices at each other — this is called “open outcry.” A human broker would literally yell “I’ll buy 100 shares at 50!” and another would yell back a price, and a deal would be struck by eye contact and a nod. This worked when trading volumes were small.

As the number of trades exploded through the 1980s and 1990s, exchanges moved to electronic systems. The New York Stock Exchange, Nasdaq, the London Stock Exchange, and later the National Stock Exchange and BSE in India all built computerized matching engines to replace or augment the trading floor. By the 2000s, “high frequency trading” firms started competing on microseconds, and matching engines had to get faster and faster to stay fair and useful. Today, the fastest matching engines process orders in single-digit microseconds — far below one millisecond.

Why is this hard? Let’s break down the actual requirements a real stock trading platform must satisfy, and why each one is difficult.

2.1 Requirement: Strict Ordering Guarantees

If two people both want to buy the last 100 shares of a stock at the best price, and orders come in one millisecond apart, the platform must always give the trade to whoever’s order arrived first — not whoever’s request happened to get processed first due to some random network delay or thread scheduling quirk. This is called price-time priority and it is a legal and regulatory requirement in most markets, not just a nice-to-have.

This means the system cannot simply process orders on multiple threads or multiple machines in parallel without a rule for who “wins” when a race occurs. Ordering must be deterministic and auditable.

2.2 Requirement: Sub-Millisecond Matching Latency

Modern trading platforms are expected to match an eligible order against the order book in well under one millisecond — often in tens of microseconds. This rules out many “normal” software design choices: garbage-collected languages with unpredictable pause times, network calls in the hot path, disk writes that block the caller, and even ordinary locks that cause thread contention.

2.3 Requirement: Massive Throughput

The system must handle a scenario of millions of requests per minute — that works out to roughly 16,000 to 20,000+ requests per second sustained, with bursts far higher during market open, market close, or major news events (sometimes 5 to 10 times the average rate in a few seconds).

2.4 Requirement: Zero Data Loss and Perfect Auditability

Every single order, cancellation, and trade must be durably recorded. If the matching engine crashes one microsecond after accepting an order, that order cannot simply disappear — regulators and customers will ask for it later, sometimes years later.

2.5 Requirement: Determinism and Replayability

Beyond just recording data, regulators and internal risk teams frequently need to replay a sequence of historical orders and get exactly the same trades out, byte for byte, as happened live. This means the matching logic itself must be completely deterministic — no reliance on random tie-breaking, no dependence on wall-clock time for decisions (only on the sequencer’s assigned order), and no floating-point rounding differences between runs. This requirement quietly shapes many low-level implementation choices, such as using fixed-point integer arithmetic for prices instead of floating-point numbers.

Let’s now design the full system, box by box. Every component below is labeled clearly so you can see exactly what role it plays, including the load balancer and API gateway the platform needs at its edge.

Component-by-Component Breakdown

Load Balancer

The Load Balancer sits at the very front of the system. It is a Layer 4 (TCP level) load balancer, often using anycast IP addressing, so that a client’s connection is automatically routed to the nearest healthy data center. For a trading platform, the load balancer must add almost no latency — typically under 100 microseconds — and must support very fast health checks so it can pull a failing node out of rotation within milliseconds.

API Gateway

The API Gateway is the single entry point for all client traffic after the load balancer. It handles authentication (is this really who they say they are), authorization (are they allowed to trade this instrument), rate limiting (stop one client from flooding the system), request validation (is this even a well-formed order), and protocol translation (converting REST/WebSocket calls from retail apps and FIX protocol messages from institutional clients into one internal format).

Order Management Service (OMS)

This service receives the validated order and enriches it — attaching account information, instrument metadata, and timestamps. It is the “front office” brain that decides an order is well-formed enough to proceed further.

Risk Check Service

Before any order reaches the matching engine, it must pass pre-trade risk checks: does the account have enough buying power, is the order size within allowed limits, does it violate any regulatory circuit breakers. This check must also be extremely fast (single-digit microseconds), so it’s usually backed by in-memory account state, not a database round trip.

Sequencer Service

This is one of the most important and most overlooked components. The Sequencer assigns a strictly increasing, globally unique sequence number to every incoming order before it is handed to a matching engine shard. This sequence number is what guarantees deterministic ordering — even if two orders for the same stock arrive at nearly the same instant from different gateway nodes, the sequencer decides, once and for all, which one is “first.”

Matching Engine (Core)

This is the heart of the system. Each matching engine instance owns the order book for a specific set of stock symbols and runs as a single-threaded event loop per symbol shard. This sounds counter-intuitive — why not use many threads? — but it is precisely how sub-millisecond, correctness-guaranteed matching is achieved. We cover this in detail in the Internal Working section.

Write Ahead Log (WAL) Store

Before the matching engine even processes an order, or immediately after (depending on the durability model chosen), the event is appended to a durable, append-only log. If the matching engine process crashes, it can replay this log to rebuild the exact order book state it had before the crash.

Market Data Publisher

Every trade, and every change to the order book, must be broadcast out to all subscribers — other trading systems, market data vendors, and the exchange’s own public feeds — with minimal delay. This uses a fan-out/multicast pattern.

Message Queue

A durable commit log style message queue decouples the matching engine from slower downstream consumers like clearing, settlement, compliance reporting, and analytics, so those systems never slow down the matching engine itself.

Clearing and Settlement Service

After a trade is matched, someone has to actually move the shares and money between accounts (this can take T+1 or T+2 days in many real markets). This service handles that “back office” process asynchronously.

Databases and Caches

A relational database stores account, position, and reference data (this doesn’t need microsecond latency). A time-series database stores every trade tick for historical queries, charting, and compliance. An in-memory cache stores a fast-access snapshot of the current order book for read-only queries (like “show me the current best bid/ask”) without hitting the matching engine’s hot path.

Monitoring Service

Collects metrics, logs, and distributed traces from every component so operators can see latency percentiles, error rates, and system health in real time.

Now let’s go inside the matching engine itself and understand exactly how it decides which orders trade with which.

4.1 The Order Book Data Structure

An order book has two sides: bids (buy orders) and asks (sell orders). Each side needs to answer, extremely fast: “what is the best available price right now, and who is waiting at that price, in what order did they arrive?”

The classic data structure is:

• A price level map — often a balanced tree or a sorted array/skip-list, keyed by price, so the best price (highest bid, lowest ask) can be found in O(log n) or even O(1) with clever bucketing for common tick sizes.
• At each price level, a FIFO queue of orders, so that within the same price, whoever arrived first gets matched first (time priority).

4.2 Why Single-Threaded Wins

This is the single most important architectural decision in a matching engine, and it surprises many engineers coming from a “scale with more threads” mindset.

Why does this work at scale? Because we don’t need one giant thread handling every stock — we shard by symbol. AAPL’s order book and TSLA’s order book have zero business rules connecting them, so they can each be owned by a separate single-threaded matching engine instance, running on a separate CPU core, completely independently. This gives us both correctness (no locking races within a symbol) and horizontal scalability (add more shards/cores for more symbols).

4.3 The Matching Algorithm Itself

When a new order arrives at its shard, the matching engine follows this loop:

1. Look at the opposite side of the book (a buy order looks at asks, a sell order looks at bids).
2. Find the best price on that opposite side.
3. If the incoming order’s price crosses that best price (a buy at or above the best ask, or a sell at or below the best bid), a trade can happen.
4. Match against the oldest order at that price level first (time priority).
5. Reduce both orders’ remaining quantity by the matched amount. If an order is fully filled, remove it from the book.
6. Repeat until the incoming order is fully filled, or no more crossing prices exist — at which point any remaining quantity rests in the book as a new resting order.

Java Example: Price-Time Priority Order Book

public class OrderBook {
    // Bids: highest price first. Asks: lowest price first.
    private final TreeMap<Long, ArrayDeque<Order>> bids =
        new TreeMap<>(Comparator.reverseOrder());
    private final TreeMap<Long, ArrayDeque<Order>> asks = new TreeMap<>();

    // Called from a single dedicated thread only - no locks needed.
    public List<Trade> submit(Order incoming) {
        List<Trade> trades = new ArrayList<>();
        TreeMap<Long, ArrayDeque<Order>> opposite =
            incoming.isBuy() ? asks : bids;

        while (incoming.remainingQty() > 0 && !opposite.isEmpty()) {
            Map.Entry<Long, ArrayDeque<Order>> bestLevel = opposite.firstEntry();
            long bestPrice = bestLevel.getKey();

            boolean crosses = incoming.isBuy()
                ? incoming.limitPrice() >= bestPrice
                : incoming.limitPrice() <= bestPrice;
            if (!crosses) break;

            ArrayDeque<Order> queue = bestLevel.getValue();
            Order resting = queue.peekFirst();

            long fillQty = Math.min(incoming.remainingQty(), resting.remainingQty());
            trades.add(new Trade(incoming.id(), resting.id(), bestPrice, fillQty));

            incoming.reduceQty(fillQty);
            resting.reduceQty(fillQty);
            if (resting.remainingQty() == 0) queue.pollFirst();
            if (queue.isEmpty()) opposite.remove(bestPrice);
        }

        if (incoming.remainingQty() > 0) {
            TreeMap<Long, ArrayDeque<Order>> same =
                incoming.isBuy() ? bids : asks;
            same.computeIfAbsent(incoming.limitPrice(), k -> new ArrayDeque<>())
                .addLast(incoming);
        }
        return trades;
    }
}

Notice there are no locks anywhere in this code. That’s only safe because exactly one thread ever calls submit() for this particular order book instance. This is the core trick behind achieving microsecond-level latency: eliminate contention by eliminating sharing.

4.4 Getting Orders Into a Single-Threaded Engine Without Blocking It

If only one thread is allowed to touch the order book, how do dozens of network I/O threads (handling thousands of incoming client connections) hand orders off to it without causing contention? The answer is a lock-free multi-producer, single-consumer ring buffer (often implemented using something like the LMAX Disruptor pattern in Java).

Network threads write incoming order events into pre-allocated slots in a circular buffer using atomic compare-and-swap operations to claim a slot — no traditional mutex is involved. The single matching thread reads from this ring buffer in a tight spin-loop, picking up new events the instant they appear, often within a few hundred nanoseconds.

Java Example: Lock-Free Intake Queue

public class OrderRingBuffer {
    private final Order[] slots;
    private final int mask;
    private final AtomicLong writeCursor = new AtomicLong(0);
    private volatile long readCursor = 0;

    public OrderRingBuffer(int capacityPowerOfTwo) {
        this.slots = new Order[capacityPowerOfTwo];
        this.mask = capacityPowerOfTwo - 1;
    }

    // Called by many network I/O threads concurrently
    public void publish(Order order) {
        long slot = writeCursor.getAndIncrement();
        slots[(int) (slot & mask)] = order;
    }

    // Called only by the single matching engine thread
    public Order poll() {
        long available = writeCursor.get();
        if (readCursor >= available) return null; // nothing new yet
        Order next = slots[(int) (readCursor & mask)];
        readCursor++;
        return next;
    }
}

Notice this design deliberately avoids synchronized blocks and traditional locks. The AtomicLong compare-and-swap operation is the only synchronization primitive, and it is extremely cheap on modern CPUs — typically tens of nanoseconds, not microseconds.

4.5 Memory Layout and Cache Efficiency

At microsecond-scale latency budgets, even CPU cache misses matter. A cache miss that goes all the way to main memory can cost 100+ nanoseconds — a meaningful fraction of the entire matching budget. Production matching engines pay close attention to:

• Data locality: storing an order book’s price levels and order queues in contiguous memory (arrays) rather than scattered objects linked by pointers, so the CPU can prefetch efficiently
• False sharing avoidance: padding frequently-updated variables (like the ring buffer’s read/write cursors) onto separate cache lines, so that one CPU core updating its cursor doesn’t invalidate another core’s cache unnecessarily
• Object pooling: reusing pre-allocated Order and Trade objects instead of creating new ones per event, which avoids triggering garbage collection pauses in managed-memory languages like Java

4.6 Handling Order Cancellations and Modifications

Cancels and modifies (“amend this order’s price or quantity”) must also go through the exact same single-threaded, sequenced path as new orders — otherwise a cancel could race against a match and either arrive too late (client gets an unexpected fill) or too early (a valid trade gets skipped). The matching engine typically maintains an index from order ID to its exact position in the price-level queue, so a cancel is an O(1) or O(log n) removal, not a linear scan through the book.

Let’s trace exactly what happens to one order, end to end, including where durability and sequencing occur.

Order States

An order moves through a well-defined set of states during its life:

Every architectural decision in a system like this involves giving something up in exchange for something else — there is no version of this design that is simultaneously the fastest, the cheapest, and the simplest to operate. The table below makes these trade-offs explicit, which is exactly the kind of reasoning worth walking through out loud in a system design interview, since interviewers are usually more interested in why a decision was made than in the decision itself.

6.1 Key Trade-offs Table

6.2 CAP Theorem in the Context of a Matching Engine

The CAP theorem states that a distributed system can only guarantee two out of three properties during a network partition: Consistency (every read sees the latest write), Availability (every request gets a response), and Partition tolerance (the system keeps working despite network failures between nodes).

A matching engine sidesteps much of this dilemma cleverly: by keeping each order book’s authoritative state on a single node (not distributed across multiple nodes that need to agree in real time), there is no distributed consistency problem to solve during normal operation — one node is simply the source of truth. The CAP trade-off only becomes relevant during a failover, when the standby must decide whether to take over (favoring availability, at the risk of a tiny consistency gap if the last few log entries hadn’t replicated yet) or wait for absolute certainty (favoring consistency, at the cost of availability during the wait).

This is why the design in Figure 5 uses synchronous replication of the write-ahead log specifically — it converts what would otherwise be an availability-versus-consistency trade-off into something closer to “both,” at the cost of a small, bounded amount of added latency on every order (waiting for the standby’s acknowledgment before confirming to the client).

Let’s now do the actual math for the “millions of requests per minute” scenario stated in the requirements.

7.1 Sizing the Problem

Assume a peak scenario of 3 million requests per minute. That is:

• 3,000,000 / 60 = 50,000 requests per second average
• Real-world traffic is bursty — market open/close can see 5–10x spikes, so we must design for roughly 250,000–500,000 requests per second at peak for brief windows

7.2 Where the Time Goes

For sub-millisecond matching, we need a latency budget. A realistic breakdown for the “order to acknowledgment” path might look like this:

7.3 Scaling Techniques Used

Horizontal Sharding by Symbol

As covered earlier, splitting the ~5,000+ actively traded symbols on a typical exchange across dozens of matching engine shards means each shard only needs to handle a fraction of total volume — often just a few thousand orders per second for a given symbol, well within a single core’s capability.

Batching for the Write-Ahead Log

Instead of doing a disk flush (fsync) per single order — which can take milliseconds and would blow the entire latency budget — production systems use group commit: batch several microseconds’ worth of events together and flush them in one disk operation, amortizing the fixed cost of durability across many orders.

Kernel Bypass Networking

The fastest matching engines avoid the standard OS networking stack (which adds context-switch overhead) and use kernel-bypass techniques like DPDK or specialized NICs to shave off tens of microseconds per message at the network layer.

Backpressure and Admission Control

During extreme bursts (like a flash crash or major news event), the API Gateway applies backpressure — queuing and shedding lower-priority traffic (like market data queries) while protecting order submission capacity.

7.4 Capacity Planning Walkthrough

Let’s work through a concrete capacity plan for the stated scenario of millions of requests per minute.

• Assumption: 3,000,000 orders/cancels/modifies per minute at peak, across roughly 3,000 actively traded symbols.
• Average per symbol: 3,000,000 / 3,000 = 1,000 requests per minute per symbol on average — roughly 17 per second. Trivial for a single core.
• Skew assumption: the top 50 “hot” symbols might account for 40% of total volume. That’s 1,200,000 requests/minute across 50 symbols = 24,000/minute per hot symbol = 400/second. Still comfortably within a single core’s capacity, which can typically handle well over 100,000 simple order-book operations per second.
• Shard count: With 3,000 symbols spread across, say, 40 matching engine shards (roughly 75 symbols per shard, grouped to balance volume rather than alphabetically), each shard handles a manageable, bounded load even during bursts.
• Gateway/OMS tier: These are stateless and can scale horizontally far more easily — simply add more instances behind the load balancer to absorb the 250,000+ requests/second burst capacity needed at market open.

Java Example: Group Commit for the Write-Ahead Log

public class GroupCommitWriter {
    private final BlockingQueue<LogEntry> pending = new LinkedBlockingQueue<>();
    private final FileChannel logFile;
    private static final int MAX_BATCH = 500;
    private static final long MAX_WAIT_MICROS = 200;

    public void run() throws Exception {
        List<LogEntry> batch = new ArrayList<>(MAX_BATCH);
        while (true) {
            long deadline = System.nanoTime() + MAX_WAIT_MICROS * 1000;
            batch.clear();
            LogEntry first = pending.poll(MAX_WAIT_MICROS, TimeUnit.MICROSECONDS);
            if (first != null) batch.add(first);
            while (batch.size() < MAX_BATCH && System.nanoTime() < deadline) {
                LogEntry next = pending.poll();
                if (next == null) break;
                batch.add(next);
            }
            if (!batch.isEmpty()) {
                writeBatch(batch);      // single sequential write
                logFile.force(false);   // single fsync for the whole batch
                for (LogEntry e : batch) e.ackFuture.complete(true);
            }
        }
    }
}

This pattern trades a tiny amount of added latency (waiting up to 200 microseconds to accumulate a batch) for a massive reduction in the number of expensive disk flush operations — turning what would be thousands of individual fsync calls per second into a much smaller number of batched flushes, each covering hundreds of orders.

7.5 Vertical Scaling Still Matters

Even with horizontal sharding, the single core running each matching engine shard should be chosen carefully: high single-thread clock speed matters more than core count here, since each shard is fundamentally single-threaded. Production trading infrastructure often uses CPU pinning (dedicating specific cores exclusively to matching engine threads) and disables features like CPU frequency scaling and hyper-threading on those cores to eliminate latency jitter caused by the operating system scheduler.

A stock exchange cannot simply “go down” — even a few seconds of downtime during market hours is a major incident that regulators investigate.

8.1 Techniques for High Availability

• Hot standby replicas: A secondary matching engine instance continuously replays the write-ahead log from the primary, keeping an identical in-memory order book, ready to take over instantly.
• Consensus-based failover: A failover controller (often using a consensus protocol like Raft) detects primary failure via missed heartbeats and promotes the standby, avoiding a “split-brain” where two nodes both think they’re primary.
• Synchronous log replication: For the strictest guarantees, the primary doesn’t acknowledge an order until the write-ahead log entry has been replicated to at least one standby — trading a small amount of latency for zero data loss on failover.
• Circuit breakers and kill switches: Automatic and manual mechanisms to halt trading on a symbol (or the whole exchange) if abnormal conditions are detected — this protects against cascading failures like the Knight Capital incident mentioned earlier.
• Disaster recovery site: A geographically distant data center holds a warm/cold copy of the system that can be activated within minutes to hours if an entire region becomes unavailable.

8.2 Consensus and Leader Election

The failover controller cannot simply be “the first node that notices the primary is down,” because network partitions can cause multiple nodes to independently believe they should become primary — a dangerous condition called split-brain, where two matching engines might both accept and match orders for the same symbol simultaneously, producing two conflicting versions of the truth.

To avoid this, production systems use a consensus protocol such as Raft for the failover controller itself (not for the matching engine’s hot path, which stays single-threaded and fast). Raft ensures that a new leader can only be elected if it has the votes of a strict majority of controller nodes, and that only one leader can be active at any given term — mathematically preventing split-brain as long as a majority of nodes can still communicate.

8.3 RPO and RTO Targets

Two standard metrics define how good a disaster recovery plan is:

8.4 Regular Failover Drills

A standby that has never actually been promoted to primary in a real (or realistic simulated) failover is a liability, not a safety net — subtle bugs in the promotion path (like a missed configuration flag or an untested network route) often only surface the first time failover is actually exercised. Mature trading platforms run scheduled, sometimes even unannounced, failover drills to validate that the standby genuinely can take over within its RTO target.

Trading platforms are high-value targets, so security spans multiple layers:

9.1 Authentication and Authorization

• Multi-factor authentication for retail clients; mutual TLS and dedicated leased-line connections for institutional/FIX clients
• Fine-grained authorization: which accounts can trade which instruments, with what size limits

9.2 Pre-Trade Risk Controls (also a security concern)

• Fat-finger checks — rejecting an order that is wildly outside the normal price/size range (like accidentally typing an extra zero)
• Self-trade prevention — stopping a client from accidentally matching against their own order
• Maximum order rate limits per client to prevent a malfunctioning algorithm from flooding the exchange

9.3 Network and Infrastructure Security

• Strict network segmentation — the matching engine’s internal network should never be directly reachable from the public internet
• DDoS protection at the edge, before traffic even reaches the load balancer
• Encrypted transport (TLS) for all client-facing traffic; encryption at rest for stored order and account data

9.4 Auditability

Every action must be traceable to a specific authenticated user, with an immutable, timestamped audit trail — this is both a security control (detecting misuse) and a regulatory requirement (proving what happened during an investigation).

Java Example: Pre-Trade Risk Check

public class RiskCheckService {
    private final Map<String, AccountLimits> limitsCache; // in-memory, replicated

    public RiskDecision check(Order order) {
        AccountLimits limits = limitsCache.get(order.accountId());
        if (limits == null) {
            return RiskDecision.reject("Unknown account");
        }
        long notional = order.limitPrice() * order.quantity();

        if (notional > limits.maxOrderNotional()) {
            return RiskDecision.reject("Order notional exceeds per-order limit");
        }
        if (order.isBuy() && notional > limits.availableBuyingPower()) {
            return RiskDecision.reject("Insufficient buying power");
        }
        if (isFatFinger(order, limits.referencePrice())) {
            return RiskDecision.reject("Price deviates more than 10% from reference");
        }
        if (limits.ordersInLastSecond() > limits.maxOrderRatePerSecond()) {
            return RiskDecision.reject("Order rate limit exceeded");
        }
        return RiskDecision.approve();
    }

    private boolean isFatFinger(Order order, long referencePrice) {
        double deviation = Math.abs(order.limitPrice() - referencePrice)
            / (double) referencePrice;
        return deviation > 0.10; // more than 10% away from last traded price
    }
}

9.5 Cost Optimization

Even in latency-obsessed systems, cost matters. Practical levers include:

• Running the matching engine tier on dedicated, right-sized hardware (since this tier benefits most from performance), while running elastic, bursty tiers like the API Gateway and OMS on auto-scaling cloud infrastructure that scales down during off-market hours
• Tiered storage for historical trade data — recent data in fast time-series storage, older data moved to cheaper cold storage after a retention window, while still meeting regulatory retention requirements
• Compressing market data feed payloads and using efficient binary encodings (like SBE — Simple Binary Encoding) instead of verbose text formats such as JSON, reducing both bandwidth cost and serialization latency

Because latency is the product here, monitoring must be extremely fine-grained.

10.1 Key Metrics to Track

10.2 Approach

• Use high-resolution timestamps (nanosecond precision) at every hop, tagged with a correlation ID so a single order’s journey can be reconstructed end-to-end
• Emit metrics via a lightweight, non-blocking mechanism (for example, writing to a lock-free ring buffer that a separate thread drains) so monitoring itself never adds latency to the hot path
• Use distributed tracing (like OpenTelemetry) for the slower parts of the pipeline (gateway, risk, clearing) where a few extra microseconds don’t matter
• Real-time alerting dashboards for latency SLA breaches, replication lag, and abnormal order rejection spikes

10.3 Service Level Objectives (SLOs)

A trading platform typically publishes internal SLOs like the following, which drive alerting thresholds and capacity planning decisions:

Note how much stricter the tail latency (p99.9) requirement is compared to a typical consumer web application, where p99 under a few hundred milliseconds might be perfectly acceptable. This difference is exactly why trading infrastructure requires specialized engineering practices throughout the entire stack, not just in the matching engine itself.

10.4 Distributed Tracing Example

For the slower, non-hot-path portions of the system (gateway, OMS, risk, clearing), a trace might look like this when reconstructed from logs tagged with a shared correlation ID:

trace_id=8f3a21 order_id=CL-2024-88213
  [gateway]    auth+validate     : 82us
  [oms]        enrich            : 41us
  [risk]       pre-trade-check   : 37us
  [sequencer]  assign-sequence   : 12us
  [matching]   match-and-fill    : 29us
  [wal]        group-commit-ack  : 118us
  ---------------------------------------
  total order-to-ack latency      : 319us

Having this breakdown available per order (sampled, not for every single order, to avoid overhead) is invaluable for diagnosing exactly which stage regresses when overall latency creeps up after a deployment or during a traffic spike.

Latency-critical trading infrastructure is deployed differently from typical web services.

11.1 On-Premises vs Cloud

Many exchanges keep matching engines in dedicated, co-located data centers physically near the exchange itself (this is why “co-location services” — renting rack space right next to the exchange — is a real business). Cloud deployment is more common for the surrounding services: risk analytics, historical data, client-facing web apps, back-office/clearing systems, and disaster recovery.

11.2 Deployment Practices

• Blue-green deployment for gateway and OMS layers, so new versions can be rolled out with zero downtime and instant rollback
• Extremely cautious deployment for the matching engine itself — often deployed only during scheduled maintenance windows outside market hours, with extensive pre-production testing in a shadow environment that mirrors live traffic
• Infrastructure as Code to make every environment (dev, staging, DR) reproducible and auditable
• Canary releases for non-critical services, gradually shifting a small percentage of read-only traffic (like market data queries) to new versions before full rollout

11.3 Shadow Testing Before Go-Live

Before any change to the matching engine reaches production, it is typically run in a shadow environment — a replica system that receives a mirrored copy of real production order flow (with trades never actually settling) and its output is compared, order for order and trade for trade, against the current production system. Any divergence is investigated before the new version is ever allowed near real client funds. This is a much higher bar than typical staging environment testing, and it exists precisely because the cost of a matching engine bug reaching production is measured in real, often irreversible, financial loss.

12.1 Database Choices

12.2 Caching Strategy

A read-only cache (updated asynchronously from the matching engine’s trade/book-update stream) serves queries like “what’s the current best bid/ask for AAPL” without ever touching the matching engine directly. This isolates read-heavy client dashboard traffic from the latency-critical write path entirely.

12.3 Load Balancing Approach

Beyond the entry-point load balancer, internal load balancing routes orders to the correct matching engine shard using consistent hashing on the stock symbol — not round robin — because correctness requires that every order for a given symbol always reaches the exact same shard, in submission order.

12.4 Partitioning and Replication for the Relational Database

The relational database holding account and position data faces a very different scaling challenge than the matching engine: it needs to support complex queries (joins across accounts, positions, and order history) rather than raw sequential throughput. Typical techniques include:

• Partitioning by account ID range or hash — spreading accounts across multiple database shards so no single database instance becomes a bottleneck for the whole client base
• Read replicas — since position and account lookups vastly outnumber writes (a client checks their portfolio far more often than they trade), read replicas offload query traffic from the primary write node
• Asynchronous replication from the matching engine’s trade stream — positions are updated in the database only after a trade is confirmed and published, keeping the database decoupled from the matching engine’s timing constraints entirely

12.5 Time-Series Database Considerations

Every trade tick, every book update, and every order event is timestamped and append-only by nature — a perfect fit for a time-series database, which is optimized for exactly this write pattern (mostly sequential appends, queries filtered by time range). These systems typically use techniques like columnar storage and time-bucketed partitioning to keep both write throughput and historical query performance high, even as the dataset grows into billions of rows over years of trading history.

The platform exposes multiple protocols to different types of clients:

• REST/WebSocket API — used by retail trading apps for order submission and real-time order/trade updates
• FIX Protocol (Financial Information eXchange) — the industry-standard binary/text protocol used by institutional trading systems and brokers, optimized for low latency and well understood by trading software worldwide
• Market Data Feed (multicast/UDP) — a very high-throughput, low-latency, one-way broadcast feed for price and trade updates, consumed by thousands of subscribers simultaneously

13.1 Microservices Boundaries

Each service in the architecture has one clear responsibility and communicates via well-defined, versioned interfaces:

• OMS never directly writes to the order book — it only ever talks to the matching engine via its narrow submission interface
• Risk service is stateless per-request but reads from a fast, replicated in-memory account-limits cache
• Clearing/settlement is entirely decoupled via the message queue, so it can be slow, retried, or even briefly down without affecting live trading

13.2 A Closer Look at the FIX Protocol

FIX (Financial Information eXchange) has been the backbone of institutional trading connectivity since the 1990s. Unlike a typical REST API where each request is a self-contained HTTP call, FIX operates over a persistent, long-lived TCP session between client and exchange, exchanging compact, tag-value encoded messages.

A simplified new-order message looks conceptually like this (real FIX uses numeric tags, shown here in a readable form):

MsgType=NewOrderSingle
ClOrdID=CL-2024-88213
Symbol=AAPL
Side=Buy
OrderQty=100
OrdType=Limit
Price=189.50
TimeInForce=Day
TransactTime=2026-08-04T09:15:00.123456Z

The exchange responds with an ExecutionReport message type confirming acceptance, partial fill, full fill, rejection, or cancellation — the same state machine described earlier in the Data Flow & Lifecycle section, just expressed in FIX’s specific vocabulary.

13.3 Market Data API Design

Unlike the order-submission APIs (request-response, one client at a time), the market data API is fundamentally a one-to-many broadcast. Every trade and book update the matching engine produces must reach potentially thousands of subscribers with minimal and, crucially, fair delay — no single subscriber should get the update meaningfully earlier than another, since that would create an unfair trading advantage. This is why exchanges often use UDP multicast rather than many individual TCP connections: multicast lets the network itself replicate the data to every subscriber at effectively the same instant, rather than the server serially sending copies to each one.

14.1 Patterns Used

14.2 Idempotency and Exactly-Once Semantics

Network retries are inevitable — a client’s acknowledgment might get lost even though the order was actually accepted, tempting the client to resend it. If the system isn’t careful, this could result in the same order being submitted twice. The standard solution is an idempotency key: the client generates a unique client order ID for every new order, and the OMS keeps a short-lived deduplication cache. If the same client order ID arrives again within a reasonable window, the system returns the original result instead of creating a second order.

14.3 Anti-Patterns to Avoid

15.1 Best Practices

15.2 Common Mistakes

16.1 Lessons Learned Across the Industry

Studying these real-world incidents reveals a consistent pattern: almost every major trading system failure traces back to one of the same handful of root causes covered throughout this tutorial — a deployment process that wasn’t uniform across servers, a missing or bypassed pre-trade risk check, insufficient testing of failover paths, or automated systems interacting in ways nobody had modeled. This is precisely why the architecture in this tutorial treats risk checks, circuit breakers, deployment discipline, and failover drills as core, non-negotiable components rather than afterthoughts bolted on once the “happy path” works.

17.1 Frequently Asked Questions

Q: Why not just use a distributed database with strong consistency for the order book?

Distributed consensus protocols (like Paxos or Raft across multiple nodes) typically add single-digit-millisecond latency due to network round trips between nodes — far too slow for sub-millisecond matching. Instead, we achieve strong consistency cheaply by keeping each order book on a single node/thread, and only replicate asynchronously (or semi-synchronously) for durability, not for the matching decision itself.

Q: How do you scale beyond one CPU core per symbol if one stock becomes extremely popular?

In practice this is rare because even the busiest single stock’s order flow (tens of thousands of orders per second) comfortably fits on one modern core running an optimized event loop. If it ever became a bottleneck, the two options are pinning that shard to a more powerful dedicated core, or a much more complex price-range sub-partitioning scheme — most real systems never need the second option.

Q: What happens if the sequencer itself becomes a bottleneck?

The sequencer only needs to do one very cheap operation (increment a counter) per order, so it can typically handle far more throughput than the rest of the pipeline. If needed, it can be sharded by symbol as well, since orders for different symbols don’t need to be ordered relative to each other — only orders for the same symbol need a shared, strict sequence.

Q: Is this architecture over-engineered for a smaller trading platform?

For a platform truly needing sub-millisecond latency and millions of requests per minute, no — every component here earns its place. For a much smaller platform (say, a niche brokerage with modest volume), many of these pieces (like symbol sharding or kernel-bypass networking) can be simplified or deferred until the traffic actually demands them.

Q: Why use price-time priority instead of some other matching rule?

Price-time priority (best price first, then earliest arrival within the same price) is the fairest and most widely mandated rule across global markets because it rewards two things everyone can understand and verify: offering a better price, and being willing to commit to that price earlier than others. Alternative rules (like pro-rata allocation, common in some derivatives markets) exist for specific product types, but price-time priority remains the default for standard equity trading because of its simplicity and fairness.

Q: How is this different from a typical e-commerce “add to cart and checkout” system?

An e-commerce checkout deals with one buyer and one fixed price at a time, and can tolerate hundreds of milliseconds of latency without anyone noticing. A matching engine must continuously reconcile many competing buyers and sellers against each other, with a strict fairness rule about ordering, at a speed where even one millisecond is considered slow. The core algorithmic problem — continuous double auction matching — simply doesn’t exist in most other kinds of software systems.

Q: Can this design be adapted for cryptocurrency or derivatives exchanges?

Yes — the core matching engine design (single-threaded, symbol-sharded, price-time priority order book) is nearly identical across equities, crypto, and many derivatives markets. What typically changes is the settlement layer (crypto settles on-chain or via custodial ledgers instead of traditional T+1/T+2 clearing) and some risk-check specifics (like margin and liquidation logic for derivatives).

17.2 Summary

Designing a stock trading platform is one of the richest problems in software architecture because it forces every classic distributed-systems concern — latency, consistency, availability, durability, security, cost — to be considered simultaneously, with financial and regulatory consequences for getting any one of them wrong. The architecture above is not the only correct one, but it captures the pattern that has independently emerged across virtually every serious exchange in the world: a single-threaded, symbol-sharded matching engine sitting behind a carefully layered ingress path, backed by write-ahead logging and hot standbys, monitored down to the microsecond.