Imagine you order a phone charger online. Somewhere, in a few hundred milliseconds, a decision is made: which warehouse, out of dozens spread across a country or the world, will pack and ship this exact item to you? That decision — invisible to the customer — is one of the most consequential pieces of engineering in modern e-commerce. Get it wrong, and a customer in Delhi waits five extra days for a package that could have shipped from a warehouse forty kilometers away. Get it wrong at scale, and a company loses millions of dollars a year in unnecessary shipping costs and unhappy customers.

This system is called order fulfillment routing, sometimes called warehouse selection, sourcing logic, or ship-from-store/ship-from-warehouse optimization. In plain English: it is the brain that looks at an order, looks at every warehouse that might be able to fulfill it, and picks the best one — or the best combination, if the order has to be split. Different companies give this capability different internal names, but the underlying engineering problem, and the trade-offs involved in solving it well, remain remarkably consistent across the industry.

1.1 A short history

Each generation of e-commerce added one more constraint on top of the previous one, and today’s routing systems are the compressed accumulation of thirty years of that evolution.

• 1990s — Single Warehouse Era. Early online retailers like the first version of Amazon operated out of a single distribution center. Routing was trivial: there was only one place an order could come from. The main engineering challenge was inventory accuracy, not selection.
• Early 2000s — Regional Distribution Centers. As order volumes grew, companies opened regional warehouses to cut shipping time and cost. This introduced the first real routing problem: given two or three candidate warehouses, which one should fulfill an order? Rule-based logic (“always prefer the closest warehouse with stock”) was enough at this scale.
• 2010s — Marketplace and Multi-Seller Complexity. Marketplaces like Amazon Marketplace, Flipkart, and Alibaba introduced third-party sellers, each with their own inventory pools, some using the platform’s fulfillment network (like Fulfilled by Amazon) and some shipping themselves. Routing now had to account for seller-owned inventory, platform-owned inventory, and mixed carts with items from different sources.
• 2015–2020 — Real-Time Optimization. Companies began treating warehouse selection as a live optimization problem, incorporating real-time carrier rates, warehouse load, promised delivery dates, and even carbon footprint. Machine learning models started predicting delivery time and cost instead of relying purely on static rules.
• 2020s — Ship-From-Store and Dark Stores. The pandemic accelerated “ship from store” and micro-fulfillment models, where a retail store or small local warehouse (“dark store”) could also be a fulfillment source. This exploded the number of candidate locations for any given order, making the routing decision far more computationally demanding and time-sensitive (some quick-commerce platforms now route in under 50 milliseconds).

1.2 Scope of this guide

This guide is written for engineers, students, and interview candidates who already understand basic web application concepts (clients, servers, databases) but want a thorough, from-first-principles walkthrough of how a real, production-grade warehouse routing system is designed and reasoned about. Every technical term used is explained the first time it appears, with a plain-English definition, a real-life analogy, and a concrete example, so no prior background in supply chain or logistics is assumed. By the end, you should be able to not only describe the architecture from memory, but explain the specific failure mode each component exists to prevent — which is exactly what a strong system design interview answer requires.

Let’s define the problem precisely. A customer places an order containing one or more line items (products and quantities). The company operates N warehouses, each holding a different subset of inventory in different quantities. The system must answer, in real time, four hard questions at once.

2.1 Why naive approaches fail

A beginner’s first instinct might be: “just pick the closest warehouse with stock.” This fails quickly in practice for several reasons:

• Stock is a moving target. Thousands of orders can hit the same popular item within seconds. If routing decisions aren’t coordinated, ten different orders might all be told “yes, Warehouse A has it,” when Warehouse A only had one unit left — this is called an oversell.
• Distance is not the same as delivery time or cost. A warehouse 50 km away next to no highway access might be slower and pricier to ship from than one 300 km away next to a major logistics hub.
• Orders often have multiple items. If a single warehouse doesn’t have all items, should the system split the order into two shipments (more cost, more packaging, more carbon footprint) or wait for restock?
• Business rules change constantly. A company may want to prioritize clearing aging inventory in one warehouse, or avoid a warehouse that’s currently overloaded and behind on packing orders.

2.2 A concrete scenario

Consider a mid-sized online retailer running 40 warehouses across a country, processing 2 million orders a day, with a peak of roughly 5,000 orders per second during a major sale event. Each order, on average, references 1.8 line items, and roughly 8% of items are held in fewer than three warehouses at any given time, meaning contention on those items is common. If the routing decision takes even 50 milliseconds too long per order at peak, checkout latency creeps upward for every single customer, and if the reservation logic has even a rare one-in-ten-thousand race condition, that still translates into hundreds of oversold orders a day at this volume — each one a canceled order, a refund, and a damaged customer relationship. These numbers are why routing correctness and speed are treated as first-class, heavily tested engineering concerns rather than an afterthought bolted onto checkout.

Before diving into architecture, let’s build vocabulary. Each term below includes what it means, why it matters, a real-life analogy, and a concrete example.

3.1 Inventory Availability

What: The count of sellable units of a product physically present (and not already reserved) in a given warehouse. Why: Without knowing what’s available, you cannot route correctly — this is the single source of truth the whole system depends on. Analogy: Like checking how many seats are left in a movie theater before selling a ticket. Example: Warehouse A has 40 units of a t-shirt; 5 are already reserved for other pending orders, so 35 are actually available to sell.

3.2 Reservation (Soft Hold)

What: A temporary claim on inventory made the moment an order is being routed, before payment is even confirmed, to prevent two orders from claiming the same unit. Why: Prevents overselling during the window between “decided to buy” and “payment confirmed.” Analogy: Like a restaurant holding a table for 15 minutes after you call to reserve it — if you don’t show up, the table is released. Example: When checkout starts, the system reserves 1 unit at Warehouse A for 10 minutes; if payment fails, the reservation expires and the unit becomes available again.

3.3 Scoring / Weighted Decision Function

What: A formula that converts multiple factors — distance, cost, stock health, warehouse load — into a single comparable number per warehouse. Why: You cannot compare “distance” and “cost” directly; they’re different units. A scoring function normalizes and weighs them. Analogy: Like a college admissions committee scoring applicants on grades, essays, and interviews using a weighted rubric instead of comparing raw numbers directly. Example: score = 0.4 × distance_score + 0.35 × cost_score + 0.25 × stock_health_score.

3.4 Split Shipment

What: Fulfilling one order from more than one warehouse because no single warehouse has all items in stock. Why: Improves fulfillment rate but increases shipping cost and complexity. Analogy: Ordering a burger and fries from two different food trucks because one ran out of fries. Example: An order with a phone and a phone case ships the phone from Warehouse A and the case from Warehouse B.

3.5 Service Level Agreement (Delivery Promise)

What: The delivery date/time promised to the customer at checkout (for example, “arrives in 2 days”). Why: Routing must respect this promise — picking a cheap but slow warehouse can break the SLA. Analogy: A pizza chain’s “30 minutes or free” promise constrains which branch can take your order. Example: If the promised delivery is next-day, only warehouses within same-day-dispatch-and-overnight-carrier range are eligible candidates.

3.6 Warehouse Load / Capacity

What: How many orders a warehouse is already processing relative to its packing and shipping capacity for the day. Why: Routing everything to the “best” warehouse on paper can overload it, causing delays for everyone. Analogy: A popular restaurant that keeps accepting reservations until the kitchen can’t keep up — smart hosts spread reservations across time slots or sister restaurants. Example: Warehouse A is scoring highest but is at 95% of its daily packing capacity, so the router deprioritizes it in favor of Warehouse B at 60% capacity.

3.7 Serviceable Area (Geo-fencing)

What: The geographic boundary within which a warehouse can realistically deliver, based on carrier coverage and delivery time promises. Why: A warehouse might technically have stock, but if it sits outside the carrier’s next-day delivery zone for a given address, it should never even be considered — including it would waste computation and risk a bad delivery-time promise. Analogy: Think of a pizza chain’s delivery radius — even if a branch has every ingredient, it won’t take an order from a customer forty kilometers away because the food would arrive cold and late. Example: Warehouse C is only 60 km from the customer, but sits just outside the carrier’s guaranteed next-day zone due to a mountain pass with no direct highway; it is excluded from candidate generation entirely, even though a straight-line distance calculation might suggest otherwise.

3.8 Backorder

What: A state where an ordered item is not currently available anywhere, but the order is accepted anyway with a promise to ship once stock is replenished. Why: Rather than rejecting a sale outright when everything is technically out of stock, some businesses choose to accept the order and set expectations accordingly, capturing demand instead of losing it entirely. Analogy: Like a bookstore that takes your name and number for a bestseller that’s sold out, promising to call you when the next shipment arrives. Example: A customer orders a laptop with zero units across all warehouses; the system marks it as backordered with an estimated ship date of five days out, tied to an incoming supplier shipment already logged in the Inventory Service.

3.9 Business Rule Overrides (Priority Tiers)

What: A layer of business-defined rules that can override or adjust the pure scoring result for strategic reasons unrelated to distance or cost, such as prioritizing a warehouse that is clearing aging seasonal stock, or deliberately deprioritizing a warehouse currently undergoing a system migration. Why: Not every fulfillment decision should be purely mathematical — real businesses have goals like reducing write-offs on aging inventory or supporting a new warehouse’s ramp-up period that a pure distance-and-cost formula would never account for on its own. Analogy: Like a store manager occasionally choosing to feature and sell older stock first even if newer stock is more conveniently placed, simply because the older stock needs to move before it expires or goes out of season. Example: A merchandising team flags a particular warehouse’s winter jacket inventory as “priority clearance,” and the Routing Engine applies a small positive score boost to that warehouse for winter jacket orders for the next thirty days, gently nudging more of those orders its way without completely overriding distance and cost considerations.

3.10 Fulfillment Rate

What: The percentage of ordered line items that get fulfilled without cancellation or indefinite delay, often tracked per warehouse and system-wide. Why: It is one of the most important business health metrics for this entire system — a routing engine that is fast but frequently fails to find a valid warehouse is not actually doing its job well. Analogy: Like a restaurant tracking what percentage of menu items it actually had in stock when customers ordered them, rather than just how quickly it took orders. Example: If 985 out of 1,000 ordered line items ship successfully within the promised window, the fulfillment rate for that period is 98.5%.

Let’s zoom out and look at the full system. Every box in the diagram below is labeled with exactly what it does, so you can see how a request flows from a customer’s click to a confirmed, warehouse-assigned order.

4.1 Component responsibilities

4.2 Why these service boundaries specifically

It’s worth pausing on why the architecture is split exactly this way, since the boundaries themselves are a design decision, not an accident. The Inventory Service is separated from the Routing Engine because inventory data has fundamentally different consistency and durability requirements — it is a strongly consistent, transactional resource that many other systems beyond routing also depend on, such as merchandising dashboards and supplier replenishment tools, so it earns its own dedicated ownership boundary. The Distance and Cost Service is kept separate from the Routing Engine because it frequently depends on third-party providers (mapping APIs, carrier rate APIs) with their own latency and reliability characteristics quite different from the company’s own internal databases; isolating it behind a clean interface means a slow or flaky external provider can be swapped, cached, or circuit-broken without touching the core decision logic at all. The Capacity Service is separated because warehouse operational data (staffing levels, current throughput) often comes from an entirely different internal system — sometimes even a separate team’s on-premises warehouse operations software — and treating it as its own bounded service avoids tightly coupling the Routing Engine to that system’s specific data model and release cycle.

This pattern of drawing service boundaries around differences in consistency needs, data ownership, and external dependencies, rather than simply grouping by team org chart or arbitrary convenience, is one of the more subtle but important skills in real-world system design, and interviewers often probe exactly this reasoning rather than just asking you to draw more boxes.

Let’s open up the Routing Engine and see exactly how it decides a warehouse. The process has four stages: candidate generation, scoring, selection, and reservation.

5.1 Stage 1 — Candidate Generation

Not every warehouse is a candidate for every order. The engine first filters down the full warehouse list using cheap, fast checks: does this warehouse carry this SKU at all? Is it within a shipping radius that could still meet the promised delivery date? Is it operational (not closed for maintenance)? This step typically shrinks hundreds of warehouses down to a handful of realistic candidates before any expensive computation happens.

5.2 Stage 2 — Scoring

Each remaining candidate warehouse is scored using a weighted function. A common, simple, and interview-friendly formula:

score(warehouse) =
    w1 * normalize(1 / distance_km)
  + w2 * normalize(1 / shipping_cost)
  + w3 * stock_health_score
  + w4 * (1 - current_load_ratio)

where w1 + w2 + w3 + w4 = 1

Each factor is normalized to a 0–1 range so they can be fairly combined despite being measured in different units (kilometers vs. rupees vs. a ratio). The weights (w1…w4) are business decisions — a company optimizing for speed might weight distance and delivery-time-feasibility heavily; one optimizing for margin might weight cost more.

5.3 Stage 3 — Selection

The candidate with the highest score wins for single-item orders. For multi-item orders, the engine runs a more involved process: it first checks if any single warehouse can fulfill 100% of the cart (preferred, since it avoids split-shipment cost). If none can, it evaluates split combinations, still trying to minimize the number of shipments and total cost, similar to a small bin-packing or set-cover problem.

5.4 Stage 4 — Reservation

Once a warehouse is selected, the engine immediately calls the Reservation Service to place a soft hold with a time-to-live (TTL), typically 5–15 minutes. This closes the race-condition window between “we decided” and “payment is confirmed.”

public class WarehouseScorer {

    private final double distanceWeight = 0.40;
    private final double costWeight = 0.30;
    private final double stockHealthWeight = 0.15;
    private final double loadWeight = 0.15;

    // Higher score is better. All inputs are pre-normalized to 0.0 - 1.0
    public double score(WarehouseCandidate w) {
        double distanceScore = 1.0 - w.getNormalizedDistance();
        double costScore = 1.0 - w.getNormalizedShippingCost();
        double stockHealthScore = w.getStockHealthScore();
        double loadScore = 1.0 - w.getCurrentLoadRatio();

        return (distanceWeight * distanceScore)
             + (costWeight * costScore)
             + (stockHealthWeight * stockHealthScore)
             + (loadWeight * loadScore);
    }

    public WarehouseCandidate selectBest(List<WarehouseCandidate> candidates) {
        return candidates.stream()
            .filter(WarehouseCandidate::isEligible) // meets SLA, is operational
            .max(Comparator.comparingDouble(this::score))
            .orElseThrow(() -> new NoEligibleWarehouseException());
    }
}

5.5 Handling split shipments algorithmically

When no single warehouse can cover the entire cart, the engine needs to decide how to split it. This is a small instance of the classic set cover problem — find the minimum number of warehouses whose combined stock covers every item in the order. Solving set cover perfectly (optimally) is computationally expensive as the number of items and warehouses grows, so in practice most systems use a fast greedy approximation rather than an exact solver, since checkout latency budgets don’t allow for a slow, perfectly optimal answer.

public class SplitShipmentPlanner {

    // Greedy approach: repeatedly pick the warehouse covering the most
    // remaining items, until every item is assigned or none can be covered.
    public List<ShipmentPlan> plan(List<OrderItem> items, List<WarehouseCandidate> candidates) {
        Set<OrderItem> remaining = new HashSet<>(items);
        List<ShipmentPlan> plans = new ArrayList<>();

        while (!remaining.isEmpty()) {
            WarehouseCandidate best = null;
            Set<OrderItem> bestCovered = Collections.emptySet();

            for (WarehouseCandidate candidate : candidates) {
                Set<OrderItem> covered = candidate.itemsItCanFulfill(remaining);
                if (covered.size() > bestCovered.size()) {
                    best = candidate;
                    bestCovered = covered;
                }
            }

            if (best == null) {
                throw new UnfulfillableOrderException(remaining);
            }

            plans.add(new ShipmentPlan(best.getWarehouseId(), bestCovered));
            remaining.removeAll(bestCovered);
        }
        return plans;
    }
}

This greedy strategy runs in polynomial time relative to the number of items and candidate warehouses, which is fast enough to fit comfortably inside the routing latency budget. It does not always produce the mathematically fewest possible shipments, but in practice it gets close, and the small loss in optimality is an acceptable trade for predictable, low latency.

5.6 Greedy approximation vs. linear programming

A more mathematically rigorous alternative is to model warehouse selection as a linear programming (LP) or mixed-integer programming (MIP) problem, where the solver simultaneously minimizes total cost across all items and warehouses subject to stock and capacity constraints. This produces a truly optimal plan, but general-purpose LP/MIP solvers can take tens to hundreds of milliseconds or more for larger problems, and their runtime becomes less predictable as the number of variables grows. Most consumer-facing checkout systems use fast greedy or heuristic scoring for the real-time path, and reserve full optimization solvers for offline or near-real-time batch processes — for example, recomputing an optimal warehouse-to-region assignment plan once a day rather than for every single order.

5.7 CAP theorem, consensus, and failure recovery

The CAP theorem states that a distributed data system can only guarantee two out of three properties at any moment during a network partition: consistency (every read sees the latest write), availability (every request gets a response), and partition tolerance (the system keeps working despite network failures between nodes). Since network partitions are a fact of life in any system spread across multiple servers or data centers, the real choice in practice is between consistency and availability when a partition actually occurs.

This system deliberately makes different choices for different parts of the data. For product browsing and general availability checks, it favors availability — showing a customer a slightly stale “in stock” indicator from cache is far better than showing an error page just because one database node is briefly unreachable. But for the actual reservation write — the moment inventory is decremented — it favors consistency, using the version-checked atomic update described later, because serving an incorrect “yes, reserved” response when it isn’t true directly causes an oversell.

Where multiple nodes must agree on a single fact — for example, which of several database replicas is currently the primary and allowed to accept writes — the system relies on a consensus algorithm such as Raft, run by the underlying managed database or coordination service. Consensus algorithms let a cluster of nodes agree on one true value (like “which node is primary”) even if some nodes are slow or temporarily unreachable, as long as a majority of nodes are healthy and can communicate. This is what allows automatic failover to happen safely: when the primary Inventory Database node fails, the remaining nodes use consensus to elect a new primary without any node acting on stale or conflicting information about who is in charge.

Failure recovery in this system is layered: transient failures are handled by retries with backoff at the calling service; a fully failed dependency is handled by a circuit breaker falling back to degraded logic; and a fully failed database node is handled by consensus-driven failover to a healthy replica, with the previously described reservation TTL mechanism acting as a final safety net that guarantees any reservation that never gets a definitive outcome (success or failure) automatically expires and releases its hold on inventory, rather than leaking stock indefinitely.

5.8 Time complexity of the decision pipeline

It helps to reason about this pipeline in terms of computational complexity, since that reasoning is exactly what separates a design that stays fast at ten warehouses from one that quietly falls apart at a thousand. Candidate generation, using a geo-index, narrows the search space in roughly logarithmic time relative to the total number of warehouses, rather than checking every warehouse one by one. Scoring each remaining candidate is a constant-time operation per candidate, so scoring overall runs in time proportional to the number of surviving candidates, which is why aggressive candidate filtering matters so much — it directly bounds how much scoring work has to happen afterward. The greedy split-shipment algorithm runs in time proportional to the number of items multiplied by the number of candidates, since in the worst case it re-evaluates all remaining candidates each time it assigns one warehouse to a subset of items. None of these steps individually are expensive, but understanding their complexity is exactly what lets an engineer predict, ahead of time, how the system’s latency will behave as the business grows to more warehouses, more SKUs, and more simultaneous orders, rather than discovering the answer painfully during a future outage.

Here is the full lifecycle of a single order, from the moment a customer clicks “Buy Now” to the moment the warehouse is notified to start packing.

Notice the split between synchronous and asynchronous work. Everything the customer waits for — validation, routing, reservation — happens quickly and directly. Everything that can happen “in the background” — telling the physical warehouse system to start packing, sending confirmation emails — happens through the message queue. This is a critical design decision: it keeps checkout fast even if the warehouse’s on-site systems are briefly slow or unavailable.

6.1 What happens on failure

If payment fails after a reservation was made, the reservation simply expires after its TTL and the inventory becomes available again — no manual cleanup needed. If the Fulfillment Service cannot reach a particular warehouse’s WMS, the message stays in the queue and is retried with backoff, rather than being lost.

No architecture is free. Being honest about what this one gives up in exchange for what it delivers is what turns a design into a defensible one under interviewer pressure or an architecture review.

7.1 Key trade-off: consistency vs. speed

The routing decision needs fresh inventory data, but checking the single source of truth (the database) for every request under heavy load is slow. Most systems trade a small amount of consistency for speed: they read from a fast cache that might be a few hundred milliseconds stale, then verify and lock the actual reservation against the strongly consistent database only at the final step. This is a classic instance of the CAP theorem in action — under network partition, most retailers choose availability and eventual consistency for browsing/read paths, but require strong consistency at the exact moment of reservation.

7.2 Key trade-off: latency vs. optimality

A perfectly optimal warehouse selection, computed by exhaustively evaluating every combination of warehouses against every constraint, would almost always beat a fast greedy approximation on paper — slightly lower cost, slightly better delivery time, slightly fewer split shipments. But that perfect answer might take an extra 200 to 500 milliseconds to compute, which is often not acceptable during a live checkout flow where customers expect a near-instant response. Every mature routing system accepts a small, bounded loss in optimality in exchange for a predictable, low latency answer, reserving the truly optimal, slower computation for offline or batch planning where there is no impatient customer waiting on the other end.

7.3 Where different parts of the system sit on the consistency spectrum

This table is worth internalizing well beyond this specific system: almost every large distributed system makes exactly this kind of case-by-case consistency decision, rather than applying one blanket policy everywhere. Recognizing which pieces of data genuinely need strong consistency, versus which can tolerate being slightly stale in exchange for speed and availability, is one of the most transferable skills in all of system design.

At scale, this system must handle sudden spikes — a flash sale can push order volume up 50x within minutes. Let’s look at how each layer scales.

8.1 Horizontal scaling of the Routing Engine

The Routing Engine should be stateless: every instance can handle any request, with no in-memory data that only one node knows about. This means during a traffic spike, you simply add more instances behind the load balancer. Statelessness is what makes horizontal scaling nearly free from a design standpoint — the hard scaling problem is really in the Inventory layer, not the Routing Engine’s own compute.

8.2 Caching strategy

Inventory reads vastly outnumber writes — for every purchase, there might be hundreds of “is this in stock” checks from users just browsing. A read-through cache (Redis) in front of the inventory database absorbs this load, with a short TTL (a few seconds) or event-driven invalidation whenever stock changes.

8.3 Reducing candidate set size

A naive implementation might score every warehouse in the country for every order. A well-designed system uses a geo-index (like a geohash or an R-tree) to instantly narrow candidates to warehouses within a reasonable radius, cutting the scoring workload from hundreds of warehouses to a handful before any expensive computation runs.

8.4 Little’s Law applied to warehouse capacity

Little’s Law (L = λ × W, average number of orders in a warehouse’s processing pipeline equals arrival rate times average processing time) is directly useful here: if a warehouse can process orders at a known average rate, the Capacity Service can predict how many orders are already “in flight” and avoid over-assigning new orders that would blow past the warehouse’s promised turnaround time.

8.5 Capacity planning

Capacity planning for this system means answering, ahead of time, exactly how many Routing Engine instances, database connections, and cache nodes are needed to handle both steady-state traffic and known future spikes, such as a planned promotional sale. Teams typically run scheduled load tests that simulate the expected peak multiplier (for example, 20 times normal order volume) well before the actual event, using those results to pre-provision additional capacity rather than relying purely on reactive auto-scaling, since auto-scaling alone can lag behind a truly sudden spike by the time new instances boot up, warm their caches, and start serving traffic. For entirely unplanned spikes — an item unexpectedly going viral on social media — the system relies on a combination of fast auto-scaling, aggressive caching to reduce per-request database load, and, if truly necessary, a controlled queueing or waiting-room mechanism at the API Gateway that smooths a sudden burst of traffic into a rate the backend can safely absorb, rather than letting every request hit the Routing Engine simultaneously and risk a cascading failure.

8.6 Database sharding and connection pooling

As order and inventory volume grows, a single database instance eventually cannot keep up with the write throughput of reservations across thousands of SKUs and warehouses. The Inventory Database is typically sharded by warehouse ID, so each shard only needs to handle the write load for the SKUs stored at that particular warehouse, rather than every write in the entire company funneling through one instance. Each application instance maintains a connection pool to the database rather than opening a new connection per request, since establishing a fresh database connection is relatively expensive and would add unnecessary latency to every single reservation attempt under load.

8.7 Message queue partitioning

The Kafka-based event bus that carries fulfillment events is partitioned, commonly by warehouse ID, so that events destined for the same warehouse are processed in order by the same consumer, while events for different warehouses can be processed fully in parallel by different consumer instances. This matters because a warehouse’s Fulfillment Service instance often needs to see pick-and-pack instructions for that warehouse in the order they were generated, to avoid confusing or conflicting instructions arriving out of sequence.

8.8 Networking considerations

Because a single routing decision fans out into several internal network calls, connection reuse matters a great deal. Services communicating over gRPC or HTTP keep long-lived, pooled connections open between instances rather than establishing a fresh TCP connection (and, for encrypted traffic, a fresh TLS handshake) for every single request, since that setup cost alone can eat a meaningful chunk of the overall latency budget if paid repeatedly. DNS lookups for internal service addresses are cached aggressively and refreshed in the background, since a slow or failing DNS resolution at the wrong moment can silently add hundreds of milliseconds to what should be a fast internal call. For the public-facing side, a Content Delivery Network sits in front of static assets and cacheable read-only endpoints (like general product availability pages), keeping that traffic away from the core routing infrastructure entirely so the Routing Engine’s capacity is reserved for the requests that actually need its real-time decision-making.

Network topology also matters at a physical level: placing the Routing Engine, Inventory Service, and Reservation Service within the same availability zone or data center minimizes the network round-trip time between them, since even a few milliseconds of extra latency per hop adds up across the multiple internal calls a single order triggers. Cross-region calls are avoided entirely on this synchronous, latency-sensitive path; if a customer’s request lands in a region whose local read replica is unavailable, the system fails over to another healthy region rather than reaching across a slow, long-distance link for every single request.

An outage in this system doesn’t just slow down a webpage — it can stop checkout entirely, or worse, cause overselling. Reliability here means designing for both partial failures and total region failures.

9.1 Multi-AZ vs. multi-region

It’s worth distinguishing two levels of redundancy that are often conflated. Deploying across multiple availability zones (physically separate data centers within the same broad geographic region, connected by fast, low-latency links) protects against a single data center losing power or network connectivity, and is the baseline expectation for any production system of this kind. Deploying across multiple full regions (geographically distant, sometimes on different continents) protects against a much larger-scale event, like an entire region’s cloud provider infrastructure going down, but comes with real trade-offs: cross-region data replication is slower and more expensive, and keeping inventory consistent across regions in real time is significantly harder than within a single region’s low-latency network. Many companies choose multi-AZ as their standard operating posture and reserve full multi-region failover for their most business-critical paths only, such as this exact routing and reservation flow, given how directly it affects revenue and customer trust.

9.2 Failure handling strategies

9.3 Disaster recovery and backup

Beyond day-to-day failover, the system needs a plan for a full regional disaster — a data center outage, a natural disaster affecting a cloud provider’s region, or a catastrophic misconfiguration. This typically involves regular, automated backups of the Order and Inventory databases, a documented and periodically tested recovery time objective (RTO — how long recovery is allowed to take) and recovery point objective (RPO — how much data loss, measured in time, is acceptable). For a system where a few minutes of lost reservation data could mean real oversold inventory, most companies aim for an RPO measured in single-digit minutes or less, backed by continuous or near-continuous replication rather than nightly backups alone.

9.4 Chaos engineering

Because this system’s failure modes (an oversold item, a stuck reservation, a warehouse silently overloaded) are subtle and business-critical, many mature engineering teams practice chaos engineering — deliberately injecting failures like killing a Routing Engine instance mid-request, or artificially slowing down the Inventory Service, in a controlled environment to verify that circuit breakers, retries, and fallbacks actually behave as designed, rather than only working in theory. This proactive testing catches gaps that normal unit and integration tests often miss, since those gaps typically only appear under real concurrent load and partial failure conditions.

Order fulfillment routing touches customer addresses, payment-adjacent workflows, and internal business logic (pricing, cost data) — all of which need protection.

10.1 Encryption in transit and at rest

All traffic between the customer’s device and the API Gateway is encrypted using TLS, and all internal service-to-service traffic uses mutual TLS as mentioned earlier, so no request — external or internal — ever travels across the network in plain text. Data at rest, including customer shipping addresses and order history stored in the databases, is encrypted using standard symmetric encryption, with encryption keys managed by a dedicated key management service rather than being embedded in application code or configuration files, and rotated periodically according to the company’s security policy.

10.2 Compliance and data residency

When a company operates warehouses and serves customers across multiple countries, customer address and order data may be subject to regulations like GDPR in Europe, which can require that certain personal data be stored and processed within specific geographic regions. This means the architecture sometimes needs region-specific deployments of the Order Database, with the Routing Engine treating “which region’s database can legally hold this customer’s data” as an additional constraint alongside distance, cost, and stock — a compliance concern that directly shapes technical architecture, not just legal policy.

10.3 Secure internal API design

Internal service-to-service calls (Routing Engine to Inventory Service, Fulfillment Service to WMS) are protected using mutual TLS so both sides of every call cryptographically verify each other’s identity, preventing a compromised or rogue internal service from impersonating a trusted one. Secrets and API keys used for these calls are never hardcoded — they are pulled at runtime from a secrets manager, rotated on a regular schedule, and scoped narrowly so a leaked credential for, say, the Distance and Cost Service cannot be reused to access the Order Database.

Because a routing mistake costs real money and real customer trust, this system needs deep observability, not just basic uptime checks.

Structured, correlated logging (every log line carrying an orderId and traceId) combined with distributed tracing lets an engineer follow a single order’s journey across every service it touched — essential when investigating “why was I charged extra shipping” tickets from customer support.

Beyond the technical metrics listed above, it’s worth tracking a handful of business-facing metrics side by side with the engineering ones, since a system can be technically healthy — low latency, no errors — while still making poor business decisions. Average shipping cost per order, average promised-versus-actual delivery time, and warehouse utilization balance (whether load is spreading roughly as intended across the network rather than concentrating unexpectedly) all belong on the same dashboard as latency and error rate, because a regression in any of them is just as real a production incident as a spike in 500 errors, even though no alert-triggering exception was ever thrown.

11.1 Defining SLAs and SLOs

A Service Level Objective (SLO) is an internal target the engineering team holds itself to, such as “99.9% of routing decisions complete in under 300 milliseconds” or “oversell rate stays below 0.01% of reserved units.” A Service Level Agreement (SLA) is a related but distinct concept: a formal, often externally communicated commitment, such as a promised delivery window shown to the customer at checkout. The Routing Engine’s internal SLOs exist specifically to make sure the externally-facing SLA (the delivery promise) can actually be kept; if the routing decision itself is too slow or too often wrong, the customer-facing delivery promise becomes unreliable regardless of how good the underlying logistics network is. Tracking SLO compliance over time, and treating a sustained SLO breach as seriously as a full outage, is what keeps a growing system’s reliability from quietly eroding as complexity increases.

11.2 Distributed tracing

A single order touches the API Gateway, Order Service, Routing Engine, Inventory Service, Distance and Cost Service, Reservation Service, and eventually the Fulfillment Service — potentially seven or more hops. Distributed tracing tools (implementing standards like OpenTelemetry) attach a single trace ID to the request as it enters the system and propagate it through every downstream call, so an engineer can later pull up one screen showing exactly how long each hop took and where time was spent, rather than manually cross-referencing separate logs from seven different services.

11.3 Alerting strategy

Not every anomaly deserves to wake someone up at 2 a.m. A well-designed alerting strategy distinguishes between metrics that need an immediate page (like a sudden spike in oversell rate, which directly costs money and trust) and metrics that are fine to review the next morning on a dashboard (like a slow week-over-week increase in split shipment rate). Alert thresholds are typically based on rate-of-change and statistical baselines rather than fixed numbers, since “normal” order volume on a random Tuesday looks very different from “normal” during a planned flash sale, and a good alerting system needs to account for that context rather than firing constant false alarms.

This system is typically deployed as a set of independently deployable microservices on a container orchestration platform such as Kubernetes, spread across at least two cloud regions or availability zones for resilience.

12.1 Infrastructure as Code

Warehouse and region configuration (which warehouses exist, their coordinates, capacity limits) is managed as versioned configuration rather than hardcoded, deployed through Infrastructure as Code tools so a new warehouse can be onboarded through a reviewed pull request rather than a manual production change. This approach also means the entire environment — every service, every network rule, every database instance — can be recreated identically in a disaster recovery scenario or a new region simply by re-running the same code against a fresh cloud account, rather than depending on any single engineer’s memory of manual configuration steps performed months or years earlier. Treating infrastructure this way additionally gives the team a full, auditable history of every configuration change, which becomes invaluable when investigating exactly when and why a particular warehouse’s capacity limit or serviceable radius was last adjusted.

12.2 Blue-green for the Reservation Service

Because the Reservation Service touches inventory correctness directly, many teams prefer blue-green deployment for it specifically — running the new version fully in parallel and switching traffic over only after full validation, rather than a gradual canary that could leave two logic versions reserving stock simultaneously in subtly inconsistent ways.

12.3 Cloud service models applied here

This system typically mixes cloud service models depending on the component. The core microservices (Routing Engine, Order Service, Inventory Service) run on Infrastructure as a Service or a managed container platform (IaaS/PaaS style, giving the team control over scaling and deployment configuration). The message queue and databases are frequently consumed as managed Platform as a Service offerings (a managed Kafka or managed PostgreSQL service), so the team doesn’t need to operate the underlying servers, patching, and replication by hand. Third-party components like carrier rate lookups or geocoding are consumed as Software as a Service through external APIs, since building and maintaining a global mapping and geocoding system in-house rarely makes sense compared to using an established provider.

12.4 Cost optimization

At the scale this system operates, infrastructure cost becomes a real engineering concern, not just a finance line item. Common levers include using cheaper spot or preemptible compute instances for non-latency-sensitive background jobs (like nightly reservation cleanup sweeps or offline route optimization), right-sizing the Routing Engine’s instance count based on observed traffic patterns rather than provisioning for worst-case peak at all times, and tiering data storage so old, rarely-accessed order history moves to cheaper cold storage instead of staying on the same fast, expensive database used for live orders.

Different data has different needs. Inventory, orders, cache, and analytics each get their own storage story.

13.1 Inventory data model

Inventory is typically sharded by warehouse ID or by a hash of the SKU, since inventory operations are naturally scoped to a single warehouse at a time. Each row commonly uses an optimistic concurrency version number to detect conflicting concurrent updates.

public class InventoryReservationService {

    private final InventoryRepository repository;

    // Optimistic locking: retries on version conflict instead of holding a lock
    public ReservationResult reserve(String warehouseId, String sku, int quantity) {
        int maxRetries = 3;
        for (int attempt = 0; attempt < maxRetries; attempt++) {
            InventoryRecord record = repository.find(warehouseId, sku);

            if (record.getAvailableQuantity() < quantity) {
                throw new InsufficientStockException(warehouseId, sku);
            }

            int expectedVersion = record.getVersion();
            int newAvailable = record.getAvailableQuantity() - quantity;

            boolean updated = repository.updateIfVersionMatches(
                warehouseId, sku, newAvailable, expectedVersion);

            if (updated) {
                return ReservationResult.success(warehouseId, sku, quantity);
            }
            // Version changed under us; another request updated first. Retry.
        }
        throw new ReservationConflictException(warehouseId, sku);
    }
}

The updateIfVersionMatches call is a single atomic database statement, roughly equivalent to UPDATE inventory SET available = ?, version = version + 1 WHERE warehouse_id = ? AND sku = ? AND version = ?. If zero rows are affected, someone else updated the record first, and the code retries — this is optimistic concurrency control, well-suited to inventory because conflicts, while possible, are relatively rare for any single SKU-warehouse pair compared to the read volume.

13.2 Partitioning strategies compared

Beyond sharding by warehouse ID, teams sometimes consider partitioning inventory data by product category or by geographic region instead, and it’s worth understanding why warehouse-based partitioning usually wins out for this specific workload. Partitioning by product category would mean a single order touching multiple categories (a phone and a phone case, say) would need to coordinate a reservation across two separate shards even when both items happen to sit in the very same physical warehouse, adding unnecessary cross-shard coordination for what is, physically, a single-location transaction. Partitioning by geographic region has a similar issue in reverse, since a single warehouse’s inventory would then be split across the boundary of whatever regional partitioning scheme was chosen. Partitioning by warehouse ID aligns the data partition boundary with the natural transactional boundary of the business operation itself — a reservation is always scoped to one physical warehouse — which is generally the right heuristic to reach for when choosing a partitioning key for any system, not just this one.

13.3 Consistent hashing for cache distribution

The Redis cache layer itself is typically distributed across multiple nodes rather than a single instance, using consistent hashing to decide which cache node holds the data for a given warehouse-SKU key. Consistent hashing arranges cache nodes and keys on a conceptual ring, so that when a cache node is added or removed — which happens routinely during scaling events — only a small fraction of keys need to move to a different node, rather than the entire cache needing to be reshuffled and effectively invalidated all at once. Without consistent hashing, scaling the cache layer up or down during a traffic spike would itself cause a temporary flood of cache misses hitting the database directly, which is exactly the kind of load the cache exists to prevent in the first place.

13.4 Caching layer

A Redis cache sits in front of the inventory database for read-heavy “is this in stock” checks shown to browsing customers. Writes always go to the database first; cache entries are invalidated (not just updated) on every successful reservation to avoid serving stale “in stock” data that leads to an oversell. Cache invalidation is deliberately event-driven rather than purely time-based, meaning the moment a reservation succeeds and changes a warehouse’s available count, an invalidation event is published so the corresponding cache entry is removed immediately, rather than waiting for a fixed time-to-live to expire naturally and risking a window where stale data could still be served to a browsing customer.

13.5 Load balancing approach

Layer 7 (application-aware) load balancing is used at the edge so requests can be routed based on path (for example, sending order-placement traffic to a differently-scaled pool than read-only product browsing traffic). Internally, service-to-service calls (Routing Engine to Inventory Service) typically use client-side load balancing with health-aware routing, so a slow instance is automatically deprioritized. This distinction between edge and internal load balancing matters because the traffic patterns and failure characteristics differ significantly: edge load balancing must defend against untrusted, highly variable external traffic including potential abuse, while internal load balancing operates in a more controlled environment and can therefore make more aggressive assumptions about instance health and afford tighter timeout and retry budgets.

13.6 Replication strategy

Each inventory shard typically has one primary node accepting writes and one or more read replicas serving lower-priority read traffic, such as analytics queries or admin dashboards that don’t need the absolute latest data. Replication between primary and replicas is usually asynchronous for performance, meaning there is a small, typically sub-second, window where a replica could serve slightly stale data — acceptable for dashboards, but never used for the reservation write path itself, which always reads and writes against the primary to guarantee correctness.

13.7 Indexing strategy

The Inventory table is indexed on the combination of warehouse ID and SKU, since nearly every query and update in the hot path looks up inventory by exactly that pair. A secondary index on SKU alone supports the common “which warehouses have this item” query used during candidate generation. Over-indexing is avoided deliberately, since every additional index slows down writes — and this table experiences extremely high write volume during peak sales, so the team is careful to add only indexes that earn their keep against real query patterns rather than speculative ones.

The routing capability is exposed internally as a well-defined API so other services (Order Service, admin tools, customer support tooling) can call it consistently.

{
  "orderId": "ord_8f21a",
  "destination": { "lat": 28.61, "lng": 77.20 },
  "items": [
    { "sku": "SKU-1001", "quantity": 1 },
    { "sku": "SKU-2044", "quantity": 2 }
  ],
  "promisedDeliveryBy": "2026-08-02T18:00:00Z"
}

// Response
{
  "orderId": "ord_8f21a",
  "plan": [
    { "warehouseId": "WH-04", "items": ["SKU-1001"], "reservationId": "res_991" },
    { "warehouseId": "WH-04", "items": ["SKU-2044"], "reservationId": "res_992" }
  ],
  "splitShipment": false
}

Notice the API is synchronous and idempotent — calling it twice with the same orderId returns the same plan rather than creating duplicate reservations, which matters given retries at the network layer are inevitable.

14.1 Why microservices here

Splitting Routing, Inventory, Capacity, and Fulfillment into separate services lets each scale and evolve independently. The Inventory Service might need to scale to handle millions of reads per second, while the Fulfillment Service scales with order volume, not browsing traffic — bundling them would force one bottleneck to dictate the scaling of the whole system.

14.2 Internal communication: REST vs. gRPC

Customer-facing APIs (Order placement) typically use REST/JSON for compatibility with web and mobile clients. Internal, high-frequency calls (Routing Engine to Inventory Service, potentially thousands of calls per second) often use gRPC for its lower serialization overhead and built-in support for streaming and strict typed contracts via Protocol Buffers.

14.3 Service discovery

With multiple instances of each microservice running behind the scenes and instances being added, removed, or replaced constantly as the system auto-scales, no service can afford to have a hardcoded list of its dependencies’ network addresses. Instead, a service discovery mechanism (often built into the container orchestration platform) maintains a live, continuously updated registry of which instances of, say, the Inventory Service are currently healthy and reachable. When the Routing Engine needs to call the Inventory Service, it asks the service discovery layer or a client-side load balancer for a healthy instance, rather than relying on a static configuration file that would quickly go stale as the fleet of instances changes throughout the day.

14.4 API versioning and backward compatibility

The routing API is versioned explicitly in its path (notice the /v1/ prefix in the example above), so that when the scoring logic or request shape needs to change in a breaking way, a new /v2/ endpoint can be introduced while existing callers on /v1/ continue working unaffected. New optional fields (like an added sustainabilityPreference flag) are added in a backward-compatible way whenever possible, meaning older clients that don’t send the field simply get the previous default behavior, rather than forcing every caller across the company to update simultaneously on a hard deadline. This matters enormously in a large organization where dozens of internal tools and services may depend on this API, each on their own release schedule.

Patterns are the shorthand experienced architects use to communicate a whole design decision in two words. Anti-patterns are the shorthand for the mistakes those same architects have already made themselves at least once.

15.1 Patterns used

15.2 Anti-patterns to avoid

Every point in this section comes from the scars of a real production incident somewhere in the industry — either directly, or as an obvious extrapolation from one.

16.1 Testing strategy

A system this sensitive to correctness needs a layered testing approach. Unit tests cover the scoring function itself, checking that specific inputs produce the expected relative ordering of candidates. Integration tests exercise the full routing flow against a real (test) database and message queue, verifying that a reservation is correctly created, expires correctly, and correctly rolls back on failure. Load tests simulate realistic peak traffic, including the specific popular-item contention patterns described earlier, since a system that performs fine under smooth, evenly distributed synthetic load can still fail badly the moment ten thousand people try to buy the same limited-edition item within the same thirty seconds. Finally, concurrency-focused tests specifically try to trigger race conditions on purpose — for example, firing many simultaneous reservation requests against a single unit of inventory in a test environment — to confirm that the optimistic concurrency logic genuinely prevents an oversell rather than merely making it statistically unlikely.

16.2 Common mistakes

• Treating distance as a proxy for delivery time without accounting for actual carrier network and road/rail access — a nearby warehouse in a remote area can be slower than a farther one on a major logistics corridor.
• Not accounting for warehouse capacity, leading to a “best” warehouse being flooded with orders it can’t actually pack on time.
• Recomputing the full routing decision on every retry instead of reusing the original reservation via an idempotency key, causing duplicate holds.
• Underestimating how often SKU-level inventory truth drifts from what a warehouse’s physical count actually is (a problem generally solved by periodic cycle counts and reconciliation jobs, not purely software).
• Rolling out a new scoring algorithm to 100% of traffic at once instead of gradually, making it hard to isolate whether a sudden change in fulfillment rate or cost was caused by the new logic or something else entirely.
• Allowing the Routing Engine to silently swallow errors from a downstream dependency instead of surfacing a clear signal, making production issues far harder to diagnose after the fact.
• Treating the scoring weights as a one-time decision made at launch rather than a living configuration that needs periodic revisiting as the warehouse network, carrier contracts, and business priorities inevitably change over time.

Looking at how the largest retailers and logistics platforms actually solve this problem is the fastest way to sanity-check your own design. Every one of them converges on the same broad shape.

The questions that come up over and over in interviews, architecture reviews, and Slack threads five minutes before a launch. Each answer traces back to a decision made earlier in this tutorial.

Multi-warehouse order fulfillment routing is, at its heart, a real-time optimization problem wrapped inside a distributed systems problem. The optimization is simple to describe — pick the best warehouse based on distance, cost, and stock — but the distributed systems challenges around consistency, availability, and scale are what make it genuinely hard to build well. Mastering this one problem deeply pays dividends well beyond warehouse routing itself, since so much of the reasoning here transfers directly to any system that must make a fast, high-stakes decision using data spread across many independent sources.

If you take one idea away from this guide, let it be this: the hardest part of warehouse routing is not the scoring math — it’s making sure the world hasn’t changed between the moment you make a decision and the moment you act on it. Every design choice in this system, from reservation TTLs to optimistic concurrency to circuit breakers, exists to manage that gap safely.

19.1 Where to go from here

For anyone preparing for a system design interview on this topic, it helps to practice narrating the system in layers rather than jumping straight to a diagram: start with the problem and why naive solutions fail, walk through the four-stage decision pipeline, explain how correctness is protected under concurrency, and only then bring in scaling, availability, and monitoring concerns. Interviewers consistently reward candidates who can explain why a design choice exists — the specific failure mode it prevents — over candidates who can only recite component names from a memorized diagram. The warehouse routing problem is an excellent vehicle for demonstrating exactly that kind of reasoning, because nearly every component in this architecture exists to solve one very specific, nameable failure mode: the cache exists because the database can’t handle read volume alone, the reservation TTL exists because payment confirmation isn’t instant, the circuit breaker exists because dependencies fail, and the message queue exists because fulfillment shouldn’t block checkout. Understanding those individual “whys” is what separates a memorized diagram from genuine system design intuition, and it is that same habit of asking “what specific problem does this piece solve” that will serve you well well beyond this one topic, across essentially every system design conversation you will ever have in your career.