A gift card looks like one of the simplest products in retail: someone pays money, gets a card or a code, and later someone spends it. But underneath that simplicity sits a genuinely hard distributed systems problem. A gift card is, at its core, a small bank account that must be perfectly accurate, must work identically whether it’s swiped at a physical cash register or typed into a checkout page on a phone, and must survive network failures, duplicate requests, and even fraud attempts — all without ever letting the balance drift by even one cent.

Gift cards started as simple paper certificates in the early 20th century, redeemable only at the single store that printed them. The shift to magnetic stripe cards in the 1990s let large retail chains track balances electronically for the first time, using a central mainframe that every store register called over a private network. That worked because there was only one channel: the physical store.

Everything changed when e-commerce, mobile apps, and marketplace partnerships arrived. Today, the same gift card might be purchased on a retailer’s website, redeemed partially at a physical checkout counter, checked for balance through a mobile app, and eventually used to pay for a purchase made through a third-party marketplace integration. The “one channel, one system” model breaks down completely. This is exactly the kind of system we will design in this tutorial: a gift card platform built to be the single source of truth across every sales channel a business operates.

By the end of this tutorial you will understand how to design the full system: how cards are issued and paid for, how balances are tracked with financial-grade accuracy, how redemption is made safe against double-spending and network retries, and how the platform stays available and fast even during the busiest shopping days of the year.

1.1 Why gift cards are a favorite system design interview topic

Interviewers love asking candidates to design a gift card or wallet system because it forces a decision on almost every fundamental distributed systems trade-off in a single, easy-to-explain domain. Unlike a social media feed or a URL shortener, there is no ambiguity about what “correct” means: the numbers must add up, always. This makes it an excellent lens for evaluating whether an engineer understands consistency models, idempotency, and failure handling, rather than just knowing how to draw boxes and arrows.

It also mirrors a huge number of real production systems that engineers will actually build in their careers: digital wallets, loyalty point balances, prepaid mobile recharge systems, in-app currency for games, and corporate expense credit systems all share the exact same underlying shape as a multi-channel gift card platform. Mastering this design pattern transfers directly to many other domains, which is exactly why it keeps appearing in interviews at companies that have nothing to do with retail or gift cards at all.

1.2 What “multi-channel” really means

It is worth pausing on the word “multi-channel” because it changes the design in subtle but important ways compared to a single-channel gift card system. A single-channel system, like the original mall gift card that only worked at mall registers, can get away with a simple centralized database and a slow, batch-oriented reconciliation process, because there is only one place transactions originate from. A multi-channel system has to assume that transactions can originate from anywhere, at any time, from clients with wildly different reliability characteristics — a retail POS terminal on a spotty in-store Wi-Fi connection behaves very differently from a mobile app on a fast cellular network, which behaves very differently from a batch webhook call from a third-party marketplace partner. The architecture we build in this tutorial has to serve all of these clients through one unified, consistent core, while still being flexible enough at the edges to speak each channel’s native protocol.

Before drawing any boxes and arrows, we need to be precise about what problem we are solving. A vague goal like “build a gift card system” leads to vague architecture. Let’s break the real business and technical problems apart.

2.1 The core business problem

• Money must never leak. If a $50 gift card can somehow be redeemed for $60 total across two channels because of a race condition, that’s real financial loss, multiplied across millions of cards.
• Every channel must agree on the truth. A retail cashier and a mobile app must show the exact same balance at the exact same moment, even though they talk to completely different front-end systems.
• Retries must be safe. Point-of-sale terminals and mobile networks are unreliable. If a redemption request times out and the terminal automatically retries, the customer’s balance must not be charged twice.
• Regulations must be respected. Many countries require unredeemed gift card balances to eventually be reported as unclaimed property (“escheatment”), and some regions ban expiry dates altogether. The system must be flexible enough to encode different rules per region.

2.2 The core technical problem

Strip away the business language and the technical challenge is: design a highly available, strongly consistent ledger system that can be written to and read from many different front-end channels simultaneously, at high throughput, with strict idempotency guarantees. This sits at the intersection of distributed systems (consistency, partitioning, replication), financial engineering (double-entry bookkeeping), and API design (supporting many heterogeneous clients).

2.3 Functional requirements

Before architecture, it helps to write down exactly what the system must do. These functional requirements drive every later design decision:

• Issue a new gift card with a specified monetary amount, currency, and optional design/theme, triggered from any supported channel.
• Allow a customer to check the current balance of a card instantly, from any channel.
• Allow full or partial redemption of a card’s balance at checkout, across web, mobile, retail POS, call center, and partner marketplace channels.
• Support refunds that credit money back to a card, for example when a purchase made with a gift card is returned.
• Support voiding a card entirely, for example when fraud is confirmed after issuance.
• Provide a full, queryable transaction history for any card, for customer support and audit purposes.
• Enforce regional rules around expiry dates and unclaimed-property escheatment.
• Give business stakeholders real-time and historical reporting on issuance volume, redemption volume, and outstanding liability.

2.4 Non-functional requirements

Just as important as what the system does is how well it must do it. These non-functional requirements are what actually separate a toy design from a production-grade one:

2.4.1 The cost of getting this wrong

It helps to be concrete about what failure actually looks like in this domain, because it clarifies why the design choices throughout this tutorial lean so heavily toward correctness over convenience. If the Redemption Service allows a race condition where two simultaneous requests both read a $10 balance and both successfully debit $10, the business has just given away $10 it never received payment for, multiplied across every card affected by the same bug before it is caught. If the Ledger Service silently drops an event during a deploy, a customer’s card balance in the cache might show funds that do not actually exist in the authoritative ledger, leading to an awkward and reputation-damaging conversation at checkout when the real balance is finally checked. If reconciliation between the ledger and the payment processor’s settlement reports is only run once a week instead of continuously, a systematic bug could run undetected for days, quietly compounding losses the entire time. None of these are hypothetical edge cases in the gift card industry — they are the well-known failure modes that shape why experienced teams treat idempotency, double-entry bookkeeping, and continuous reconciliation as non-negotiable requirements rather than nice-to-have polish.

2.5 Constraints and assumptions

Every real design also has to state its assumptions explicitly, because they shape which trade-offs are acceptable. In this tutorial, we assume the platform serves a large retailer or marketplace operator with multiple owned channels (web, mobile, in-store) plus third-party partner integrations; that transaction volume can spike far above baseline during predictable seasonal events; that different regions have different legal requirements around expiry and escheatment; and that the platform must integrate with an existing, separately-owned payment processing system for the actual capture of the customer’s original payment method, rather than owning card payment processing itself.

Let’s design the system layer by layer, starting from how a request enters the platform all the way down to where data is stored. Every box in the diagram below includes the specific component responsibility, including the entry-point components like the load balancer and API gateway, because in real interviews and real production systems, “where does the request land first” is often the most under-specified part of a design.

3.0 Reading the diagram, top to bottom

Follow a single request through this diagram to understand why each layer exists. A customer opens the web storefront, which loads its static assets from the CDN for speed. When that customer submits a request — say, buying a $50 gift card — the request first hits the WAF, which strips out anything that looks like an attack. It then reaches the Load Balancer, which picks a healthy API Gateway instance to handle it. The API Gateway checks the customer’s authentication token, applies rate limits, validates the request shape, and only then forwards it inward to the Channel Adapter Service, which normalizes it into the platform’s canonical internal format before handing it to the Issuance Service. Every one of these layers exists to answer one specific question — is this traffic well-formed, is it authenticated, is it within safe limits, is it in the right shape — before the request is allowed anywhere near the code that actually touches money.

This layered filtering is deliberate and important to call out in an interview: it is far cheaper to reject a malformed or malicious request at the edge (the WAF or Gateway) than to let it reach the Issuance or Redemption Service and discover the problem there. Each layer should assume the layers before it did their job, but never assume it can skip its own checks — defense in depth, not defense in one place.

3.1 Component-by-component breakdown

3.2 Data ownership per service

A rule this design follows strictly: every piece of data has exactly one service that owns writes to it, even though many services may read a copy or a projection of it. The Issuance Service owns the card record and its metadata (design, denomination, expiry rule). The Ledger Service owns the append-only transaction log and is the only service permitted to write a debit or credit entry. The Catalog Service owns gift card designs and denominations available for purchase. The Fraud Service owns risk scores and rule configurations. This clear ownership prevents the classic distributed-monolith trap where five different services all write to the same table and nobody can safely change its schema.

3.2.1 Handling partial failures across services

A common early mistake when drawing a diagram like this is to assume every arrow always succeeds. In reality, the Issuance Service’s call to the Fraud Service can time out, the Ledger Service’s database write can momentarily fail during a leader election, and the Kafka publish can be delayed during a broker rebalance. The design treats every one of these arrows as a potential failure point with an explicit, documented behavior: fraud check timeouts fall back to a conservative default rather than blocking indefinitely, ledger writes retry with exponential backoff a small, bounded number of times before surfacing a clear error to the caller, and Kafka publishes go through the outbox pattern described later so they are never silently lost. Drawing the happy path is the easy ten percent of the design; deciding what happens when each arrow fails is the harder, more valuable ninety percent, and it is exactly what separates a junior design from a senior one in an interview setting.

3.3 Why a Channel Adapter Service instead of channel-specific gateways

An alternative design would give each channel — web, mobile, POS, partner — its own dedicated backend-for-frontend service. That works, but it tends to duplicate business logic across four or five nearly-identical services, and any bug fix in, say, the redemption validation logic then has to be replicated everywhere. Instead, this design uses one thin Channel Adapter Service whose only job is translating each channel’s request and response format into one canonical internal contract, while all business logic lives once, centrally, in the Issuance, Balance, and Redemption services. The trade-off is that the adapter service must be carefully versioned so that a change for one channel’s protocol quirk does not silently break another channel.

Now let’s go one level deeper and look at exactly what happens inside the platform for the two most important operations: issuing a gift card and redeeming one. These two flows are where correctness matters most.

A useful mental model when reading these sequence diagrams is to ask, at every arrow, three questions: what happens if this call never returns, what happens if it returns an error, and what happens if the caller retries the entire operation from scratch after either of those. Answering all three questions for every single arrow is what turns a diagram from a nice picture into an actual, implementable, production-grade design. We will call out the answers to these questions explicitly as we walk through each flow below, rather than leaving them as an exercise for later, because in practice “later” is exactly when these gaps turn into incidents.

4.1 Issuance flow, step by step

Notice that the fraud check happens before the card number is generated. This ordering matters: generating and storing a card number is a real resource commitment, so we reject risky requests as early as possible, before we do meaningful work.

4.2 Redemption flow, step by step

The idempotency key is the single most important detail in this flow. Every redemption request carries a client-generated unique key. If the POS terminal’s network connection drops after the debit succeeds but before the response arrives, the terminal will retry with the same idempotency key. The Redemption Service recognizes the key, skips re-debiting the ledger, and simply returns the previously computed result.

4.3 Optimistic locking on the ledger

Because many redemption requests for popular cards can arrive close together (think of a corporate gift card shared by a whole team), the Ledger Service uses optimistic concurrency control: every card row carries a version number. A debit only succeeds if the version read at the start of the transaction still matches the version at commit time; otherwise it retries with the freshly read balance.

@Transactional
public LedgerResult debit(String cardId, BigDecimal amount, String idempotencyKey) {
    GiftCard card = cardRepository.findByIdForUpdate(cardId)
        .orElseThrow(() -> new CardNotFoundException(cardId));

    if (card.getBalance().compareTo(amount) < 0) {
        throw new InsufficientBalanceException(cardId, amount, card.getBalance());
    }

    int updatedRows = cardRepository.debitWithVersionCheck(
        cardId, amount, card.getVersion());

    if (updatedRows == 0) {
        throw new OptimisticLockException("Card " + cardId + " was updated concurrently");
    }

    LedgerEntry entry = LedgerEntry.builder()
        .cardId(cardId)
        .type(LedgerEntryType.DEBIT)
        .amount(amount)
        .idempotencyKey(idempotencyKey)
        .balanceAfter(card.getBalance().subtract(amount))
        .createdAt(Instant.now())
        .build();

    ledgerRepository.save(entry);
    return new LedgerResult(entry.getBalanceAfter(), entry.getId());
}

findByIdForUpdate takes a row-level lock (SELECT ... FOR UPDATE) so two concurrent transactions on the same card cannot both read a stale balance. This trades a small amount of latency under contention for absolute correctness, which is exactly the right trade-off for money.

4.4 Balance check flow

Balance checks are the highest-volume read in the entire system — customers check balances far more often than they redeem — so this path is optimized purely for speed and is explicitly designed to tolerate a small amount of staleness.

4.5 Refund flow

Refunds are a credit back onto a card, typically triggered when a purchase made partly or wholly with a gift card is returned. This flow reuses the same ledger and idempotency infrastructure as redemption, just with the entry direction reversed.

Notice that the Refund Service is deliberately kept separate from the Redemption Service even though both eventually write to the same Ledger Service. Refunds usually originate from support staff or automated return systems rather than a customer at checkout, carry different fraud-risk characteristics (refund abuse is a distinct attack pattern from redemption abuse), and often need additional authorization checks. Splitting them keeps each service’s logic focused and its risk profile easy to reason about independently.

A gift card moves through a well-defined set of states from the moment it is requested until it is fully spent, expired, or voided. Modeling this explicitly as a state machine, rather than as loose boolean flags, prevents an entire category of bugs where a card ends up in an impossible combination of states.

5.1 The ledger as the single source of truth

A recurring theme in financial systems design is: never trust a mutable “current balance” column as the primary record. Instead, treat the balance as a derived value, computed by summing an append-only ledger of transactions. The giftcards table can still keep a cached current_balance column for fast reads, but the ledger table is what auditors, reconciliation jobs, and dispute investigations trust.

This is standard double-entry bookkeeping applied to software. When a card is issued, the business receives cash (debit) and takes on a liability to the customer (credit) — the business owes that customer $50 of goods or services in the future. When the customer redeems $20, the liability shrinks (debit) and it becomes recognized revenue (credit). Every entry always balances to zero, which makes the whole ledger self-checking: if debits and credits ever stop matching, something is broken and reconciliation will catch it automatically.

5.2 Cross-channel consistency

Because balance reads can come from a Redis cache for speed, there is a small window where a channel could read a slightly stale balance. The design mitigates this in two ways: first, redemption itself never trusts the cache for the final decision — it always re-validates against the authoritative ledger inside the same transaction that performs the debit. Second, every successful debit or credit publishes a BalanceChanged event to Kafka, which invalidates or refreshes the cache within milliseconds across every service instance.

5.3 Worked example: a card’s full lifecycle

Walking through one concrete card end to end makes the abstract state machine much easier to internalize. Imagine a $100 gift card purchased on the web storefront as a birthday present.

1. The card is created in the Requested state the moment checkout begins, before payment is confirmed.
2. The payment processor confirms the $100 charge succeeded, so the Issuance Service transitions the card to Active and the Ledger Service records a $100 credit (liability) entry.
3. Two weeks later, the recipient spends $35 at a physical store. The Redemption Service debits $35, and the card moves to PartiallyRedeemed with a $65 balance remaining.
4. A month later, the recipient returns part of that purchase, and a $10 refund is credited back onto the card through the Refund Service, bringing the balance to $75, while the card remains in PartiallyRedeemed.
5. Over the following year the recipient spends the remaining $75 gradually across the mobile app and a partner marketplace. Once the balance hits zero, the card automatically transitions to FullyRedeemed.

If, instead, the recipient had never spent the remaining balance and the region’s expiry rule allowed a five-year expiry, the card would transition to Expired once that period elapsed, and — depending on local law — eventually to Escheated, where the outstanding balance is reported and remitted to the relevant state or national unclaimed-property authority rather than being kept as revenue by the business.

5.4 Regional rule configuration

Because expiry and escheatment rules vary so widely — some US states prohibit gift card expiry entirely, others allow it only after a minimum number of years, and many countries have no escheatment concept at all — the platform stores these rules as data, not code. A RegionRuleSet record, keyed by issuing region, defines the minimum legal expiry period, whether dormancy fees are permitted, and the escheatment reporting cadence. The Issuance Service looks up the correct rule set at card creation time based on the purchasing customer’s region, so a single global codebase can correctly serve dozens of different legal jurisdictions without a code deployment every time a law changes.

6.0.1 Weighing the trade-offs from a business perspective

It is easy to evaluate these advantages and disadvantages purely from an engineering lens, but a mature system design discussion also weighs them against business impact. The strong consistency guarantees this design insists on directly protect revenue recognition accuracy and audit outcomes, which matters enormously to finance and legal stakeholders even if it is invisible to the end customer. The operational overhead of running many microservices, on the other hand, is a real, ongoing engineering cost — measured in on-call burden, infrastructure spend, and onboarding time for new engineers — that has to be justified against the scale and channel diversity the business actually has today, not the scale it might have someday. A senior engineer presenting this design should be able to articulate both sides of that trade-off clearly rather than presenting microservices as an unquestioned best practice.

6.1 Key trade-off: consistency versus availability

This system deliberately favors consistency over availability for the write path (issuance and redemption), following a CP (Consistent, Partition-tolerant) stance from the CAP theorem for money-moving operations. For read-only operations like “show me my balance” or “browse gift card designs,” the system instead favors availability, tolerating a few milliseconds of staleness in exchange for speed and resilience — an AP (Available, Partition-tolerant) stance. This split is intentional and should be called out explicitly in any interview discussion.

6.2 Key trade-off: microservices versus a modular monolith

It is worth honestly weighing the alternative to everything described so far. A well-structured modular monolith — one deployable application internally organized into clean modules for issuance, ledger, redemption, and fraud — would be simpler to operate, easier to debug with a single stack trace, and would avoid the network latency and partial-failure complexity of service-to-service calls. For a business launching in a single country with a single sales channel, this is often the right starting point, and it is a mistake to reach for a ten-service architecture before the scale or organizational structure actually demands it.

The tipping point toward microservices in this design is specifically the combination of multiple independent channels, each with different traffic patterns, plus a fraud detection component that needs to iterate much faster than the core ledger. When those forces are present, the operational cost of microservices buys real benefits: independent scaling, independent deployment risk, and independent team ownership. When they are not present yet, that same cost is pure overhead.

6.3 Key trade-off: synchronous fraud checks versus asynchronous post-hoc review

The design in this tutorial checks fraud risk synchronously, before completing issuance or redemption, which adds latency to every request but blocks bad transactions before money moves. An alternative is to approve transactions optimistically and run fraud detection asynchronously afterward, flagging suspicious ones for reversal or account freezing. The synchronous approach is safer for money movement but slower and occasionally blocks legitimate customers on a false positive; the asynchronous approach is faster and more forgiving to genuine customers but accepts that some fraud will briefly succeed before being caught. This design chooses synchronous checks for the core money-movement path because reversing a completed gift card redemption after the fact is operationally painful and often impossible once goods have left the store.

Gift card traffic is famously spiky: it multiplies many times over during the holiday shopping season and around major sale events. The architecture must handle both the steady daily baseline and sudden multi-times traffic surges without falling over.

7.1 Horizontal scaling of stateless services

The API Gateway, Channel Adapter Service, Issuance Service, and Balance Service are all stateless — they hold no in-memory session state between requests — so they scale horizontally simply by adding more container replicas behind the load balancer. Auto-scaling policies watch CPU usage, request queue depth, and p99 latency, and add pods proactively before latency degrades.

7.2 The redemption hot path

The Redemption Service is the most latency-sensitive component, because a slow gift card redemption directly stalls a physical checkout line. Three techniques keep it fast:

• Connection pooling to the database, so each request reuses an existing connection instead of paying the cost of establishing a new one.
• Read-through caching for balance pre-checks, falling back to the database lock only for the final commit.
• Sharding the ledger database by card ID hash, so that write load is distributed across many database partitions rather than a single write node becoming a bottleneck.

7.3 Capacity example

Consider a retailer expecting 5,000 redemptions per second at Black Friday peak. If each redemption requires roughly 8 milliseconds of database transaction time on a single shard, one shard alone can theoretically support around 125 transactions per second before saturating — so the design needs at least 40 logical shards to comfortably handle peak load with headroom, plus caching to absorb balance-check reads separately from the write path.

7.4 Database indexing strategy

Every redemption request needs to find a card by its ID almost instantly, so the card table’s primary key index is the single most important index in the entire system, and it should be a clustered or otherwise physically-organized index on the sharded card ID. The ledger table additionally needs a composite index on (card_id, created_at) to make “show me this card’s transaction history” queries fast, and a separate unique index on idempotency_key to make duplicate-key lookups an O(1) operation rather than a table scan.

7.5 Connection pooling and backpressure

Database connections are a finite, relatively expensive resource, so every service maintains a bounded connection pool (commonly sized using the formula: number of CPU cores multiplied by a small constant, then tuned empirically) rather than opening a new connection per request. When traffic exceeds what the pool and downstream database can handle, the API Gateway applies backpressure: it starts returning fast, explicit 429 Too Many Requests responses rather than letting requests queue indefinitely and eventually time out anyway. A fast, honest “try again shortly” is far better for both the customer experience and system stability than a slow, silent failure.

7.6 Read replica lag and its implications

Because the Balance Service reads from PostgreSQL read replicas for non-authoritative balance checks, there is inherent replication lag — typically single-digit milliseconds under normal load, but potentially seconds under heavy write pressure. The design explicitly documents this lag as an accepted trade-off for balance-check reads, while ensuring the actual redemption decision never uses a replica; it always reads and locks against the primary. This distinction — which reads can tolerate replica lag and which absolutely cannot — should be written down explicitly in any system design document, since it is exactly the kind of detail that is easy to get wrong under time pressure.

7.7 Batching and bulk operations

Partner marketplace integrations frequently need to check the balance of many cards at once, for example when reconciling their own nightly settlement reports against the platform. Exposing a dedicated batch balance-lookup endpoint, backed by a single multi-row query instead of forcing partners to make thousands of individual sequential requests, dramatically reduces both partner-side latency and the total request volume hitting the API Gateway. The general principle is worth stating explicitly: whenever a client’s real-world workflow naturally involves many related items, the API should offer a batched shape for that workflow rather than only exposing single-item endpoints and hoping clients batch responsibly on their own.

7.8 Load testing methodology

Before every major seasonal peak, the platform runs load tests that replay realistic, anonymized traffic patterns from the previous year’s peak day at increasing multiples of volume, rather than synthetic uniform traffic, since real gift card traffic clusters heavily around specific minutes (for example, immediately after a marketing email send) rather than spreading evenly across a day. These tests validate not just raw throughput numbers but also confirm that auto-scaling policies react quickly enough, that the database connection pool does not become a bottleneck before the compute layer does, and that alerting correctly fires if any of these limits are approached during the test itself.

A gift card platform effectively becomes part of the critical path of checkout for any retailer that uses it. If it goes down, checkout lines stall and online carts cannot complete. Reliability is not optional here — it is the product.

8.1 Multi-region deployment

The database primary lives in one region, with synchronous replication to a same-region standby (for fast, zero-data-loss failover) and asynchronous replication to a secondary region (for disaster recovery if the whole region goes down). Global DNS health checks detect a regional outage and redirect traffic, accepting a small amount of potential data loss for the rare case of a full regional failure — a deliberate, documented trade-off.

8.2 Graceful degradation

Not every failure needs to become a full outage. The design defines explicit degraded modes:

• If the Fraud Detection Service is slow or unavailable, the Issuance Service falls back to a conservative default (approve small amounts automatically, queue larger ones for manual review) rather than blocking all issuance.
• If Redis is unavailable, balance reads fall back directly to the read replica database, at higher latency but without an outage.
• If Kafka is unavailable, the Ledger Service still commits transactions to the database (the source of truth) and queues events locally to publish once Kafka recovers, so notifications are delayed but no money is lost.

8.3 Circuit breakers between services

@Service
public class FraudServiceClient {

    private final CircuitBreaker circuitBreaker;
    private final RestTemplate restTemplate;

    public RiskDecision evaluate(RiskRequest request) {
        return circuitBreaker.executeSupplier(() ->
            restTemplate.postForObject("/fraud/evaluate", request, RiskDecision.class)
        );
    }

    // Fallback used when the circuit is open (Fraud Service is unhealthy)
    public RiskDecision fallbackDecision(RiskRequest request, Throwable t) {
        return RiskDecision.conservativeDefault(request.getAmount());
    }
}

The circuit breaker trips after a configured error threshold, stops sending requests to the failing Fraud Service for a cooldown period, and routes to a conservative fallback instead — preventing one struggling dependency from cascading into a full platform outage.

8.4 Backup and disaster recovery

Beyond live replication, the platform takes continuous write-ahead log backups of the primary database, allowing point-in-time recovery to any second within the retention window, alongside periodic full snapshots. Recovery Point Objective (RPO) — the maximum acceptable data loss — is targeted at under one second for same-region failures thanks to synchronous replication, and under a few seconds for full regional failover thanks to near-real-time asynchronous replication. Recovery Time Objective (RTO) — how long recovery takes — is targeted at under five minutes for same-region failover, driven primarily by automated health-check and DNS propagation timing rather than manual intervention.

8.5 Chaos testing

Reliability claims are only as good as the tests behind them, so the platform runs regular chaos engineering exercises in a staging environment that mirrors production: randomly killing service instances, injecting artificial network latency between the Redemption Service and the Ledger Service, and simulating a full database failover mid-transaction. These exercises verify, on a recurring schedule rather than just once at launch, that circuit breakers actually trip, that failover actually completes within the target RTO, and that no transaction is ever left in an ambiguous half-committed state.

8.6 Idempotency as a reliability mechanism, not just a correctness one

It is worth explicitly connecting idempotency back to availability. Because every write is idempotent, the platform can be aggressive about client-side and gateway-level retries during transient failures — a dropped connection, a brief database failover — without ever risking a double charge. This means the system can recover from many classes of transient failure automatically, simply by retrying, rather than requiring a human to intervene or a customer to notice a failed transaction. Idempotency is as much a reliability feature as it is a correctness feature.

8.7 Defining and measuring SLOs

Service Level Objectives turn “the system should be reliable” into something measurable and actionable. This platform defines its redemption path SLO as 99.99 percent of requests succeeding within 200 milliseconds, measured over a rolling 28-day window, with an error budget of roughly four minutes of allowed downtime-equivalent degradation per month. When the error budget is being consumed faster than expected, the team deliberately slows down the pace of new feature releases and shifts focus to reliability work until the budget recovers — a discipline that keeps reliability an ongoing, data-driven practice rather than a one-time architectural decision made at launch and never revisited.

8.8 Handling planned maintenance without downtime

Database schema migrations on the Ledger Service are designed to be backward compatible at every intermediate step: a new column is always added as nullable first, backfilled in the background, and only made mandatory once every service instance has deployed code that populates it, following an expand-and-contract migration pattern. This lets the platform perform schema changes on its most critical table without any maintenance window or service interruption, which matters enormously for a system that effectively cannot afford scheduled downtime given how many checkout flows depend on it continuously across time zones.

Gift cards are effectively bearer instruments — whoever holds a valid card number and PIN can typically spend the balance. That makes security a first-class design concern, not an afterthought.

9.1 Card number and PIN generation

Card numbers must be generated using a cryptographically secure random number generator, never a predictable sequence, to prevent attackers from guessing valid card numbers (“card cracking”). PINs should be hashed at rest exactly like passwords, using a slow hash function such as bcrypt or Argon2, never stored in plaintext.

public GiftCardCredentials generateCredentials() {
    SecureRandom secureRandom = new SecureRandom();

    String cardNumber = generateLuhnValidNumber(secureRandom, 16);
    String pin = String.format("%06d", secureRandom.nextInt(1_000_000));
    String pinHash = passwordEncoder.encode(pin); // BCrypt

    return new GiftCardCredentials(cardNumber, pin, pinHash);
}

9.1.1 What the Fraud Detection Service actually evaluates

It is worth being specific about what “fraud scoring” means in practice rather than treating it as a black box. For issuance, the Fraud Service looks at signals such as how many cards a given payment method or account has purchased in the last hour compared to its historical baseline, whether the shipping or delivery email address has been associated with previous chargebacks, and whether the purchase amount is unusually large relative to the customer’s typical order size. For redemption, it looks at signals such as how many different physical store locations a single card has been used at within a short time window (a strong signal of a cloned or stolen card number), whether the redemption device fingerprint matches previous successful redemptions on that card, and whether the transaction velocity across an entire card batch (cards issued together in one promotional campaign) suggests coordinated abuse rather than independent customer behavior. Each signal feeds into a weighted risk score, and only transactions crossing a configurable threshold are held for either a fallback default action or manual review, so that the overwhelming majority of legitimate transactions pass through with no added friction at all.

9.2 Rate limiting and bot defense against card cracking

The most common real-world attack on gift card systems is automated “balance checking” — bots trying thousands of card number and PIN combinations to find funded cards. Defenses include strict per-IP and per-account rate limits at the API Gateway, CAPTCHA challenges after repeated failures, and Fraud Service rules that flag accounts probing many different card numbers in a short window.

9.3 Transport and data security

• TLS 1.2 or higher on every network hop, including internal service-to-service calls, not just the public-facing edge.
• Field-level encryption for card numbers at rest in the database, with access controlled through a dedicated encryption service or KMS (Key Management Service), so a database breach alone does not expose usable card numbers.
• Tokenization: after issuance, most internal services reference a card by an opaque token or internal ID rather than the raw card number, minimizing the surface area that ever touches sensitive data — similar in spirit to how PCI DSS token vaults protect credit card numbers.
• OAuth2 client-credentials flow for partner marketplace integrations, with scoped API keys limited to only the operations that partner needs (for example, redemption only, no issuance).

9.4 Zero Trust between internal services

Even inside the private network, services authenticate to each other using short-lived mTLS certificates or signed service tokens, rather than assuming that “being inside the network” is enough proof of trust. This limits blast radius if any single service or node is compromised.

9.5 Compliance considerations

Because gift cards touch payment flows, the platform must be mindful of PCI DSS (Payment Card Industry Data Security Standard) scope even though it typically does not store the customer’s original credit card number itself — that is handled by a separate, dedicated payment processor. Keeping the gift card platform out of direct PCI card-data scope is a deliberate architectural choice: the Issuance Service only ever receives a payment confirmation token from the payment processor, never raw card details, which significantly narrows the compliance surface area the gift card platform itself has to maintain.

Data privacy regulations such as GDPR in Europe or India’s Digital Personal Data Protection Act also apply, since gift card records are tied to customer identities and purchase history. The design supports data subject deletion requests by separating personally identifiable information (name, email) into a distinct customer profile service, referenced by an opaque customer ID from the ledger, so a deletion request can remove identifying details without breaking the financial audit trail, which by law often must be retained regardless of a deletion request.

9.6 Preventing internal abuse

Not every threat is external. Customer support agents and store employees have legitimate reasons to view card details, but issuing a full-balance card to themselves is a known internal fraud pattern in real retail gift card programs. The design mitigates this with role-based access control limiting who can trigger manual issuance or refunds, mandatory dual-approval above a configurable dollar threshold, and an immutable audit log of every manual action tied to the acting employee’s identity, reviewed automatically for suspicious patterns like repeated self-issuance.

9.7 Encryption key rotation

Encryption keys used to protect card numbers at rest are rotated on a regular schedule through the Key Management Service, and the design supports key versioning so that data encrypted under an older key can still be decrypted during a rotation window while all new writes use the current key. A background re-encryption job gradually migrates older records to the latest key version over time, without requiring a disruptive one-time migration that would lock the card table during the switch.

9.8 A brief threat model

Thinking through this system’s threat model out loud is a strong signal in an interview. The primary external threats are card-cracking bots probing for funded card and PIN combinations, credential-stuffing attacks against customer accounts that store saved gift cards, and man-in-the-middle attempts against POS terminals on poorly secured store networks. The primary internal threats are privileged employees abusing manual issuance or refund tools, and misconfigured service permissions accidentally exposing more data than intended to a lower-trust service. Each of these threats maps directly to a specific control already described in this section: rate limiting and anomaly detection for bot activity, multi-factor authentication for customer accounts, mandatory TLS for every terminal connection, role-based access control and audit logging for internal abuse, and the principle of least privilege enforced through scoped service tokens for misconfiguration risk.

9.9 Physical card and QR code security

For physical gift cards sold on a retail shelf, the card number is often printed on the card itself, with the PIN concealed under a scratch-off panel, which introduces a distinct threat: someone tampering with cards while they still sit on the shelf, peeling the panel, recording the PIN, and resealing it before a legitimate customer purchases the card. The Fraud Service mitigates this specific pattern by flagging any card whose very first redemption attempt happens unusually soon after issuance from a device or location inconsistent with the purchasing store, and by supporting an activation step where a card only becomes spendable after being scanned at the register during purchase, rendering a pre-recorded PIN from an unsold card useless to an attacker.

Because this system moves real money, observability is not just about uptime — it is about proving, continuously, that the books balance.

10.1 The three pillars

10.2 Financial-specific monitoring

Beyond standard infrastructure metrics, the platform runs a continuous “ledger balance” metric: the sum of all liability account credits minus debits, compared every few minutes against the sum of outstanding card balances. Any nonzero drift triggers an immediate high-severity alert, because in a financial ledger, drift of even a few cents indicates a serious bug, not noise to be tolerated.

// Example scheduled reconciliation check
@Scheduled(fixedRate = 300000) // every 5 minutes
public void checkLedgerDrift() {
    BigDecimal ledgerTotal = ledgerRepository.sumOutstandingLiability();
    BigDecimal cardBalanceTotal = cardRepository.sumAllActiveBalances();
    BigDecimal drift = ledgerTotal.subtract(cardBalanceTotal).abs();

    if (drift.compareTo(BigDecimal.ZERO) != 0) {
        alertService.pageOnCall(
            "Ledger drift detected: " + drift + " - halting new issuance until resolved");
        featureFlagService.disable("issuance.enabled");
    }
}

Notice the response to detected drift is not just an alert — it is an automatic safety action (pausing new issuance) until humans confirm the books are correct again. In financial systems, this kind of automatic circuit-breaking on data integrity is often more important than automatic scaling.

10.3 Correlation IDs across channels

Every request, regardless of channel, is tagged with a correlation ID at the API Gateway and passed through every downstream call and Kafka event. This lets support engineers trace a single customer’s “why didn’t my redemption work” complaint through every service it touched, in seconds rather than hours.

10.4 Alerting thresholds and on-call design

Not every anomaly deserves to wake an engineer at 3 a.m. The platform separates alerts into severity tiers: a spike in p99 redemption latency above 500 milliseconds pages the on-call engineer immediately, since it directly affects customers at checkout right now; a gradual rise in cache miss rate over an hour creates a lower-priority ticket for the next business day, since it affects efficiency but not correctness; and any nonzero ledger drift, however small, pages immediately and simultaneously halts new issuance automatically, since it represents a potential correctness failure rather than a performance one. This tiering prevents alert fatigue while ensuring the alerts that matter most are never buried among noisy ones.

10.5 Dashboards for different audiences

Engineering teams need a technical dashboard showing service health, latency percentiles, and error rates. Finance and operations teams need a completely different view: total outstanding gift card liability by region, issuance versus redemption volume trends, and any cards flagged by fraud review, presented without any infrastructure jargon. Building both from the same underlying event stream — rather than maintaining two disconnected reporting pipelines — keeps the technical and business views of the system from ever silently diverging from each other.

10.6 A worked tracing example

Imagine a customer support ticket reporting that a redemption at a store register failed with a generic error. With correlation IDs propagated end to end, the support engineer pulls up the trace for that specific request and sees the full timeline: the request arrived at the API Gateway, passed authentication, reached the Redemption Service, and then spent an unusually long 4 seconds waiting on the Fraud Service call before that call finally timed out and fell back to the conservative default — which, in this case, was configured to decline rather than approve for the transaction’s amount tier. Without distributed tracing, diagnosing this would require manually correlating log timestamps across five separate services by hand, a process that can take hours instead of the few minutes a trace view takes. This single example is often enough, in an interview setting, to justify why distributed tracing is treated as a required capability rather than an optional add-on for any system built from more than two or three services.

10.7 Synthetic monitoring

In addition to monitoring real customer traffic, the platform runs synthetic transactions — small, real (but internally flagged) issuance and redemption requests executed on a continuous schedule against production, using dedicated test card ranges that are excluded from customer-facing reporting. These synthetic transactions catch outages and correctness regressions even during unusually quiet traffic periods, such as the middle of the night in a given region, when real customer volume alone might not be enough to reveal a problem quickly.

10.8 Capacity forecasting from monitoring data

Metrics collected throughout the year feed directly into the capacity planning work described in the deployment section. Rather than guessing at next year’s peak traffic, the team fits a seasonal growth model against several years of historical redemption and issuance volume, broken down by channel, and uses it to set auto-scaling ceilings and pre-warmed capacity targets ahead of the next major shopping event. This turns what would otherwise be an anxious, reactive scramble in the days before a predictable peak into a routine, data-driven exercise completed weeks in advance, with load testing described earlier used to validate the forecast against real infrastructure behavior before the actual peak arrives.

Each microservice is packaged as a container and deployed on a managed Kubernetes cluster, giving the platform consistent scaling, rolling updates, and self-healing across cloud environments (AWS EKS, Google GKE, or Azure AKS all work equivalently well for this design).

11.1 Deployment pipeline

Canary deployments matter enormously here: a bad Ledger Service deploy that miscalculates debits by even one paisa per transaction would be catastrophic at scale, so the pipeline routes a small percentage of real traffic to the new version first and automatically compares error rates and reconciliation drift before a full rollout.

11.2 Infrastructure choices

• Container orchestration: Kubernetes, for self-healing, rolling deploys, and horizontal pod autoscaling.
• Managed database: A managed PostgreSQL service (e.g. Amazon RDS or Cloud SQL) for automated backups, patching, and failover, so the team is not hand-rolling database operations.
• Managed Kafka: A managed streaming service (e.g. Amazon MSK or Confluent Cloud) to avoid operating Kafka brokers manually.
• Secrets management: A dedicated secrets manager or vault for database credentials and encryption keys, injected at runtime rather than baked into container images.
• Infrastructure as Code: Terraform or equivalent, so every environment (staging, production, disaster-recovery region) is reproducible and reviewable.

11.3 Cost optimization

A platform sized for Black Friday peak traffic would be wildly over-provisioned for a normal Tuesday in March. Cost optimization here relies on horizontal auto-scaling that shrinks stateless service pods back down during quiet periods, scheduled scaling that pre-emptively adds capacity ahead of known peak events (based on the previous year’s traffic curve) rather than reacting after latency already degrades, and using cheaper spot or preemptible compute instances for non-critical batch workloads like the nightly reconciliation report generation, while keeping the Ledger Service itself on stable, reserved capacity given its criticality.

11.4 Environment parity

Using Infrastructure as Code to define staging, production, and disaster-recovery environments from the same templates ensures a bug that only reproduces under production-scale load or production-like data volume can still be caught in staging before release, rather than surfacing for the first time in front of real customers. Feature flags further decouple deployment from release: new functionality can be deployed to production dark (inactive) and enabled gradually for an internal audience first, then a small customer percentage, before a full rollout — an especially valuable safety net for anything touching the issuance or redemption path.

11.5 Regional deployment for data residency

Some jurisdictions require that financial transaction data for their citizens remain stored within the country or region’s borders. Rather than treating this as an exception handled with special-case code, the platform’s Terraform-defined regional stacks are designed to be genuinely independent from the start, each with its own database cluster and its own Kafka cluster, coordinated only through a thin, asynchronous cross-region reporting layer for global business intelligence. A customer’s card and ledger data lives entirely within their region’s stack, satisfying data residency requirements structurally rather than through after-the-fact compliance patches.

11.6 Runbooks and operational readiness

A design is only as reliable as the team’s ability to respond when something inevitably goes wrong at 2 a.m. Every alert defined in the monitoring section is paired with a written runbook that a first-time on-call engineer, not just the original author, can follow: how to confirm whether ledger drift is real or a false positive from a replication delay, how to manually re-enable issuance once the underlying cause is fixed, and how to safely roll back a bad deploy of the Redemption Service without losing in-flight idempotency state. These runbooks are treated as living documents, reviewed and rehearsed during the chaos testing exercises described earlier, so that the first real incident is never also the first time anyone has actually walked through the response steps.

11.7 Multi-cloud and vendor lock-in considerations

Although this design assumes a single cloud provider for simplicity, the core building blocks — Kubernetes, PostgreSQL, Kafka, and Redis — are all available as managed offerings across every major cloud provider, and none of the business logic in the Issuance, Ledger, or Redemption services depends on a proprietary, provider-specific service. This is a deliberate choice: it keeps a future multi-cloud or cloud-migration strategy realistic without a ground-up rewrite, even though the team is not paying the extra operational complexity of actually running multi-cloud from day one. It is worth naming this trade-off explicitly in an interview — building with portable, open technologies costs a small amount of convenience up front in exchange for real optionality later.

12.1 Choosing the database

The Ledger Service uses PostgreSQL, a relational database, because the core requirement is strong ACID transactions: Atomicity, Consistency, Isolation, and Durability. A gift card redemption absolutely must be all-or-nothing — either the debit and the ledger entry both commit, or neither does. Relational databases with mature transaction support are the natural fit; a document store optimized for eventual consistency would fight against this requirement rather than support it.

12.2 Caching strategy

Redis is used for two distinct purposes, and it is worth being explicit about the difference:

• Cache-aside for balance reads: the Balance Service checks Redis first; on a miss, it reads the replica database and populates the cache with a short time-to-live.
• Distributed rate limiting and idempotency: Redis’s atomic increment and set-with-expiry operations make it a natural fit for tracking request counts per API key and storing idempotency keys with automatic expiration.

12.3 Load balancing strategy

The Layer 7 load balancer uses least-outstanding-requests balancing rather than simple round robin, because gift card requests vary significantly in cost — an issuance request that triggers a fraud check takes longer than a cached balance lookup — and least-outstanding-requests naturally routes new traffic away from instances that are still busy with slow requests.

12.4 Sharding the ledger

As covered in the performance section, the ledger database is sharded by a hash of the card ID. This keeps all operations on a single card routed to a single shard (avoiding distributed transactions across shards for the common case), while spreading overall write load across many database nodes.

12.5 Handling shard rebalancing

Over time, as data volume grows, the number of shards may need to increase. Rather than a simple modulo-based hash (which would require re-hashing almost every card when the shard count changes), the design uses consistent hashing with virtual nodes, so adding a new shard only requires moving a small, predictable fraction of cards rather than the entire dataset. Rebalancing itself runs as a background, throttled migration process that copies affected card and ledger rows to their new shard, verifies row-for-row consistency, and only then updates the routing table — with the old location kept readable until the cutover is verified, ensuring zero downtime during the migration.

12.6 Why not a single globally distributed database

Modern globally distributed SQL databases (like Google Spanner or CockroachDB) are a legitimate alternative to manually sharded PostgreSQL, offering built-in strong consistency across regions. They are worth mentioning in an interview as a valid alternative, with the trade-off being higher operational cost and, for some engines, higher write latency due to cross-region consensus on every commit. The manually sharded PostgreSQL approach in this design trades some of that built-in elegance for a well-understood, cost-effective, and widely battle-tested technology stack — a reasonable choice for most organizations, though a globally distributed database becomes increasingly attractive as the platform’s geographic footprint grows.

12.7 Time-series partitioning of the ledger table

Independent of card-based sharding, the ledger table within each shard is additionally partitioned by month using native database table partitioning. Because most queries only need recent transaction history, this keeps the “hot” partition small and fast to scan, while older partitions can be moved to cheaper storage tiers or compressed, without ever needing to touch the live write path.

13.1 Core API contract

POST /v1/giftcards
Authorization: Bearer <token>
Content-Type: application/json

{
  "amount": 50.00,
  "currency": "USD",
  "designId": "birthday-confetti",
  "recipientEmail": "friend@example.com"
}

Response 201 Created
{
  "cardId": "gc_8f92a1",
  "last4": "4821",
  "balance": 50.00,
  "currency": "USD",
  "status": "ACTIVE",
  "expiresAt": "2028-07-31T00:00:00Z"
}

POST /v1/giftcards/{cardId}/redeem
Authorization: Bearer <token>
Idempotency-Key: 3f29-a819-77bd
Content-Type: application/json

{
  "amount": 20.00,
  "channel": "POS",
  "storeId": "store-1042"
}

Response 200 OK
{
  "cardId": "gc_8f92a1",
  "amountRedeemed": 20.00,
  "newBalance": 30.00,
  "transactionId": "txn_02a9c1"
}

13.2 Why microservices instead of a monolith here

This system benefits from microservices specifically because its components have genuinely different scaling profiles, release cadences, and risk levels. The Fraud Detection Service is iterated on almost daily as new attack patterns emerge, while the Ledger Service changes rarely and is deployed with extreme caution. Bundling them into one deployable unit would force the stable, high-stakes ledger code to be redeployed every time a fraud rule changes — increasing risk for no benefit.

13.3 Synchronous versus asynchronous communication

Notice in the architecture that Redemption calls Fraud and Ledger synchronously (the customer is waiting at checkout, so we need an answer now), but publishes to Kafka for Notification and Reconciliation asynchronously (the customer does not need to wait for an SMS receipt before walking away with their purchase). Getting this split right is one of the most important microservice design decisions in the whole system.

13.4 Additional endpoints

GET /v1/giftcards/{cardId}/balance
Authorization: Bearer <token>

Response 200 OK
{
  "cardId": "gc_8f92a1",
  "balance": 30.00,
  "currency": "USD",
  "status": "PARTIALLY_REDEEMED"
}

GET /v1/giftcards/{cardId}/transactions?limit=20
Authorization: Bearer <token>

Response 200 OK
{
  "cardId": "gc_8f92a1",
  "transactions": [
    { "type": "CREDIT", "amount": 50.00, "createdAt": "2026-03-01T10:00:00Z" },
    { "type": "DEBIT",  "amount": 20.00, "createdAt": "2026-03-15T18:22:00Z" }
  ],
  "nextPageToken": "eyJvZmZzZXQiOjIwfQ"
}

POST /v1/giftcards/{cardId}/void
Authorization: Bearer <token>
Idempotency-Key: 91ac-77f2-b0d4
Content-Type: application/json

{
  "reason": "CONFIRMED_FRAUD"
}

Response 200 OK
{
  "cardId": "gc_8f92a1",
  "status": "VOIDED"
}

Every one of these endpoints follows the same conventions consistently: resource-oriented URLs, an explicit Idempotency-Key header on every state-changing request, cursor-based pagination for list endpoints rather than offset-based pagination (which behaves poorly under concurrent writes), and predictable, well-documented status codes. This consistency is what makes the API genuinely usable by third-party partner engineering teams who will never see the internal implementation.

13.4.1 Error response conventions

Every error response across every endpoint follows one consistent shape, carrying a machine-readable error code, a human-readable message safe to display to a support agent, and the same correlation ID present in the successful-response case, so a failed request is exactly as traceable as a successful one. For example, an insufficient-balance error returns HTTP 422 with an error code of INSUFFICIENT_BALANCE and the actual available balance in the response body, letting a POS terminal immediately prompt the cashier for a different payment method without an extra round trip to look up the balance separately. Distinguishing between client errors (4xx, the caller did something the API does not allow, such as requesting more than the available balance) and server errors (5xx, something went wrong on the platform’s side and the caller should retry) is treated as a strict contract, since channel integrations often build automatic retry logic keyed directly off this distinction.

13.5 API versioning strategy

The /v1/ prefix is not decorative. When a breaking change is eventually needed — for example, changing the shape of the transaction history response — the platform introduces /v2/ alongside the existing /v1/, keeping both live for a documented deprecation window measured in months, not weeks, because partner marketplace integrations are often maintained by external teams who cannot redeploy on short notice. Non-breaking changes, like adding a new optional field, never require a version bump at all.

14.1 Patterns used in this design

14.2 Anti-patterns to avoid

14.3 A closer look at the Saga pattern here

The issuance flow touches three services — Fraud, Issuance itself, and Ledger — and must end in a consistent state even if one step fails partway through. Rather than a fragile distributed two-phase-commit transaction spanning all three, the design uses a choreographed saga: each service performs its local step and publishes an event; the next service reacts to that event. If the Ledger Service fails to record the opening entry after a card has already been marked active, a compensating IssuanceFailed event triggers the Issuance Service to roll the card back to a Failed state and, if payment was already captured, triggers an automatic refund through the payment processor. This keeps each service’s local transaction simple and fast, while still guaranteeing the overall business process reaches a consistent end state, accepting a brief window of temporary inconsistency in exchange for avoiding a much more fragile distributed lock across three services.

14.4 Outbox pattern for reliable event publishing

A subtle failure mode in event-driven systems is committing a database transaction successfully but then failing to publish the corresponding Kafka event, for example if the service crashes in the tiny window between the two operations. The Ledger Service avoids this using the transactional outbox pattern: instead of publishing directly to Kafka after committing, it writes the event into an outbox table within the exact same database transaction as the ledger entry itself. A separate, simple relay process polls the outbox table and publishes pending events to Kafka, marking them as sent. Because the outbox write and the ledger write share one atomic transaction, it becomes impossible for a ledger entry to exist without its corresponding event eventually being published, even across crashes.

14.5 How reconciliation actually consumes these events

The Reconciliation Service is itself a consumer of the same event stream the outbox relay publishes, alongside its scheduled batch comparison job described earlier. For every BalanceChanged event, it maintains a running, independently computed total of outstanding liability, entirely separate from the Ledger Service’s own internal bookkeeping. Comparing two independently derived totals — one from the Ledger Service’s live transactional state and one from the Reconciliation Service’s event-stream-derived state — creates a genuine cross-check rather than a single system simply confirming its own arithmetic against itself, which is a meaningfully weaker guarantee. A discrepancy between the two signals either a bug in the event publishing pipeline itself or, more rarely, a bug in the ledger’s transactional logic, and either case deserves the same halt-and-investigate response described in the monitoring section.

15.1 A note on testing financial systems

Unit tests alone are not sufficient for a ledger. This design relies heavily on property-based testing for the Ledger Service: instead of writing individual test cases, the test suite generates thousands of random sequences of credits and debits, including deliberately concurrent ones, and asserts a small set of invariants that must always hold — the sum of all entries for a card always equals its current balance, a debit never succeeds against an insufficient balance, and the total of all liability accounts across the entire ledger always exactly equals the total of all outstanding card balances. Property-based tests catch entire categories of concurrency bugs that hand-written example-based tests tend to miss, because they explore the state space far more broadly than a human writing test cases would think to.

15.2 Documentation as a first-class deliverable

Because this platform is consumed by multiple internal teams and external partners, API documentation, including explicit idempotency semantics, error code meanings, and rate limit headers, is generated directly from the API contract (for example, an OpenAPI specification) rather than maintained separately by hand. This keeps documentation from silently drifting out of sync with the actual implementation, which is a common and costly failure mode in systems with many external integrators.

15.3 Migrating from a legacy single-channel system

Very few gift card platforms are built greenfield; most are designed as a replacement for an older, single-channel system that a retailer has relied on for years. The safest migration path uses the strangler pattern mentioned earlier: introduce the new Channel Adapter Service in front of the legacy system first, initially just proxying requests through unchanged, so the new platform is exercised by real production traffic with zero behavior change. Next, migrate read-only operations like balance checks to the new Balance Service while writes still flow to the legacy system, validating that the new read path produces identical results to the old one. Only once reads are proven correct does write traffic — issuance and redemption — begin shifting over, store by store or channel by channel, with the legacy system kept running in a shadow, read-only capacity as a safety net until the migration is fully validated and the old system can finally be decommissioned. Attempting a single big-bang cutover on a financial system this sensitive is one of the most common and most damaging mistakes a team can make during a platform migration.

15.4 Common mistake: underestimating support tooling

Engineering teams often invest heavily in the customer-facing path and underinvest in the internal tools customer support and finance teams need daily — looking up a card’s full transaction history, manually reissuing a lost card with appropriate approvals, or investigating a discrepancy a customer has reported. Treating these internal tools as a genuine first-class product, built on the same underlying APIs as the customer-facing channels rather than as ad hoc scripts against the production database, both reduces operational risk and dramatically improves how quickly real customer issues get resolved.

16.6 What these examples have in common

Stepping back across Starbucks, Amazon, airline loyalty programs, digital wallets, and corporate incentive platforms, the same underlying pattern repeats every single time: a stored-value balance, touched from more channels than the original designers ever anticipated, protected by an authoritative ledger, idempotent writes, and continuous reconciliation. Recognizing this recurring shape is genuinely more valuable, both in interviews and in real engineering work, than memorizing any single company’s specific implementation, because it means the design skills built while studying this tutorial transfer directly to the next stored-value system you encounter, whatever industry it happens to sit in.

A balance column alone cannot answer “how did we get here” during a dispute or audit. An append-only ledger of debits and credits, from which the balance is derived, gives a complete, tamper-evident history and makes automated reconciliation possible.

Every redemption acquires a row-level lock on the card during its transaction and uses optimistic version checks, so two simultaneous redemption attempts on the same card — regardless of which channel they came from — cannot both succeed against the same balance.

Depending on regional regulation, the card either simply remains active indefinitely, expires after a configured period (where legally allowed), or eventually enters an escheatment process where the unclaimed balance is reported and remitted to the relevant government authority.

Add a currency field to both the card and every ledger entry, store amounts in the smallest currency unit (cents, paise) as integers to avoid floating-point issues, and ensure the Balance Service and Redemption Service always compare amounts within the same currency, rejecting cross-currency redemption unless an explicit, auditable conversion step is added.

Treating the ledger as the append-only source of truth and making every write path idempotent. Almost every other component in this design — caching, sharding, reconciliation, monitoring — exists to support and protect that one core guarantee.

Wrap that partner’s traffic in the Channel Adapter Service, generating a deterministic idempotency key server-side from a stable combination of fields the partner does send, such as their own order ID and card ID, rather than trusting the partner to generate one. This isolates the workaround to a single adapter component instead of weakening the idempotency guarantee for the whole platform.

Start with a number well above current needs, such as sixty-four or one hundred twenty-eight logical shards, even if they initially run on far fewer physical database instances. Logical shards can be re-mapped onto more physical instances later without re-hashing any data, whereas increasing the number of logical shards later requires the more disruptive rebalancing process described earlier. Over-provisioning the logical shard count is cheap; under-provisioning it is expensive to fix later.

Start with a modular monolith containing the same logical boundaries described in this tutorial (issuance, ledger, redemption, fraud) as internal modules with clean interfaces, backed by a single, un-sharded PostgreSQL database with the same double-entry ledger schema. Keep interfaces clean enough that any module can be extracted into its own service later, but do not pay the operational cost of ten separate deployable services before the traffic and organizational structure genuinely justify it.

17.1 Internationalization in practice

Expanding this platform into a new country typically touches four areas at once: the RegionRuleSet described earlier for expiry and escheatment law, the currency and rounding rules in the Ledger Service (some currencies, like the Japanese yen, have no minor unit at all, which the amount-handling code must account for explicitly rather than assuming every currency has cents), the Fraud Service’s regional risk model (velocity patterns considered suspicious in one market may be entirely normal in another), and localization of customer-facing notifications and receipts. Treating these four concerns as independently configurable, rather than hardcoding US-centric assumptions anywhere in the core services, is what makes genuine international expansion a configuration change rather than a rewrite.

17.2 How this design would evolve at ten times the scale

If traffic grew by an order of magnitude, three areas would need the most attention. First, the ledger shard count would need to grow well beyond its initial sizing, exercising the consistent-hashing rebalancing approach described earlier for real. Second, Kafka topic partitioning would need to scale alongside the shard count to keep event processing parallelism matched to write volume, so that Notification and Reconciliation consumers do not fall behind during peak load. Third, the Fraud Service, which is the most computationally expensive synchronous dependency in the redemption path, would likely need to move from synchronous rule evaluation to a pre-computed, cached risk score refreshed asynchronously per customer, only falling back to a live synchronous check for higher-risk transactions above a configurable amount threshold — trading a small amount of fraud-catching precision for the latency headroom needed at that scale.

17.3 Where to go from here

If you are studying this design for an interview, the best way to internalize it is to redraw the architecture diagram from memory, then deliberately try to break your own design: ask what happens if the Ledger Service’s database fails mid-transaction, what happens if two redemption requests for the same card arrive within the same millisecond from two different channels, and what happens if a partner’s webhook retries the same request ten times in a row. If you can answer each of those confidently using the tools covered in this tutorial — the ledger as source of truth, idempotency keys, row-level locking, circuit breakers, and continuous reconciliation — you have genuinely understood the design, not just memorized its diagram. And if you are building a system like this for real, start smaller than this tutorial: a modular monolith with a correct double-entry ledger and strict idempotency from day one will take you much further than a premature microservices architecture with a shaky data model underneath it. Whichever path you take, keep coming back to the same question this entire tutorial has circled around: if you drew a line through every arrow in your architecture diagram right now, would the numbers on both sides still add up.