When an attacker gains control of a legitimate merchant’s account — through stolen credentials, a hijacked API key, a malicious insider, or a takeover of the merchant’s point-of-sale software — they inherit that entire chain of trust instantly. They can push transactions that look, on the surface, exactly like the merchant’s normal business.

This is fundamentally different from classic card fraud, where a stolen card number is used at many different merchants. Here, the merchant account itself is the compromised asset, and the tell-tale sign is usually a sudden, dramatic change in that merchant’s own behavior: a coffee shop that normally processes fifty $8 transactions a day suddenly pushes ten thousand $400 transactions in an hour. The fraud detection problem becomes: how do you notice that a merchant is no longer acting like itself, fast enough to stop the bleeding, without constantly crying wolf on legitimate business spikes like a viral sale or a holiday rush?

A short history of merchant fraud detection

The history of this problem tracks the history of electronic payments itself. In the early days of card processing, fraud detection was almost entirely manual — a settlement report reviewed days later, chargebacks arriving weeks after the fact. As real-time authorization networks matured through the 1990s and 2000s, card networks like Visa and Mastercard built rules-based fraud engines (Visa’s Advanced Authorization, for instance) that could flag suspicious transactions within the authorization flow itself, in milliseconds. The 2010s brought machine learning into this space at scale — companies like Stripe (Radar), PayPal, and Adyen built models trained on billions of transactions to catch subtler patterns that fixed rules missed. Compromised-merchant detection specifically became a bigger focus as merchant onboarding became self-service and instantaneous (a merchant can start accepting payments online within minutes of signing up), which also made it faster and easier for a compromised or fraudulently created merchant account to start moving money before anyone notices.

In this tutorial, we will design a real-time fraud detection system focused specifically on catching a compromised merchant account processing fraudulent transactions at unusually high volume. We will build both a rules-based layer (fast, explainable, tunable) and a machine-learning scoring layer (adaptive, catches subtler anomalies), feeding into a human analyst review workflow and an automated risk-response system — all sized to handle a scenario where transaction volume can spike to a million requests in a single minute, whether that spike is a legitimate flash sale or the very fraud pattern we’re trying to catch.

1. Baseline drift per merchant. There is no single global definition of “too many transactions.” A large national retailer processing ten thousand transactions an hour is completely normal; a local bakery doing the same volume is almost certainly compromised. Every merchant needs its own learned baseline.
2. Real-time decisioning under extreme load. The whole point of catching a compromised account “at unusually high volume” is that the fraud itself generates a burst of traffic — potentially the exact million-requests-a-minute scenario we need our infrastructure to survive, and it needs to survive it while still scoring every single transaction, not falling behind.
3. The false positive trade-off. Block too aggressively, and you shut down a legitimate merchant’s biggest sales day, causing real business harm and reputational damage. Block too loosely, and fraudulent money keeps flowing until a human notices, by which point it may be unrecoverable.
4. Explainability for human review. A model that outputs “0.94 fraud probability” with no reasoning is nearly useless to a fraud analyst who has minutes to decide whether to freeze a merchant’s account. The system must surface why it thinks something is wrong.
5. Fast, safe response actions. Detecting the problem is only half the job — the system must be able to take a graduated action (soft hold, transaction-level block, full account freeze) quickly and reversibly, because these actions have real financial and legal consequences if wrong.

It’s also worth being explicit about why this problem resists a purely reactive, “review after the fact” approach. Traditional settlement-based fraud review, where suspicious activity is caught days later during reconciliation, might have been acceptable when card processing itself was slower and less automated. Today, payouts to merchants can happen within hours or even instantly through some payment platforms’ fast-payout features, meaning a compromised account can extract real, unrecoverable cash well before any batch review process would ever look at it. This is precisely why the detection system described in this tutorial must operate in near real time against the live transaction stream, not as an overnight batch job.

The two core guarantees we must design for

3.1 Merchant velocity

Velocity refers to the rate and volume of transactions a merchant processes over a rolling time window — for example, transaction count in the last 5 minutes, or total dollar volume in the last hour. Velocity is the single most important signal for catching a compromised account at high volume, because a hijacked account almost always tries to extract as much money as possible before it gets shut down, which shows up as an abrupt spike against the merchant’s own historical baseline.

3.2 Merchant baseline / behavioral profile

A baseline is a statistical summary of a merchant’s normal behavior — typical transaction count per hour, typical average ticket size, typical geographic distribution of customers, typical time-of-day pattern — learned from that merchant’s own historical data. Detection is fundamentally a comparison: is what’s happening right now consistent with this specific merchant’s baseline, or not?

3.3 Rules engine vs machine learning scoring

A rules engine evaluates simple, explicit, human-authored conditions (e.g., “flag if transaction count in 5 minutes exceeds 10x the merchant’s 30-day average”). Rules are fast, transparent, and easy for compliance teams to audit, but they’re rigid — a clever attacker can often find patterns just under the threshold. A machine learning scoring model learns subtler, higher-dimensional patterns from historical labeled fraud cases, catching things no human would think to write a rule for, at the cost of being harder to explain and requiring ongoing retraining.

3.4 Feature store

A feature store is a fast-access data layer holding precomputed signals (“features”) about each merchant — rolling transaction counts, average ticket size, velocity ratios — kept fresh in near real time so both the rules engine and the ML model can read them with very low latency instead of recomputing expensive aggregates from raw transaction history on every single request.

3.5 Fraud case

A fraud case is the record created when a merchant’s activity crosses a suspicion threshold. It bundles the triggering transactions, the rule and model evidence, and becomes the unit that a human analyst reviews, decides on, and closes. Cases are what turn a stream of scores into an auditable investigation trail.

3.6 Graduated risk response

Graduated response means the system doesn’t jump straight to “freeze the account” the moment suspicion crosses zero. Instead it escalates through steps — increased monitoring, a temporary hold on payouts (money is still collected but not released to the merchant), a full transaction block, and finally full account suspension — matching the severity of the response to the strength of the evidence. This mirrors a principle common across many operational risk domains: the punishment, or in this case the intervention, should scale with the confidence of the evidence, not jump straight to the most severe available action on the first sign of trouble.

Component reference

5.1 The sequence of a single transaction being scored

5.2 Why the scoring path is asynchronous, off the authorization path

Notice something crucial in Figure 2: the merchant receives their authorization response before the Rules Engine or ML Scoring Service has finished scoring. This decoupling is deliberate. Payment authorization latency is a strict, aggressive SLA (typically well under 300ms end-to-end). Fraud scoring, if placed synchronously on that path, would either (a) blow the latency SLA when the ML model is slow or the feature store hiccups, or (b) force the scoring layer to be so conservative that it misses subtle patterns. Instead, the authorization decision uses only the cheapest, most conservative checks (basic validation, card status), and full compromise scoring runs asynchronously against the stream, catching problematic merchants within seconds — fast enough to stop the bleeding, but never at the cost of merchant-facing latency.

5.3 How the stream is partitioned

The streaming queue partitions events by merchantId, meaning every transaction for a given merchant lands in the same partition, in order. This preserves the property that when the Aggregation Service updates a merchant’s velocity counter, no other consumer is racing it for the same merchant’s state — a fundamental guarantee for correct per-merchant windowing. Because the merchant space is large (millions of merchants), this same partitioning gives us essentially unlimited horizontal parallelism: doubling the partition count doubles the scoring throughput with no code changes.

5.4 How velocity is actually computed

The Aggregation Service maintains sliding-window counters per merchant. A naive implementation would look up every transaction from the last N minutes on every request, but that’s far too expensive at a million-transactions-a-minute scale. Instead, we use a tumbling-bucket structure — e.g., 60 one-minute buckets stored in Redis — and the 5-minute count is just the sum of the last 5 buckets. Updating one bucket is O(1); reading the sliding value is O(window_size), which is small and bounded.

public class MerchantVelocityRule implements Rule {

    private final MerchantBaselineStore baselines;
    private final FeatureStore featureStore;

    // A merchant is flagged if their transaction count in the last 5 minutes
    // exceeds their own 30-day-average-per-5-minute-window by a configurable
    // factor (e.g., 10x). This is per-merchant, not a global constant.
    public RuleResult evaluate(String merchantId) {
        long recentCount = featureStore.getCount(merchantId, "5m");
        double baselinePerFiveMin = baselines.getPerFiveMinuteAverage(merchantId);
        double factor = configuredFactorFor(merchantId); // e.g., 10.0

        boolean triggered = recentCount > baselinePerFiveMin * factor;

        return new RuleResult(
            "velocity_spike_5min",
            triggered,
            Map.of(
                "recent_count_5m", recentCount,
                "baseline_per_5m", baselinePerFiveMin,
                "factor_applied",  factor
            )
        );
    }
}

The important thing here isn’t the exact rule — it’s that the rule is per-merchant, learned from that merchant’s own history, and its output is a structured object carrying not just “triggered yes/no” but the exact numbers behind that decision, so any human reviewing the case knows what the rule saw.

5.5 Key algorithms in the detection pipeline

Under the covers, several well-known algorithms make this pipeline hold up at scale:

• Consistent hashing for stream partitioning. Merchant IDs are consistently hashed to Kafka partitions and Redis cluster slots, so adding or removing brokers or Redis nodes only reshuffles a bounded fraction of merchants, not the entire keyspace.
• Sliding window aggregation with tumbling buckets. Rolling counts (1m, 5m, 1h, 24h) are maintained as fixed-size arrays of tumbling buckets; each new event increments exactly one bucket and the current window value is the sum of the last N buckets — O(1) writes, tiny bounded-time reads.
• Count-Min Sketch for cheap approximate signals. For features where an exact count isn’t critical — “how many distinct card BINs has this merchant seen in the last hour?” — a Count-Min Sketch gives a very small memory footprint per merchant with a small, controllable overestimation bias, keeping the hot feature store affordable.
• Ensemble scoring at Case Management. The Rules Engine returns a hard rule outcome plus a rule strength; the ML Scoring Service returns a probability plus a top contributing feature; Case Management combines them with a simple weighted ensemble (with weights that themselves are auditable and tunable), giving a final risk severity that’s more robust than either signal alone.

6.1 Walking through the lifecycle

Every merchant starts and stays in Monitoring under normal conditions — scored continuously, but not the subject of any active case. When a rule fires or the ML score crosses its threshold, the merchant enters Flagged, which automatically opens a new record in Under_Review in the Case Management Service. The case is now durable, assigned, and visible in analyst tooling.

An analyst reviews the bundled evidence — the triggering transactions, the exact rule numbers, the model score, the top contributing feature, the merchant’s recent history. From there, three outcomes are possible: the case is closed as False_Positive (analyst clears the merchant, feedback flows back into rule tuning and future ML training data), the case is escalated to Confirmed_Fraud (evidence supports a compromised account), or, for extreme high-confidence signals, the system may proactively enter Account_Frozen even before analyst review is complete, applying an automated but explicitly-reversible protective hold.

Confirmed fraud cases either move to Under_Investigation for deeper compliance and chargeback follow-up, or, in the clearest cases with evidence of ongoing abuse, to Account_Terminated. If subsequent investigation exonerates the merchant, the case ends in Account_Reinstated and the merchant returns to normal monitoring.

6.2 Why the “never skip a state” property matters

The case lifecycle is intentionally strict: every state transition emits an immutable CASE_EVENT record (see Chapter 13) containing who or what triggered the change, when, and why. This gives us two things a payment platform absolutely needs: a full audit trail for compliance and dispute resolution, and a rich labeled dataset (“this case was ultimately confirmed / cleared / reinstated”) that feeds directly back into future model retraining and rule tuning. Systems that skip explicit state modeling and just “block or don’t block” lose both.

7.1 Detection speed vs false-positive rate

This is the central trade-off the whole system exists to manage. Pushing rule thresholds down or ensemble weights higher catches more fraud faster but pulls in more legitimate merchants during ambiguous surges. Graduated response gives us a middle ground: at moderate confidence we escalate to human review or a payout hold (both reversible), only at extreme confidence with strong corroborating evidence do we take the harder, more disruptive actions like a full transaction block. This lets us tune for catch rate more aggressively than a “block or don’t block” system could ever afford.

8.1 Techniques that keep this system scalable

• Merchant-partitioned streaming. Kafka partitions keyed by merchantId let us process the stream with thousands of parallel consumers without any per-merchant ordering ever being violated. Adding capacity is a matter of adding consumer pods, not rearchitecting.
• Precomputed features in the hot store. Both scoring paths read merchant velocity from Redis at sub-millisecond latency. They never recompute rolling counts from raw transaction history on the request path.
• Independent auto-scaling per service. Rules Engine, ML Scoring, Aggregation, Case Management, and Risk Action all scale independently on their own metrics — rules on CPU, ML on GPU or model-server throughput, Aggregation on queue lag, Case Management on case-open rate. Nothing shares a scaling group with something it doesn’t share load characteristics with.
• Multi-region deployment behind a Global Load Balancer. Traffic is served from whichever region is closest and healthy, keeping merchant-facing latency low and giving us a natural failover path when a region degrades — the shared partitioned stream ensures per-merchant ordering survives the region hand-off.

8.2 Capacity numbers at the target scale

9.1 No single point of failure

Every stateful and stateless component in the design is deployed in multiple replicas across availability zones. The Load Balancers themselves are managed regional services with built-in HA. Kafka brokers replicate every partition across three brokers with rack-aware placement. Redis feature-store shards run primary-replica pairs with automatic failover. Postgres runs primary-plus-replicas with synchronous replication for the primary’s WAL to ensure zero data loss on planned failover. No component is a single hop between the merchant and detection going dark.

9.2 Handling partial failures gracefully

Just as important as “no single point of failure” is “the system does something sensible when a piece does fail.” The design assumes partial failure will happen and responds by degrading gracefully rather than either fully blocking transactions or fully going blind:

• If the ML Scoring Service is degraded or slow, Case Management continues on rule-only evidence and clearly flags scored-with-rules-only cases so analysts know coverage is reduced — better than either blocking every transaction or missing every fraud signal.
• If the Rules Engine is unavailable, ML scoring alone still runs, and cases opened during that window are marked so a rule-based post-mortem re-scoring can happen after recovery.
• If the Hot Feature Store is degraded, scorers fall back to computing a bounded, less-fresh feature set directly from the last N minutes of the stream, at higher latency but without losing coverage entirely.
• If the Case Management Service or Risk Action Service is briefly unavailable, scoring continues, and cases are buffered on their own durable queue for delayed processing rather than being dropped.

9.4 CAP-theorem posture

Different parts of this system make different CAP choices, deliberately. The Primary DB storing merchants, cases, and risk actions is CP — correctness matters more than availability, we’d rather brief unavailability than a case that says two different things in two different regions. The Hot Feature Store is closer to AP — a briefly stale velocity counter is fine, dropped scoring reads are not. Kafka gives us strong per-partition ordering and durability guarantees within its replication factor, which is what merchant-scoped ordering actually needs. This isn’t a single global choice; it’s picked per subsystem based on what the subsystem is actually protecting.

9.5 Consensus and split-brain protection

Case Management and Risk Action must never operate under a split-brain scenario (two disagreeing primaries), because a merchant being simultaneously “frozen” in one region and “monitoring” in another is a correctness disaster. The Primary DB uses synchronous replication with fenced automatic failover (the old primary is fenced from writes before a new primary is promoted), and case-write operations require quorum from at least a majority of replicas. Redis, being AP-leaning, tolerates split-brain more permissively for velocity counters — the worst case is briefly-inconsistent counters that reconverge quickly, which is acceptable for that role.

9.6 Disaster recovery

Beyond zone-level HA, the system supports region-level disaster recovery via the multi-region deployment shown earlier: if an entire region becomes unhealthy, the Global Load Balancer redirects merchant traffic to another healthy region within seconds. The shared partitioned stream and centrally consistent case data mean the surviving region can pick up scoring for affected merchants without any manual failover of application state. Regularly rehearsed game-days validate this path works end-to-end, not just in theory.

10.1 Protecting the detection logic itself

A fraud detection system’s own rule thresholds and model weights are sensitive assets — if an attacker learns exactly where the velocity threshold sits, they can structure fraudulent transactions to stay just under it. Access to rule configuration, model artifacts, and even detailed scoring logs is tightly restricted via the API Gateway’s authorization layer, following least-privilege principles, and changes to thresholds go through an audited approval workflow rather than being editable ad hoc.

10.2 API Gateway as a security boundary

Every transaction submission and every internal service-to-service call passes through the API Gateway (or a service mesh with equivalent guarantees), enforcing authentication, authorization, and rate limiting. This also protects against a compromised merchant’s terminal itself being used to flood the detection pipeline directly, bypassing the intended transaction flow.

10.3 Securing the risk action path

The Risk Action Service, which can freeze real merchant accounts and hold real money, is one of the most sensitive components in the entire system. It requires mutual TLS between Case Management and Risk Action, strong service-to-service authentication, and every action it takes is itself logged as an immutable event — an audit trail of who or what triggered every freeze, hold, or block, and when.

10.4 Data privacy for merchant and transaction data

• Transaction data used for feature computation and model training is handled under strict data governance policies, with personally identifiable customer information tokenized or excluded from the features that don’t need it.
• All data in transit uses TLS 1.2 or higher; all data at rest, especially in the Data Lake used for training, is encrypted with managed or customer-managed keys.
• Access to raw historical transaction data for model training is scoped to the ML platform team through audited, time-limited credentials rather than broad standing access.

10.5 Adversarial robustness

Because the entities being detected are, by definition, motivated to evade detection, the ML Scoring Service is designed with adversarial robustness in mind: features are chosen to be expensive or impossible for an attacker to directly observe and game (like cross-merchant behavioral correlations), model performance is monitored for signs of concept drift that could indicate attackers adapting their patterns, and rule thresholds are periodically and unpredictably adjusted within a safe range to avoid becoming a static target.

11.1 The three pillars

• Metrics — transaction throughput, rules and ML scoring latency (p50/p95/p99), case creation rate, false positive rate (from analyst feedback), and queue consumer lag, all scraped by Prometheus and visualized in Grafana.
• Logs — structured, correlation-ID-tagged logs from every service in the detection pipeline, centralized for search, so any transaction’s full scoring journey can be reconstructed.
• Traces — distributed tracing (via OpenTelemetry) following a single transaction from ingestion through both the Rules and ML paths to case creation, useful for diagnosing exactly where latency is being spent under load.

11.2 Model-specific monitoring

Beyond standard service metrics, the ML Scoring Service requires its own specialized monitoring: feature distribution drift (are the inputs the model is seeing today statistically different from training data), prediction score distribution drift (is the model suddenly scoring everything higher or lower than usual, which can indicate a pipeline bug rather than a genuine change in fraud patterns), and model staleness (how long since the model was last retrained on recent data).

11.3 Critical alerts

12.1 Containerized microservices on Kubernetes

Each service — Transaction, Rules Engine, ML Scoring, Aggregation, Case Management, Risk Action, Notification — is packaged as a Docker container deployed on Kubernetes, with horizontal pod auto-scaling driven by both CPU/memory and custom metrics like queue consumer lag.

12.2 Canary deployments for model and rule changes

Changes to the ML model or rule thresholds are especially risky to deploy carelessly — a bad model update could either miss real fraud or flood analysts with false positives. New model versions and threshold changes are rolled out as shadow deployments first (scoring in parallel without triggering real actions, purely for comparison against the live version), then canaried to a small percentage of merchant traffic, before full rollout, with automated rollback if false positive or miss rates deviate significantly from the baseline.

12.3 Infrastructure as code

Kubernetes clusters, Kafka clusters, database instances, load balancer configuration, and IAM roles are all defined declaratively with Terraform, ensuring environments are reproducible and every infrastructure change is reviewable through pull requests.

12.4 Multi-region deployment

For platforms operating globally, detection infrastructure is deployed across multiple regions with a Global Load Balancer routing merchant traffic to the nearest healthy region, as shown in Figure 4, while case management and risk action data remains centrally consistent to avoid a merchant being frozen in one region but not another.

12.5 Cost optimization

Detection infrastructure needs to handle rare, extreme bursts (a compromised account spiking to a million transactions a minute) while normally running at a much lower baseline. Aggressive auto-scaling with a modest always-on floor keeps steady-state costs reasonable, while the heavier offline model training and historical analysis workloads run on spot or preemptible compute, since they can tolerate interruption in a way the real-time scoring path cannot.

13.1 Data model

13.2 Why a hybrid data store approach

No single database technology fits every need here. The Primary Database (merchant, case, risk action records) uses PostgreSQL for strong relational integrity and ACID transactions, since case and action data has the same correctness requirements we’d want in any financial system. The Hot Feature Store uses Redis for its sub-millisecond read latency on velocity counters, prioritizing speed over durability (counters can be reconstructed from the stream if lost). The Data Lake uses a columnar, append-friendly store (like Parquet files on object storage, queried through Spark or a similar engine) optimized for cheap bulk storage and large-scale offline analytics rather than low-latency point lookups.

13.3 Sharding strategy

The Primary Database is sharded by merchant ID, keeping a given merchant’s transactions, velocity windows, and cases co-located on the same shard, which keeps the most common query patterns (like “show this merchant’s recent case history”) a single-shard operation.

13.4 Caching strategy

• Velocity counters live in Redis with short TTLs matching their window size, continuously refreshed by the Aggregation Service.
• Merchant baselines (computed less frequently, e.g., daily) are cached with longer TTLs since they change slowly, avoiding repeated expensive recomputation on every scoring request.
• Cache invalidation for baselines happens explicitly when a merchant’s risk tier changes or after a confirmed fraud case closes, since stale baselines right after a confirmed incident could undercount genuinely risky behavior.

13.5 Load balancing algorithms

The Load Balancer uses least-connections routing for the Rules Engine and ML Scoring Service tiers, since scoring latency can vary based on feature complexity and model load, and least-connections keeps work evenly spread across instances under this kind of variable latency, similar to the reasoning used for payment processing tiers in other high-stakes financial systems.

14.1 Key internal APIs

POST /internal/v1/transactions/score
Body: {
  "transactionId": "txn_7743a",
  "merchantId": "merch_2291",
  "amount": "412.00",
  "currency": "USD",
  "processedAt": "2026-08-04T09:12:00Z"
}

Response 200 OK
{
  "ruleScore": 0.91,
  "modelScore": 0.87,
  "topReason": "velocity_spike_5min",
  "caseCreated": true,
  "caseId": "case_5581"
}

Both the rule score and model score are always returned together where available, rather than a single combined opaque score, so downstream Case Management and analysts can see exactly how much of the suspicion came from an explainable rule versus a harder-to-interpret model signal.

14.2 Why microservices instead of a monolith here

Splitting Transaction, Aggregation, Rules, ML Scoring, Case Management, and Risk Action into separate services allows the two very different workload types — high-throughput low-latency scoring versus lower-throughput but high-stakes case and action management — to scale and be operated independently, with different teams (data science for ML Scoring, risk operations for Case Management and Risk Action) able to own and deploy their own services without stepping on each other.

14.3 Synchronous vs asynchronous API boundaries

14.4 The transactional outbox pattern for case events

Just as in other financial systems, writing a case status change to the database and publishing a corresponding event (say, to notify an analyst dashboard in real time) uses the transactional outbox pattern — the event is written to an outbox table in the same transaction as the state change, and a relay process publishes it separately, avoiding the dual-write problem where a database commit succeeds but the downstream notification silently fails to fire.

15.1 Patterns used in this design

15.2 Anti-patterns to avoid

16.1 Best practices

• Build per-merchant baselines from day one, even if crude initially, rather than launching with only global thresholds and retrofitting personalization later.
• Always pair a fast, explainable rules layer with an adaptive ML layer — neither alone covers the full space of compromise patterns well.
• Treat detection and enforcement as separate, decoupled concerns, connected through an auditable case workflow rather than direct, unreviewed automation for anything beyond the highest-confidence, most reversible actions.
• Close the feedback loop from analyst decisions back into both rule thresholds and model training data, systematically, not ad hoc.
• Instrument the full chain from detection to enforcement, not just detection accuracy alone — a correctly detected but unenforced case is still a fraud loss.
• Design for graceful degradation, so losing one detection path (rules or ML) reduces coverage without blinding the system entirely.

16.2 Common mistakes

16.3 Testing strategy

Given the asymmetric cost of false negatives (real fraud continuing) versus false positives (blocking legitimate merchants), testing must explicitly evaluate both error types rather than a single blended accuracy number.

• Unit tests cover rule logic, sliding window calculations, and state machine transition rules in isolation.
• Backtesting replays historical confirmed fraud cases and confirmed false positives against candidate rule and model changes before deployment, measuring precision and recall separately.
• Shadow deployment testing, as described earlier, compares a new model’s live decisions against the current production model on real traffic without affecting real outcomes, surfacing disagreements for review before rollout.
• Load tests simulate the million-transactions-a-minute burst scenario, verifying scoring latency and accuracy both hold up, not just that the system stays up.
• Chaos tests deliberately degrade or kill the ML Scoring Service, the feature store, and streaming brokers individually to confirm graceful degradation behaves as designed rather than as hoped.

17.5 The common thread

Across every one of these platforms, the same underlying pattern repeats: fast, asynchronous scoring that never blocks the core payment flow; a combination of explainable rules and adaptive machine learning; per-merchant or per-cohort baselines rather than global thresholds; and a graduated, auditable response process that treats detection and enforcement as related but distinct concerns. The specific technology choices vary, but the architectural shape converges because the underlying correctness and risk-management problem is the same one we solved in this tutorial. Whether the platform is a global card network scoring billions of transactions a day or a smaller SaaS payment processor serving a niche vertical, the same fundamental building blocks — a behavioral baseline per merchant, a fast parallel scoring layer, and a graduated response mechanism — form the backbone of a trustworthy fraud detection system.

Q: What is the single most important design decision in this whole system?

Decoupling transaction authorization from fraud detection scoring. Every other design choice — the streaming architecture, the parallel rules-and-ML paths, the graduated response — exists to support fast, accurate detection running asynchronously, without ever holding up the merchant’s actual checkout flow.

Q: How do you avoid the system being too slow to catch fraud that happens in a fast burst?

By keeping detection latency (transaction ingest to case creation) in the low single-digit seconds even at peak load, through precomputed features, lightweight real-time models, and merchant-partitioned parallelism, and by using automated, high-confidence temporary holds for the most severe cases rather than always waiting for full human review before any action is taken.

Q: Why not just require every merchant to have a hard transaction volume cap?

A hard cap either has to be set low enough to catch fraud (which then blocks legitimate growth and big sales days for real merchants) or high enough to accommodate legitimate growth (which then fails to catch fraud that stays under it). Per-merchant learned baselines, combined with multiple orthogonal signals beyond just volume, handle this far better than any single fixed number could.

Q: How would you test that the false-positive feedback loop actually improves the system over time, rather than just trusting it works?

Track precision and recall on a held-out, continuously updated set of confirmed cases over time, and run periodic backtests comparing the current rules and model configuration against past confirmed outcomes to verify the false positive rate is trending down (or at least not up) as more feedback is incorporated, rather than assuming the feedback loop is helping just because it exists.

Q: Does this design change for a platform that supports many countries and currencies?

The core engine — streaming ingestion, rules plus ML scoring, feature store, case management, graduated response — stays the same. What changes is that baselines and features need to account for regional norms (average ticket sizes and typical velocity patterns vary significantly by country and industry), and risk actions may need to route through region-specific compliance review processes given differing financial regulations, similar to how a multi-region billing system needs pluggable, region-aware logic layered on top of a shared core.

Q: How would you handle a case where the merchant themselves, not an attacker, is the one committing fraud (a “friendly fraud” or bust-out scheme) rather than a genuine account compromise?

The detection signals overlap significantly — both show as an abnormal spike against the merchant’s baseline — but the response differs. A genuine compromise typically warrants immediate account protection and merchant notification, since the merchant is a victim too. A suspected bust-out scheme (a merchant intentionally maximizing charges before disappearing, often on a newer account) instead routes to a specialized fraud investigation queue with additional signals like time-since-onboarding, payout destination changes, and support contact patterns, since the appropriate response — investigation before any merchant notification — is quite different from a compromise where the merchant is expected to cooperate.

Taken together, these principles form a system that is fast enough to matter, accurate enough to trust, and humble enough to admit — through its graduated, reversible response design — that no detection system, however well built, will ever be perfectly certain on every single case it flags. That humility, encoded directly into the architecture rather than left as an afterthought, is ultimately what separates a fraud system merchants trust from one they fear.

Closing thought

Fraud detection system design interviews reward the same kind of judgment as billing system interviews, but with an added adversarial dimension: whatever pattern you design to catch, you should assume a motivated attacker will eventually try to evade it. A candidate who can explain not just how the rules and ML paths work, but why they’re deliberately kept parallel and independent, why response is graduated rather than instant, and why the feedback loop from human review matters as much as the initial detection logic, is demonstrating the kind of systems thinking that keeps a real fraud platform effective as attackers adapt around it. The architecture in this tutorial is a solid foundation — the reasoning behind each trade-off is what’s worth carrying into your own designs.