Before we talk about the knock, we need to understand the switch. This chapter explains feature flags, Flagsmith, and the real-world pain that webhooks remove.

What is a feature flag?

A feature flag is a switch inside your software that lets you turn a feature on or off without releasing new code. Instead of writing the rule “show the new checkout page” directly into the program and shipping it, developers write the rule as “show the new checkout page only if the flag called new_checkout is on.” The flag itself lives outside the program, in a tool that can be changed in seconds by a person with a browser.

What is Flagsmith?

Flagsmith is a feature flag and remote configuration service. It is open source, which means anyone can read its code, and you can run it yourself on your own servers or use the hosted version that the Flagsmith team operates. It gives you a dashboard where people manage flags, and it gives your applications a way to ask, “is this flag on for this user right now?” Besides simple on-and-off switches, Flagsmith can store values, such as a text message, a number, or a small piece of configuration, so you can change how an app behaves without redeploying it. That second ability is called remote configuration.

The silent switch problem

Flags are powerful because they change behaviour instantly. That same power creates a risk: changes happen quietly. Without some form of announcement, the rest of your company only finds out about a flag change when something looks different. Consider a few examples of what goes wrong when changes are silent:

• A caching layer holds on to old answers because nobody told it that a flag changed, so some users see the new behaviour while others see the old one.
• A compliance team needs a record of every production change, but the only record lives inside the flag tool itself, where nobody checks.
• A release manager wants a chat message in the team channel whenever a risky flag is switched on in production, but has to ask developers to remember to post it.
• A data team wants to line up flag changes with spikes in error rates, but has no timeline of when each change happened.

Pull versus push

There are two basic ways for a system to learn about a change. In the pull style, the interested system keeps asking, over and over, “Has anything changed?” In the push style, the system where the change happens sends a message the moment it occurs. Webhooks are the push style. Flagsmith pushes a message to your chosen address immediately after a flag change, so you do not have to keep asking.

If the word webhook has always sounded mysterious, this chapter removes the mystery. A webhook is just a polite, automatic web request.

The web in thirty seconds

Every time you open a website, your browser sends a request to a computer somewhere, and that computer sends back a reply. The address you type is called a URL. The request has a type, called a method. The two methods that matter here are GET, which means “give me something,” and POST, which means “here is something for you.” Replies carry a three-digit status code. Codes that begin with 2, such as 200, mean success. Codes that begin with 4 mean you made a mistake, and codes that begin with 5 mean the other computer had a problem.

So what is a webhook?

A webhook is an automatic POST request that one system sends to a URL you provide, whenever a certain event happens. The message inside the request is usually written in JSON, a simple text format for describing information with names and values. The system that sends the message is the sender. The system that receives it, at the URL you gave, is the receiver. You are the one who builds and hosts the receiver. Flagsmith is the sender.

Webhooks compared with polling

Polling is the opposite of a webhook. A polling system asks again and again on a timer. Polling is simple, but it has two weaknesses. First, it wastes effort, because most of the time the answer is “nothing new.” Second, it is slow, because a change that happens one second after a check will not be noticed until the next check. Webhooks avoid both weaknesses because the message is sent only when there is something to say, and it is sent right away.

Webhooks compared with a regular API

An API is a menu of questions you can ask a service on demand. A webhook is the service volunteering an announcement. Most real systems use both. You might use the Flagsmith API to list all flags on demand, and you might use a webhook to be told when one of them changes. The two work well together, and we will see later how teams combine them to stay safe when a message is lost.

Flagsmith organises everything in layers. Learning the layers makes webhook settings and payloads easy to read.

Flagsmith groups your work in a tree. At the top is the organisation, which usually represents your company. Inside an organisation live projects, which usually represent one product or application. Inside each project live environments, such as development, staging and production. Inside the project you create features, which are your flags and configuration values. Each environment keeps its own state for each feature, so a flag can be on in development and off in production at the same time.

Why this matters for webhooks

Webhooks attach to specific layers. One kind attaches to an environment and reports changes to flag states inside that environment. Another kind attaches to the organisation and reports entries written into the audit log. Knowing which layer you care about tells you which kind of webhook to create, which is exactly what the next chapter covers.

Flagsmith sends webhooks at two different levels. Picking the right one is the first real design decision.

Environment webhooks

An environment webhook belongs to one environment. It fires when something changes in the state of a feature inside that environment. That includes switching a flag on or off, changing its value, adding or editing a segment override, adding or editing an identity override, and deleting the feature. You configure it in the settings page of the environment. If you want to hear about production changes only, you create it on the production environment and leave the others alone.

Organisation webhooks

An organisation webhook belongs to the whole organisation and is tied to the audit log. Whenever Flagsmith writes a new audit log entry, it can notify your receiver. Audit entries cover a wider range of activities than flag state changes, such as creating or editing features, segments and environments, so this kind of webhook gives you a broader view of what people are doing in Flagsmith. You configure it in the organisation settings.

Webhooks are not the same as integrations

Flagsmith also offers built-in integrations with popular tools, for example chat, monitoring and analytics products. An integration is a ready-made connection where Flagsmith already knows how to talk to the other product. A webhook is a general-purpose, do-it-yourself connection that you can point anywhere. If a built-in integration exists for your tool and it does what you need, it can save you work. If not, or if you need custom logic, a webhook is the flexible choice. We compare these options in more detail later.

Let us look at the moving parts that take a click in the dashboard and turn it into a message at your door.

flowchart LR
    A["Admin using Dashboard"] --> B["Flagsmith API"]
    A2["Automation using Admin API"] --> B
    B --> C["Primary Database"]
    B --> D["Audit Log Writer"]
    B --> E["Background Task Queue"]
    D --> C
    E --> F["Webhook Dispatcher"]
    F --> G["Environment Webhook Receiver"]
    F --> H["Organisation Webhook Receiver"]
    G --> I["Cache Refresh Service"]
    G --> J["Chat Alert Service"]
    H --> K["Compliance Archive"]
    H --> L["Security Monitoring"]

Walking through the diagram

The diagram reads from left to right. Changes begin at the left, either because a person uses the dashboard or because an automation script calls the Flagsmith admin interface. Both paths end up at the Flagsmith API, which is the front door of the service.

The API does three things when it accepts a change. It saves the new state in the primary database, which is the permanent storage. It asks the audit log writer to record who made the change, what changed and when. And it places work orders on a background task queue. A queue is simply a waiting line for jobs. The important idea is that sending webhooks is placed on the queue instead of being done on the spot, so the person who clicked the switch does not have to wait for the slowest receiver on earth.

A worker picks jobs from the queue and hands them to the webhook dispatcher. The dispatcher looks up which webhooks apply, builds the message, signs it if a secret is configured, and sends the POST request to each receiver. Your receivers then pass the information to the tools that actually care: a cache service, a chat room, a compliance archive or a security monitor.

The components in plain words

Now we follow a single flag change on its journey, including what happens when the receiver is having a bad day.

sequenceDiagram
    participant U as Engineer
    participant F as Flagsmith API
    participant Q as Task Queue
    participant D as Dispatcher
    participant R as Your Receiver
    U->>F: Turn flag off in production
    F->>F: Save new state and write audit entry
    F->>Q: Enqueue webhook job
    F-->>U: Change confirmed
    Q->>D: Deliver job to worker
    D->>R: POST message with signature
    alt Receiver answers with success code
        R-->>D: 200 OK
        D->>D: Mark delivery complete
    else Receiver fails or times out
        R-->>D: Error or no reply
        D->>Q: Schedule retry after a delay
        Q->>D: Run retry later
        D->>R: POST message again
    end

Why the dashboard stays fast

The most important design choice in this journey is that delivery happens in the background. If Flagsmith waited for every receiver before telling you the change worked, one slow receiver could make the whole dashboard feel broken. Background delivery separates the speed of making a change from the speed of announcing it. The price of that separation is that announcements are not instant to the millisecond, and they can occasionally be delayed when the queue is busy.

A word on background workers

Background work needs something to do it. In the hosted Flagsmith service, the Flagsmith team runs that machinery for you. In a self-hosted installation, how background tasks run depends on how you deployed it. Some setups run tasks inside the main application process, while larger setups run a dedicated task processor so heavy background work never slows the web servers. We return to this in the chapter about self-hosting, because it changes how you think about reliability.

A webhook is only as useful as the message inside it. Let us open the envelope and understand each piece.

The envelope and the letter

Every webhook has two parts. The first part is the set of headers, which are small labels attached to the request, such as the content type and, when you have configured a secret, a signature. The second part is the body, which is the actual letter. Flagsmith writes the body in JSON, so any programming language can read it. The exact fields can evolve between versions, so a good receiver reads the fields it needs and politely ignores the rest.

The event type

The body includes a name that says what kind of event this is. For the two families of webhooks, the main event names you will meet are a flag-updated event, a flag-deleted event and an audit-log-created event. Your receiver should look at this name first and decide what to do. A receiver that treats every message the same way is a receiver that will one day do something silly with a deletion.

The data inside

For environment webhooks, the data section describes the change in a before-and-after style. You can expect to find the following ideas, although the exact names and nesting are best confirmed against the current Flagsmith documentation for your version:

• Who made the change. A name or identifier of the user or key responsible.
• The new state. Whether the feature is enabled, what value it holds, which environment and feature it belongs to, and whether the change applies to a specific identity or to the general setting.
• The previous state. The same information as it was before the change, so you can see exactly what moved.
• A timestamp. When the change happened, which is vital for ordering messages correctly.

For organisation webhooks, the data section describes the audit log entry: a short human-readable description of what happened, who did it, when, and which object was touched.

Reading the message the safe way

The setup is short, but each small choice affects security and reliability later. Here is the full journey from nothing to a working webhook.

Choosing the receiver address wisely

The address you give Flagsmith becomes a permanent doorway into your systems. Treat it with care. Use a dedicated path just for Flagsmith rather than a general-purpose address. Avoid putting secret tokens in the web address itself, because addresses tend to leak into logs. Use the signature instead of obscurity to prove who is calling.

One webhook or many?

You can create more than one webhook on the same environment, each pointing at a different receiver. This is useful when different teams care about the same events. A cache service, a chat bot and a compliance archive can each have their own address. The alternative is a single receiver that fans out to several downstream tools. Separate webhooks keep failures isolated, because a broken chat bot will not delay the cache refresh. A single fan-out receiver keeps configuration simpler and centralises security checks. Both patterns are valid, and we compare them in the chapter on design patterns.

A webhook receiver is a public door. Anyone on the internet can knock on it. Security is how you make sure only the real Flagsmith gets in.

The threats in simple terms

If your receiver trusts every message it receives, a stranger can send a fake message claiming that a flag changed. If the receiver reacts by clearing caches, deploying code or paging people, the stranger has just gained a remote control. Three threats matter most: forgery, where someone fakes a message; tampering, where someone changes a real message on the way; and replay, where someone records a real message and sends it again later.

Encryption in transit

Always use https for your receiver address. Encryption keeps eavesdroppers from reading the message and prevents casual tampering along the route. It protects the content of the flag change from anyone who can see network traffic between Flagsmith and you.

Signatures: proving the message is genuine

When you configure a secret on a webhook, Flagsmith uses it to compute a signature from the exact bytes of the message body and sends that signature in a header. The signature is produced with a method called HMAC, which combines a message and a secret key using a hash function, in this case SHA-256, into a short fingerprint. Only someone who knows the secret can produce the right fingerprint for a given message. Flagsmith documents the header name and the exact method, and you should confirm both for your version, since your receiver must match them precisely.

Your receiver repeats the same calculation using its own copy of the secret and compares the result with the header. If they match, the message is genuine and unchanged. If they do not match, the receiver rejects the message with an error code and does nothing else. Two details matter a great deal. First, calculate the signature over the raw, unmodified bytes you received, not over a re-formatted version, because even a tiny change such as extra spaces produces a different fingerprint. Second, compare fingerprints using a constant-time comparison function so that attackers cannot learn the answer from tiny timing differences.

Replay protection

A signature proves a message is real, but it does not prove the message is fresh. An attacker who records a genuine message could send it again tomorrow. To defend against this, receivers can check the timestamp inside the message and reject anything that is too old, and they can remember identifiers of messages they have already processed so that repeats are ignored. Treat replay protection as an extra layer, particularly for receivers that trigger powerful actions.

Protecting the secret

Network-level defences

Beyond signatures, you can limit who can reach the receiver at all. A web application firewall can block obviously malicious traffic. If Flagsmith publishes the network addresses it sends from, you can restrict the receiver to accept connections only from those addresses. For self-hosted setups the sender is your own server, so private networking can keep the receiver off the public internet entirely. Remember that address lists change, so combine network rules with signatures, never replace signatures with them.

Least privilege inside the receiver

Give the receiver only the permissions it needs. A receiver that posts to a chat channel should not have the power to deploy code. A receiver that refreshes a cache should not have access to the customer database. If a forged or buggy message ever slips through, least privilege limits the damage to something small and fixable.

The internet is unreliable. Servers restart, networks hiccup and receivers get overloaded. This chapter explains what to expect when things go wrong, and how to design for it.

What does delivered even mean?

From the sender’s point of view, a delivery is successful when the receiver answers with a success status code in the two-hundreds range within the allowed time. Anything else, including an error code, a timeout or a refused connection, counts as a failure. The sender cannot see inside your receiver, so a success code is a promise from you that you have safely taken responsibility for the message.

Retries

When a delivery fails, Flagsmith does not give up immediately. It retries a limited number of times, waiting longer between attempts, a technique known as backoff. Backoff prevents a struggling receiver from being flooded by constant retries at the exact moment it needs room to recover. In self-hosted installations, the number of retries and the waiting pattern are typically configurable through settings, and the exact defaults are worth confirming in the current documentation. Importantly, retries are limited. After the final attempt fails, the message is not delivered, and you should plan for that.

At least once, and the duplicate problem

Systems that retry offer what is called at-least-once delivery. The message will arrive one or more times, but not necessarily exactly once. Consider a case where your receiver processes a message successfully but the success reply is lost on the way back. The sender sees a timeout, assumes failure and sends the message again. Your receiver now sees the same message twice. This is not a bug. It is a natural result of unreliable networks, and your receiver must be ready for it.

Idempotency: the golden rule

An operation is idempotent when doing it twice has the same result as doing it once. Setting a cache entry to a value is idempotent. Adding one to a counter is not. Design your receiver so that repeated messages are harmless. Two common techniques are available. The first is to remember identifiers or fingerprints of messages already processed and skip repeats. The second is to make the action itself naturally repeatable, for instance by always writing the full new state rather than applying a delta.

Ordering

If a person flips a flag on and then off a few seconds later, two messages are produced. They may not arrive in that order. A retry of the first message might land after the second message was delivered, leaving your system believing the flag is on when it is actually off. To defend against this, compare the timestamp inside each message with the timestamp of the last message you processed for the same feature and environment, and ignore anything older.

Reconciliation: the safety net

Because messages can be lost, mature teams treat webhooks as a fast signal and not as the only source of truth. They add a slow background check, perhaps every few minutes or hours, that asks the Flagsmith API for the real current state and fixes any difference. Webhooks give speed. Reconciliation gives certainty. Together they give both.

The receiver is where most webhook problems are born. A small amount of structure turns a fragile endpoint into a dependable one.

flowchart TD
    A["Incoming POST request"] --> B{"Signature valid?"}
    B -- No --> C["Reject with client error and log attempt"]
    B -- Yes --> D{"Message already seen?"}
    D -- Yes --> E["Reply success and skip work"]
    D -- No --> F["Save raw message to durable queue"]
    F --> G["Reply success immediately"]
    G --> H["Worker reads from queue"]
    H --> I{"Event type known?"}
    I -- No --> J["Log and discard safely"]
    I -- Yes --> K["Apply action such as cache refresh or alert"]
    K --> L{"Action succeeded?"}
    L -- Yes --> M["Mark message complete"]
    L -- No --> N["Retry later, then move to dead letter store"]

The principle: acknowledge fast, work later

The most valuable habit is to answer the sender quickly. A receiver should verify the message, save it somewhere durable, and reply with a success code within a moment. The real work, such as calling other services or updating big data stores, happens afterwards in a separate background worker. This protects you in two ways. The sender never times out because your work was slow, and a failure in your downstream tools does not make the sender think the delivery failed.

Walking through the flow

The diagram starts when a request arrives. The first gate is the signature check. A bad signature is rejected right away and logged, because repeated bad signatures may signal an attack. The second gate is the duplicate check. If the receiver has already handled this message, it simply replies with success and does nothing more, which is what makes retries harmless.

A fresh message is written to a durable queue, which is a waiting line that survives restarts. Only after the message is safely stored does the receiver reply with success. A worker then reads from the queue, checks whether it understands the event type, and carries out the action. If the action fails, the worker tries again later. After too many failures, the message is parked in a dead letter store, a special holding area where humans can look at problem messages without blocking everything else.

Choosing a durable queue

Many technologies can act as the durable queue: a managed cloud queue service, a message broker, or even a database table used as a to-do list. The choice matters less than the properties: messages must survive restarts, workers must be able to retry, and you must be able to see how many messages are waiting. Start with whatever your team already operates well.

Keeping the receiver healthy

Webhooks become exciting when they connect flag changes to real outcomes. Here are practical patterns that teams use every day.

Webhooks are one tool among several. Knowing the alternatives helps you choose the simplest tool that does the job.

The options on the table

Flagsmith offers several ways to keep other systems informed or in sync. Each has a different personality.

flowchart TD
    A["I need to react to flag changes"] --> B{"Is the reader my application code?"}
    B -- Yes --> C["Use a Flagsmith SDK"]
    B -- No --> D{"Does a built-in integration cover my tool?"}
    D -- Yes --> E["Use the integration"]
    D -- No --> F{"Do I need near real-time reaction?"}
    F -- Yes --> G["Use a webhook with reconciliation"]
    F -- No --> H["Poll the Admin API or read the audit log on a schedule"]

Reading the decision path

The first question separates application code from everything else. Your application should always read flag values through an SDK, because SDKs handle caching, targeting and performance for you. If the reader is not an application, check whether a built-in integration already speaks to your tool, because the least code is the best code. If neither applies and you need a fast reaction, a webhook is the answer, ideally paired with the safety net of periodic reconciliation. If speed does not matter, a scheduled read of the API or audit log is the calmest and simplest choice.

Where Flagsmith runs changes who is responsible for what. Webhooks feel a little different depending on your choice.

With the hosted service

When you use the hosted Flagsmith service, the sender lives on the public internet and is operated by the Flagsmith team. Your receiver therefore needs to be reachable from the internet. That means a public address, a valid certificate for https, and protection against abuse. You do not need to worry about the machinery that queues and retries messages, because that belongs to the provider, but you should read the documentation about delivery behaviour so your expectations match reality.

With a self-hosted installation

When you run Flagsmith yourself, the sender is your own system, so you control more of the journey. You can keep the receiver on a private network, reach it through internal addresses, and use your own certificates. In return, you own the reliability of the background machinery. Webhook delivery needs working background task processing. If background tasks are processed inside the main application, heavy bursts of webhook work can compete with web traffic. In larger setups, teams run a dedicated task processor so webhook delivery, audit processing and other background duties are separated from the web servers.

Network paths and outbound rules

On a self-hosted installation, the Flagsmith server must be allowed to make outbound web requests to your receiver. Strict firewalls and outbound proxies are a classic reason for webhook silence. Check that the worker process, not just the web process, has the right network permissions, since the worker is the one that sends the message.

You cannot fix what you cannot see. A webhook pipeline that nobody watches will eventually fail without anyone noticing.

Why silence is the scariest failure

When a web page crashes, people complain. When a webhook pipeline stops, nothing happens, and nothing happening feels like peace. A cache stays stale, an audit archive misses a week and nobody realises until the day it matters. Monitoring turns silent failure into loud alerts.

What to measure on the receiver

Logging that actually helps

Write one structured log line per message that includes the event type, the feature and environment names, the message timestamp, the decision taken, and a short identifier that links the entry to the stored raw message. Avoid logging secrets and avoid logging full flag values if they might be sensitive. When someone asks, “Did the cache hear about the 3 p.m. change?” you should be able to answer in a minute.

The sender’s side

Also look at what Flagsmith itself tells you. Review recent deliveries and failures wherever the product exposes them for your plan or deployment, and on a self-hosted system watch the health of the background task machinery, including backlog size and error rates. A rise in failed deliveries on the sender side that is not visible on the receiver side often points to network, certificate or firewall problems that stop requests from ever arriving.

Heartbeat testing

A clever safeguard is the heartbeat. Create a harmless flag in a quiet environment and flip it automatically on a schedule, for example once an hour. Your monitoring then expects a message at that rhythm. If the heartbeat message does not appear, you know the whole chain, from the sender through the network to your receiver, is broken, even when no human has changed a real flag.

When a webhook does not arrive, follow the path of the message from the beginning to the end. Most problems fall into a few familiar groups.

Common symptoms and likely causes

A safe way to look at real messages

While debugging, point a temporary webhook at a trusted request-inspection tool that simply shows incoming requests. You will see the headers and the body exactly as Flagsmith sends them. Use a non-production environment and harmless flags so that no sensitive configuration ends up in a third-party tool. Remove the temporary webhook when you are finished.

Teams that run webhooks for years tend to settle on the same few good habits, and fall into the same few traps.

Patterns that work

Single hub versus separate receivers

Anti-patterns to avoid

This chapter gathers the most valuable habits in one checklist, and the mistakes that most often cause pain.

Good hygiene over time

Webhook setups age. People leave, services move, addresses change, and certificates expire. Schedule a short review every quarter. List every webhook, confirm each still has an owner and a purpose, rotate secrets, delete the ones nobody uses and confirm your monitoring still fires. A webhook that points to a decommissioned server is not just useless, it can become a security risk if someone later takes over that address.

Respecting the human side

Automation that reacts to flag changes must still respect people. If a receiver posts to chat, keep messages short and clear. If it triggers a deployment, require approval for risky environments. If it pages someone at night, make sure the message explains what changed and how to undo it. The best webhook pipelines make people calmer and better informed, not more anxious.

A webhook you have never tested is a webhook you cannot trust. Testing is cheap when you do it in the right order.

Test in layers

Good testing starts small and grows. First, test the receiver on its own by sending it hand-made sample messages, including valid ones, ones with a wrong signature, ones with unknown event types and ones that are far too large. Second, test the connection by creating a webhook in a non-production environment and changing a harmless flag. Third, test failure by deliberately making the receiver slow or broken for a few minutes and watching what the sender does. Finally, test recovery by bringing the receiver back and checking that retries, duplicates and reconciliation behave the way you designed.

What to check in each test

• The message was verified using the correct secret.
• The receiver replied quickly with a success code.
• The raw message was stored and can be found again.
• Sending the same message twice produced only one real action.
• An older message arriving after a newer one did not undo the newer state.
• An unknown event type was logged and ignored without errors.
• Alerts fired when the heartbeat message was deliberately withheld.

Once the basics are solid, a few deeper ideas separate a good webhook pipeline from a great one.

Rotating secrets without downtime

Secrets should change from time to time, but a careless change can break delivery. Because Flagsmith signs with the secret you configured, there is a moment between changing the secret in Flagsmith and changing it in your receiver when the two disagree. A safe approach is to teach the receiver to accept either of two secrets for a short window. First, add the new secret to the receiver alongside the old one. Second, change the secret in Flagsmith. Third, watch for messages that verify with the new secret. Finally, remove the old secret from the receiver. If your setup allows two webhooks pointing at the same receiver, you can also create a second webhook with the new secret, confirm it works and then delete the old one.

Handling bursts

Automation can change hundreds of flags in seconds, producing a burst of webhooks. A receiver that processes each message with heavy work will fall behind, and retries will pile on. The cure is the durable queue from earlier, combined with a worker pool sized for steady throughput. Workers pull at their own pace, so the queue absorbs the spike. Consider also collapsing work: if twenty changes to the same feature arrive within a second, only the newest matters, so a worker can skip the older ones.

Concurrency and race conditions

When several workers process messages at once, two messages about the same feature may be handled at the same moment. Without care, the older one can finish last and overwrite the newer result. There are three common defences. You can process messages for the same feature and environment in order, by routing them to the same worker. You can use conditional updates that only apply a change when its timestamp is newer than what is stored. Or you can fetch the real current state from the API after each signal and write that, which makes order irrelevant.

Evolving message formats

Software changes. A new version of Flagsmith may add new fields or new event types. Receivers that insist on an exact shape break the first time something is added. Build tolerant readers: ignore fields you do not know, treat missing optional fields as normal, and send unknown event types to a log. Keep a library of recorded real messages and replay them in your automated tests before upgrading, so format surprises appear in a test, not in production.

Multi-region and high availability

If the receiver is important, run it in more than one place. Place copies in different zones behind a load balancer so that one failure does not stop reception. If you span regions, remember that the same message might be delivered to either region on a retry, which makes the duplicate protection store a shared concern. Use a shared fast store for processed-message memory, or accept that rare duplicates can reach both regions and keep actions idempotent so duplicates are harmless.

Governance and ownership

Every webhook should have a named owner, a documented purpose, a list of downstream systems it touches and a recovery plan. Keep that information in a simple register. When a person leaves, review the webhooks they owned. When an incident review asks, “Who gets notified when this flag changes?” the register answers in seconds.

Cost and noise control

Every message has a cost, in compute, in storage and in human attention. Filter early. If a chat room only needs production changes for a handful of important flags, drop everything else at the receiver. If an archive needs everything, keep it compact and use cheap long-term storage. Fewer, clearer messages beat a flood that people learn to ignore.

Flagsmith is one of many products that send webhooks. The industry has been refining these ideas for years, and their lessons apply directly to your Flagsmith receivers.

Ideas become clear when they are attached to a story. Let us follow an imaginary online bookshop as it adds webhooks to its Flagsmith setup.

The company and its problem

Imagine a growing online bookshop with a website, a mobile app and a warehouse system. The team uses Flagsmith to control new features such as a redesigned basket page, a faster search box and a gift-wrapping option. They have three environments: development, staging and production. Over the last quarter they met four recurring pains. Product managers do not know when a flag goes live. The search service caches flag answers and sometimes shows old behaviour for several minutes. The security team has no easy list of production changes. And the on-call engineer wastes time during incidents hunting for what changed recently.

The design they chose

The team decided on one environment webhook for production and one organisation webhook for the audit feed. Both point to a single fan-out hub owned by the platform team. The hub verifies signatures, stores each message in a durable queue and replies quickly. Three small consumers read from the queue: a chat notifier, a cache refresher and an archive writer. A fourth job, the reconciler, runs every fifteen minutes and compares the real Flagsmith state with the cache.

flowchart LR
    A["Flagsmith Production Environment"] --> B["Environment Webhook"]
    A --> C["Organisation Audit Webhook"]
    B --> D["Fan-out Hub"]
    C --> D
    D --> E["Durable Queue"]
    E --> F["Chat Notifier"]
    E --> G["Cache Refresher"]
    E --> H["Archive Writer"]
    I["Reconciler every 15 minutes"] --> G
    I --> J["Flagsmith Admin API"]
    F --> K["Team Chat Channel"]
    G --> L["Search Service Cache"]
    H --> M["Write-once Audit Store"]

Reading the picture

On the left, the production environment produces two streams of news. The environment webhook reports feature state changes. The organisation webhook reports audit entries. Both streams enter the fan-out hub, which is the single public door. The hub puts everything on a durable queue, and the three consumers each take what they need. The chat notifier writes a friendly sentence into the team channel. The cache refresher clears the right entries in the search service. The archive writer copies the audit entries into a store that cannot be edited afterwards. Finally, the reconciler works on a timer. It asks the Flagsmith admin interface for the current state, compares it with what the cache believes, and repairs any difference. If a webhook is ever lost, the reconciler catches the gap within fifteen minutes.

A day in the life

What the team learned

When something goes wrong at three in the morning, nobody wants to invent a plan. A short runbook turns panic into a checklist. Here are four scenarios every webhook team should rehearse.

Scenario one: the receiver is down

The hub stops answering because of a bad release or a failed server. Flagsmith keeps retrying for a short while, then gives up on the messages it could not deliver. The first job is to restore the receiver, ideally by rolling back the latest change. Once the receiver is healthy, look at your own logs to find the time window when it was unreachable. Then run reconciliation immediately for that window instead of waiting for the next scheduled run, and read through the audit log in Flagsmith to confirm which changes happened during the gap. Finally, write down how long the outage lasted and whether alerts fired on time.

Scenario two: the secret may have leaked

Someone pasted the secret into a public chat, or a log file containing it was shared. Treat this as urgent but calm. Create a new secret and follow the safe rotation steps so that the receiver temporarily accepts both. Change the secret in Flagsmith, confirm that messages verify with the new one, and remove the old one. Review the receiver’s logs for unusual messages that were signed with the old secret during the exposure, and check what actions those messages could have triggered. Afterwards, find out how the secret escaped and fix that habit, whether it was a shared document, a screenshot or a careless log line.

Scenario three: a storm of messages

An automation script goes wrong and flips hundreds of flags back and forth. The hub receives a flood of messages, the queue grows and the consumers fall behind. Resist the urge to switch off the hub, because the messages are evidence. First stop the misbehaving script. Next, let the queue drain while workers collapse repeated changes to the same feature, since only the newest matters. Watch the queue depth and the age of the oldest message. If the chat notifier is spamming a channel, pause that single consumer rather than the whole pipeline. After the storm, run reconciliation to confirm that the final state everywhere matches Flagsmith.

Scenario four: an accidental deletion

Someone deletes a feature in production, and a flag-deleted message flows to downstream systems. Perhaps the cache refresher now removes the flag from the cache. The immediate task is to restore the feature in Flagsmith, which may involve recreating it with its previous settings using the audit log as a guide. Then check that downstream systems recovered. This scenario is a reminder of why receivers should never take destructive actions based only on a single message. A careful receiver treats a deletion as a request to re-check the real state first.

Keeping the runbook alive

A runbook that nobody reads is as useless as no runbook at all. Keep it short, keep it where on-call engineers can find it quickly, and rehearse it in calm moments. After every real incident, update it with what you learned. Include contact details for the owners of each webhook, links to dashboards, the location of the stored raw messages and the steps for reconciliation. Assign someone to review it every quarter, together with the list of webhooks and secrets.

Capacity planning in plain words

Capacity planning sounds grand but boils down to three questions. How many messages arrive on a normal day, and how many on the busiest day? How long does one message take to process? How long can you tolerate a backlog? Multiply the busiest rate by the processing time to see how many workers you need, and add a healthy margin. Then test with a burst larger than anything you have seen. Flag changes are usually modest in number, but automation, bulk imports and busy release days can create sudden peaks. Planning for the peak means normal days feel effortless.

A quick reference for every important word used in this guide, explained in plain language so you can look things up without rereading whole chapters.

Short answers to the questions that beginners ask most often.

Everything in this guide comes down to a few ideas you can carry into your next project.

Webhooks turn Flagsmith from a quiet switchboard into a talkative member of your engineering team. When a flag changes, the right people and the right systems find out immediately. We started with the silent switch problem and ended with a design for receivers that are secure, dependable and easy to operate. The core story is simple: Flagsmith saves the change, queues a background job, builds and signs a message, and sends it to your address. Your receiver verifies it, stores it, replies quickly and then works at its own pace.