Before we talk about evaluation, we need to be clear about what is being evaluated and why a simple on/off switch is not so simple.

The simplest possible flag

A feature flag is a named setting. It has a name, such as “new-checkout-page”, and a state, which is usually “on” or “off”. Your application asks a question like “is new-checkout-page on?” and then behaves differently depending on the answer. If the answer is yes, the user sees the new checkout page. If the answer is no, the user sees the old one.

The magic is that the answer can change without anybody changing the application. No new version is built. No server is restarted. Someone simply flips the switch in a dashboard, and the next time the app asks, it hears the new answer.

Why one global answer is not enough

If every user got the same answer all the time, a flag would be a blunt tool. Real teams want finer control. They want to say “turn it on for our own employees first”, or “turn it on for ten percent of users”, or “turn it on only for customers on the premium plan”, or “turn it on for everybody except people in one country where a law is different”.

To honour these wishes, the system has to look at who is asking and what is known about them, and then pick the right answer for that person. That decision process has a name: flag evaluation.

Why this matters in production

Companies that release software many times a day cannot afford to show every new idea to every customer at once. Large engineering organisations such as Facebook, Google, Netflix, and Etsy have publicly described using flag-like systems for gradual rollouts, internal testing, and experiments. The basic benefit is the same everywhere: you separate deploying code from releasing a feature. Code can reach production quietly and be switched on later, for the right people, at the right speed.

That separation only works if evaluation is trustworthy. A wrong answer can show a half-finished feature to every customer, or hide a paid feature from someone who paid for it. So evaluation must be predictable, explainable, and fast.

Flagsmith has a small set of ideas. Learn these eight words and the rest of the article will feel natural.

Flagsmith is an open-source feature flag and remote configuration service. You can run it yourself or use the hosted version. It gives you a dashboard where people manage flags, and it gives your applications a way to ask for flag values. Here are the building blocks, from the largest to the smallest.

Environments deserve a second look

Beginners often assume a flag is either on or off everywhere. In Flagsmith, a flag has a separate state in each environment. You can have “new-checkout-page” switched on in Development so engineers can test it, off in Staging, and off in Production until the team is ready. Each environment also has its own environment key, a long random text string that your application sends to prove which environment it is asking about. Flipping the Development switch can never accidentally change Production, because they are different rooms with different keys.

Flags carry values, not only on/off

A feature state has two parts. The first is the enabled setting, which is true or false. The second is an optional value, which can be a piece of text, a number, or a true/false setting. This second part is called remote configuration. For example, a flag called “max-upload-size” might be enabled with a value of 25, meaning 25 megabytes. Change the value to 50 in the dashboard and every app that asks next will receive 50. Evaluation returns both parts together.

Evaluation is a function: inputs go in, one answer per flag comes out. Seeing it that way removes most of the mystery.

The inputs

Flagsmith needs three kinds of input to evaluate a flag.

• Which environment is asking. This comes from the environment key your application presents.
• Who is asking. This is the identity, a unique text label such as a user ID. It is optional. If you leave it out, you only get the environment-wide defaults.
• What is known about them. These are the traits, the small facts such as plan, country, or app version.

The output

For every feature in the environment, the result contains the feature name, whether it is enabled, and its value. Your app usually reads one or two of these flags and ignores the rest. Often the app receives the whole list at once, and then asks local questions such as “is this one enabled?” without making another trip across the network.

The three layers of the answer

Flagsmith builds the answer in layers, and the most specific layer always wins. We will study this in detail in a later chapter, but here is the short version.

Evaluation is deterministic

The word deterministic means the same inputs always give the same output. If you evaluate the same user against the same settings twice, you get the same answer both times. This matters a lot for percentage rollouts. A user who is in the “ten percent” group today should not drop out tomorrow just because they refreshed the page. Flagsmith achieves this by turning identities into numbers using a stable calculation, which we will explain in the chapter on percentage rollouts.

Where does the answer come from, and which pieces of machinery are involved? Here is the whole picture in one diagram.

At a high level, Flagsmith has a place where humans manage flags, a place where data is stored, a place where flag requests are served, and the libraries inside your apps that talk to it. Let us name each part.

flowchart LR
    ADMIN["Team member using the Dashboard"] --> DASH["Flagsmith Dashboard"]
    DASH --> CORE["Core API - manages flags, segments, identities"]
    CORE --> DB[("Primary database")]
    CORE --> AUDIT["Audit log and webhooks"]
    CORE --> EDGE["Flag serving layer - fast read path"]
    APPC["Client app - browser or mobile"] --> CSDK["Client-side SDK"]
    APPS["Backend service"] --> SSDK["Server-side SDK"]
    CSDK -->|"environment key"| EDGE
    SSDK -->|"remote evaluation"| EDGE
    SSDK -->|"server-side key - environment document"| CORE
    SSDK --> LOCAL["Local evaluation engine inside the SDK"]
    EDGE --> CACHE["Cache"]
    CACHE --> DB

The pieces explained

Read path versus write path

Engineers like to split a system into a write path and a read path. The write path is what happens when a person changes a flag. It is rare, perhaps a few times a day, and it can afford to be careful and slow. The read path is what happens when an app asks for flags. It happens all the time, perhaps millions of times per minute, and it must be extremely quick. Flagsmith’s design keeps these two paths separate in spirit: changes flow through the Core API into storage, and then are published to a layer built to answer reads.

The heart of flag evaluation is a strict order of precedence. Memorise this order and you can predict almost any answer Flagsmith gives.

When Flagsmith evaluates a flag for an identity, it walks through a short list of questions. It stops as soon as one of them gives an answer.

flowchart TD
    START["Request: environment key plus identity plus traits"] --> Q1{"Does this identity have its own override for the feature?"}
    Q1 -->|"Yes"| R1["Return the identity override"]
    Q1 -->|"No"| Q2{"Does the identity match any segment that overrides the feature?"}
    Q2 -->|"Yes"| PICK["Pick the matching segment with the highest priority"]
    PICK --> R2["Return that segment override"]
    Q2 -->|"No"| Q3{"Is the feature multivariate?"}
    Q3 -->|"Yes"| HASH["Hash the identity into a number from 0 to 100"]
    HASH --> R3["Return the matching variation or the control value"]
    Q3 -->|"No"| R4["Return the environment default"]

How segment priority works

One identity can belong to many segments at once. A person might be in “premium customers”, “Indian users”, and “beta testers” all at the same time. If more than one of those segments overrides the same feature, which one speaks first? Flagsmith lets you order the segment overrides for each feature, and the one at the top of the list wins. In the dashboard you can drag overrides up and down to change this. Only the winner matters. The others are not blended or averaged.

A worked example in words

A streaming service has a flag called “new-player”. The environment default is off. A segment called “beta-testers” has an override turning it on. A segment called “old-smart-tvs” has an override turning it off, because the new player does not run on old televisions. The old-smart-tvs override is placed above beta-testers in priority.

Now consider three people. Asha is a beta tester on a new phone: she matches beta-testers only, so she sees the new player. Ben is a beta tester on an old television: he matches both segments, but old-smart-tvs sits higher, so he sees the old player. Chitra is a regular user on any device: she matches neither, so she gets the environment default, off. Finally, imagine the support team gives Ben an identity override turning the flag on to investigate a bug. Now Ben sees the new player, because identity overrides sit above all segments.

Identities and traits are the raw material for every targeted decision. Treat them with care and your rules stay simple.

What is an identity?

An identity is a label that tells Flagsmith “this is the same entity as before”. Most teams use their own user ID. Some use an account ID, a device ID, or an organisation ID. The choice matters because the identity is the unit of consistency. If you use a user ID, every device that user owns will get the same flag answers. If you use a device ID, the same human on a phone and a laptop could see different experiences.

What is a trait?

A trait is a named fact about an identity: a key and a value. Examples are “plan” with the value “premium”, “country” with the value “IN”, “app_version” with the value “4.2.0”, or “signup_days” with the value 12. Traits can be text, whole numbers, decimals, or true/false.

Traits can reach Flagsmith in two ways. Your application can send them at the moment it asks for flags, which keeps them fresh. Or they can be stored with the identity in Flagsmith so they are remembered next time. When the hosted service evaluates remotely and traits are sent along with the request, those traits are saved against the identity, so later requests and segment rules can use them. Newer versions also support transient identities and traits, which are used for a single evaluation and not saved. That is handy for sensitive or fast-changing facts you do not want stored.

Anonymous users

What if the person is not logged in? You have two options. You can generate a random identifier for the visitor and keep it in the browser, so the same visitor keeps seeing the same experience on that browser. Or you can skip the identity altogether and ask only for environment-level flags. In that case no segment or percentage logic can be used, because Flagsmith has nobody to look at. The user gets the environment default.

Choosing good traits

A segment turns a vague wish like “our power users in Europe” into a precise, repeatable test that the engine can run in a blink.

How a segment is built

A segment is made of one or more rules. A rule contains one or more conditions. A condition looks at one trait and compares it with a value using an operator. For example: the trait “plan” equals “premium”. Each rule says how its conditions combine:

• All: every condition in the rule must be true, like saying “and”.
• Any: at least one condition must be true, like saying “or”.
• None: no condition may be true, like saying “not any of these”.

When a segment has more than one rule, an identity must satisfy all of the rules to be in the segment. Rules can also be nested inside rules for more complex shapes, though simpler is almost always better.

The operators you can use

What happens when a trait is missing?

If a rule looks at a trait the identity does not have, the condition generally does not match, with the “is not set” operator being the natural exception. This is usually what you want, but it can surprise you. If you write “country is not equal to US” and a user has no country trait at all, do not assume they will be included. Always test with real examples that have missing data.

Segments are reusable

A segment is defined once at the project level and can then be used by overrides on many features. Change the segment’s definition and every feature using it changes behaviour together. That is powerful and slightly dangerous: editing “premium customers” to also include trial users affects every feature that targets it. Teams keep segment names descriptive and review changes with the same care they apply to code.

How do you give a feature to exactly ten percent of people, and make sure the same ten percent stay in tomorrow? The answer is stable hashing.

Two different ideas that sound alike

Flagsmith offers two ways to spread something across a population. They are easy to mix up.

How the same person gets the same answer

A hash function is a recipe that turns any piece of text into a number in a repeatable way. The same text always gives the same number, but similar-looking texts give wildly different numbers. Flagsmith feeds identifying details of the feature or segment and of the identity into such a recipe and converts the result into a position between 0 and 100. If the position falls inside the allowed range, the identity is in. Because the recipe never changes, the position of a given identity never changes either.

This has a lovely side effect. If you raise a rollout from ten percent to twenty percent, the people who were already inside generally stay inside, and new people are added on top. Nobody who had the feature loses it just because you widened the net.

Multivariate options

A multivariate feature has a control value, which is the normal environment value, and several variations, each with a percentage. The percentages add up to at most 100. Whatever is left over keeps the control value. If you give variation A 20 percent and variation B 30 percent, then 50 percent of identities stay on the control. Each identity is placed in the same bucket every time they are evaluated, so a visitor does not see the red button on Monday and the blue button on Tuesday.

There is an important limit. Multivariate allocation needs an identity to hash. If you ask for flags without an identity, there is nobody to place into a bucket, so you get the control value.

Why stickiness matters for experiments

If you are measuring whether a new button increases purchases, you must show each person the same version for the whole experiment. Otherwise the numbers become meaningless, because one person could be counted in both groups. Stable hashing gives you that consistency without keeping a giant table of who saw what. Flagsmith itself decides the variation; the measuring of results is usually done by connecting Flagsmith to your analytics tool, which receives information about which variation each user saw.

There are two places where the decision can be made: on Flagsmith’s servers, or inside your own application. This choice shapes speed, cost, and freshness.

Remote evaluation: ask the expert each time

In remote evaluation, your SDK sends the identity and traits to Flagsmith over the internet. Flagsmith’s servers run the rules and send back the finished list of answers. This is the default and the only mode available to client-side SDKs (the ones in browsers and mobile apps). It keeps rules private and your app simple, at the price of one network round trip for each identity you evaluate.

sequenceDiagram
    participant U as User
    participant A as App with SDK
    participant F as Flagsmith serving layer
    participant D as Stored settings
    U->>A: Opens the app
    A->>F: Request flags with environment key, identity, traits
    F->>D: Load environment state, segments, overrides
    D-->>F: Settings
    F->>F: Run the evaluation order
    F-->>A: Flag names, enabled states, values
    A->>A: Keep the answers in memory
    A-->>U: Shows the right experience

Local evaluation: carry the rulebook with you

In local evaluation, a server-side SDK downloads a complete description of the environment, often called the environment document. It contains all the features, their values, the segments and their rules, multivariate options, and in recent versions the identity overrides. The SDK then makes decisions on its own machine, with no network call per decision. It refreshes the document on a timer, which by default is about once every sixty seconds, so changes arrive shortly after they are made.

flowchart TD
    S["Backend service starts"] --> K["SDK uses server-side key"]
    K --> DL["Download environment document"]
    DL --> MEM["Hold document in memory"]
    MEM --> REQ["Incoming request with identity and traits"]
    REQ --> ENG["Local engine runs the evaluation order"]
    ENG --> OUT["Answer in microseconds, no network call"]
    MEM --> TIMER["Refresh timer about every 60 seconds"]
    TIMER --> DL

Side-by-side comparison

Let us follow a single request from the moment a user taps a button to the moment the screen changes, so you can see every hand-off.

Why fetching all flags at once is smart

Many beginners expect one network call per flag. In practice the SDK fetches the whole set for the identity in one trip, and later questions are answered from memory. This design means that adding a fiftieth flag to your app does not add a fiftieth network call. It also means all flags you read in one screen are consistent with each other, because they came from the same moment in time.

Where your code runs decides which key you may use and which evaluation modes you may choose. Getting this wrong is the most common security slip.

Why two kinds of key exist

The client-side key is meant to be embedded in an app that thousands of strangers will download. Anyone can pull it out and read it. So Flagsmith designs it to be low-power: it can ask “what are the flags for this identity?”, and nothing more sensitive. The server-side key is like a staff pass. If it leaks, an outsider could download your entire environment document, including your segment definitions and the names of your experiments. Treat it like a database password.

Who should be allowed to see which flag?

Remember that the flags you send to a browser are visible to the user. If a flag’s name or value hints at a secret plan, such as “launch-competitor-killer-feature”, a curious person can read it in their browser tools. A good habit is to use neutral, boring flag names for anything that reaches the client, and to keep truly confidential switches on the server side only. Also never use a client-side flag as the only protection for something sensitive. A user can alter what their own browser displays, so real access checks must happen on your server.

Trust boundaries in one sentence

Anything running on a device the user owns can be inspected, so it can only be trusted to display things. Anything running on your servers can be trusted to decide and protect things.

A flag check sits in the hot path of your application, so it has to be nearly free. Here is how the whole stack keeps it that way, even at millions of requests per minute.

Why flag reads are easy to scale

Flag evaluation has a very friendly pattern: reads are enormously more common than writes. A flag might be read ten million times a day and changed twice. Systems with this pattern respond well to caching, which means keeping a ready-made copy of an answer close to where it is needed so it need not be rebuilt each time.

The layers of caching

The freshness trade-off

Every cache creates a gap between “the truth in the database” and “what the app currently believes”. A shorter refresh interval reduces the gap but increases traffic. A longer interval does the reverse. For most features, a delay of seconds to a minute is perfectly fine. For urgent switches, such as turning off a broken feature, teams want the change to land quickly. That is where polling intervals and, for browsers and apps, real-time updates come in. Flagsmith supports streaming updates to some client SDKs over a one-way connection called server-sent events, so a change in the dashboard can reach open apps without waiting for the next poll. Availability depends on the SDK and plan, so check what your own setup supports.

Scaling to huge traffic

Picture a service with a hundred instances, each handling a thousand requests per second. If every request called Flagsmith remotely, that would be a hundred thousand flag calls every second. With local evaluation, the same fleet makes a few hundred small refresh calls per minute, each downloading one document. The load on Flagsmith drops by several orders of magnitude, and your request latency no longer includes a hop to a third-party system. Large engineering organisations follow the same idea with their own flag systems: push the rules to the edge of the fleet and decide in memory.

Watch your payload size

A flag response contains every feature in the environment, so hundreds of unused flags make every response larger and every device parse more. Remove dead flags regularly. Smaller payloads mean faster starts, especially on slow mobile networks.

Any network call can fail. A good flag setup makes sure that a flag problem never becomes a customer-facing outage.

The golden rule: flags must fail safe

Imagine Flagsmith becomes unreachable for a few minutes. What should your app do? It must have a sensible answer ready. For a risky new feature, the safe answer is “off”. For something that should stay visible, like a core menu, the safe answer is “on”. The decision of what is safe has to be made by humans ahead of time, flag by flag.

Tools that protect you

Failure scenarios and what you will see

Do not forget human mistakes

The most frequent flag outage is not a server failing but a person flipping the wrong switch in the wrong environment. Flagsmith helps with colour-coded environments, an audit log, permissions, and, in higher plans, change requests that require approval and scheduled changes. Use them for Production.

Flags decide who sees what. That makes them a security-relevant system, and the data you feed into them a privacy matter.

The security checklist

• Keys: keep server-side keys in a secrets manager, not in code. Rotate them on a schedule and after any suspected leak.
• Least privilege: give team members only the permissions they need. Not everyone should be able to change Production.
• Audit trail: every change to a flag is recorded with who, what, and when. Review it after incidents.
• Transport: all traffic to Flagsmith should use encrypted connections.
• Server-side enforcement: never rely on a flag in a browser as the only barrier protecting paid or private features.

Privacy: choose your traits wisely

Traits are sent to Flagsmith and, in remote mode, may be stored with the identity. Under privacy laws such as GDPR, personal data needs a clear reason to be stored and a way to be deleted. A safe pattern is to send coarse, non-identifying facts, such as “plan = premium” or “region = EU”, rather than names or emails. Use a pseudonymous identifier, like an internal ID, instead of an email address as the identity. If you must use a sensitive value for one decision, a transient trait lets you use it without saving it.

Self-hosting versus hosted

Because Flagsmith is open source, organisations with strict data rules can run it inside their own network, so identities and traits never leave their environment. Teams that prefer less work use the hosted version. The evaluation logic is the same either way.

When a user says “I do not see the new feature”, you need a calm, repeatable way to find out why.

What you can observe

A calm way to debug “why did this user get that value?”

Watch the health of the flag system itself

Track how long flag fetches take, how often they fail, and how often your app falls back to defaults. A rising fallback rate is an early warning that something is off, long before users complain.

Flags are used in a few well-known shapes. Pick the right shape and the flag does its job cleanly and then goes away.

A safe rollout recipe

Start with the flag off everywhere. Add an override for an internal staff segment and test in Production with real data. Next, add a small percentage segment, perhaps one percent, and watch your monitoring. Step up gradually: five, twenty-five, fifty, one hundred. At any sign of trouble, set the default back to off. When the feature has been fully on for a comfortable period, delete the flag and the old code path.

Anti-patterns to avoid

These habits come from teams who have lived with feature flags for years. Follow them and you skip a lot of pain.

Habits worth building

• Name flags clearly. Use names that describe the feature, not the ticket number. “new-checkout-page” beats “flag-4417”.
• Write a description. One sentence on why the flag exists and who owns it.
• Use tags. Label flags by team or purpose so you can filter and clean up easily.
• Choose defaults on purpose. Decide what the app does when a flag cannot be fetched.
• Test both states. Automated tests should cover the flag on and the flag off.
• Roll out in small steps. Never jump from zero to everyone for a risky change.
• Protect Production. Use permissions, approvals, and the audit log.
• Clean up. Remove temporary flags soon after a rollout finishes.

Common mistakes

The same few ideas show up inside well-known engineering organisations. Here is how the patterns look in practice.

Feature flags are powerful, but power has a price. A wise team knows both sides.

The big trade-offs, plainly stated

There is no single best answer. The right balance depends on your traffic, your risk tolerance, and how fast your team needs changes to land.

Short answers to the questions that come up again and again.