Releasing to everyone at once is the riskiest way to ship software. Releasing to a slice first is the habit that makes modern teams calm.

The cost of a big bang

In a traditional release, new behaviour reaches all users at the same moment. If it contains a mistake, every user meets that mistake together, the support queue floods and the team scrambles to fix or reverse it under pressure. Even a small fault becomes large simply because of how many people it touches. Software teams call this the blast radius: the number of users affected when something goes wrong.

Shrinking the blast radius

Gradual exposure shrinks the blast radius on purpose. If the new payment screen goes first to one percent of customers, a hidden fault harms one percent, not all of them. The team sees the problem in graphs, switches the feature off for that small group and fixes it calmly. Nobody else ever knew. The cost of learning about the fault drops by a factor of one hundred, and learning early is what a rollout is for.

What is a percentage split?

A percentage split is a rule that is true for a chosen share of users, such as five percent or fifty percent. It does not look at a user’s plan, country or age. It looks at the user’s identifier, turns it into a repeatable number and checks whether that number falls inside the chosen share. The result looks random, yet it is completely predictable for any one person, which is exactly the combination a rollout needs.

What does “within a segment” add?

A split on its own selects a slice of everyone. That is often too blunt. You may not want a random tenth of all users, including people whose apps cannot show the screen or who live in a country where the feature is not licensed. A segment lets you describe the eligible people first, and the split then selects a slice of those. The first layer protects you from showing the feature to the wrong people. The second layer protects you from showing it to too many of the right ones.

A small set of terms describes everything in this guide. Learn them once and every later chapter reads easily.

flowchart TD
    SEG["Segment: Premium customers in India"] --> RG1["Rule group ALL: eligibility"]
    SEG --> RG2["Rule group ALL: the slice"]
    RG1 --> C1["Condition: plan equals premium"]
    RG1 --> C2["Condition: country equals IN"]
    RG1 --> C3["Condition: app version at least 4.2.0"]
    RG2 --> C4["Condition: percentage split is 10"]
    SEG -. "used by" .-> OV["Segment override on a flag in one environment"]

Read the figure from the top. One segment holds two rule groups. The first group describes who is eligible: premium plan, in India, on a recent app version. The second group holds the percentage split, which picks a share of people. Because top-level rule groups combine like the word AND, a person must pass both groups to belong to the segment. The dotted arrow shows that a flag uses the whole segment through an override, so the flag never needs to know how the audience was built.

The phrase hides two separate decisions. Keeping them apart in your mind prevents most confusion.

Decision one: who is eligible?

Eligibility is answered by ordinary conditions on traits. A premium customer in India on a recent app is eligible. A free customer in the United States is not. Eligibility depends on facts about the person, facts that can change over time: a customer may upgrade, move country or update their app. When a fact changes, eligibility changes with it.

Decision two: are they in the slice?

The slice is answered by the percentage split and depends only on the person’s identifier, not on any trait. A person who is in the ten percent slice today is in the ten percent slice tomorrow, whatever their plan or country. The slice is a fixed property of the identifier, mixed with something unique to the segment. It changes only when you change the threshold.

Putting the two decisions together

A person belongs to the segment only when both answers are yes: eligible, and inside the slice. Since the slice has nothing to do with traits, it picks roughly the same share of every group. So if you ask for ten percent and there are twenty thousand eligible customers, you can expect about two thousand of them, give or take a small statistical wobble that Chapter 9 explains. You do not get ten percent of everyone, you get ten percent of the eligible group.

The table shows all four combinations. Only Asha passes both checks. Ben is eligible but happens to sit outside the slice, so he sees nothing yet. Chitra sits inside the slice but is not eligible, so the slice cannot help her. Dev fails both. This is why the order of thinking matters: the slice never overrides eligibility.

Why not simply make the segment smaller by rules?

You might ask why anyone needs a split at all, since rules alone can shrink an audience. A rule such as “customer number ends in 7” would do something similar. The reasons to prefer the split are practical. It is designed to spread people evenly and fairly, it is simple to widen from ten to twenty percent in one edit, and it does not require any special trait to exist. Rules based on real traits describe meaning. A split describes a quantity.

The calculation behind a percentage split is short, repeatable and worth understanding, because every surprising behaviour follows from it.

What is a hash?

A hash function takes any input, such as a name or a number, and returns a scrambled number of fixed size. Two properties make it useful here. It is repeatable: the same input always gives the same output. And it is evenly spread: different inputs land all over the range with no pattern, so similar inputs, such as user-100 and user-101, produce unrelated outputs. You cannot guess the output by looking at the input, but you can always recompute it.

flowchart LR
    A["Identity identifier"] --> M["Mix with the segment's own ID"]
    S["Segment ID"] --> M
    M --> H["Hash function scrambles the mix"]
    H --> B["Scale to a bucket from 0 to 100"]
    B --> Q{"Bucket at or below the threshold?"}
    Q -- "Yes" --> IN["Condition is true"]
    Q -- "No" --> OUT["Condition is false"]

Follow the figure from the left. The identity’s identifier and the segment’s own internal ID are mixed together into one piece of text. The hash function scrambles it into a large number, which is then scaled to a bucket between zero and one hundred. Finally the bucket is compared with the threshold you typed. If the bucket is at or below the threshold, the condition is true. Everything on this path is repeatable, which is why a person’s answer never changes unless the threshold, the identifier or the segment changes.

Why the segment’s ID is mixed in

If only the identifier were hashed, every split in the whole system would pick the same people. The same lucky ten percent would see every new feature, and the same unlucky ninety percent would never be the first to see anything. Mixing in the segment’s own ID gives each segment its own private scramble. Two segments, each set to ten percent, pick different tenths of the population. This is fair, and it stops unrelated experiments from landing on the same group of customers.

Threshold, in plain terms

Think of the bucket as a position along a line from zero to one hundred. The threshold is a line drawn across it. Everyone whose position is to the left of the line is in. Set the threshold to zero and nobody is in. Set it to one hundred and everyone is in. Set it to ten and about one tenth of the people, those who landed in the leftmost tenth of the line, are in.

Anyone can pick ten percent of users at random. The hard part is picking the same ten percent every time.

The flicker problem

Imagine deciding on every page load, using a fresh random number, whether to show the new feature. A single visitor would see the new design on one page, the old on the next, and the new again after a refresh. The experience would feel broken. Bug reports would be impossible to reproduce, because the person who saw the problem could not see it again. Any measurement of the feature’s effect would be meaningless, since the same person would be counted on both sides.

Three promises of a good split

Why the nesting promise is so valuable

Think about what you would feel as a customer. You are one of the first to receive a new feature. A week later the team widens the audience, and your feature vanishes. You are confused, perhaps annoyed. With stable nesting, widening the audience never takes anything away from early users. They keep a continuous experience, and the team can compare how the same people behave over time.

What breaks the promises

A split rarely stands alone. Its power comes from what you pair it with, and so do its pitfalls.

The split is just another condition

Inside a segment, the percentage split is one condition among others. Flagsmith evaluates each condition for the person, and the rule type, “all”, “any” or “none”, decides how the answers combine. The split does not have any special authority. It is true or false for a person, like “plan equals premium” is true or false.

The usual pattern: eligibility AND the slice

The most common arrangement uses an “all” rule, or several top-level rules, so that the person must satisfy the eligibility conditions and the split. Because rules combine like AND, a person who fails either part is outside. In practice, this means that the split picks about the chosen share of the eligible group, because the scrambling has nothing to do with plan, country or any other trait.

The last row shows the danger of choosing the wrong connector. With “any”, the split no longer narrows the audience, it adds to it. A person who is premium qualifies automatically, and a person who is not premium qualifies by luck of the slice. Many accidental large releases begin with an “any” where an “all” was meant.

Order does not matter, structure does

Whether the split sits first or last inside a rule makes no difference to the answer, since each condition is evaluated for the same person and the results are combined. What matters is how conditions are grouped. A split placed in the same “all” rule as the eligibility conditions behaves the same as one placed in a separate top-level rule. Choose the layout that makes the segment easiest to read, usually with eligibility in one group and the slice in another, as in Figure 1.

Excluding test and internal accounts

Real customers and test accounts should rarely share a rollout. Test accounts distort measurements, and internal accounts may already have the feature through a staff override. A common structure puts the split and eligibility in an “all” rule, and adds a separate “none” rule that excludes accounts with an is_test trait set to true. The person must pass every top-level rule, so a test account is always outside the segment, even when the split would have included it.

A single threshold picks the left end of the line. With a little care you can cut the line into several separate slices, each for a different purpose.

The line is shared inside one segment

Every percentage split condition in the same segment uses the same scramble, so a person has one bucket for that segment. If one condition says “bucket at or below ten” and another in the same segment says “bucket at or below fifty”, then everyone in the first is automatically in the second. The two conditions are not independent. They are nested, like Russian dolls.

Making a band

That nesting lets you cut a band out of the middle of the line. Suppose you want people whose bucket lies between ten and thirty. You need “bucket at or below thirty” and also “not at or below ten”. In Flagsmith’s rule language, that is an “all” rule containing a split of thirty, together with a “none” rule containing a split of ten. People in the first ten percent are removed by the “none” rule, and people above thirty are removed by the split. What remains is the band from ten to thirty, which is twenty percent of the population.

flowchart LR
    L0["Bucket 0 to 10: Band A"] --> L1["Bucket 10 to 30: Band B"]
    L1 --> L2["Bucket 30 to 60: Band C"]
    L2 --> L3["Bucket 60 to 100: Band D"]
    L0 -. "split 10" .-> T1["Threshold at 10"]
    L1 -. "split 30" .-> T2["Threshold at 30"]
    L2 -. "split 60" .-> T3["Threshold at 60"]

The figure shows the line from zero to one hundred divided by three thresholds at ten, thirty and sixty. The first band covers ten percent, the second twenty, the third thirty and the last forty. Each band can be its own segment, built from one split and, for all but the first, a “none” rule excluding the band to its left. Because all of them are built from the same line, the bands never overlap and together they cover everyone.

Why bands are useful

Cautions with bands

Bands rely on all conditions sharing one segment’s scramble. If you put each band in a different segment, each has its own scramble, and the bands will overlap randomly. Build all the bands you want to be disjoint from the same segment’s line, or accept the overlap. Also, test every band edge with sample users. The edges are where off-by-one errors in thresholds live. And keep band layouts simple: more than three or four bands in one design is a sign you may be rebuilding an experimentation tool by hand.

Real products run several rollouts at once. Knowing how their audiences overlap prevents accidental pile-ups on the same unlucky people.

Different segments, different scrambles

Each segment mixes its own internal ID into the scramble, so two segments produce unrelated buckets for the same person. A customer who sits at bucket four in one segment might sit at bucket eighty-one in another. The two lines are independent of each other. That is a deliberate design, and it is good news: your ten percent for the new checkout and your ten percent for the new search page are different ten percents.

The arithmetic of overlap

Independence lets you predict overlap with simple multiplication. If one rollout includes ten percent of customers and a separate rollout includes ten percent, the share of customers in both is ten percent of ten percent, which is one percent. The share in neither is ninety percent of ninety percent, which is eighty-one percent. The remaining eighteen percent are in exactly one of the two.

Read any row by multiplying. For twenty and thirty percent, both is 0.2 times 0.3, which is six percent. In exactly one is twenty plus thirty minus twice the overlap, which is thirty-eight percent. In neither is eighty percent of seventy percent, which is fifty-six percent. These are expected shares, and small numbers of users will vary, as the next chapter shows.

When overlap is a problem

Independence is usually healthy, but two situations call for care. The first is when two risky features interact. If both the new checkout and the new payment provider reach the same one percent, a fault might appear only when they meet, and you will not know which caused it. The second is when you are running two experiments that measure the same outcome. Overlapping groups can mix their effects. In both cases, consider running the rollouts in sequence, or build them as bands of one line, so they are guaranteed never to overlap.

When you want exact overlap

Sometimes you want the opposite: two flags that must always reach exactly the same people, such as a new screen and the new service behind it. Never build two separate segments with the same threshold, because they will not match. Build one segment and attach it to both flags, or ask whether the two flags should really be one flag. A single segment is the only reliable way to guarantee the same audience.

Ten percent does not mean exactly ten percent. Understanding the wobble prevents false alarms and false comfort.

A target, not a guarantee

The percentage split spreads people evenly in the long run, but it does not count them. For each person, the bucket is effectively a fair random number. Over a large crowd, about ten percent land below ten. Over a small crowd, the share can drift well above or below ten. Flip a fair coin ten times and you will often get six heads, or four. Flip it a million times and you will be very close to half. Percentage splits behave the same way.

How big is the wobble?

A simple rule describes it. For a group of n eligible people and a chosen share p, the expected number included is n times p. The typical deviation from that number, called the standard deviation, is the square root of n times p times one minus p. About two thirds of the time, the real number lands within one standard deviation of the expected one, and about ninety-five percent of the time within two.

The table shows two lessons. First, the relative wobble shrinks as groups grow: for a hundred people it is about thirty percent of the target, for a hundred thousand it is about one percent. Second, a tiny share of a small group is risky. One percent of a thousand people is about ten, but you might get seven or thirteen, and a single user more or less is a huge change in relative terms.

Practical consequences

The whole mechanism depends on one piece of text. Pick it well and the split is invisible. Pick it badly and everything feels random.

What makes a good identifier?

A good identifier is unique to one subject, never changes and is available wherever flags are requested. A permanent account ID fits all three. An email address fits uniqueness but can change when a person updates their profile, and it is personal data you may prefer not to share. A session ID changes constantly and is almost always a poor choice. A device ID is stable on one device but differs between a person’s phone and laptop.

The anonymous-to-logged-in problem

Many applications have visitors before they log in. Giving them a temporary ID is sensible, but if the same person then logs in and receives a different permanent ID, their bucket changes. A visitor in the slice before login might be outside it afterwards, so the feature disappears at the moment they sign in. There are two good fixes. Use the permanent ID after login and accept that features may change at that moment. Or keep the same stored ID before and after login for the same person, which requires linking the two at your end.

Splitting by company, not by person

Some products sell to organisations, and everyone in a company should see the same thing. A feature that appears for one colleague and not another causes confusion and support requests. In that case, use the organisation’s ID as the identifier the split hashes. Everyone in the same company then lands in the same bucket, and the whole company is in or out together. The trade-off is that your slice is made of companies, so a few very large ones can swing your numbers a lot.

Do identifiers carry across environments?

Do not assume that the same person lands on the same bucket in Staging and Production. Flagsmith’s rule engine may combine the identifier with environment-specific information before hashing, in which case the same text produces a different bucket in each environment, and in any case the segment may carry different internal details. Treat the buckets as belonging to one environment, and do not use Staging results to predict exactly who will be in a Production slice. Confirm the behaviour of your version in its documentation.

Changing the threshold is the most common edit you will make. Its effects are asymmetric, and the difference is worth knowing before the first emergency.

Raising: calm and additive

When you raise the threshold, the barrier on the line moves right, and people between the old and new positions become members. Nobody who was already a member loses the feature. Raising is therefore the gentle direction. Customers experience either no change, if they were already in, or the arrival of something new, if they were just added.

Lowering: abrupt and uneven

When you lower the threshold, the barrier moves left, and the people between the new and old positions lose the feature. Lowering from fifty to ten removes eighty percent of the members at once, and they are the people who were most recently added. They may have started to rely on the feature, may have unsaved work involving it or may have already given feedback about it. A sudden removal can be as disruptive as a bug.

flowchart TD
    A["Current threshold is 25"] --> B{"What do you want?"}
    B -- "Widen audience" --> C["Raise threshold, for example to 50"]
    C --> C1["Earlier members keep the feature"]
    C1 --> C2["New members arrive in the band 25 to 50"]
    B -- "Narrow audience" --> D["Lower threshold, for example to 10"]
    D --> D1["Members in the band 10 to 25 lose the feature"]
    D1 --> D2["Check that removal is safe for them"]
    B -- "Stop everything" --> E["Switch the flag or override off"]

The figure starts from a rollout at twenty-five percent and asks what you want. Widening means raising the threshold, which keeps everyone and adds a new band. Narrowing means lowering it, which removes the band between the new and old thresholds, so you must check that removal is safe for those people. And if the aim is to stop a problem quickly, switching the override or flag off is cleaner and faster than lowering the number, because it treats everybody the same way.

When lowering is the right tool

A ramp is a promise to yourself: we will widen only when the evidence says it is safe. Writing it down turns a hope into a procedure.

The typical shape

Most ramps follow a similar shape, starting very small and roughly doubling or more at each step. A common sequence is staff only, then one percent, five percent, twenty-five percent, fifty percent and everyone. Early steps are small because uncertainty is highest. Later steps are larger because evidence has accumulated and the cost of waiting grows. The exact numbers depend on how risky the feature is and how many users you have.

flowchart TD
    A["Staff only"] --> B["Raise to 1 percent"]
    B --> C{"Healthy for a full day?"}
    C -- "No" --> X["Switch off and investigate"]
    C -- "Yes" --> D["Raise to 5 percent"]
    D --> E{"Healthy?"}
    E -- "No" --> X
    E -- "Yes" --> F["Raise to 25 percent"]
    F --> G{"Healthy?"}
    G -- "No" --> X
    G -- "Yes" --> H["Raise to 50 percent, then 100 percent"]
    H --> I["Retire the split and the flag"]

The figure shows the principle: after every increase there is a pause and a question. Is the feature healthy for long enough to trust? A yes sends you up one rung. A no sends you to the same place every time, which is switching off and investigating. That single, well-known exit is the point. In the middle of an incident, nobody wants to choose between several options.

How long should each step last?

A good step is long enough to include at least one full cycle of your users’ behaviour. If your customers shop mostly on weekends, a one-day step on a Tuesday teaches you little about weekends. If your business has a monthly billing cycle, some problems only appear at month end. Choose waits based on the pattern of your own traffic, not on a generic template.

A ramp is only as safe as the signals that guard it. Choose them before you start, not while you are anxious.

Guardrail metrics

A guardrail metric is a measurement that must not get worse while you try something new, such as error rate, response time, crash rate or the share of orders that complete. You are not trying to improve guardrails, you are trying to protect them. Before the rollout, write down each guardrail, its normal range and the value that triggers a pause. Having the line drawn in advance removes the temptation to explain away a bad number.

Success metrics

A success metric is the number you hope the feature improves, such as conversion rate or time to finish a task. Success metrics need more data and more time to read than guardrails, since the improvements are usually small and noisy. Do not widen a rollout merely because a success metric is slightly up in a small slice. Use guardrails to decide whether it is safe, and use success metrics to decide whether it is worthwhile.

Assignment is not exposure

A person who is in the split is assigned to receive the feature. But assignment is not the same as exposure. Someone may be assigned yet never visit the page where the feature appears. If you compare all assigned users against all unassigned users, the difference is diluted by people who never saw anything. Where you can, record an event when the feature is actually shown, and measure among those who were exposed, while still reporting how many were assigned.

Compare like with like

Always compare the slice against a group that is similar and measured over the same period. Comparing this week’s slice with last month’s overall numbers mixes the effect of the feature with the effect of the calendar, holidays, marketing campaigns and seasons. The group that is eligible but outside the slice is a natural comparison, because the scramble made it statistically similar. This is a useful, rarely appreciated gift of a percentage split.

Flagsmith has two mechanisms that both talk about percentages. They solve different problems, and it helps to know which is which.

Two mechanisms, two questions

A percentage split inside a segment answers: “Is this person in or out of an audience?” It produces a yes or no. A multivariate flag’s percentage allocation answers a different question: “Among the people who get this flag, which variant does each person receive?” It divides people between several values, such as blue, green and orange buttons, with a share for each. Both use repeatable scrambling, so assignments are stable in both cases.

Using them together

The two often work side by side. A flag controls a new checkout design with three variants. The multivariate allocation decides which variant a person sees. A segment with a percentage split decides who gets any treatment at all, with everyone else keeping the old design through the default. Reading this as two layers helps: the segment says “who participates”, the multivariate setting says “which version they see”.

A caution about overrides and variants

A segment override typically supplies a fixed value for its members, not a new split between variants. If you want part of an audience to see variant A and part variant B from within a segment, the usual pattern is to build separate segments, as bands of one line, each with its own fixed value. Check how your version handles segment overrides on multivariate flags, because the details of how variants are chosen for segment members can differ.

A split segment is rarely alone on a flag. Its place in the priority list decides who it can reach and who it can never touch.

How overrides are chosen

When a flag has several segment overrides and a person belongs to more than one of the segments, Flagsmith needs a tie-breaker. It checks the overrides from the top of the priority list and uses the first segment the person belongs to. A personal identity override beats all segments. If no segment matches, the environment’s default applies. So the position of your percentage segment in the list controls how much of the audience it can actually influence.

flowchart TD
    A["Flag requested for an identity"] --> B{"Personal identity override?"}
    B -- "Yes" --> R1["Use the personal override"]
    B -- "No" --> C{"In Internal staff segment?"}
    C -- "Yes" --> R2["Use the staff override"]
    C -- "No" --> D{"In Excluded accounts segment?"}
    D -- "Yes" --> R3["Use the exclusion override, feature off"]
    D -- "No" --> E{"In Ten percent rollout segment?"}
    E -- "Yes" --> R4["Use the rollout override, feature on"]
    E -- "No" --> R5["Use the environment default"]

Follow the figure from the top. A person with a personal override gets that answer and nothing else is considered. Next comes the staff segment, whose members always receive the feature. Then an exclusion segment turns the feature off for accounts that must never see it. Only after those does the percentage rollout segment get its turn. Anyone who matches none of the above gets the environment default. Because staff and exclusions come first, they are never affected by the rollout’s threshold.

Ordering principles

A common mistake with order

A frequent error is placing the rollout segment above the staff segment. Staff who are not in the slice then fall through to the staff override, which is fine, but staff who are inside the slice get the rollout’s answer, which may differ from the staff’s. If the two overrides carry different values, staff experience becomes unpredictable. Put staff above the rollout, and make sure both overrides agree on the main value so nobody has to remember which rule applied.

Watching one request travel from the application to Flagsmith and back makes every earlier chapter click into place.

sequenceDiagram
    participant APP as Application
    participant SDK as Flagsmith SDK
    participant FS as Flagsmith
    participant ENG as Rule Engine
    APP->>SDK: Identify user with traits
    SDK->>FS: Request flags for this identity
    FS->>ENG: Evaluate segments for the identity
    ENG->>ENG: Check eligibility conditions
    ENG->>ENG: Compute bucket and compare with threshold
    ENG-->>FS: Return matching segments
    FS->>FS: Apply overrides in priority order
    FS-->>SDK: Return flag values for this user
    SDK-->>APP: Flags ready to use

The application tells the SDK who the user is and what is known about them. The SDK asks Flagsmith for the flags that apply. Flagsmith hands the identity to its rule engine, which checks the eligibility conditions of each relevant segment and, for any segment with a percentage split, computes the person’s bucket and compares it with the threshold. The engine reports matching segments, and Flagsmith applies the overrides in priority order before returning the final values. The whole journey normally takes a fraction of a second.

No identity, no split

The split needs an identifier to scramble. If the application asks for flags without identifying the user, there is nothing to hash, and segments are not evaluated at all. The application then receives the environment’s default values. This is the most common reason a first rollout appears to “do nothing” in testing: the request was anonymous.

Local evaluation on your own server

For busy backend services, Flagsmith’s server-side SDKs offer a local evaluation mode. The SDK downloads the whole rule set, including segments, thresholds and overrides, and runs the same rule engine in your own service. Decisions become extremely fast and keep working if the network to Flagsmith is briefly interrupted. The engine is designed to give the same answers as the hosted service, so the same person should land in the same place either way, but check your SDK documentation for any differences. The trade-off is a short delay before changes arrive, because the SDK refreshes on a timer.

Saving a new threshold in the dashboard is not the same as every user seeing it. Knowing the delay prevents confused panic.

Layers of memory

Between the dashboard and a user’s screen sit several layers that remember recent answers to avoid asking the same question again and again. Your application may keep flags in its own memory for a time. Client SDKs on phones and browsers may store results on the device. Server SDKs in local evaluation mode download rules and refresh them on a timer, often around once a minute by default. Any proxy or delivery network in front of Flagsmith may hold answers briefly too. The sum of these delays is the time from saving a change to the last user seeing it.

A short mixed period is normal

For a short time after you raise the threshold, some servers and devices see the old state and some the new. A person who is newly inside the slice might see the feature on one server and not on another when they reload. The mix ends once every cache has refreshed. Good applications are designed to tolerate this, behaving safely whichever answer arrives. If your feature cannot work correctly when two servers briefly disagree, the problem is the feature’s design, not the flag.

Planning around the delay

A split looks like a single number, but it is a rule, and rules deserve tests. The tests are cheap and catch the quiet mistakes.

Two kinds of test

Two different questions need two different kinds of test. The first is about individual people: does a specific person get the right answer? The second is about the crowd: does the share included come out close to the threshold? A split can pass one and fail the other. The first is checked with named test identities. The second is checked with many generated identities and a count.

Named test identities for the individual

Create a set of test identities, each designed to prove one thing, and keep them permanently in a non-production environment. For each, you know in advance what the flag should return. Because assignment is repeatable, the answer for a given identifier at a given threshold never changes, so you can pick, once, identities known to sit low on the line and others known to sit high, and then use them as fixed reference points.

A crowd test for the distribution

To check the share, generate a few thousand synthetic identifiers in a non-production environment, ask Flagsmith for the flag for each, and count how many receive the feature. The count should land near the expected number, within the wobble Chapter 9 described. A count wildly different from expectation signals a problem with the rules, the identifiers or the environment. This check is worth repeating after any change to the segment’s structure, though not after a simple change of threshold.

Sooner or later someone will ask why a particular person sees or does not see a feature. A calm, ordered method answers it quickly.

flowchart TD
    A["Report: wrong flag value for a user"] --> B{"Was the user identified?"}
    B -- "No" --> B1["Identify the user and retry"]
    B -- "Yes" --> C{"Right environment key?"}
    C -- "No" --> C1["Use the key for the intended environment"]
    C -- "Yes" --> D{"Eligibility conditions all true?"}
    D -- "No" --> D1["Correct the traits or the rules"]
    D -- "Yes" --> E{"Higher priority override winning?"}
    E -- "Yes" --> E1["Reorder overrides or adjust the winner"]
    E -- "No" --> F{"Identifier same as usual?"}
    F -- "No" --> F1["Fix the identifier so the bucket is stable"]
    F -- "Yes" --> G{"Cache or refresh delay?"}
    G -- "Yes" --> G1["Wait one refresh period and retest"]
    G -- "No" --> H["The person's bucket is simply above the threshold"]

The figure moves from the most common causes to the least. First, confirm the user was identified, because anonymous requests skip segments. Second, confirm the key belongs to the environment you think you are looking at. Third, check each eligibility condition by hand with the person’s real traits. Fourth, look for a higher override that is winning. Fifth, check that the identifier is the one you expect. Sixth, allow for caching delay. If every check passes, the answer is often simply that the person’s bucket sits above the threshold, which is correct behaviour: they will be included when you widen.

Common symptoms and likely causes

When a split is used to compare versions, not just to protect users, the standard of fairness rises. Small flaws can ruin the whole test.

What is an experiment?

An experiment compares two or more groups that differ in one thing, to learn whether that thing changes an outcome. The groups must be alike in every other way, or any difference in results might come from the groups themselves. This is why random assignment is the gold standard: if the groups are chosen by chance, any systematic difference between outcomes can reasonably be credited to the thing you changed.

Why a split is a good fit

A percentage split inside a segment gives you repeatable, evenly spread groups, which is exactly what an experiment needs. The group inside the slice receives the change, and the eligible group outside the slice acts as the comparison, usually called the control. Because the scramble ignores traits, the two groups should resemble each other in plan, country and behaviour.

What is sample ratio mismatch?

Sample ratio mismatch is a warning sign. If you designed a fifty-fifty split and one group ends up with noticeably more people than the other, beyond what chance explains, something is wrong with how people are being assigned or counted. Perhaps one version crashes and loses events, perhaps a bot is hitting one side, or perhaps the identifier is not what you think it is. Whatever the cause, you cannot trust the results until you find it, because the groups are no longer comparable.

Rules of a fair experiment

A rollout usually climbs from Development to Staging to Production. The split makes the climb easier in some ways and trickier in others.

The segment is shared, the override is local

A segment is defined once for the whole project, so the split rule, with its threshold, is the same in every environment. The override that uses the segment, and the flag’s default, are stored per environment. This means you may set a ten percent split in the segment while the Staging override is switched on and the Production override is not yet created. The threshold is shared, the consequence is local.

flowchart LR
    SEG["Segment: Rollout, split defined once"] --> DEV["Development: override on"]
    SEG --> STG["Staging: override on"]
    SEG --> PRD["Production: override not yet created"]

The figure shows a single segment feeding three environments. The audience definition, including the threshold, is identical everywhere. Staging and Development already have the override switched on, and Production has not been touched, so nobody in Production receives the feature, whatever the segment says. Creating the override in Production is the real act of release.

The shared threshold is a hazard

Because the threshold lives in the shared segment, raising it in Development changes it in Production too. The Production override may still be off, which protects you today, but the moment someone switches it on, the audience is whatever the shared threshold says, perhaps far larger than intended. To avoid this, either keep the rollout’s segment dedicated to one feature and treat its edits as Production edits, or use separate segments per stage with clear names.

Buckets differ between environments

As Chapter 10 noted, do not assume the same person has the same bucket in Staging and Production. Test identities that sit “in the slice” in Staging may sit outside it in Production. Always verify Production behaviour with identities tested in Production, not by transferring conclusions from another environment.

A safe promotion routine

A single number in a dashboard can decide how many real customers see something new. That deserves the same care as a code change.

Why a threshold is a Production change

Raising a percentage looks small, a few keystrokes in a box. But the effect is real: thousands of customers may suddenly see different behaviour. A mistaken threshold, such as typing fifty where five was meant, can expose ten times more people than planned. Many teams treat threshold edits in Production the same way as deployments: reviewed, recorded and reversible.

Who should be able to edit what?

Approvals

On plans that support change requests, an edit to a Production flag can be proposed, reviewed by a colleague and applied only after approval. That second pair of eyes catches typing slips and misunderstandings. Remember that a threshold lives in a shared segment, and your version may treat segment edits differently from flag edits. Confirm whether segment changes pass through approval on your plan, and add a team habit of review where they do not.

The audit trail

An audit log is a diary kept by the system, recording who changed what and when. It answers the questions that always follow a surprise: who changed the threshold, from what to what, and at what time. Pair it with a short comment on each change, explaining the reason. A diary that says “threshold 5 to 25” is useful. One that adds “after 24 healthy hours at 5 percent” is far more useful to the next reader.

A split decides who gets something first. Random does not automatically mean harmless, and the people on the wrong side of a draw are still people.

Send the least data that works

The split itself needs only an identifier. Segment eligibility needs a few traits. Sending a whole customer profile to a flag service “just in case” adds risk without adding capability. Data minimisation, meaning collecting and sending only what is needed for a stated purpose, reduces privacy risk, speeds up evaluation and makes review simpler. Prefer an opaque account number over an email address, and a yes-or-no trait over a detailed history.

Fairness of who goes first

If a feature improves people’s experience, such as a faster checkout, then the slice receives a benefit that others wait for. If a feature carries risk, such as an untested payment path, the slice carries that risk on behalf of everyone else. A random split spreads both benefits and risks evenly, which is a fair default. It is worth pausing when the feature affects prices, access, eligibility or opportunity, because then who goes first has real consequences and may raise legal or ethical questions.

Client-side visibility

When an application running on a user’s device asks for flags, it can see the answers it receives. A determined person inspecting the app may see which flags are on for them and which values they carry. Do not place secrets or private business information in values delivered to browsers and phones, and do not treat the existence of a split as a way to hide a feature that must remain confidential.

An anti-pattern is a habit that looks reasonable and causes trouble later. These five are the most common with percentage splits.

A percentage split inside a segment is a strong, simple tool with real limits. Seeing both sides helps you use it where it pays.

Choosing the right tool for the job

Experienced teams repeat a short list of habits. Newer teams repeat a short list of mistakes. Here are both.

Habits worth building

Mistakes to catch early

To finish, here is one imaginary feature taken from idea to full release, using everything in this guide.

BrightCart, the imaginary online shop, has built a one-tap “express reorder” button. The team wants staff to try it first, then ten percent of loyal customers on recent mobile apps, widening to everyone loyal. Test accounts must never count. About forty thousand customers are eligible. The application sends a permanent account ID, plus the traits orders, platform, app_version, is_staff and is_test.

What would have gone wrong without the routine?

Suppose the team had used a session ID. Customers would have seen the button appear and vanish between visits, and support would have been unable to reproduce complaints. Suppose they had used “any” for the rule type. Every loyal customer would have received the button immediately, whatever the split said. Suppose they had rebuilt the segment mid-ramp to “tidy up”. Early users would have lost the feature, and a new random group would have gained it. Each of these is an ordinary mistake, and each is caught by the habits in this guide.

Short, direct answers to the questions people ask most often about percentage splits inside segments.

A percentage split inside a segment turns “show it to some of the right people” into a precise, repeatable and reversible instruction. A few ideas carry nearly all of the value.