Before learning any settings, let us understand the problems that usage tracking solves.

Flags are easy to create and easy to forget

Creating a flag takes a minute. Removing it needs someone to be sure that no code still reads it. That asymmetry means flags pile up. Old flags are not harmless. Each one is a small piece of hidden behaviour that a future engineer must understand, test and work around.

Questions that usage data answers

• Is this flag still used? If nobody asks for it, it may be safe to remove.
• Which flags are busiest? Heavily used flags deserve extra care and testing.
• Did the rollout reach people? If a new feature is on, are users actually evaluating it?
• Which value are people getting? A flag that is on, yet always returns the same value, may be finished business.
• Which apps use which flags? Useful when planning changes that affect specific products.
• How much traffic does Flagsmith serve? Important for capacity and cost.

Four groups who benefit

A few Flagsmith words appear in every conversation about usage. Let us make sure they are clear.

Organisation, project, environment

An organisation is your company account. It holds projects, usually one per product. Each project holds environments, such as Development, Staging and Production. Each environment has its own key that apps use to ask for flags.

Feature, identity and trait

A feature is a named switch. An identity is a record that stands for one user. A trait is one fact about that user. Apps can ask for flags with or without an identity.

SDK and client

An SDK is a ready-made code library that lets your app talk to Flagsmith. The app is the client. SDKs exist for browsers, mobile apps and server languages.

Evaluation

An evaluation is the moment when your app asks “what is the value of this flag for this user?” and gets an answer. Evaluations are the raw events of usage tracking.

Usage tracking is the habit of counting how flags are used, and reading the counts.

The simple definition

Flag usage tracking means collecting numbers about when and how often your apps evaluate each flag, and presenting those numbers in a way that people can read. The numbers may be shown per flag, per environment and over time.

Three levels of tracking

What usage tracking is not

• It is not the audit log. The audit log records changes to flags. Usage tracking records reads of flags.
• It is not a full product analytics system. It tells you that a flag was asked for, not whether the user clicked a button afterwards.
• It is not a guarantee of every single read. Some counts are sampled, batched or approximate, depending on setup.

Flagsmith has two kinds of events, and mixing them up causes confusion.

Writes: changes to flags

When a person or a program changes a flag, that is a write. It is rare, deliberate and recorded in the audit log. A busy team may make a few dozen writes per day.

Reads: apps asking for flags

When an app asks for flags, that is a read. It is frequent and automatic. A popular app may make millions of reads per day. Reads are what usage tracking counts.

Why the difference matters

If you want to know who turned a flag on, look at writes. If you want to know whether anyone still asks for a flag, look at reads. A flag can have no recent writes and many reads, which means stable and important. Or no reads at all, which means forgotten.

Flagsmith offers several windows into usage. Which ones you have depends on your version and plan, so treat this as a map to look for, not a promise.

Window 1: Organisation or project usage

A view of the number of requests the whole organisation or project made to Flagsmith, often by day. It helps with capacity and with plan limits. It does not say which flag was asked for.

Window 2: Flag analytics

For each flag, a count of evaluations over a period, often shown as a chart. This is the view that tells you whether a flag is alive. It usually needs the apps to report their evaluations, a setting we will explain later.

Window 3: Integrations

Flagsmith can send information about which flags a user received to other tools, such as analytics platforms. This links flag values to user behaviour in the tools your analysts already use.

Window 4: Your own application logs and metrics

Your apps can record flag evaluations in their own logs, traces or metrics. This gives you the most freedom and the most detail, and works with any version.

Window 5: Code references

Some setups can connect Flagsmith to your code repository to show where a flag is mentioned in source code. This does not count runtime use. It shows whether the code still contains the flag name. Together with runtime counts, it is powerful.

Let us follow a single question from your app to a number on a chart.

The journey in plain steps

sequenceDiagram
    participant U as User action
    participant A as Your app
    participant S as SDK
    participant F as Flagsmith
    participant D as Dashboard
    U->>A: Opens a page
    A->>S: Is checkout_v2 on?
    S-->>A: Yes, value X
    S->>S: Count one evaluation
    S->>F: Send batched counts every interval
    F->>F: Add to totals for flag and environment
    D->>F: Ask for usage chart
    F-->>D: Counts by time

Why batching matters

If every evaluation sent its own message, the tracking would cost more than the flag itself. Batching keeps the cost low. The price is a small delay: a number on the chart may be a little behind real time.

Many SDKs do not report usage unless you ask them to. This is a deliberate choice, since reporting costs some traffic.

The usual idea

Most Flagsmith SDKs have a setting that enables analytics or evaluation reporting. When it is off, the SDK still works, but the usage views for flags may stay empty. When it is on, the SDK records evaluations and sends counts back in the background.

What to check

• The setting exists in your SDK and version, and is switched on in the environments you care about.
• The reporting interval is acceptable. Longer intervals mean fewer messages and slower charts.
• Short-lived programs still get a chance to send their counts before they stop.
• Privacy rules are respected. Counts are about flags, not about people, but check your own policy.

Which apps should report

You do not need every app to report in every environment. A sensible start is Production for all customer-facing apps, plus Staging for the apps you test most. Development usually adds noise.

Browser and mobile apps

Apps running on user devices may close at any moment. They may lose counts that have not yet been sent. Treat their numbers as good estimates, not exact totals.

Server apps

Backend programs run for a long time and send counts regularly, so their numbers are usually more complete. If they use local evaluation, the counts come from the server’s own evaluations.

A chart is only useful if you know how to read it. Here is a simple way to look at the shape of a flag’s life.

What the chart shows

Time runs along the bottom, from older to newer. The height shows how many evaluations happened in each period. Most often the period is a day, and sometimes an hour.

Common shapes

Beware of the sudden drop

A sudden drop to zero has two very different causes. Perhaps a developer removed the flag from code, which is good. Or perhaps tracking stopped because of a deployment problem, which is bad. Always check the app’s own health before celebrating a clean-up.

Compare with the calendar

Many patterns are explained by events: a holiday sale, a release, a weekend, an outage. Keep a simple event calendar beside your charts, and read both together.

Numbers feel exact, but they have limits. Knowing them prevents bad decisions.

What counts can tell you

• Whether a flag is being evaluated at all.
• Roughly how busy a flag is, compared with others.
• When use started, grew or stopped.
• Which environments use a flag.

What counts cannot tell you

• Exact totals. Batching, short-lived apps and sampling make small differences.
• Individual users. Totals do not name people, unless another tool is used.
• Business value. A flag evaluated a million times may be useless, and a flag used rarely may protect critical money.
• Code use that never runs. A rare path, such as a yearly report, may evaluate the flag once a year.
• Safety of removal. Zero recent counts suggest a flag is unused, but do not prove it.

The yearly flag trap

Imagine a flag used by a tax report that runs once a year. Over a month, its chart is flat zero. A clean-up based on one month of data would remove it, and next year the report would break. The lesson: choose a look-back period that matches the slowest rhythm of your business, and combine with code references.

Every environment has its own story. A flag busy in Production may be silent in Staging, and that is perfectly normal.

Counts follow the environment key

Apps ask Flagsmith for flags using an environment key. The usage numbers are attached to that environment. So if your Production app uses the Production key, its counts appear under Production. If somebody accidentally puts the Staging key into a Production app, the counts, and the flag values, will land in the wrong place.

What healthy patterns look like

Comparing environments

Put the charts of Staging and Production side by side. If a flag is evaluated in Production but never in Staging, it means the code path runs only for real customers, or that your tests do not touch that path. Both are worth knowing.

Key hygiene

Because counts depend on the key, keep keys in configuration, not in code. Name configuration clearly. Check that each deployed app uses the key for its own environment, and regenerate keys if you suspect a mix-up.

Flagsmith SDKs can work in two broad ways. The way they work changes what gets counted and where.

Remote evaluation

The SDK asks the Flagsmith service for the answer each time or each few moments. Flagsmith works out the value on its own servers. Because Flagsmith is involved in every question, it can see the requests, and it can count them at the service level.

Local evaluation

A trusted server program downloads the rules for the whole environment once, and then answers questions itself, without asking Flagsmith. This is fast and saves traffic. But because Flagsmith is no longer asked each time, it cannot count the questions by itself. The SDK must report counts back, if analytics is switched on and supported.

Caching also hides reads

Many apps keep flags for a short time instead of asking repeatedly. A flag might be read by code a thousand times, but fetched from Flagsmith only once per minute. The service-level number counts the fetches. The flag-level number, if available, counts the evaluations. They answer different questions.

Which number do I need?

• For capacity and plan limits, the number of requests to Flagsmith is the right one.
• For whether a flag is alive, the flag-level evaluation count is the right one.

flowchart TD
    Q["Code asks: is flag X on?"] --> C{"Cached or local rules?"}
    C -- "Cached answer" --> A1["Answered inside the app"]
    C -- "Local evaluation" --> A2["Answered by SDK with downloaded rules"]
    C -- "Needs fresh answer" --> A3["Request goes to Flagsmith"]
    A1 --> R["SDK may report flag count"]
    A2 --> R
    A3 --> R
    A3 --> S["Service counts the request"]

Sometimes the question is not “how many” but “what did this particular user see?” Flagsmith has ways to help with that, and so do your own tools.

Looking up an identity

If your app sends identities and traits to Flagsmith, the dashboard can often show an identity’s traits and the flag values that identity would receive now, including the effect of segments and overrides. This is a current view, not a history. It tells you what the user would get if they asked right now.

The history gap

What the user saw yesterday at 10:05 may differ from what they would see now, because traits or flags may have changed. To answer historical questions you need your own record, such as application logs that note the flag value served with each request.

Support use

For customer questions, the identity view is valuable. A support agent can check: which segments is this customer in, and what value would they receive? Combined with the audit log, they can explain changes of behaviour.

Privacy care

• Send the least personal data that works. An opaque customer ID is better than an email address.
• Do not store sensitive facts as traits unless truly needed.
• Limit who can view identities, since traits may reveal personal details.
• Follow your own rules for deletion requests, and remember that logs also hold identifiers.

The most valuable question in product work is: “Did the people who saw the new feature behave differently?” To answer it, flag values must meet user behaviour data.

The idea of integrations

Flagsmith offers integrations with several analytics and data platforms. In general, an integration sends information, such as the flag values and traits of an identity, to another tool. The other tool can then split its reports by flag value. For example, it can compare the completion rate of checkout for users with the new page and users without it.

Why this matters

Without this link, a flag is only a switch. With it, a flag becomes an experiment: you can measure what the change did. Product managers use this to decide whether to widen a rollout, or stop.

What to send

• The flag name and the value served.
• A stable user identifier that the other tool also knows.
• The time of evaluation, so that behaviour before and after can be compared.

Cautions

• Identifier match. If the two tools use different IDs, they cannot be joined.
• Exposure versus evaluation. A user may evaluate a flag without actually seeing the feature, for example if the page never rendered. Prefer to record exposure at the moment the feature is truly shown.
• Volume and cost. Sending every evaluation may produce large data bills. Send on change or on first exposure.
• Privacy. Treat data sent to other tools by the same rules as the original.

flowchart LR
    APP["Your app"] --> FS["Flagsmith: serves flag value"]
    APP --> EXP["Exposure event: user saw variant B"]
    FS --> INT["Integration: flag values and traits"]
    EXP --> AN["Analytics platform"]
    INT --> AN
    AN --> REP["Report: behaviour by flag value"]

When a flag serves different values to different groups, usage tracking helps you check that the plan matches reality.

Percentage rollouts

Suppose a flag is on for 10 percent of users. Usage numbers can help you confirm that roughly one in ten users receives the new value. If you see 40 percent or 1 percent, something is wrong: perhaps a segment override is overriding the split, or traits are missing.

Multivariate flags

With several values, such as A, B and C, you can check that each value is served about as often as planned. Large gaps may show that one variant is blocked by a segment or an identity override.

Segment reach

A segment-based launch, such as “all premium users in India”, should reach a number of evaluations that matches the size of that segment. If the number is far below, check the segment rules and the traits your app sends.

A simple reach check

The most common use of usage tracking is clean-up. Done carefully, it keeps your system understandable. Done carelessly, it breaks things.

What is a stale flag

A stale flag is a flag whose job is finished. Typical cases: a fully launched feature where the flag is always on, a cancelled experiment where it is always off, or a flag whose code has been removed. Keeping it adds confusion and risk.

A safe five-step process

flowchart TD
    A["Candidate: low or zero usage"] --> B{"Look-back long enough?"}
    B -- "No" --> A2["Wait or extend the period"]
    B -- "Yes" --> C{"Flag name still in code?"}
    C -- "Yes" --> D["Plan code removal first"]
    C -- "No" --> E{"Owner agrees it is finished?"}
    D --> E
    E -- "No" --> F["Keep, and note the reason"]
    E -- "Yes" --> G["Remove from code and deploy"]
    G --> H["Watch for errors and usage"]
    H --> I["Delete flag in Flagsmith and record it"]

Why the order matters

If you delete the flag first, code that still asks for it may receive nothing, or a default, and behave oddly. If you remove the code first, the flag becomes unused and can be deleted calmly.

Keep the permanent value

Before removing a flag that is always on, make sure the code contains the “on” behaviour permanently. Before removing one that is always off, remove the “on” code entirely. Mixing these up leaves the product behaving differently than before.

Runtime counts show what happens. Code references show what could happen. Together they give a much better picture than either alone.

Two signals, four situations

Code references in practice

Flagsmith can connect to some code hosting platforms to scan repositories and show where each flag name appears. Even without that, a simple search of your repositories for the flag name gives a similar answer. Make the search part of the clean-up routine.

Beware of built-up names

If code builds flag names from pieces, such as joining a prefix and a country code, a plain search will not find them. Avoid building flag names dynamically, or keep a list of such patterns.

Many repositories

A flag may be used by several services. Search all of them, including mobile apps, scripts and infrastructure configuration. A flag unused in one repository may be busy in another.

quadrantChart
    title Runtime use versus code presence
    x-axis "Name absent from code" --> "Name present in code"
    y-axis "No runtime use" --> "High runtime use"
    quadrant-1 "Live and healthy"
    quadrant-2 "Old versions or other repos"
    quadrant-3 "Likely dead"
    quadrant-4 "Dormant or untracked"

The built-in views are a good start. Many mature teams also record flag evaluations in their own observability tools, which gives deeper and longer-lasting answers.

What to record

• Metrics. A counter named after the flag and the value, such as evaluations of checkout_v2 by value, per service.
• Logs. A short note in each request log: which flags were evaluated and which values were served.
• Traces. A tag on a trace span that records the flag values affecting a request.

Why this helps

• You can find out exactly which flag values were active during an incident.
• You can correlate flag values with latency or error rate.
• You keep the data as long as your own policy allows, even if the dashboard keeps less.

The cardinality trap

Cardinality means the number of different label values in a metric. A counter labelled by flag name and value is fine. A counter labelled by user ID would create millions of series and overload your monitoring system. Never put user IDs, or other unbounded values, into metric labels. Put them in logs instead, where unbounded values are expected.

Sampling

For very busy flags, record every evaluation in counters, but write only a sample of them to logs. A one percent sample is often enough to understand patterns, and keeps costs sane.

Let us look behind the screen. Understanding the parts explains why numbers behave the way they do.

The big picture

flowchart TD
    APP["Apps with SDKs"] --> EDGE["Flag serving layer"]
    APP --> COLLECT["Usage collector"]
    EDGE --> CORE["Flagsmith core: rules and flag states"]
    COLLECT --> QUEUE["Buffer or queue"]
    QUEUE --> AGG["Aggregator: counts per flag, environment, time"]
    AGG --> STORE[("Usage store")]
    STORE --> VIEW["Dashboard charts and APIs"]
    CORE --> INTEG["Integrations to other tools"]
    APP --> OWN["Your own logs and metrics"]
    OWN --> OBS["Your observability platform"]

The components in plain words

Why the paths are separate

Serving flags must be fast and reliable, because customers wait for it. Counting usage is less urgent. So counts travel on a separate path, with buffers, and can be delayed or even dropped under heavy load without hurting customers. This is a deliberate design: the product comes first, the statistics come second.

Self-hosted notes

If you host Flagsmith yourself, you also decide how usage data is stored, how long it stays and how much capacity the collector has. Plan storage growth. Heavy traffic multiplied by many flags can create a lot of rows, so ask for aggregation, retention and clean-up settings in the documentation.

A usage number is born, grows, is summarised and finally retired. Following this life helps you know how far back you can look.

Birth

The count is born inside an SDK, as a tiny tally: “flag X, once”. It exists only in the memory of your app.

Early life: batching

For a short time, the tally sits with others, waiting for the next reporting interval. If the app crashes during this time, the tally is lost.

Arrival

The batch reaches Flagsmith. The collector accepts it, places it in a buffer and acknowledges it.

Summarising

The aggregator adds the counts to totals, usually by flag, environment and time bucket such as an hour or a day. Individual tallies are no longer visible, only the totals.

Storage and reading

The totals sit in the usage store and are shown in charts. Recent data may be detailed. Older data may be rolled up into coarser buckets to save space.

Retirement

After the retention period, the totals are removed. The length of this period depends on plan and, for self-hosted use, on your settings. If you need a longer history, export totals into your own system.

flowchart LR
    B["Tally in SDK memory"] --> BA["Batched until next interval"]
    BA --> AR["Arrives at collector"]
    AR --> SU["Summed into time buckets"]
    SU --> ST["Stored and charted"]
    ST --> RO["Older data rolled up"]
    RO --> RE["Removed after retention"]
    ST --> EX["Exported to your own store"]
    EX --> LT["Kept per your policy"]

Counting is never free. Good tracking keeps the cost so small that nobody notices it.

Costs inside the app

Recording a tally takes a few microseconds, a tiny amount of memory and, now and then, a network message. For almost all apps this is invisible. But in code that evaluates flags millions of times per second, even tiny costs add up, so measure before assuming.

Costs on the network

Each batch is one small message. If an app sends one every ten seconds, that is six per minute, which is nothing. If thousands of app copies do the same, the collector sees thousands of messages. Longer intervals reduce the load, at the price of slower charts.

Costs at the service

The service must accept, buffer, add and store the counts. Flagsmith handles this for the hosted service, so the cost is part of your plan. In self-hosted setups, it is your hardware and your database.

Ways to keep costs low

• Use batching with a sensible interval, such as every few seconds to a minute.
• Track only the environments and apps that need it.
• Avoid evaluating the same flag again and again inside tight loops. Read it once per request, and reuse the value.
• Sample logs for the busiest flags, while keeping counters complete.
• Keep labels in your own metrics few and bounded.

Fail quietly

Tracking should never break the product. If the usage message fails, the SDK should drop or retry later, without errors reaching the customer. Check that your SDK and your own wrappers do this.

Flagsmith plans often count requests. Understanding where requests come from helps you keep traffic sensible and bills predictable.

Where requests come from

• Every page load of a browser app that asks for flags.
• Every mobile app start and each refresh.
• Each server request that asks for flags on the spot.
• Polling by local evaluation servers that refresh the rules.
• Scripts and tests that call the API.

Ways to reduce unnecessary requests

• Cache flags in the app for a short time, instead of asking each moment.
• Fetch once per page or session and reuse.
• Use local evaluation in servers, which answers questions without a network request.
• Increase polling intervals where freshness is not critical.
• Stop test scripts from hammering Production.

Balance freshness against volume

A long cache means fewer requests, but slower arrival of changes. A short cache gives quick changes but more traffic. Pick the interval based on how quickly a change must reach customers. A kill switch may need seconds. A marketing banner can wait several minutes.

Watching the trend

Plot request volume per week. A steady climb in line with users is normal. A sudden jump with no traffic growth often means a bug: a loop calling Flagsmith repeatedly, a cache switched off, or a new app version that asks too often.

Usage data looks harmless, yet it can reveal secrets. Treat it with care.

What usage data can reveal

• Product plans. Flag names such as “new_pricing_2027” tell outsiders about future work.
• Customer behaviour. Identity-level data can show who has which feature.
• System structure. Which services exist and how busy they are.

Protecting the keys

Environment keys that apps use to read flags are not as sensitive as admin keys, but still matter. A leaked server-side key may let someone read all flag rules, including segments. Keep server keys secret, use client-side keys only where needed, and rotate if in doubt.

Keeping personal data out

• Use opaque identifiers instead of emails or names.
• Do not put sensitive details such as health or finance facts in traits.
• Do not put personal data in flag names or values.
• Keep user-level tables and logs behind access controls.

Who may see usage

Give read access to the people who need it: engineers, product managers and support. Limit identity-level data to fewer people than aggregate charts.

Exports and integrations

When usage or flag values flow into other tools, the same rules apply there. A strict policy in Flagsmith does not help if the analytics platform is open to everybody.

Rules about data often apply to usage records. This chapter is general information, not legal advice, so your own advisers should confirm what applies to you.

Aggregate versus personal

Aggregated counts such as “checkout_v2 was evaluated 40,000 times” usually contain no personal data. Identity-level records, such as “user 123 got variant B at 10:05”, often do. Privacy laws like GDPR care about the second kind.

Retention

Decide how long you keep each kind of data, and write it down.

Deletion requests

If a person asks you to delete their data, remember that identifiers may live in several places: Flagsmith identities, your application logs, analytics tools and backups. Keep a short list of where identifiers can appear, and a process to delete them.

Purpose limits

Data collected to run flags should not quietly be reused for unrelated purposes. If you want to use evaluation data for marketing, check your policy and the law.

Evidence for audits

Auditors may ask how you ensure that personal data in tracking is minimal, protected and deleted on request. Have a one-page description and a recent example ready.

Usage data becomes a safety net when it triggers alerts on strange behaviour.

Useful alerts

Metrics for the tracking system itself

• Share of apps reporting. If only half of your servers report, counts understate use.
• Delay between evaluation and chart. Should stay within the expected interval.
• Dropped batches. If many are dropped, investigate networks and limits.

Avoid alert fatigue

Start with three alerts: core flag drops, request spike and plan limit. Add more only when real incidents show the need. Too many alerts teach people to ignore them.

Weekly walk-round

Once a week, someone looks at the top ten busiest flags, the ten flags with zero use and the request trend. Ten minutes is enough, and it often spots oddities that no alert thought of.

Tracking itself can go wrong. Knowing the usual failures lets you respond calmly.

Scenario 1: The chart is empty

The flag is clearly used, yet the chart is flat. Possible causes: analytics is switched off in the SDK, the version does not support it, local evaluation without reporting, the wrong environment is open, or the apps have not yet sent their first batch. Check the setting first, then the environment, then wait for the next interval.

Scenario 2: Counts are too low

Browser tabs close before batches are sent, mobile users go offline, short jobs stop early. Accept small undercounting for such apps, and use server-side numbers for decisions where you can.

Scenario 3: Counts suddenly double

Two copies of the SDK may report the same evaluations, or a new app version evaluates the flag twice per request. Compare with request volume and with deployment times.

Scenario 4: A flag was deleted, but still used

An old app version, a forgotten script or another repository still asks for it. Errors or defaults appear. Recreate the flag with the old permanent value to stop the damage, then find and update the callers, and delete again later.

Scenario 5: The request bill jumps

A loop or a disabled cache floods the service. Look at request volume by hour, find the first hour of the jump, match it to a deployment, and fix or roll back.

Scenario 6: Integration data does not match

Counts in Flagsmith and in the analytics tool disagree. Different definitions, delays and identifier mismatches are common reasons. Compare the definitions before blaming either side.

Scenario 7: Self-hosted collector is overloaded

Batches queue up and charts lag. Add capacity, lengthen the SDK interval or sample, and verify retention clean-up is running.

Every tool has benefits and costs. An honest comparison helps you decide how much tracking you need.

Trade-offs worth thinking about

Detail versus cost

Recording every evaluation for every user is rich but expensive. Counting per flag and per day is cheap but coarse. Most teams use cheap counts widely and rich detail only where needed.

Freshness versus load

Short reporting and caching intervals give quick data and quick flag changes, but increase traffic. Long ones save traffic, but make decisions slower.

Convenience versus privacy

Identity-level views make support easier. They also create sensitive data. Use opaque identifiers, limit access and set short retention.

Built-in views versus own metrics

Built-in views are quick to adopt. Your own metrics integrate with your whole observability stack and keep data longer, but need work. Many teams start with the first and add the second as they grow.

Teams that track usage well follow similar patterns. Teams that struggle repeat similar mistakes.

Good patterns

Anti-patterns

Here is a practical list of habits to adopt and traps to avoid.

Best practices

1. Turn on tracking in Production first for all customer-facing apps.
2. Verify after enabling. Look at the chart one day later.
3. Give every flag an owner and a purpose in its description.
4. Review quarterly for unused and constant-value flags.
5. Use a long look-back that matches your slowest business cycle.
6. Use two signals: runtime usage and code search.
7. Remove from code first, then from Flagsmith.
8. Keep personal data out of traits, names and metric labels.
9. Export aggregate counts if you need a longer history.
10. Alert on drops, spikes and limits with a small, trusted set of rules.

Common mistakes

Troubleshooting quick guide

• No data at all. Check SDK analytics setting, environment key and plan features.
• Data only from some apps. Check which apps have the setting, and which are short-lived.
• Numbers lag. Check the reporting interval and any queue delay.
• Numbers look doubled. Check for duplicate SDK instances.
• Cannot see identity data. Your role may lack permission, or the app may not send identities.

Let us follow an imaginary company, “Acme Cart”, through a quarterly flag clean-up to see all the ideas working together.

The situation

Acme Cart has 180 flags in its main project. Engineers complain that nobody knows which are safe to remove. The platform team, led by Meera, decides to run a clean-up using usage tracking. First, she checks that analytics is switched on in every Production app, and finds that the mobile app and one background job had it off. They turn it on and wait three weeks to gather data.

Stage 1: Build the candidate list

Meera exports flag-level counts for the last 90 days. She finds:

Stage 2: Check the look-back

Meera remembers that the finance team runs a year-end report. She extends the look-back to a year for the zero-use group. Five flags that looked dead turn out to be used once a year. She moves them to the “keep” list and adds notes to their descriptions.

Stage 3: Search the code

For the remaining 35 zero-use flags, engineers search all repositories. For 28 of them, the name no longer appears anywhere. For 7, the name is still in code, in paths that never run. Those seven need code removal first.

Stage 4: Ask owners

Meera posts the lists in team channels. Owners confirm 24 flags as finished. Four are still needed for a planned feature, and are kept with an expiry date.

Stage 5: Remove in the right order

For the 35 busy-and-constant flags, teams remove the flag checks from code and keep the permanent behaviour. After deployment, usage for those flags falls to zero, as expected. A week later, with no errors, they delete the flags in Flagsmith. For each deletion, they write a short note in the clean-up ticket.

flowchart TD
    S["180 flags"] --> A["Enable tracking in all Production apps and wait"]
    A --> B["Export counts and group flags"]
    B --> C["Extend look-back to a year for zero-use flags"]
    C --> D["Search code in all repositories"]
    D --> E["Owners confirm finished flags"]
    E --> F["Remove from code and deploy"]
    F --> G["Watch usage and errors for a week"]
    G --> H["Delete flags and record in ticket"]
    H --> R["About 70 flags removed, 110 remain"]

Stage 6: A near miss

One of the “zero-use” flags had a few evaluations in the previous year, tied to an old app version on a few tablets in warehouses. The team checks usage by app version, sees the old version still in use, and delays removing the flag until the tablets are upgraded. The extra check saved a warehouse outage.

Stage 7: Make it a habit

Meera adds an expiry date and an owner to the template for new flags, and schedules the same review every quarter. The count of flags falls from 180 to about 110, and stays under control.

Numbers do not clean anything by themselves. People do. A few written rules turn usage data into a steady habit of tidiness.

Every flag has a type

Not all flags live the same length of time. Agree on a few types, and tag each flag with its type when it is created.

Why types matter for usage

A kill switch should show almost no changes and sometimes no reads of the “on” path, yet is vital. A release flag with no changes for six months is a warning sign. Without types, every quiet flag looks suspicious, or every one looks innocent.

A one-page flag lifecycle policy

• Every flag has an owner, a type and a purpose in its description.
• Release and experiment flags have an expected end date.
• Usage and constant-value reports are reviewed every quarter.
• Removal follows the safe process: two signals, owner agreement, code first.
• Exceptions, such as kill switches, are marked and tested.

Owners for each piece

When a company grows, usage questions grow too. Here are ideas for larger setups.

Many projects

Each project has its own flags and its own usage. For a company-wide view of flag hygiene, collect counts from all projects into one place. Add a field for the project name and the environment, and then one dashboard can answer questions like “how many unused flags exist across the company?”

Shared flags

Some flags are read by several services owned by different teams. Usage by service helps you see who depends on a flag. Before changing or removing it, tell every team that appears in the data.

App version awareness

Mobile and desktop apps keep running old versions for months. If usage can be split by app version, you can see when the last old version stops asking for a flag. If your tracking cannot split it, add the app version to your own logs or metrics.

Cost allocation

Large companies may split Flagsmith costs between teams. Request counts per project or per key give a fair basis. Use separate keys per service if you want finer detail.

Automation of hygiene

A scheduled job can fetch usage counts, list flags that match rules, such as “no reads in 180 days and no code references”, and open tickets for owners. Automation does the boring work, and people make the decisions.

Forecasting

With a year of request volume, you can forecast next year’s plan needs, and spot unusual growth early. Combine with product growth plans for better estimates.

flowchart LR
    P1["Project A usage"] --> C["Central usage store"]
    P2["Project B usage"] --> C
    P3["Project C usage"] --> C
    CODE["Code search results"] --> J["Hygiene job"]
    C --> J
    J --> T["Tickets for flag owners"]
    C --> D["Company dashboard"]
    C --> F["Forecast and cost reports"]

A fuel gauge you have never checked may lie. Test your tracking before you rely on it.

The first-time test

1. In a test environment, create a flag called “tracking_test”.
2. Run a small test app that reads it exactly 100 times.
3. Wait for the reporting interval, plus a little extra.
4. Look at the usage view. Does it show about 100?
5. Repeat with local evaluation, if you use it.

The short-lived job test

Run a job that reads a flag and exits at once. See whether the count appears. If not, add a step that flushes counts before exit, or accept that such jobs are not counted.

The wrong-key test

Temporarily start a test app with the Staging key and see that its usage appears under Staging, not Production. This confirms that the environment mapping works as you believe.

The alert test

Switch off a test flag’s reads and see that your “drops to zero” alert fires. An alert that has never fired may not work.

The restore test

If you export usage totals, run a test restore or a test query on the exported data once a year, to prove it is complete and readable.

Nobody builds perfect tracking in a day. Here is a path with four levels, so you can see where you are and what comes next.

How to move up

The same numbers serve different people. Knowing their questions helps you prepare the right views.

The engineer

“Can I delete this flag?” The engineer needs evaluation counts over a long period, code references and the list of services that read it.

The product manager

“Did the new feature reach the people we wanted?” The manager needs reach compared with the plan, and a link to behaviour data, such as completion rates by flag value.

The on-call engineer

“Which flag values were active when the errors started?” On-call staff need flag values in logs and traces, and the audit log for recent changes.

The support agent

“Why does this customer see something different?” The agent needs the identity view, the customer’s segments and the recent changes.

The platform engineer

“How much traffic do we send to Flagsmith, and is it growing?” The platform engineer needs request volume by project and key, cache behaviour and plan limit tracking.

The security lead

“Which data about people flows through flags?” The lead needs a list of traits, integrations and exports, plus retention settings.

The finance partner

“What will next year’s bill look like?” The partner needs request trends and growth plans.

A single page that shows the health of your flags is worth more than a hundred scattered charts. Here is a plan for a simple one that any team can build.

Principles

• Few panels. Six panels that people really read beat thirty that nobody opens.
• Every panel leads to an action. If a number cannot lead to a decision, leave it out.
• Same words as the team. Use flag names and owner names that people know.

The six panels

Where the data comes from

• Flag lists, owners and types from Flagsmith through its API or exports.
• Usage counts from the Flagsmith usage views or your own exports.
• Code search results from your repository tools.
• Request volume from the service level usage view.

flowchart LR
    F["Flag list, owners, types"] --> J["Daily job"]
    U["Usage counts"] --> J
    C["Code search results"] --> J
    V["Request volume"] --> J
    J --> S[("Small reporting store")]
    S --> D["Flag health dashboard"]
    D --> W["Weekly walk-round"]
    D --> Q["Quarterly clean-up"]

Keep it honest

Show the date of the last successful update on the dashboard. A dashboard that silently stops updating is worse than none, because people trust it. Also show how many apps are reporting, so that readers know how complete the numbers are.

Reading charts is a skill, and skills grow with practice. Here are five imaginary situations, with a careful reading of each.

Chart 1: The cliff on a Tuesday

A flag called “new_search” shows 200,000 evaluations per day, then drops to zero on a Tuesday afternoon. A careless reader might say “the team finished the clean-up”. A careful reader checks the release calendar and the health of the search service. It turns out that the new version of the service shipped without the analytics setting, so reporting stopped. The flag is still busy. Fix: restore the setting.

Chart 2: The staircase growth

A flag called “checkout_v2” grows in steps: 5 percent of traffic, then 25, then 50, with a flat line between steps. This matches a careful rollout plan. The careful reader compares the steps with the change history and sees that each step followed a request approval and a check. Good practice, nothing to do.

Chart 3: The sawtooth

A flag called “report_export” is evaluated heavily on the first day of each month, and almost never otherwise. A reader using a 30-day look-back might call it unused in the middle of the month. A careful reader sees the monthly rhythm and keeps it.

Chart 4: The twin lines

Counts for “profile_banner” double overnight, with no change in user traffic. The careful reader checks the deployment calendar. A new version evaluates the flag twice per page, once in the header and once in the footer. Not a problem for users, but a small waste. Fix: evaluate once and pass the value.

Chart 5: The ghost

A flag called “legacy_pricing” was deleted last month in code and in Flagsmith, yet a small steady trickle of requests for its name keeps arriving. A careful reader looks at usage by app version and finds old tablets in a warehouse. The flag must return, with its old permanent value, until those tablets are updated.

The habit behind all five

• Ask what else changed around the same time.
• Look at the shape, not only the level.
• Check the tracking before trusting the number.
• Confirm with a second source, such as code or logs.

Tracking what is used, and tidying what is not, is an old discipline. The descriptions below are general patterns that are widely discussed, not details of any private system.

Dead code and feature cleanup in software

Large software companies have written publicly about the cost of “dead” code and stale configuration. Tools that measure which code paths actually run in production help engineers remove unused parts safely. Flag usage tracking is the same idea applied to switches.

Observability platforms

Modern monitoring tools show request rates, error rates and latency. Many teams add flag values to traces and logs, so that during an incident they can see which flag variants were active. This is now a common practice at companies that run feature flags at scale.

Experimentation at large internet companies

Companies such as large search engines, social networks and video services have described experimentation systems that record which variant each user was exposed to, and link it to behaviour. The key lesson is to record exposure carefully and to keep a stable identifier. Flag usage tracking and integrations support the same pattern at a smaller scale.

Retail and stock control

Shops track sales by product to decide what to restock and what to discontinue. Products that have not sold for a long time move to a clearance rack. Stale flags are the unsold stock of a software company.

Libraries and archives

Libraries keep borrowing counts to decide which books to keep on open shelves and which to move to storage. They also keep rare items that are seldom borrowed but valuable, just like kill switches and yearly-use flags.

Utilities and metering

Water and electricity companies read meters to bill customers and to find leaks. A sudden increase at night suggests a leak. A sudden jump in Flagsmith request volume suggests a loop or a missing cache.

Copy these into your team wiki. Short lists, used often, beat long manuals read once.

Setting up tracking

• SDK analytics is switched on in all Production apps.
• Each app uses the key for its own environment.
• Reporting interval is chosen and documented.
• The chart is checked one day after enabling.
• Short-lived jobs flush counts or are marked as uncounted.
• Personal data is kept out of traits and labels.

Before removing a flag

• Look-back covers the slowest business cycle.
• Usage is zero, or the value is constant.
• Usage by app version shows no old versions asking.
• The flag name is gone from all repositories, or code removal is planned.
• The owner confirms and names the permanent behaviour.
• Remove from code first, deploy, watch, then delete the flag.
• Write the reason in the ticket.

After launching a rollout

• Compare actual reach with the plan.
• Check that each variant is served as expected.
• Check that exposure events reach the analytics tool.
• Look at error and latency graphs by flag value.

Every quarter

• List zero-use and constant-value flags.
• Check request volume trends and plan limits.
• Review flags past their expiry date.
• Review who can see identity-level data.
• Test the alert for a core flag dropping to zero.

A short reference for the words used in this guide.

Quick answers to the questions that come up most often.

Let us pull the whole story together in a few lines you can remember.