Cost Attribution via Telemetry: Knowing Who Spends What

Cost Attribution via Telemetry: Knowing Who Spends What

How the traces, metrics and logs you already collect can turn one giant, confusing cloud bill into a fair and explainable answer to the question every business eventually asks: who is spending this money, and what are we getting for it?

Imagine the monthly cloud bill lands in your finance team’s inbox. It is a very large number with six digits, and underneath it sits a long list of lines with names like “compute,” “storage,” “data transfer” and “managed database.” The chief financial officer asks a simple question: which product is costing us the most, and which customers are we losing money on? Nobody in the room can answer. The bill knows what was bought. It has no idea why.

This guide is about closing that gap. Cost attribution via telemetry means using the evidence your running software already produces, its metrics, traces and logs, to work out which team, product, feature or customer caused each part of the bill. Done well, it replaces arguments and guesses with numbers that everyone can inspect.

You do not need to know anything about finance or monitoring to follow along. Every new word is explained the first time it appears, every big idea has an everyday comparison, and the worked examples use small numbers you can check with a calculator.

1The Vocabulary: Bills, Resources, Tags and Telemetry

Six everyday words carry the whole topic. Learn them once and the rest is easy.

Most modern software does not run on computers that a company owns. It runs on computers rented from a cloud provider such as Amazon Web Services, Google Cloud or Microsoft Azure. You rent only what you need and pay by the hour, by the gigabyte or by the request. That is wonderfully flexible, and it also means the bill changes every single day depending on what your software did.

Everyday Analogy

Picture four friends sharing one apartment with a single electricity meter. At the end of the month a bill arrives for the whole flat. One friend works from home with three monitors, one is away for two weeks, and one runs a space heater all winter. Splitting the bill four ways feels unfair to someone. Fairness needs a sub-meter on each room. In the cloud, telemetry is the sub-meter.

Money

The Cloud Bill

A list of charges from the provider, grouped by the provider’s own product names, covering a period such as a month. It tells you what was bought and how much it cost.

Money

Resource

Any single thing you rent: a virtual server, a database, a storage bucket, a message queue. Each resource produces cost lines on the bill.

Label

Tag or Label

A small note you attach to a resource, such as “team = payments.” Providers can group the bill by tag. Tags are the cheapest form of attribution, and they have limits you will meet soon.

Telemetry

Metrics

Numbers measured over time, such as processor seconds used, requests served or bytes sent. They are the meter readings.

Telemetry

Traces

The recorded story of one request as it travels through many services. Each step is called a span. Traces show who called whom.

Telemetry

Logs

Text records of events, such as “order 4821 created for customer Acme.” Logs carry rich detail but are bulky.

OpenTelemetry, usually written OTel, is the free, vendor-neutral standard for producing and moving metrics, traces and logs. It gives every language the same way of attaching descriptive labels to telemetry, which is exactly what attribution needs. The labels are called attributes, and they are the thread that ties a cost to a cause.

T
Remember Two Words

Cost is the how much. Telemetry is the who and why. Attribution is the act of joining them.

2What Is Cost Attribution?

A plain definition, a restaurant, and the difference between guessing and measuring.

Cost attribution is the practice of assigning each unit of spend to the thing that caused it. That “thing” is called the cost object, and it can be a team, a product, a customer, a feature, an environment such as testing or production, or even a single type of request. The goal is to be able to say, with evidence, “this much of last month’s bill was caused by that.”

You will often hear two related words. Cost allocation is the act of splitting a cost among owners by some rule. Cost attribution puts the stress on cause: the split should follow what each owner actually consumed. In everyday speech the two are used almost interchangeably, and this guide does the same, but it keeps the stress on evidence.

Everyday Analogy

Eight colleagues eat lunch together, and the restaurant brings one bill. One person had a salad. Another ordered the lobster and two desserts. The easy option is to divide by eight. The fair option is to look at what each person ordered. Looking at the order slips is exactly what telemetry does for your cloud bill: it is the record of who asked the system to do what.

Three ways to split a bill

  • Even split. Divide by the number of owners. Easy to do and easy to resent.
  • Tag-based split. Charge each resource to the owner named in its tag. Fair when a resource has exactly one owner.
  • Usage-based split. Measure how much of a shared resource each owner used, then divide the cost in that proportion. This is where telemetry becomes essential.

Real organizations use all three, in layers. Resources with a clear single owner are charged directly. Shared resources are split by measured usage. Whatever cannot be measured is split by a simple, openly stated rule. A good attribution system also tells you how much of the bill falls into each layer, so you know how much of the answer is measured and how much is estimated.

Why this matters more every year

Three trends make attribution harder to skip. First, systems are split into more and smaller services, so a single customer action may touch thirty of them, and nobody sees the whole cost of that action. Second, cloud spending has moved from a quiet line in the technology budget to one of the largest items a company pays, which means finance leaders now ask engineers detailed questions. Third, new kinds of usage-based services, such as language models that charge per chunk of text, can multiply a bill within days when a feature becomes popular.

Without attribution, the usual reaction to a surprising bill is a company-wide request to “reduce cloud spend by ten percent.” That sounds firm, but it punishes everybody equally, including the teams that were already efficient. With attribution, the reaction can be precise: the cost of one feature rose, here is the cause, and here is who can fix it.

i
Not the Same as Saving Money

Attribution does not reduce the bill by itself. It makes the bill understandable, which is what lets people decide where to reduce it. Teams that can see their own cost tend to ask better questions about it.

3Why the Bill Alone Cannot Answer the Question

The bill is organized the way the provider sells things, not the way your business works.

A cloud bill is a perfectly honest record of what the provider charged. The trouble is the shape of that record. It is organized by the provider’s catalog: so many hours of this server type, so many gigabytes of that storage class. Your business is organized differently, around products, customers and teams. The two views do not line up on their own.

The business asksWhat the bill showsWhat is missing
How much does the Checkout product cost to run?One line for the shared Kubernetes clusterWhich part of that cluster Checkout used
Are we losing money on customer Acme?One line for the shared databaseHow many queries Acme caused
Did last week’s release make us more expensive?Daily totals that rise and fallWhich service and which change caused the rise
What does one order cost us in infrastructure?Total spend for the monthThe number of orders, and their share of each resource
Who is wasting money in the test environment?Servers with no clear ownerWho created them, and whether anyone uses them

Four reasons tags cannot do it alone

1. A shared resource has many owners

A tag holds one value. A database used by twelve services can be tagged with only one team, and the other eleven look free.

2. Tags are applied by humans, so they drift

Someone forgets, someone misspells, someone copies an old template. Over months, a growing share of spend becomes “untagged,” and nobody can say whose it is.

3. Some costs have no tag at all

Data transfer between regions, support charges and some managed-service fees may not carry the tags you set on your own resources.

4. Tags describe ownership, not consumption

A team may own a cluster but serve ten other teams’ traffic from it. The tag shows who owns it. Only telemetry shows who used it.

!
Watch Out

Many organizations discover that a large slice of the bill is tagged with nothing, or with a single “platform” owner. That slice is where the surprises live, and it is the slice telemetry is best at explaining.

4Showback, Chargeback and Unit Economics

Three words from the world of cloud financial management, and what each one asks of your data.

The discipline that deals with cloud money is called FinOps, a blend of “finance” and “operations.” It is a practice, not a product, and it has an industry body, the FinOps Foundation, that publishes shared vocabulary. Attribution sits at its heart because every other FinOps activity, such as budgeting, forecasting and optimization, needs to know whose money is being discussed.

PracticeWhat happensAccuracy needed
ShowbackEach team is shown the cost it caused, but nobody is billed. It builds awareness.Good enough to be believed
ChargebackEach team’s budget is actually reduced by the cost it caused. It builds accountability.High, because money moves and disputes follow
Unit economicsCost is divided by a business measure, such as cost per order or per customer, to see if growth is healthy.Good, plus a trustworthy count of the business unit
Everyday Analogy

Showback is the receipt your phone company sends that lists each family member’s usage, with a note that says “for information only.” Chargeback is when each family member actually pays their own share. Unit economics is asking “what does one call cost us?” so you can decide whether the plan is any good.

The industry also maintains a common data layout for billing records, called FOCUS, short for the FinOps Open Cost and Usage Specification. Its purpose is to let cost data from different providers look alike, so a single attribution system can read all of them. If you join telemetry to billing data, using a common layout saves a great deal of translation work.

Why telemetry is the key to the third one

Showback and chargeback need to know who. Unit economics needs to know how many. The number of orders, the number of active customers and the number of messages processed are all business events, and a well-instrumented system already counts them, because the same counts feed its dashboards. Telemetry therefore supplies both halves of the fraction: the cost on top, split by cause, and the unit count underneath.

5Telemetry as the Missing Meter

Each kind of telemetry measures something different. Knowing their strengths tells you which to use for which cost.

No single signal answers every attribution question. Think of them as different instruments in a workshop.

Signal

Metrics: The Meter Reading

Cheap, compact and regular. Processor time, memory held, requests per second, bytes stored. Best for splitting the cost of shared compute and storage. Weakness: they usually lack detail about who asked.

Signal

Traces: The Receipt

Show the path of each request and how long each step took. Best for working out which customer or feature triggered work deep inside the system. Weakness: usually sampled, so counts are incomplete.

Signal

Logs: The Detail

Hold rich facts such as customer names, query text and sizes. Best for hard cases that need detail. Weakness: bulky, costly to store and easy to over-collect.

Context

Resource Attributes: The Nameplate

Facts about where telemetry came from: service name, cluster, region, team. They connect a number to a piece of infrastructure.

Context

Business Events: The Order Count

Counts such as orders placed or messages processed. They provide the “per unit” in cost per unit.

Money

Billing Export: The Price List

The provider’s detailed cost records, usually delivered to a data warehouse. They supply the dollars to be divided.

Notice that the last card is not telemetry at all. Attribution needs two data sources: usage evidence from your own software, and cost data from your provider. The craft is in joining them over a shared key, such as a resource identifier, a namespace or a service name, and over a shared time window, such as an hour or a day.

flowchart LR
A["Metrics of resource use"] --> J["Join on resource and time window"]
B["Traces and business events"] --> J
C["Cloud billing export"] --> J
J --> D["Cost per team, customer and feature"]
    
FIG 1. Attribution is a join between what was used and what it cost
T
Start With What You Have

You do not need new instrumentation to begin. Many organizations already collect processor, memory and request metrics for every service. That alone can attribute the biggest slice of a typical bill, which is shared compute, within a few weeks.

6Identity: The Attributes That Make Attribution Possible

A cost can only be assigned to an owner if the telemetry names that owner. This chapter is about the labels.

Every piece of telemetry can carry attributes, which are simple name and value pairs. For attribution, a handful of attributes matter far more than the rest. They answer the questions “which team,” “which service,” “which customer,” “which feature” and “which environment.”

QuestionTypical attributeWhere it is set
Which service is this?Service nameOnce per process, as a resource attribute
Which team owns it?Team or ownerOnce per process, ideally from a service catalog
Where does it run?Cluster, namespace, regionAdded automatically by the platform or the Collector
Which environment?Production, staging, testOnce per process
Which customer?Tenant or account identifierPer request, on spans and events
Which feature?Feature or endpoint namePer request

Three places an attribute can live

Resource attributes describe the thing producing telemetry and stay the same for the life of a process. Span attributes describe one particular operation and can differ on every request. Baggage is a standard way of carrying small name and value pairs along with a request as it hops from service to service, so that a service deep in the chain still knows which customer started the request.

Everyday Analogy

Think of a hospital wristband. The patient’s name and allergies are written on it once, and every department the patient visits, from reception to X-ray to the pharmacy, can read it without phoning the front desk. Baggage is the wristband of a request.

sequenceDiagram
participant C as Customer App
participant G as Gateway
participant O as Orders Service
participant D as Database
participant W as Queue Worker
C->>G: Place order
Note over G: Gateway sets baggage tenant and feature
G->>O: Call with baggage in headers
O->>D: Query tagged with tenant
O->>W: Message with baggage in its headers
Note over W: Worker reads tenant and records usage
    
FIG 2. Baggage carries the customer identity through every hop, including the queue

Where identity gets lost

  • Queues and background jobs. The request ends, and the work continues later with no connection to the original caller unless the identity is copied into the message.
  • Caches. A cache hit serves one customer with data that another customer paid to compute.
  • Batch jobs. A nightly job processes everyone at once and needs its own per-customer accounting.
  • Services without instrumentation. One silent service breaks the chain for everything beyond it.
!
Be Careful What You Carry

Baggage travels in network headers, and it can be sent to third-party services you call. Never put secrets or personal data in it. Use an opaque identifier, and strip it at the edge of your system.

7The Allocation Math: Direct, Proportional, Shared and Idle

Four rules cover almost every case. A small worked example shows how they fit together.

Every line of the bill gets sorted into one of four buckets, each with its own rule.

BucketRuleExample
DirectCharge the whole cost to the single known owner.A database used only by the Payments team
ProportionalSplit by each owner’s measured share of usage.A shared cluster split by processor time
SharedSplit by a fixed, openly stated rule because nobody can measure usage.Security tooling split by headcount
IdleCapacity paid for but not used. Show it separately, then decide how to spread it.Servers running at one-quarter load
flowchart TB
C["A line on the cloud bill"] --> T{"Does it carry a trusted owner tag?"}
T -- "Yes" --> D["Direct allocation to that owner"]
T -- "No" --> U{"Can telemetry measure who used it?"}
U -- "Yes" --> P["Proportional allocation by measured usage"]
U -- "No" --> S{"Is it shared by everyone?"}
S -- "Yes" --> F["Shared pool split by a fixed rule"]
S -- "No" --> X["Unallocated bucket to investigate"]
    
FIG 3. How each line of the bill is sorted

A worked example

A shared cluster costs $30,000 for the month. Over that month it could have supplied 100,000 core-hours of processing, where a core-hour is one processor core running for one hour. Telemetry shows three teams used 80,000 core-hours in total.

$30,000
monthly cluster cost
$0.30
price of one core-hour of capacity
20%
of capacity sat idle

The price of one core-hour of capacity is $30,000 divided by 100,000, which is $0.30. Now charge each team for what it used.

OwnerCore-hours usedCost at $0.30Share of idle costFinal attributed cost
Checkout40,000$12,000$3,000$15,000
Search25,000$7,500$1,875$9,375
Recommendations15,000$4,500$1,125$5,625
Idle20,000$6,000spread above$0
Total100,000$30,000$6,000$30,000

The idle cost of $6,000 is spread in proportion to use. Checkout used 50 percent of the 80,000 used core-hours, so it carries 50 percent of the idle cost, which is $3,000. Search used 31.25 percent and carries $1,875. Recommendations used 18.75 percent and carries $1,125. Add the final column: 15,000 plus 9,375 plus 5,625 equals 30,000. Every dollar is explained, and none is counted twice.

T
Idle Is a Decision, Not a Fact

Whether to spread idle cost to the teams or to keep it in a central bucket is a policy choice. Spreading it makes each team feel the real price of running the platform. Keeping it central shows how well the platform team is filling its servers. Many companies report both views.

8The Architecture of an Attribution Pipeline

Five stations take raw telemetry and raw billing records and turn them into a number a finance team will trust.

An attribution system is not one program. It is a short pipeline, much like a factory line, in which each station does one job. Here is the whole picture.

flowchart LR
A["Applications with OTel SDK"] --> B["OpenTelemetry Collector"]
B --> C["Usage Store"]
E["Cloud Billing Export"] --> F["Cost Warehouse"]
H["Service Catalog of owners"] --> G["Allocation Engine"]
C --> G
F --> G
G --> I["Dashboards and Reports"]
G --> J["Budget Alerts"]
    
FIG 4. Telemetry and billing data meet in the allocation engine
1

Produce

Applications emit metrics, traces and logs carrying the identity attributes from Chapter 6, such as service, team, tenant and feature.

2

Enrich

The Collector adds facts the application does not know, such as the cluster, namespace and cloud region, and cleans up messy values.

3

Aggregate

Raw telemetry is summed into compact usage records, for example “tenant Acme made 1.2 million requests to the Orders service between 10:00 and 11:00.”

4

Join

The allocation engine matches usage records with cost records on resource and time, and consults the service catalog to learn who owns what.

5

Publish

Results flow to dashboards, monthly reports and alerts, with the unattributed share shown openly.

The Collector does much of the heavy lifting

The OpenTelemetry Collector is a standalone program that receives telemetry, changes it and forwards it. Several of its parts are especially useful here. A resource detection processor adds cloud and host facts automatically. A Kubernetes attributes processor labels telemetry with the pod, namespace and deployment it came from. An attributes or transform processor renames values and fixes inconsistent labels, for example mapping “prod,” “production” and “PRD” to one spelling. A span metrics connector turns traces into counters and timing summaries, grouped by chosen labels. Component names and options change between releases, so check the documentation for your version.

Why aggregate early

Raw traces and logs are enormous. Attribution does not need every individual event. It needs totals by owner and hour. Summing close to the source cuts storage and speeds up every later step. The trade-off is that an aggregated record cannot be unpicked later, so keep a short window of raw data for investigations.

Choosing the join key

The join is the hardest step, so it is worth planning. A join key is a value that appears on both sides and lets you match a usage record to a cost record. The best keys, from strongest to weakest, are these. A resource identifier, such as the unique name of a server or database, matches exactly, but only works when telemetry records that identifier. A cluster and namespace pair works well for container platforms. A service name works when the owner catalog maps service names to billing groups. A tag value works when tags are consistent. When no strong key exists, fall back to a weaker one and mark the result as lower confidence.

Time is the second key. Billing records usually cover an hour or a day, while telemetry arrives every few seconds. Bring both to a common window, for example one hour, before matching. If one side is late, hold the window open until it arrives instead of joining half-finished data, because a half-joined hour produces a number that looks precise and is wrong.

i
Owners Need a Home

The service catalog is the list that says “service Orders belongs to team Commerce.” Without it, every team name lives in somebody’s memory. Open-source developer portals such as Backstage, which began at Spotify, are one popular way of keeping this list.

9Shared Clusters and Kubernetes

Containers make resources efficient and bills opaque. Here is how telemetry cuts through.

Kubernetes is software that packs many programs, each running in a small box called a container, onto a pool of shared servers called nodes. This is efficient, because ten teams can share one pool. It is also confusing for billing, because the provider charges for the nodes and knows nothing about the containers on them.

flowchart LR
N["Node pool cost from the bill"] --> A["Namespace Checkout"]
N --> B["Namespace Search"]
N --> C["Namespace Recommendations"]
N --> I["Idle capacity"]
I --> A
I --> B
I --> C
    
FIG 5. The node bill is split among namespaces, and idle capacity is then spread across them

Two ways to measure a container

Each container usually declares a request, which is the amount of processor and memory it asks the cluster to reserve, and then actually uses some amount. Both numbers appear in standard Kubernetes metrics. Processor use is commonly exposed by the node’s container monitor, and requests are exposed by a state-reporting add-on. A request-based charge pays for room that was reserved, and a usage-based charge pays for room that was consumed.

MethodExample: asks for 4 cores, uses 1 on average, for a 720-hour monthCost at $0.30 per core-hour
By request4 cores x 720 hours = 2,880 core-hours$864
By usage1 core x 720 hours = 720 core-hours$216

Which is right? Tools in this area, including the open-source OpenCost project, commonly charge the larger of the two. The reasoning is fair: if a team reserved four cores, nobody else could use them, so the team should pay for blocking them, even if it used only one. This gives teams a direct reason to size their requests honestly, and it turns a vague complaint about “wasteful teams” into a measurable gap of $648 per month in this example.

Everyday Analogy

You book a table for eight at a restaurant, but only three people turn up. The restaurant held the other five chairs empty and turned other customers away. Charging for three would be unfair to the restaurant, and charging for eight tells you to book realistically next time.

What else lives on the nodes

Not everything in a cluster belongs to a team. The cluster’s own system programs, monitoring agents and networking components consume resources too. Treat these as a shared platform cost and spread them by a stated rule, usually in proportion to each team’s own consumption. Namespaces, labels and annotations on workloads give you the owner. If a workload has no owner label, that is a finding in itself, and it belongs in the unallocated bucket until someone claims it.

!
Autoscaling Moves the Goalposts

When a cluster automatically adds and removes nodes, the price of one core-hour changes through the day. Compute the price per hour and allocate hour by hour, then add up. A single monthly average can make a team that runs at peak look cheaper than it was.

10Beyond Compute: Storage, Network, Databases and AI

Servers are only part of the bill. Every other category needs its own meter.

The previous chapters used processor time as the measuring stick. Real bills contain many other kinds of cost, and each one needs a different piece of evidence.

Storage

Files and Buckets

Charged by gigabytes held. Measure bytes stored per owner prefix or bucket, and add request counts if reads and writes are billed separately.

Network

Data Transfer

Charged by gigabytes leaving a region or the cloud. Measure bytes sent per service from network metrics or access logs. It is often the most surprising line on a bill.

Database

Shared Databases

One database, many callers. Measure query counts and query time per calling service, using database spans from traces, and split in proportion.

Messaging

Queues and Streams

Charged by messages or throughput. Count messages published and consumed per team, using the identity copied into message headers.

Outside

Third-Party APIs

Maps, payments, email or identity vendors that charge per call. Count outbound calls per feature with client spans.

Serverless

Functions

Charged by invocations and run time. Telemetry already records both per function, so attribution is nearly direct.

AI

Language Model Calls

Charged by tokens, which are small chunks of text. Record input and output token counts on each call, per feature and customer.

Meta

Observability Itself

Monitoring tools bill by data volume. Attribute that bill too, by the service that produced the data, so teams see what their own logging costs.

An AI example

Language models bill by tokens, with different prices for what you send in and what comes back. OpenTelemetry’s semantic conventions for generative AI define standard attribute names for these token counts, although the area is still evolving, so check the current status. Suppose an “email summary” feature makes 2,000 calls a day, each sending 1,500 input tokens and receiving 500 output tokens. Using made-up prices of $3 per million input tokens and $15 per million output tokens: input is 2,000 x 1,500 = 3,000,000 tokens, or $9 a day. Output is 2,000 x 500 = 1,000,000 tokens, or $15 a day. The feature costs $24 a day, about $720 for a 30-day month. Without per-feature token counts, that spend would be one anonymous line labeled “AI provider.”

T
Pick the Cost Driver

For every cost, ask “what makes this number go up?” The answer is the metric you should capture per owner. Storage grows with bytes, transfer with bytes sent, queues with messages and models with tokens.

11Unit Economics in Practice: Cost per Customer and per Order

Once costs are attributed, dividing them by business activity shows whether you are making or losing money.

Imagine a software company whose platform costs $90,000 a month to run. It serves three customers, called tenants, and telemetry counts the requests each made.

CustomerRequestsAttributed costMonthly revenueMarginMargin percent
Acme60 million$54,000$70,000$16,00022.9%
Borealis30 million$27,000$25,000-$2,000-8.0%
Cinder10 million$9,000$20,000$11,00055.0%
Total100 million$90,000$115,000$25,00021.7%

The cost per million requests is $90,000 divided by 100, which is $900. Acme made 60 of those millions, so $54,000. Borealis made 30, so $27,000. Cinder made 10, so $9,000. The three add to $90,000, as they must. The surprise sits in the Borealis row: the company earns $25,000 from a customer that costs $27,000 to serve. Without attribution, that customer looks healthy because it brings in real revenue. With attribution, it is a conversation about pricing, limits or efficiency.

Everyday Analogy

A delivery company charges every customer a flat fee per parcel. Then it notices that one customer always sends parcels to remote islands and another always to the next street. The flat fee hides that the island customer costs ten times more. Attribution is the act of looking at the destination on each parcel.

Not every request costs the same

Counting requests treats a simple page load and a heavy report generation as equals. A more accurate approach gives each request type a weight based on measured resource use. If a report request uses five times the processing of a simple one, count it as five units. Traces provide the timings to calculate these weights, and you can refresh them every few months.

Other useful unit costs

  • Cost per order, per booking or per message for product teams.
  • Cost per active user for consumer products.
  • Cost per thousand requests for platform and API teams.
  • Cost per environment, to see how much the test and staging copies add on top of production.
A rising bill is not a problem if the cost per order is falling. A flat bill is a problem if the orders are not.

12Sampling, Cardinality and Accuracy

Two properties of telemetry quietly distort the numbers. Both are easy to handle once you know about them.

Sampling: the missing receipts

Storing every trace is expensive, so most systems keep only a fraction. This is called sampling. It is fine for debugging, but dangerous for counting. If you keep one trace in ten, a naive count of stored traces is ten times too small.

flowchart LR
S["Spans from services"] --> M["Span metrics connector counts every span"]
M --> U["Usage metrics for attribution"]
S --> T["Tail sampler keeps about 10 percent"]
T --> B["Trace backend for debugging"]
    
FIG 6. Count before you sample, so attribution sees every request

Two safe approaches exist. The first is to count before sampling, as the diagram shows, so the usage numbers see every request, and only the stored traces are reduced. The second is to record the sampling probability with each kept trace and multiply by its inverse. If 1,000 sampled spans each represent a one-in-ten chance, the estimate is 10,000. The first approach is simpler and exact. The second is useful when only sampled data exists, but it adds statistical error, especially for small customers who appear in few samples.

Cardinality: too many labels

Cardinality is the number of different label combinations a metric can have. Every combination becomes a separate series that a monitoring system must store and usually charges for. Adding a tenant label to a metric is tempting, because it is exactly what attribution wants. But consider 5,000 tenants, 50 endpoints and 5 status codes. That is 5,000 x 50 x 5 = 1,250,000 separate series from one metric, and a big bill for your monitoring system on top of the cloud bill you were trying to understand.

!
The Observer Effect

Measuring cost can create cost. Always estimate how many series a new label will create before turning it on, and attribute your monitoring bill like any other.

Ways to keep cardinality under control

  • Separate a usage stream. Emit one compact usage record per tenant per hour, rather than putting tenant on every request metric.
  • Group the small ones. Label the top fifty tenants individually and put the rest in an “other” group, or use a plan tier.
  • Keep the identifier in traces and logs. Those systems handle detailed fields better than metric stores do.
  • Drop labels nobody uses. Every label should answer a question someone actually asks.

13Reconciliation: Earning Trust in the Numbers

An attribution report is only useful if finance believes it. Trust comes from a few simple checks.

The first rule is the one the worked examples already followed: the attributed amounts must add up to the bill. After allocation, sum every owner’s cost, plus any unallocated bucket, and compare it with the provider’s invoice total for the period. If the two differ, something was dropped or counted twice. This single check catches a surprising share of bugs, and it is what turns a clever dashboard into an auditable report.

Which price are you dividing?

Cloud providers publish several versions of cost, and they can differ a lot. Unblended cost is what each line was charged on the day. Amortized cost spreads upfront commitments, such as a prepaid discount for committing to a year of usage, across the months that benefit from them. Using unblended cost means the team that happens to be running on the day a big payment is made looks expensive. Amortized cost is usually fairer for attribution, and finance teams should agree which view the reports use, and write it down.

Timing issues

  • Billing data arrives late. Final charges, credits and corrections can take days after the month ends. Publish a provisional report early, mark it provisional, and replace it when the bill is final.
  • Time zones and windows. Billing records and telemetry may use different clocks and windows. Align them to the same hour boundaries before joining.
  • Credits and discounts. Decide whether they are shared across everyone or credited to the team that earned them.

Showing confidence

BucketHow it was decidedConfidence
DirectA trusted owner tag on the resourceHigh
ProportionalMeasured usage split from telemetryMedium to high, depending on instrumentation
SharedA fixed rule such as headcountLow by nature, but transparent
UnallocatedNobody has claimed itNone, and a target to shrink

When someone disputes a number

Sooner or later a team will say “that cost is not ours.” Disputes are healthy, because they find real mistakes, but they need a calm routine. Keep every step of the calculation reproducible, so that anyone can see which usage records and which cost lines produced a given figure. Offer the team a view of the raw usage behind its number, for example the hourly processor use by namespace. Most disputes end there, because the data answers the question.

  • Missing identity. The team is right, a service did not carry its label, and the fix is to repair the label and re-run the month.
  • Shared rule disagreement. The measurement is right, but the team disagrees with the rule for sharing a common cost. That is a policy conversation, not a data bug, and it belongs with finance and the platform owners.
  • Timing difference. A late credit or a correction has moved the final bill. Update the report and note the change.

Record every correction with a date and a reason. Over time that log becomes the best evidence that the system is honest, because it shows mistakes being found and fixed in the open.

T
Publish the Unknown

Report the unallocated percentage on every dashboard. A report that says “92 percent attributed, 8 percent unknown” earns more trust than one that quietly hides the gap. Aim to shrink the unknown each month.

14Security, Privacy and Governance

Cost data names customers and teams. Handle it with the same care as any other business data.

Attribution data is sensitive in two ways. Per-customer cost and margin can reveal how much each contract is worth, which is information a company rarely wants everyone to see. And per-customer identifiers inside telemetry can be personal data, which privacy laws regulate.

Privacy

Use Opaque Identifiers

Record a tenant code, not a name or an email address. Keep the lookup table separate and tightly controlled.

Access

Limit Who Sees Margin

Engineers need cost per service. Only a small group may need cost per customer, and revenue should live in a separate system.

Retention

Keep Aggregates Longer

Raw events can be deleted after weeks. Hourly usage totals are small and can be kept for years to support trend reports.

Policy

Enforce Tagging Rules

Write the required tags as rules that are checked automatically when infrastructure is created, rather than as a document nobody reads.

Audit

Keep the Method Versioned

Store the allocation rules in version control, so that anyone can see why a number changed between two months.

Ownership

Name an Owner for Every Service

A catalog entry with a team and a contact makes unallocated cost a short, solvable conversation.

!
A Small Shortcut With a Large Risk

It is tempting to put customer names or emails into span attributes because it makes debugging easy. Those values then flow into every telemetry vendor you use. Decide once, centrally, which identifiers are allowed.

15Monitoring the Attribution System Itself

An attribution system that quietly breaks is worse than none, because people keep trusting it.

Like any pipeline, attribution can decay. A team renames a service, a deployment drops a label, a new region appears with no tags. Each change moves a little more money into the “unknown” bucket, and nobody notices until a report looks odd. So the pipeline needs a small set of health measures of its own.

Health measureWhat it tells youWarning sign
Unallocated share of the billHow much money has no ownerRising month over month
Untagged resource countHow well tagging rules are followedNew resources appearing without owners
Instrumented service coverageHow many services emit identity attributesA big service missing from the usage store
Join success rateHow many usage records matched a cost recordA sudden drop after a platform change
Reconciliation differenceAttributed total minus the invoice totalAnything beyond rounding
Data freshnessHow old the newest usage record isHours of delay beyond normal
Series count growthWhether cardinality is explodingA steep rise after a release

Cost anomalies are a bonus

Once cost is attributed per service and per hour, spotting oddities becomes straightforward. A service whose cost doubles overnight while its request count stays flat has probably shipped a regression, such as a loop that retries, an unbounded query or a debug setting left on. Alerts on cost per unit, rather than on total cost, avoid false alarms when traffic simply grows.

Everyday Analogy

A household notices that the gas bill doubled although nobody cooked more. That points to a leak, not to a larger family. Cost per request is the “gas used per meal” figure that lets you tell leaks from growth.

T
Alert on Ratios

Use “cost per thousand requests” and “unallocated percent” as the alarm signals. Totals rise and fall with traffic. Ratios rise only when something changed.

16Design Patterns and Trade-offs

Eight patterns that mature teams use, and an honest look at what each choice costs.

Pattern 1

Ownership in a Catalog

Keep one list of services and their owning teams, and let every pipeline read from it. Attribution then never depends on a person remembering.

Pattern 2

Identity at the Edge

Set the tenant and feature once, where requests enter the system, and carry them inward as baggage. Services further in just read them.

Pattern 3

Enrich in the Collector

Add cluster, namespace and region centrally, so application teams do not each reinvent it and values stay consistent.

Pattern 4

Count Before You Sample

Derive usage counters from the full stream of spans, then reduce what you store for debugging.

Pattern 5

A Dedicated Usage Stream

Emit small, aggregated usage records per owner per hour, separate from debugging telemetry, so cardinality never harms your monitoring.

Pattern 6

Layered Allocation

Direct first, then proportional, then shared rules, then unallocated. Each layer is reported separately.

Pattern 7

Reconcile Every Run

Fail the pipeline loudly when attributed totals and the invoice disagree, and show the difference.

Pattern 8

Cost Next to Performance

Show cost per request on the same dashboard as latency and errors, so engineers weigh all three in one glance.

A key trade-off: measure everything, or sample?

Advantages of exact usage counting

  • Small customers are counted fairly, not guessed from a few samples.
  • Disputes are easy to settle with real numbers.
  • Counting is cheap compared with storing full traces.

Disadvantages

  • Needs instrumentation that counts every request, not just a sample.
  • Adds a little work in the request path.
  • Needs a separate stream and storage for usage records.

Another trade-off: precise or simple rules?

Advantages of fine-grained attribution

  • Pinpoints expensive features and customers.
  • Supports chargeback and pricing decisions.
  • Rewards teams that become efficient.

Disadvantages

  • More data to collect, store and secure.
  • More rules to explain and defend.
  • False precision when inputs are imperfect.

The sensible middle path is to start simple, which means owner tags and processor-time shares, and add detail only where the money justifies it. A team spending a few hundred dollars a month does not need per-request attribution. A product spending a million does.

17Anti-Patterns and Common Mistakes

Four habits that look sensible and quietly spoil the numbers.

ANTI-PATTERN 01Avoid
What Teams Do

They split every shared cost evenly across all teams, because it is quick and nobody can argue that it is biased.

Why It Hurts

Small, efficient teams subsidize large, wasteful ones. Nobody has a reason to improve, because their share does not change when their behavior does.

Do Instead

Use even splits only for costs that truly cannot be measured, and shrink that pool each quarter.

ANTI-PATTERN 02Avoid
What Teams Do

They rely only on resource tags and declare the problem solved once a tagging policy is announced.

Why It Hurts

Tags name one owner per resource and decay over time. Shared databases and clusters stay opaque, and the untagged share creeps back up.

Do Instead

Treat tags as the first layer, add usage telemetry for shared resources, and check tag coverage automatically every day.

ANTI-PATTERN 03Avoid
What Teams Do

They add the customer identifier as a label on every metric so that any chart can be split by customer.

Why It Hurts

The number of series explodes, the monitoring bill soars, and dashboards slow down. The cost-saving project becomes a cost driver.

Do Instead

Emit a small, aggregated usage stream per customer, and keep detailed identifiers in traces and logs.

ANTI-PATTERN 04Avoid
What Teams Do

They start chargeback in the first month, billing teams real money from numbers nobody has yet verified.

Why It Hurts

The first error becomes a political fight, and people stop trusting the system before it matures.

Do Instead

Run showback for several months, fix the biggest gaps in public, and switch to chargeback only when the reconciliation is clean and the owners agree on the rules.

Other frequent mistakes

  • Counting sampled traces as if they were complete.
  • Forgetting background jobs, queues and caches, so identity is lost.
  • Reporting a number to the cent when the inputs are rough estimates.
  • Changing allocation rules mid-year without versioning them.
  • Ignoring the unallocated bucket because it is embarrassing.

18A Step-by-Step Rollout Plan

A practical order of work that delivers useful answers early and adds precision later.

1

Get the bill into a warehouse

Turn on the provider’s detailed billing export and load it where you can query it. Look at the biggest lines first.

2

Build the owner catalog

List every service with an owning team and contact. Accept an imperfect first version and fix gaps as they appear.

3

Attribute the biggest shared resource

Usually the main cluster. Use processor and memory metrics by namespace. Reconcile to the invoice.

4

Publish showback

Give each team a simple monthly page with its cost, its trend and the unallocated share. Collect feedback.

5

Add customer and feature identity

Introduce baggage at the edge, a usage stream and weights. Start with one product line.

6

Add unit economics

Divide attributed cost by orders, users or requests. Share the trend with product leaders.

7

Automate guardrails

Enforce tag rules, alert on cost per unit, and block releases that introduce unowned resources.

Maturity at a glance

StageWhat you can answerTypical evidence
CrawlWhich account or project costs the mostProvider bill and basic tags
WalkWhich team and which service costs the mostTags plus processor and memory metrics
RunWhich customer and feature costs the most, and at what marginIdentity propagation, a usage stream and unit counts
i
Let Money Set the Pace

Spend the effort where the dollars are. If three resources make up most of the bill, getting those three right matters far more than perfecting hundreds of small ones.

19Real-World Scenarios

Five illustrative stories showing how attribution changes decisions.

The scenarios below are illustrative composites. The numbers are chosen for clarity and are not measurements from any named company.

Scenario 1: The streaming platform and the hidden recommender

Imagine a video streaming company, in the spirit of Netflix, whose cluster bill grows 30 percent in a quarter. Traffic grew only 8 percent. Per-namespace usage shows the recommendation service doubled its processor use after a model update, while its request count barely moved. The cost per thousand requests flagged the change within days. The team found an expensive scoring step running twice, fixed it, and the curve bent back.

Scenario 2: The ride-hailing platform and the shared database

A ride-hailing company, similar in scale to Uber, shares a large database among dozens of services. The bill shows one line. Database spans, counted per calling service and weighted by query time, reveal that one reporting service causes forty percent of the load. The team moves that service to a read replica and gives it its own budget line.

Scenario 3: The software vendor and the unprofitable customer

A business software vendor sees healthy revenue from every client. After tenant-level attribution, one large customer turns out to cost slightly more than it pays, as in the Borealis example in Chapter 11. Rather than lose the customer, the vendor introduces a usage tier for heavy API users and helps the customer cache repeated calls. Both sides spend less.

Scenario 4: The retailer and the sale-day surprise

A large online retailer, with the traffic pattern of Amazon on a sale day, expects the bill to triple with traffic. It rises sixfold. Data transfer, which nobody had measured per service, accounts for the gap. Bytes-sent metrics show an image service serving uncompressed pictures across regions. A caching change cuts the transfer line sharply before the next sale.

Scenario 5: The small startup and the viral feature

A small startup launches an AI writing helper and it goes viral. Usage grows tenfold in a week, and so does the model bill. Because each call already records the feature name, plan tier and token counts, the team sees within a day that free-tier users make eighty percent of the calls and produce almost no revenue. They add a daily limit for the free tier and a shorter default response length, and the cost per paying customer returns to a healthy level without turning anyone away.

Common cloud tooling makes a first step easy. AWS offers cost allocation tags and detailed cost and usage reports, Google Cloud exports billing records to its BigQuery warehouse, and Azure provides cost management with tag-based grouping. The open-source OpenCost project covers Kubernetes. These supply the money side. Telemetry supplies the usage side.

20Frequently Asked Questions

Short answers to the questions people ask most often.

Q1Is cost attribution the same as cost optimization?
No. Attribution tells you who is spending and why. Optimization is the work of reducing that spend. Attribution comes first because you cannot sensibly optimize what you cannot assign.
Q2Do I need OpenTelemetry to do this?
No, but it helps a great deal. It gives one consistent way to attach identity attributes and carry them across services and languages, so your attribution does not depend on a single vendor’s format.
Q3Are tags enough if we are small?
For a small setup with few shared resources, good tags may be enough for a long time. Add telemetry when shared clusters or databases start to hide who is spending.
Q4How accurate does it have to be?
Accurate enough that decisions would not change if the numbers moved a few percent. For showback, believable is enough. For chargeback and pricing, aim for audited reconciliation to the invoice.
Q5What do we do with idle capacity?
Show it as its own line first. Then choose a policy: spread it in proportion to use, or keep it with the platform team. Either is valid if it is stated openly.
Q6Why not just put the customer on every metric?
Because the number of series grows with every customer and can swamp your monitoring system. A separate aggregated usage stream gives the same answer at a small fraction of the cost.
Q7How does sampling affect attribution?
Counting only the stored sample undercounts. Count before sampling, or scale each kept item by the inverse of its sampling probability.
Q8Which cost view should reports use, unblended or amortized?
Amortized is usually fairer for attribution because it spreads upfront commitments over the months that benefit. Whichever you choose, document it and use it consistently.
Q9What about costs that nobody can measure?
Split them by a simple stated rule such as headcount or revenue, label them as shared, and keep that pool as small as you reasonably can.
Q10Can attribution cover AI model spend?
Yes. Record token counts and model names on each call along with feature and customer, then multiply by the provider’s prices. This often reveals one or two features that drive most of the AI bill.
Q11Where should I start tomorrow?
Turn on detailed billing export, build a simple owner catalog, and attribute your single biggest shared cluster by namespace. Reconcile it to the invoice and publish a first showback page.

21Summary and Key Takeaways

Key Takeaways

  • Cost attribution assigns each dollar of the cloud bill to the team, product, customer or feature that caused it, using evidence rather than guesses.
  • The bill shows what was bought. Telemetry shows who used it, and attribution is the join between the two, over a shared resource and time window.
  • Sort every cost into direct, proportional, shared and idle buckets, and always report how much falls into each.
  • Identity must travel with requests. Set tenant and feature at the edge, carry them as baggage, and copy them into queue messages.
  • Protect accuracy by counting before sampling and by using a small, aggregated usage stream instead of high-cardinality labels.
  • Reconcile to the invoice on every run, state which price view you use, and publish the unallocated share openly.
  • Start with showback on your biggest shared cluster, grow toward unit economics such as cost per order or per customer, and treat the system as something that needs its own monitoring.