AI Cost Attribution: Splitting the Bill by Feature, User, and Request

One common way AI applications die is not from having no users, but from the more it gets used, the more money it loses, while you don't even know where the loss comes from. At the end of the month, the OpenAI/Anthropic bill hits 50,000. You open the console and see — just a single total. Of that 50,000, which feature burned it? Which user? Which kind of call is the most expensive? Three questions, no answers.

This is what no cost attribution looks like. Token cost control (how to save) is the subject of another article; this one focuses on a single thing: how to turn that incomprehensible ledger into optimizable, accountable financial data.

I. Why AI costs must be attributed: variable cost + high variance

The cost structure of an AI application is fundamentally different from traditional software. The marginal cost of traditional SaaS trends toward zero — one more user adds a negligible amount to the amortized server bill. But AI applications have variable cost: every single LLM call costs real money, a few fractions of a cent per call, and millions of calls a day add up to tens of thousands.

Even more troublesome is high variance. For the same "smart assistant" feature, Zhang San asking "hello" burns 50 tokens, while Li Si dropping a 50-page PDF for summarization burns 50,000 tokens. The two differ by 1,000x. Mean-based thinking is enough for traditional software; AI costs demand distribution-based thinking — that 1% of requests in the tail often eats half your bill.

So without attribution, you face these blind spots:

• The PM says "adding an AI summary feature can't be that expensive, right?" Two weeks after launch it alone burned 20,000, and only then do you find out.
• A big customer makes 100,000 calls a day; you think they are using it normally, but the cost already exceeds what they pay for the annual plan.
• A prompt regression triples per-call tokens; the feature keeps working normally with zero errors, and you only discover it on the bill three weeks later.
• One free user running scripts burns through $8,000 in a single night.

The essence of attribution is to break the single total the vendor gives you into tens of thousands of tagged atomic records, then aggregate them along any dimension. Attribution is not the goal — optimization is. Knowing which feature loses money, which user is heavy, which kind of call is expensive tells you what to cut, what to limit, which prompt to change.

II. Tagging by request: every LLM call must carry its own metadata

The starting point of attribution is tagging. Every single LLM call must carry a set of metadata that tells the metering system: who triggered this call, which feature it belongs to, and in what context.

Without tags you only have the lump-sum total (that number on the vendor invoice). With tags you can break it apart. A workable tagging schema:

interface LLMCallTags {
  // Who
  userId: string;          // User ID
  orgId: string;           // Organization / team (required for B2B)

  // Which feature
  feature: string;         // 'chat' | 'summarize' | 'code-review' | ...
  featureVersion: string;  // 'summarize-v2', for A/B cost comparison
  experimentId?: string;   // feature flag / experiment bucket

  // What context
  sessionId: string;       // Session ID, aggregates a whole conversation
  requestId: string;       // Request ID, strings together the full call chain

  // Business context
  tier?: string;           // 'free' | 'pro' | 'enterprise'
  channel?: string;        // 'web' | 'api' | 'mobile'
}

Tagging must be done at the call entry point; it cannot be patched up afterward. The cleanest approach is to route through a unified LLM gateway — all calls must go through it, the entry injects tags, and the exit persists the billing result together with the tags. This is "single-point convergence of metering."

// Anti-pattern: business code calling the SDK directly all over the place — inevitably some calls miss tags
const res = await openai.chat.completions.create({ ... });

// Correct pattern: go through a unified gateway that enforces tags and rejects calls without them
const res = await llmGateway.chat({
  model: 'gpt-4o',
  messages,
  tags: { userId, feature: 'summarize', sessionId, tier },
});
// Inside the gateway: record prompt/completion tokens, compute cost, write tags into the metering table

As long as you let business code call openai.chat.completions.create() directly anywhere, some calls will inevitably miss tags, and the attribution data will never reconcile with the bill. A unified gateway is the prerequisite for trustworthy attribution — this is a hard constraint, not a nice-to-have.

III. Token metering: record input and output separately, convert to money using per-model coefficients

Tagging answers "who does this call belong to"; metering answers "how much is this call worth."

1. Tokens must be recorded separately

The basic rule of LLM billing: input and output have different unit prices, and output is usually 3-5x more expensive. They must be recorded separately; you cannot record just the total. Cache hits must also be recorded separately (prompt cache has a steep discount; without separate tracking you can't see the benefit).

interface UsageRecord {
  tags: LLMCallTags;
  model: string;
  promptTokens: number;       // Input tokens
  completionTokens: number;   // Output tokens
  cachedTokens: number;       // Cache-hit tokens (discounted price)
  latencyMs: number;
  timestamp: number;
}

Where do the token counts come from? Use the usage field in the vendor response; do not estimate it yourself. Your own estimates will never match the bill, and attribution becomes untrustworthy.

2. Convert to money using per-model pricing coefficients

Each model has a different unit price that also changes (vendors adjust pricing). So you need a pricing coefficient table, maintained as configuration rather than hardcoded:

const PRICING = {
  'gpt-4o':        { input: 2.5,  cached: 1.25, output: 10 },   // $ / 1M tokens
  'gpt-4o-mini':   { input: 0.15, cached: 0.075, output: 0.6 },
  'claude-sonnet': { input: 3,    cached: 0.3,  output: 15 },
};

function computeCost(u: UsageRecord): number {
  const p = PRICING[u.model];
  return (u.promptTokens - u.cachedTokens) * p.input / 1e6
       + u.cachedTokens * p.cached / 1e6
       + u.completionTokens * p.output / 1e6;
}

This table must be hot-updatable — when the vendor announces a price cut today, you can reflect it in the attribution data tomorrow, instead of waiting for the next release.

3. Aggregate across dimensions

Raw usage is an event stream; computing on the fly at query time is too slow. The common practice is pre-aggregation: do rollups along dimensions like (feature × user × model × day) and store them in OLAP / a data warehouse. Dashboards that respond in seconds rely on these pre-aggregated tables.

IV. Three layers of attribution: feature, user, request type

With tags in place and cost computed, the same data can be sliced three ways to answer three different questions.

Layer 1: By feature — which feature is burning money?

This table immediately exposes a problem: code-review has relatively few calls but its average per call is nearly 3x that of chat — why? Prompt too long? Using an expensive model? Output not reined in? Feature-level attribution directly helps you locate the feature with the highest optimization ROI. Cutting one unprofitable feature is more direct than optimizing ten prompts.

Layer 2: By user — who is burning money?

The cost distribution of an AI application is almost always heavy-tailed — a small number of heavy users consume most of the cost. This table directly determines whether the business model can work: Top 1% users cost $235 per capita, but they only pay $20/month in subscription fees, so this product loses more the more users it has.

Per-user attribution also identifies two kinds of problematic users: abusers (scripting the API, scraping your capabilities) and anomalous users (legitimate accounts but with strange usage patterns, e.g., always pasting very long context). Both must be caught by threshold alerts.

Layer 3: By request type — which kind of call is expensive?

Even within the same "chat" feature, calls can be split into those with tool calls, those with RAG retrieval, and multi-turn long conversations. This layer further breaks feature cost down into call-pattern cost:

• Requests with function calling have to feed in tool schemas, and prompt tokens are 3x those of a normal request.
• A RAG request that retrieves 20 documents stuffs 15K tokens into the prompt, while only 2K are actually useful — retrieval quality is burning money.
• Multi-turn conversations without history compression make the 10th turn's prompt 8x as long as the 1st turn's.

This layer points straight to concrete optimization targets: trim tool schemas, improve retrieval precision, compress conversation history.

V. Unit economics: figure out clearly what earns and what loses

The ultimate purpose of attribution is to compute unit economics: whether each user and each feature actually makes or loses money. This is an input for business decisions, not engineering self-indulgence.

Per user: LTV vs. cost to serve

Key insight: the higher the paid tier, the higher the cost-to-serve ratio may actually climb — because heavy users use the product the most. Enterprise has the highest revenue, yet its gross margin may be lower than Pro's. This doesn't mean you shouldn't do Enterprise; it means you must either raise the price, throttle usage, or optimize how they use the product.

Per feature: gross profit

Pair a feature's cost with the revenue it "drives." summarize burns $14K/month — how many paid conversions does it drive? If it's the core selling point of the Pro plan, contributing 30% of upgrades, then its gross profit is positive; if it's just a toy free users play with and drives no revenue, then it's a pure cost center that should be cut or charged for.

Unit economics takes you from "how much did this feature cost" to "is this feature worth building." That is the leap from an engineering perspective to a product/business perspective, and it is the most valuable hop attribution data enables.

VI. Anomaly detection: who is burning money, alert immediately

The other core value of attribution is real-time anomaly detection. AI cost bugs are more insidious than traditional software — a prompt change that doubles tokens triggers no error at all, the feature keeps working normally, and only the bill screams. You must build three classes of alerts:

• User level: a user's cost today is > 5x the daily average of the past 7 days. Catches abuse, scripts, abnormal usage. The script-user who burned $8,000 in one night above can only be caught by a month-end bill without alerts.
• Feature level: a feature's average cost per call jumps by more than 50%. Usually a prompt regression (added irrelevant context), the default model switched to a more expensive one, or an upstream bug (the retrieval system's recall exploded).
• Global level: total daily cost deviates from forecast by more than 30%. Usually a traffic surge (recommended by a KOL) or an across-the-board rise in token counts for some class of request.

Thresholds must combine absolute and relative values. Absolute ("cost per call > $0.5") catches obviously anomalous calls; relative ("5x the daily average") catches trend changes. Combined, both false negatives and false positives can be kept in check.

VII. Putting it into production: unified middleware + data warehouse + cost dashboard

String all the above together and the production architecture has three layers.

Layer 1: Metering middleware (online)

The gateway every LLM call must pass through. Responsibilities: enforce tag injection (reject without tags) → take usage and compute cost after the call → asynchronously write the event stream (Kafka/Kinesis) → synchronously run a budget check (throttle when a user's daily budget is exhausted). The key is still single-point convergence; it must become the single egress for LLM calls.

Layer 2: Data warehouse aggregation (offline)

The event stream flows into the warehouse (Snowflake/BigQuery/ClickHouse) and does two things: store raw usage (retain for 1-2 years for deep analysis) + pre-aggregate rollup tables (by dimensions like day × feature × user). Why not query the business database directly? Because cost analysis frequently runs heavy queries like "the trend of the top 1,000 users over the past 90 days," which would crush the online system if placed in the business database.

Layer 3: Cost dashboard + alerts (presentation)

At least three dashboards: Overview (daily/monthly total cost, stacked by feature, distribution by model, projected end-of-month cost), Drill-down (user cost leaderboard, feature drill-down to call type, time-series trends), Anomalies (real-time alerts, anomalous users/features, budget burn progress). The consumers are not just engineers — product, operations, and finance all need to look. Attribution is cross-functional, and the data has to be readable by non-technical roles too.

Closed loop: drive optimization with attribution results

Attribution is not the end; optimization is. Attribution data must be able to drive these decisions:

• code-review gross margin is negative → switch to a cheaper model / limit context / bundle into higher tiers.
• A Top user's cost per capita is $235 but they only pay $20 → tiered throttling / upsell to Enterprise.
• RAG requests stuff 15K tokens but use only 2K → optimize retrieval recall, trim context, and save 60% directly.
• summarize-v2 costs 40% more than v1 with no obvious quality gain → roll back, or keep tuning the prompt.

This is the closed loop of attribution: tag → meter → attribute → dashboard → decide → optimize → observe again. With each cycle, cost gets a bit more under control.

Summary

1. AI cost is variable cost + high variance; without attribution it is just an opaque ledger. The tail 1% of requests can eat half the bill, and mean-based thinking completely fails here.
2. Tagging is the starting point of attribution. Every call must carry metadata like feature/user/session/experiment, enforced through a unified gateway, otherwise the data will always have holes.
3. Record input and output tokens separately, compute cost by per-model coefficients, and pre-aggregate across multiple dimensions. Keep the raw events in full; the rollup tables serve queries — this is the foundation for second-level dashboard response.
4. The three layers of attribution answer three questions: feature (which one burns), user (who burns), request type (which kind is expensive). Unit economics converts them into "is it profitable," driving business decisions.
5. Anomaly detection prevents abuse and prevents bugs from burning money. User-level, feature-level, and global-level alerts; combine absolute and relative values; don't wait until the month-end bill to find out.

Bottom line: AI cost without attribution is just a cost; AI cost with attribution is leverage. Break the opaque ledger into tagged atomic data, and you will know what to cut, what price to raise, which prompt to optimize — AI cost goes from "an out-of-control black box" to "an optimizable financial dashboard," and that step is the watershed between burning money and making money.