AI Operating Gross Margin Reconciliation: Versioned COGS, Invoice Calibration, and Revenue Allocation

Even when a team knows how many tokens each user consumed, it may still not know whether the Pro plan is profitable. The reason is that model vendors price separately by input, output, cache, region, and model version; self-deployed inference adds GPU, storage, and shared gateway costs; and revenue is affected by discounts, refunds, credits, payment terms, and what the plan bundle includes. Treating "this month's vendor bill divided by subscription revenue" as gross margin hides losing cohorts and cannot explain the change before and after a price adjustment.

AI Cost Attribution already covers request tagging, token metering, feature/user cost splitting, and basic unit economics, and Token Cost Optimization discusses how to reduce costs. This article does not repeat "who the cost belongs to" or simple gross margin dashboards; it only covers versioned COGS, vendor invoice calibration, and revenue allocation policies. It provides signals for operating decisions and does not replace accounting policies—what counts as COGS, when to recognize revenue, and how to handle taxes must be defined by Finance and written into a metric policy.

Rate cards must carry effective intervals

Do not use a hot-updated model -> price map to recompute all history. Each rate card record should store provider, model/region/modality, meters such as input/output/cache/tool, unit, currency, effectiveFrom/effectiveTo, contract discounts, source document hash, approver, and version. Usage events match a unique rate ID by occurredAt, and that ID is written into the cost entry.

rateCardId: provider-a-model-x-2026-07
currency: USD
effectiveFrom: 2026-07-01T00:00:00Z
meters:
  input_token:  {unit: 1000000_token, rateRef: contract-line-12}
  output_token: {unit: 1000000_token, rateRef: contract-line-13}
  cached_token: {unit: 1000000_token, rateRef: contract-line-14}
sourceSha256: 6b3a...
status: approved

The example intentionally omits amounts because public pricing, enterprise contracts, and regional rates all change. Vendor responses first form a PROVISIONAL cost; when the monthly invoice or export arrives, an adjustment is appended and the status moves to RECONCILED—the original estimate cannot be overwritten. Foreign currency must store the original amount, the FX source used, the rate, and the date; later FX changes should not rewrite historical decisions.

Self-deployed and shared costs keep explainable unallocated items

GPU inference is attributed by the pool's actual accelerator seconds, VRAM, or a validated allocation unit—not simply spread by request count. Kubernetes CPU, memory, and storage can be mapped to services using OpenCost conventions and then allocated through operation IDs; FOCUS can standardize cloud billing fields. Idle capacity, the control plane, and shared costs that cannot be traced go into an explicit unallocated bucket rather than being forced onto the customer with the most traffic.

Every choice in LLM Gateway Routing Policy should record the rate card, estimated cost, and actual usage, but online routing can only use approved estimates—it cannot wait for the end-of-month invoice. The budget model must also cover retrieval, reranking, voice, images, third-party tools, and failed retries; counting only the main model makes complex agents look artificially cheap.

Maintain revenue definitions and cost granularity separately

The revenue ledger stores independent facts for subscription, usage charge, credit redemption, discount, refund, and adjustment. How fixed subscription revenue is allocated to a day, account, or feature is determined by a versioned allocation policy; do not assign a one-time cash receipt entirely to the day's requests. For operating gross margin, one auditable definition is "net revenue allocated by policy for the period minus variable COGS for the period," but whether processor fees, support labor, taxes, and R&D are included must be explicitly configured.

Each derived margin row stores the revenue policy, cost policy, rate card IDs, input ledger water level, and calculation version. The same inputs recompute to the same result; policy changes create new views instead of modifying the original revenue/usage. Plan comparisons aggregate by product, plan version, cohort, and task family, while account/request-level data stays in the data warehouse to avoid label explosion in monitoring.

Alert on margin, samples, and data completeness together

The dashboard should at least show cost per successful task, net revenue, variable COGS, gross margin amount, gross margin ratio, unallocated share, provisional share, and reconciliation variance. The gross margin ratio is unstable when net revenue is near zero, so alerts need a minimum revenue/sample gate and must check absolute loss at the same time. Thresholds come from this product's goals and historical variance; there is no universal safe percentage.

- alert: AiPlanMarginDegradation
  expr: |
    (ai_plan_net_revenue_6h
      > on(plan_version) group_left ai_plan_min_revenue_threshold)
    and on(plan_version)
    (ai_plan_gross_margin_ratio_6h
      < on(plan_version) group_left ai_plan_margin_floor)
    and on(plan_version)
    (ai_margin_data_completeness_ratio_6h > 0.98)
  for: 30m

Alerts attach plan/rate/policy versions and the top cost drivers. If completeness drops, first report "incomplete data" instead of asserting a business loss; vendor usage delays, missing FX, and a stuck exporter can all fake a margin improvement. AI Product Pricing Models provides the commercial choices, and the margin signal only provides traceable feedback for those choices.

Validate with boundary billing periods and vendor invoices

Test one second before and after a rate takes effect, cached tokens, canceled requests, multi-model fallback, multiple calls in one task, vendor rebilling, refunds, credit offsets, foreign currency, and long tasks crossing months. Hand-compute entries for a fixed sample, then reconcile against the provider invoice, internal usage net amount, revenue ledger, and margin row; any unknown rate or overlapping effective period blocks that slice from reporting.

Run a new calculator in dual-compute mode for at least one full billing period before it drives automatic repricing. Classify differences as rate, usage, allocation, FX, or external invoice, and keep the sign-off. Rollback only switches back to the previous calculation policy and dashboard view; original entries are not deleted—an incorrect rate appends a correction, recomputes the derived snapshot, and marks it superseded. The value of margin control is not a pretty percentage but the ability to trace every change back to usage, rates, revenue policy, and data completeness.

Further reading

• AI Cost Attribution
• Token Cost Optimization
• AI Product Pricing Models
• LLM Gateway Routing Policy