LLM cost dashboards.

Operations guide · July 12, 2026

By the LLM CFO team

An LLM cost dashboard is a set of views that connect spend to your product decisions: which features use which models, how much they cost, and where the waste is. A good dashboard answers "what changed and why?" A bad one shows you only totals, which tells you nothing until the bill arrives.

Why most dashboards fail

Teams usually build dashboards that are too broad and too shallow. They create a single "total spend this month" number, or they show every dimension at once without filtering. The result is visual noise that doesn't drive action. The fix is to start narrow: pick one question (like "which feature is costing the most?"), show the ranked list of answers, and drill into the top three results. Let tagging and data quality come first. Dashboards are secondary.

The five views that matter

Start with these five. They cover the most common cost drivers and are directly actionable.

View 1: Spend by feature

• Purpose. Identify which endpoints, workflows, or user-facing features are consuming the most budget.
• Telemetry fields required. Feature name (or endpoint path), total cost per request, request count, average cost per request, date/hour.
• Display. Ranked bar chart or table. Top 10 features by total spend; include count and unit cost. Example: "search_embeddings: $4,200/month, 120k requests, $0.035/req".
• Alert to pair with it. Daily spend threshold by feature. Example: alert if a feature's daily spend exceeds its rolling 7-day average by 50%.

View 2: Spend by model

• Purpose. Track which model tier is actually being used, and catch unintended model drift (e.g., a cheap endpoint now defaulting to a frontier model).
• Telemetry fields required. Model name, provider, total cost, input tokens, output tokens, request count, date/hour.
• Display. Stacked bar chart or waterfall. Show cost by model over time. Include a column for "request mix by model" so you can spot when an endpoint shifts from gpt-4-turbo to gpt-4o or claude-opus to claude-haiku.
• Alert to pair with it. Alert on unexpected model mix shift. Example: if a feature that historically used model X suddenly logs 80%+ requests to model Y, flag it for review.

View 3: Spend by customer or workspace

• Purpose. Understand profitability per customer. Identify runaway accounts and enforce quota controls.
• Telemetry fields required. Customer ID or workspace ID, organization name (optional), total cost, request count, plan type, date/hour.
• Display. Ranked table. Top 20 customers by spend. Include a column for spend-to-revenue ratio if you have it (even if rough), and a column for requests-per-customer so you can see utilization patterns.
• Alert to pair with it. Quota alert. Example: alert if a customer's daily spend exceeds their monthly budget / 30, or if any customer spends 3x more in one week than the previous month average.

View 4: Token trends (input, output, cache-read)

• Purpose. Detect prompt length growth and verbose output drift. Also track the value of prompt caching and cache-read discounts.
• Telemetry fields required. Total input tokens, total output tokens, total cache-read tokens, request count, average input length per request, average output length per request, date/hour.
• Display. Line chart over time, separate series for input, output, and cache-read. Include a rolling 7-day or 30-day average so spikes are obvious. Also show cost impact: input/output cost per 1M tokens and cache-read discount applied.
• Alert to pair with it. Prompt length regression. Example: alert if average input tokens per request increases 20% week-over-week. Also alert if cache-hit rate drops below historical baseline.

View 5: Cache-hit and retry rates

• Purpose. Track two of the easiest places to find waste: failed retries and low cache utilization.
• Telemetry fields required. Total requests, requests with retries, retry count, cache-eligible requests, cache hits, cache-read tokens, date/hour.
• Display. Two side-by-side metrics or cards. (1) Cache-hit rate: percentage of requests that hit cache, cost savings from cache-read discount vs. full-price tokens. (2) Retry rate: percentage of requests that failed and were retried, total cost of failed work, average retry count.
• Alert to pair with it. Retry storm or agent loop. Example: alert if retry rate exceeds 5%, or if any single request has >10 retries. Also alert if cache-hit rate drops below 30% for features that should have high cache utilization.

Telemetry schema: where to get these fields

OpenTelemetry's GenAI semantic conventions define the fields above and map them to both provider-native formats and observability backends. Use them as your standard because they allow you to switch from Langfuse to Helicone to your own warehouse without rebuilding your telemetry layer. The key fields are: gen_ai.request.model.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.cache_read_tokens, gen_ai.request.frequency_penalty, and custom attributes for your feature name and customer ID.

How to emit this data

• From application code. Log these fields on every LLM request. Use your observability SDK (OpenTelemetry, LangChain, LiteLLM) to emit them to Langfuse, Helicone, or directly to your backend.
• From a gateway. If you route all LLM requests through a gateway (LiteLLM, Proxy, Vercel AI SDK), the gateway can log these fields automatically, so you don't have to instrument every callsite.
• From provider billing. For cost reconciliation only. Provider data is truth, but it arrives 24–72 hours late. Pair it with real-time application telemetry so you can alert and act the same day.

Common dashboard mistakes and how to avoid them

• Overbuilding the dashboard layer before tagging. Teams often create 20 different views before they have accurate feature or customer tags in their data. Result: garbage in, garbage out. Instead, start with one question and make sure your data labels are correct. Build dashboards incrementally as your tagging improves.
• Underbuilding the tagging layer. Without good tags, your dashboards show you nothing actionable. Spend three days getting feature and customer tags right before you spend a day building the visualization.
• Vanity totals without segmentation. "We spent $150k on LLMs this month" is not useful. You need to know whether that's $100k on one customer, $30k on search, and $20k on customer support. Segment everything.
• Ignoring reconciliation. Most teams assume their application telemetry matches the provider bill. It usually doesn't, at least initially. Set up a monthly or weekly reconciliation view that compares your estimated cost (from application logs) to the provider's reported cost. The delta tells you where your instrumentation is broken.
• Not pairing alerts with dashboards. A dashboard is a after-the-fact report. Alerts let you act in real time. For every dashboard view, define at least one alert threshold and set it up in your monitoring tool (PagerDuty, Grafana, etc.).

Where to build these views

You have three options, depending on your infrastructure. (1) Observability platforms like Langfuse and Helicone offer built-in dashboards for these exact views; use their UI directly. (2) Warehouse BI tools like Grafana, Tableau, or Metabase let you query your own data; these give you the most control but require a warehouse layer. (3) Custom internal dashboards: build them if you have a data team and specific needs no vendor covers. Start with option 1 or 2 before building custom.

Related

• LLM cost monitoring: what to track and how to control it
• GenAI telemetry schema: the fields that matter
• LiteLLM vs Helicone vs LangFuse: where to instrument