What's new.

Service updates, new optimization techniques we've added to the playbook, and platform notes. Updated when something actually ships ; not on a marketing cadence.

2026-04 · Q2 playbook refresh

• Added Anthropic extended prompt caching (1-hour TTL) to the playbook. ~90% discount on cache-read tokens for steady traffic.
• Added OpenAI Batch API routing for non-realtime workloads. 50% discount on eligible jobs.
• Updated cache-read token tracking in baselines ; these are billed separately and were quietly inflating "input token" lines on some Anthropic invoices.

2026-03 · Provider arbitrage

• Added provider arbitrage to recommended optimizations: equivalent-quality models priced 30–60% lower on Bedrock / Vertex / OpenRouter for high-volume endpoints.
• Built quality-equivalence A/B harness with LLM-as-judge scoring + human spot-checks.

2026-02 · Semantic cache tuning

• Added embedding-model selection guide (`text-embedding-3-small` vs Cohere `embed-v3` vs Voyage) for semantic cache layers.
• Tuned default similarity thresholds per workload type (RAG retrieval, chat, classification).

2026-01 · Anomaly detection

• Rolled out cost anomaly detector in the Operator Console ; flags token leaks, mis-routed traffic, runaway loops within hours instead of monthly invoices.

Older

Pre-2026 updates aren't published. Email us if you want the full history.

How to read this page

Entries here are changes to the playbook we actually run on customer engagements, not product announcements. When a technique appears in this list it means it has been used on real traffic, measured against an invoice, and is now part of the default sequence — not that it looked promising in a provider's launch post. Techniques also get removed when they stop earning their place; provider pricing moves often enough that an optimization worth doing last year can become noise.

Why the updates cluster around provider changes

Most entries trace back to something a provider shipped: a new cache TTL, a batch endpoint, a cheaper model tier, or a billing line that changed shape. That is the nature of the work. The cost structure of an LLM workload is largely set by the provider's price list, and the job is to notice a change early and work out which customers it applies to before the next invoice closes. The cache-read tracking change in the Q2 refresh is the clearest example — nothing about the workloads changed, but the way the tokens were billed did, and baselines built before that point were quietly overstating input costs.

What isn't published here

Customer-specific findings, engagement outcomes, and anything that would identify a workload stay out of this list. Marketing-site copy changes and internal tooling are also omitted; if it did not change what we would recommend to a customer, it does not belong in a changelog. Research articles get their own dates on the research pages rather than an entry here.