Skip to main content

Billing & Usage

The Turing Platform recalculates every bill on the backend for each LLM call—it does not use the cost header returned by upstream providers. Instead, it sums per-category unit prices based on the model and usage breakdown. This page is the shared billing reference for all endpoints: pricing dimensions, calculation order, how to read usage from responses, and how to reconcile via the trace API.

Reading order

If you just want to know how much a request cost, jump straight to Reading Usage from Responses and Reconciliation API. To understand what each unit price represents and why fallback pricing sometimes applies, see Billing Dimensions Quick Reference and Billing Algorithm.

Billing Dimensions Quick Reference​

Every LLM model's pricing is described by LLMModelEngineInfo, covering 13 billable dimensions. When a request arrives, the platform computes each category's cost as counter (quantity returned in usage) × unit price.

Price fieldUnitCorresponding counterFallback
input_cost_per_tokenPer million tokensprompt_tokens (text + unclassified tokens after subtracting other categories)Required — no fallback
output_cost_per_tokenPer million tokenscompletion_tokens (text + unclassified tokens after subtracting other categories)Required — no fallback
input_cost_per_cached_tokenPer million tokensprompt_tokens_details.cached_tokens (cache read)→ input_cost_per_token
cache_creation_input_token_costPer million tokensprompt_tokens_details.cache_creation_token_details.ephemeral_5m_input_tokens (5-minute cache write) or flat cache_creation_tokens→ input_cost_per_token
cache_creation_input_token_cost_above_1hrPer million tokensprompt_tokens_details.cache_creation_token_details.ephemeral_1h_input_tokens (1-hour cache write)→ cache_creation_input_token_cost → input_cost_per_token
audio_input_cost_per_tokenPer million tokensprompt_tokens_details.audio_tokens→ input_cost_per_token
audio_output_cost_per_tokenPer million tokenscompletion_tokens_details.audio_tokens→ output_cost_per_token
image_input_cost_per_tokenPer million tokensprompt_tokens_details.image_tokens→ input_cost_per_token
image_output_cost_per_tokenPer million tokenscompletion_tokens_details.image_tokens→ output_cost_per_token
video_input_cost_per_tokenPer million tokensprompt_tokens_details.video_tokens→ input_cost_per_token
output_cost_per_reasoning_tokenPer million tokenscompletion_tokens_details.reasoning_tokens (reasoning tokens)→ output_cost_per_token
cost_per_web_search_requestPer thousand requestsprompt_tokens_details.web_search_requestsNo fallback — billed as 0 if not configured
warning

cost_per_web_search_request is the only dimension with no fallback. If a newly integrated model supports web search but this field is left unconfigured, the platform will silently skip billing for it. Always verify this field when onboarding a new model.

Billing Algorithm (by Category)​

Each call's bill is computed by _calculate_fixed_cost, with input and output sides accumulated independently. The algorithm processes categories in the order below, "carving out" each category from the total and pricing it at its unit rate; remaining text/unclassified tokens are covered by the base price.

Input Side​

  1. Audio (audio_tokens × audio_input_cost_per_token or fallback)
  2. Cache read (cached_tokens × input_cost_per_cached_token or fallback)
  3. Cache write
    • If 5m / 1h buckets are present (Anthropic new format):
      • ephemeral_5m_input_tokens × cache_creation_input_token_cost or fallback
      • ephemeral_1h_input_tokens × cache_creation_input_token_cost_above_1hr or fallback
    • Otherwise: cache_creation_tokens × cache_creation_input_token_cost or fallback
  4. Video (video_tokens × video_input_cost_per_token or fallback)
  5. Image (image_tokens × image_input_cost_per_token or fallback)
  6. Web search (web_search_requests × cost_per_web_search_request; 0 if not configured)
  7. Text (text_tokens × input_cost_per_token)
  8. Unclassified catch-all: remaining = prompt_tokens - (text + audio + cached + cache_creation + video + image). If > 0, billed at input_cost_per_token. Some upstream providers (e.g., Gemini) return text_tokens that do not cover all categories; this catch-all ensures the full total is billed.

Output Side​

  1. Image (image_tokens × image_output_cost_per_token or fallback)
  2. Audio (audio_tokens × audio_output_cost_per_token or fallback)
  3. Reasoning tokens (reasoning_tokens × output_cost_per_reasoning_token or fallback)
  4. Text (text_tokens × output_cost_per_token)
  5. Unclassified catch-all: remaining = completion_tokens - (text + audio + reasoning + image). If > 0, billed at output_cost_per_token.
Fallback does not mean underbilling

Many models charge the same rate for audio, image, and text (the upstream simply uses one tier). In that case audio_input_cost_per_token is null and the algorithm falls back to input_cost_per_token—this is the correct bill, not a missed charge. Only an unconfigured cost_per_web_search_request results in a genuine silent zero.

Tiered Pricing​

Some models use input-token-based pricing tiers, for example:

  • Gemini 2.5 Pro / 3 Pro / 3.1 Pro: ≤ 200K tier vs. > 200K tier, roughly double the unit price.
  • Claude Sonnet 4 / 4.5: ≤ 200K vs. > 200K; input, output, and cache prices all double.
  • Zhipu GLM / Doubao / Qwen3 series: multi-tier combinations (input × output two-dimensional).

Tier selection algorithm (see PricingTier.matches):

  • Tiers are evaluated in tiers order; the first match's unit prices override the base prices.
  • Two matching modes:
    • INPUT_OUTPUT: min_input_tokens ≤ input_tokens < max_input_tokens AND min_output_tokens ≤ output_tokens < max_output_tokens must both hold.
    • CONTEXT_WINDOW: input + output falls within a window range (less common).
  • If no tier matches, fallback_*_cost_per_token is used.
Which tier applies depends on output too

Tiers are evaluated against this request's input + output tokens, not a historical cumulative. A request with 190K input tokens but 20K output tokens from the model will still match the "input less than 200K" tier.

Time-of-Day Pricing (Peak / Off-Peak)​

Some models charge a different rate depending on when the call settles -- cheaper off-peak, dearer at peak. Today this applies to the dated DeepSeek V4 releases, and because they sit on different upstreams their windows differ too:

ModelUpstreamInput (peak/off-peak)Cached input (peak/off-peak)Output (peak/off-peak)
deepseek-v4-flash-0731DeepSeek official3 / 1.50.1 / 0.059 / 4.5
aliyun/deepseek-v4-flash-0731Aliyun Model Studio3 / 1.50.3 / 0.159 / 4.5
deepseek-v4-pro-0813Aliyun Model Studio9 / 4.50.9 / 0.4527 / 13.5

CNY per 1M tokens. The undated deepseek-v4-flash / deepseek-v4-pro keep a single flat rate.

Window definitions, per upstream:

  • DeepSeek official (deepseek-v4-flash-0731): peak is only 09:00-12:00 and 14:00-18:00 UTC+8; every other hour is off-peak.
  • Aliyun Model Studio (aliyun/deepseek-v4-flash-0731, deepseek-v4-pro-0813): off-peak is 22:00 to 08:00 the next day, UTC+8; every other hour is peak.

Both sides' intervals are start-inclusive and end-exclusive -- on the Aliyun side 22:00:00 is already off-peak and 08:00:00 is already peak.

One snapshot, two rates

deepseek-v4-flash-0731 and aliyun/deepseek-v4-flash-0731 are two routes to the same model snapshot. Input and output rates are identical; they differ on the cache-hit rate (0.1 vs 0.3) and on which hours count as off-peak. Cache-heavy traffic, or traffic concentrated in 12:00-14:00 and 18:00-22:00, is cheaper on the official route; late-night batches cost the same either way.

How it is applied:

  • The whole call is priced at its own settlement instant (the span's end time). A call is never split across the boundary: one that starts at 21:59 and ends at 22:05 is billed entirely off-peak.
  • Per-call tool prices (for example web_search) are not on the schedule.
  • Tiered pricing and time-of-day pricing never apply to the same model.
Checking which rate you got

Set turing_options.include_service_usages: true, read service_usages[].time_stats.end_timestamp, convert it to UTC+8 to see which window it lands in, and compare against bill_amount.

Reading Usage from Responses​

All LLM endpoints support turing_options.include_service_usages: true. When enabled, the response includes an additional service_usages array.

curl $TURING_BASE_URL/chat/completions \
  -H "Authorization: Bearer $TURING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "turing/claude-sonnet-5",
    "messages": [{"role": "user", "content": "hello"}],
    "turing_options": {"include_service_usages": true}
  }'

Response excerpt:

{
  "choices": [...],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 5,
    "prompt_tokens_details": {
      "text_tokens": 11,
      "cached_tokens": 0
    }
  },
  "service_usages": [
    {
      "deployment_name": "turing/claude-sonnet-5",
      "service_name": "claude-sonnet-5",
      "service_provider": "turing",
      "usage_unit": "tokens",
      "usage_amount": 16,
      "usage_detail": {
        "usage_type": "tokens",
        "prompt_tokens": 11,
        "completion_tokens": 5,
        "prompt_tokens_details": { "text_tokens": 11 },
        "completion_tokens_details": { "text_tokens": 5 }
      },
      "bill_amount": 0.00018,
      "bill_currency": "USD",
      "time_stats": { "duration_ms": 812 }
    }
  ]
}

Key fields:

  • usage_detail.prompt_tokens_details / completion_tokens_details: Token breakdown for each billing category. Fields correspond one-to-one with the "Corresponding counter" column in the Billing Dimensions Quick Reference.
  • bill_amount: The amount recalculated by the platform using the algorithm above. This is the authoritative bill.
  • bill_currency: USD or CNY, determined by the model configuration (see Currency & Region).
  • time_stats: Latency statistics (optional).

Multiple service calls within a single request (e.g., LLM + web search + preprocessing model triggered simultaneously) produce multiple records, grouped by service.

Reconciliation API​

Persistent billing records can be retrieved via the trace API for offline reconciliation, usage reporting, and cost attribution.

Endpoint: GET /api/v1/trace/{trace_id}/billing

The trace_id is returned in the response header x-turing-trace-id and is also recorded in the log for every call.

curl -X GET $TURING_BASE_URL/trace/trace_abcdef/billing \
  -H "Authorization: Bearer $TURING_API_KEY"

The response body is list[ServiceUsage], with fields identical to those in the inline service_usages return above. The difference is:

  • The reconciliation API provides persistent storage and can be queried without the client having enabled the include_service_usages flag.
  • The query time window aligns with the backend tracing policy. For critical workloads, wait at least 2 seconds after a call completes before querying (to ensure the write has finished).
bill_amount is the authoritative bill

The x-litellm-response-cost header returned by upstream providers is used only for internal observability and is not the customer bill. ServiceUsage.bill_amount is the only billing figure exposed externally.

Provider-Specific Notes​

Claude: Explicit Cache Control​

Anthropic's cache is opt-in by the caller: caching only occurs when a cache_control: {"type": "ephemeral"} annotation is added to a content block in messages or system. Subsequent cache hits are billed at input_cost_per_cached_token.

  • Cache writes cost more than base input (Sonnet 4.6: write $3.75/M vs. read $0.3/M vs. base input $3/M). A single write is not cost-effective; breakeven requires at least 2 cache hits.
  • Starting with 4.6 / 4.7, Anthropic added a 1-hour TTL option at an even higher write price (cache_creation_input_token_cost_above_1hr), intended for long-lived context such as multi-turn agent sessions.
  • See Prompt Caching (Claude) for details.

Claude: web_search Tool​

Claude's web search on Vertex is enabled via tools: [{"type": "web_search_20250305", ...}]. The search count is reported in usage.server_tool_use.web_search_requests in the response; the platform internally lifts this to prompt_tokens_details.web_search_requests and bills at cost_per_web_search_request = $10/1K.

  • Applicable models are indicated by the Web Search / built-in tool marker in Model List → Claude.
  • Pricing is identical for both /v1/messages and /v1/chat/completions; usage.server_tool_use.web_search_requests is visible in both responses.
  • The max_uses tool parameter caps the number of search calls. Sonnet on complex tasks has been observed making 20+ searches per turn; explicitly set max_uses in production.
  • See Model Built-in Web Search for details.

Gemini: googleSearch Grounding​

Gemini's grounding on Vertex is enabled via tools: [{"googleSearch": {}}]. The search count is reported in the top-level vertex_ai_grounding_metadata array in the response (one entry per grounding call); the platform internally lifts this to prompt_tokens_details.web_search_requests and bills at cost_per_web_search_request = $14/1K.

  • Pro / Flash / Lite all share the uniform rate of $14/1K.
  • Tool parameters such as max_uses can further limit call counts to control costs.
  • See Model Built-in Web Search for details.

OpenAI / DeepSeek / Gemini / Doubao: Implicit Automatic Prompt Caching​

Most mainstream non-Claude models (GPT-4o / GPT-5.x, DeepSeek V4, Gemini 2.5/3.x Pro, Doubao 1.6+, etc.) automatically cache common prefixes across requests on the upstream side—no client-side toggle required. Cache hits appear directly in usage.prompt_tokens_details.cached_tokens and are billed at input_cost_per_cached_token (approximately 10% for OpenAI 5.x series, 25–50% for 4.x; approximately 20% for DeepSeek / Gemini).

  • No cache_control parameter is needed, and there are no write costs.
  • Minimum hit length varies by provider (256 tokens for DeepSeek; ~1,024 tokens for OpenAI / Gemini).
  • TTL is managed dynamically by the upstream platform and is not controllable by the client.
  • For the full model list and unit prices, see Prompt Caching.

Reasoning Tokens​

Reasoning tokens generated by reasoning models (OpenAI o1/o3/o4/GPT-5 reasoning, Claude adaptive thinking, DeepSeek R1, Qwen thinking variants) are billed as output tokens by default and count against max_tokens together with visible output.

  • If the model has output_cost_per_reasoning_token configured (most OpenAI reasoning models and DeepSeek R1), the reasoning portion is billed at that dedicated rate; otherwise it falls back to output_cost_per_token.
  • Under Claude adaptive thinking, thinking.display: "omitted" does not save tokens—it only controls whether the reasoning text is echoed back in the response (see Reasoning Model Configuration).

Currency & Region​

bill_currency has only two values:

CurrencyCovered providers
USDOpenAI, Azure, Anthropic, Google Vertex, AWS Bedrock, and other global / international providers
CNYAlibaba Qwen, Zhipu GLM, ByteDance Doubao, Tencent Hunyuan, iFlytek Spark, Baidu, China Mobile Jiutian, and other domestic Chinese providers

The currency is determined by the currency field in the model configuration; callers do not need to specify it explicitly. Cross-currency bill aggregation is converted to USD at the daily exchange rate for cumulative totals (see the budget-related APIs).

Common Misconceptions​

  • "Claude caching always saves money": Write cost is approximately 1.25× the base input price; read cost is approximately 0.1×. If a context segment is read only once and then discarded, it actually costs more. Rule of thumb: caching is worthwhile only when the same context is hit by at least 2 requests.
  • "Fallback to text price = missed billing": Most providers charge the same rate for audio, image, and text, so fallback is expected behavior. Only cost_per_web_search_request is a true silent zero.
  • "Web search costs are unbounded": A single conversation may trigger multiple searches. Without a max_uses limit, Sonnet on complex tasks has been observed making 20+ searches. Always set max_uses in production.
  • "The upstream cost header is the bill": The Turing backend calculates its own bill and does not trust upstream headers. If bill_amount deviates from expectations, start by checking the counters in usage_detail.

Further Reading​