Skip to main content

Token Counting / Count Tokens

Estimate the number of input tokens in a request before making an actual model call — useful for cost estimation or making context truncation decisions upfront. No charges, no fallback triggered, no quota consumed.

When to Use​

  • Confirm you haven't exceeded the context window before constructing a long prompt
  • Decide how many chunks to include in a RAG pipeline
  • Display an estimated cost to end users
  • Run batch budget calculations in CI pipelines or scripts

Usage (Anthropic Messages)​

Only the Anthropic Messages protocol has a native count_tokens endpoint. (For equivalent Turing paths under other protocols, see the "Other Protocols" section below.)

from anthropic import Anthropic

client = Anthropic(
    base_url="https://live-turing.cn.llm.tcljd.com/api/v1",
    auth_token="your-api-key",
)

result = client.messages.count_tokens(
    model="turing/claude-sonnet-5",
    messages=[{"role": "user", "content": "Estimate the number of input tokens in this request"}],
)
print(result.input_tokens)  # -> integer
curl $TURING_BASE_URL/messages/count_tokens \
  -H "Authorization: Bearer $TURING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "turing/claude-sonnet-5",
    "messages": [{"role": "user", "content": "Estimate the number of input tokens in this request"}]
  }'

Complete request / response schema: /api/count-message-tokens

Supported Fields​

Largely the same as a standard /messages request, but excludes generation parameters such as max_tokens and temperature:

FieldPurpose
modelTokenizer to use for the specified model
messagesConversation history
systemSystem prompt
toolsTool definitions (counted toward tokens)
tool_choiceTool selection strategy
thinkingToken estimate when reasoning is enabled
cache_controlTop-level cache markers (affects the count)

Estimation for Other Protocols​

The OpenAI Chat Completions / Responses protocol does not have a corresponding count_tokens endpoint. Common approaches:

  1. Local tokenizer (recommended): Use the tiktoken package to tokenize locally for OpenAI-family models 
    import tiktoken
    enc = tiktoken.encoding_for_model("gpt-4")
    print(len(enc.encode("Text to estimate")))
  2. Send a request with max_tokens=1 and read usage.prompt_tokens from the response (incurs a small charge)

Notes​

  • Token counts are estimates: the exact count for image / audio / video inputs may differ slightly from actual inference
  • When cache_control is used, input_tokens returns the total count assuming the expected cache hit (still the sum of non-cached + cache read + cache write tokens)
  • Estimates do not count against quota or budget

See also​