Skip to main content

Rate Limits

The Turing Platform enforces multi-dimensional rate limits on every API Key. Exceeding a limit returns HTTP 429 with a business error code. This page explains how to identify which limit was hit, how to back off, and how to avoid hitting limits in the first place.

Dimensions​

DimensionError CodeTriggerScope
User RPM1111 USER_RPM_LIMIT_EXCEEDEDRequests per minute exceededSingle API Key
User TPM1112 USER_TPM_LIMIT_EXCEEDEDTokens per minute exceededSingle API Key
Client RPM1114 CLIENT_RPM_LIMIT_EXCEEDEDRPM exceeded for the same X-Client-IdAggregated per client
Client TPM1113 CLIENT_TPM_LIMIT_EXCEEDEDSameSame
General Rate1104 RATE_LIMIT_EXCEEDEDPlatform-level aggregate throttleGlobal protection
Service Rate1115 SERVICE_RATE_LIMIT_EXCEEDEDService-level throttleWithin service
Provider Rate1122 PROVIDER_RATE_LIMIT_EXCEEDEDUpstream vendor throttle (especially common with Gemini)Vendor side
Budget Exceeded1110 USER_EXPENSE_OVER_BUDGETInsufficient balance / spending cap reachedAPI Key
Free Quota Exhausted1102 USER_LEFT_TOKENS_EXCEEDFree quota used upFree-tier users

All of these use HTTP 429 (except 1102, which uses 400). Distinguish them by the business code field.

Error Response Example​

{
  "code": 1112,
  "message": "User TPM limit exceed",
  "data": {"limit": 100000, "window": "1m"},
  "trace_id": "tur_..."
}

How Clients Should Respond​

Basic Strategy: Exponential Backoff​

import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="your-api-key",
    base_url="https://live-turing.cn.llm.tcljd.com/api/v1",
)

def call_with_backoff(max_retries=4):
    delay = 1.0
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="turing/gpt-5.4-mini",
                messages=[{"role": "user", "content": "Hi"}],
            )
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            time.sleep(delay)
            delay *= 2  # 1, 2, 4, 8 ...

Prefer Platform-Native Retry / Fallback​

Rather than implementing complex client-side backoff, let the platform handle it — turing_options supports max_retries and fallbacks:

client.chat.completions.create(
    model="turing/gpt-5.4-mini",
    messages=[...],
    extra_body={
        "turing_options": {
            "max_retries": 2,
            "fallbacks": ["turing/claude-sonnet-5", "turing/gemini-3.1-pro-latest"]
        }
    }
)

The platform's backoff curve: 1s → 2s → 4s … up to 60s. This applies only to retriable errors such as 429, 5xx, and timeouts — 401 and 400 errors are not retried. See Timeouts, Retries, and Fallbacks for details.

Identifying Which Limit Was Hit​

codeWho is throttledBest response
1111 / 1112Your API KeyUpgrade quota / reduce request rate
1113 / 1114Same client (aggregated across keys)Same, or split into separate client groups
1104 / 1115Platform layerShort backoff + retry
1122Upstream vendor (Gemini/OpenAI/…)Enable fallback to switch providers, or wait for recovery
1110Balance / budgetTop up / raise spending cap

Viewing Your Current Quota​

Use the Turing Platform Portal or the /v1/admin/usage endpoint (requires additional permissions). For routine troubleshooting, check the balance, quota, and usage columns in the Portal.

Gemini-Specific: Provisioned Throughput​

If you encounter persistent 1122 vendor rate limits on Gemini models, consider purchasing Provisioned Throughput. See Troubleshooting / Gemini 429 Rate Limits and PT.

Best Practices​

  1. Always enable retries + fallbacks in production rather than failing hard
  2. Include X-Client-Request-Id on every request so 429s can be traced (see Request Tracing)
  3. Spread large batch jobs over time: sending 10,000 requests in 1 minute is worse than 1,000 per minute over 10 minutes
  4. Monitor your 429 rate: a sustained rate above 1% means it's time to increase your quota
  5. Streaming requests count output tokens for TPM: longer responses consume more quota — control max_tokens

FAQ​

  • "I only sent 100 requests, why am I getting 429?" → Check the business code: it's likely 1122 from Gemini (vendor-side), not your own quota limit.
  • Fallback is also 429 → Configure your two fallbacks to use models from different vendors (e.g., ["turing/claude-sonnet-5", "turing/gpt-5.4-mini"]).
  • Are tokens counted twice for streaming requests? → No. However, tokens already sent before a stream is interrupted are still billed.
  • How do I permanently avoid 1110? → Upgrade your account to post-paid billing or request whitelist status in the Portal (contact the platform).

See Also​