Rate limits and caps

Request rates

Exceeding a rate returns 429 rate_limit_error with a Retry-After header holding the seconds to wait. Limits are per key, so parallel workloads can spread across keys. Need more sustained throughput? Ask via the support page.

Per-request limits

• Output tokens: default 1,024 when max_tokens is omitted, ceiling 32,768 per request.
• Context window: per model, listed in the catalog and in GET /models. Prompts over the window are rejected by the engine and not charged.
• Batch input files: 1,000 lines, 5 MB.

Spend controls

• Prepaid balance. The hard stop. A request whose worst case exceeds your balance returns 402 before anything is sent.
• Per-key monthly caps. Set a monthly USD cap on any key; the key returns 429 monthly_key_limit_exceededfor the rest of the month once it's reached, without affecting your other keys.
• Auto top-up. Optional, on the Credits page: refill by a fixed amount when the balance drops below a threshold you choose.

Balance holds

While a request is in flight, its worst case (estimated prompt plus max_tokens, at the model's rates) is held from your balance, then settled to actual usage. Highly concurrent traffic therefore needs headroom: fifty simultaneous requests need fifty worst cases available at once, even though the settled cost will be far lower.