Rate limits

Rate limits keep the API fast and fair for everyone. Requests are limited per minute on a per-key basis. The exact numbers depend on your plan — you can see your current limits and live usage in your dashboard.

What happens when you exceed a limit

Requests over the limit are rejected with HTTP 429 and a rate_limit_error body. Nothing is queued on the server; the request simply does not run.

{
  "error": {
    "message": "Rate limit exceeded. Retry after the interval in the Retry-After header.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

The Retry-After header

Every 429 response includes a Retry-After header with the number of seconds to wait before your next attempt. Treat it as authoritative: waiting exactly that long is more efficient than guessing, and retrying sooner only earns another 429.

HTTP/1.1 429 Too Many Requests
Retry-After: 12
Content-Type: application/json

Backoff pattern

Prefer Retry-After when it is present, and fall back to exponential backoff with jitter when it is not:

async function fetchWithRateLimitRetry(url, options, maxAttempts = 5) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const res = await fetch(url, options);
    if (res.status !== 429) return res;

    const retryAfter = Number(res.headers.get('Retry-After'));
    const backoff = 1000 * 2 ** attempt + Math.random() * 250;
    const waitMs = retryAfter > 0 ? retryAfter * 1000 : backoff;
    await new Promise((resolve) => setTimeout(resolve, waitMs));
  }
  throw new Error('Still rate limited after maximum retry attempts.');
}

Concurrency

Most rate limit pain comes from unbounded parallelism: firing one request per item in a large array hits the per-minute ceiling instantly. Run requests through a small worker pool instead.

• Start with a low concurrency — a handful of parallel requests — and raise it only while you see zero 429s.
• Use one shared limiter per API key across your whole service. Ten workers with independent limiters behave like no limiter at all.
• When a 429 arrives, pause the whole pool for the Retry-After interval rather than just the one failed request; the limit is per key, not per request.

Staying under the limit on batch jobs

1. 1

   Check your budget first

   Read your per-minute limit in the dashboard and size the job against it. Requests per minute times minutes gives you the realistic throughput ceiling.
2. 2

   Spread requests evenly

   Pace requests across each minute instead of bursting at the top of the window. A steady drip completes sooner than burst-and-wait cycles.
3. 3

   Combine work into fewer calls

   One request that processes several items in a single prompt often replaces several small requests. Fewer, slightly larger calls use the same budget more efficiently.
4. 4

   Make retries resumable

   Checkpoint progress as you go so a rate-limited or failed run resumes where it stopped instead of replaying completed work against your limit.