Update error handling for 429 and 503 responses to distinguish between rapid traffic increases and model overload with specific error codes.

Summary by gemma-4-12b. It may be wrong; the diff below is what actually changed. Raw: 8f34e950c2-openai-rate-limits.diff · commit 8f34e950c2
diff --git a/snapshots/openai/rate-limits.md b/snapshots/openai/rate-limits.mdindex 52f3c19..653162a 100644--- a/snapshots/openai/rate-limits.md+++ b/snapshots/openai/rate-limits.md@@ -62,5 +62,5 @@ Responses can include the following header fields: | x-ratelimit-reset-project-tokens | 3s | The time until the project-scoped token rate limit resets to its initial state. | -Project-token headers may be present when a project-scoped token limit applies. `Retry-After` may be present on `429` responses caused by a temporary rate limit. It does not mean that quota, billing, or other errors that require user action can be resolved by retrying.+Project-token headers may be present when a project-scoped token limit applies. `Retry-After` may be present on `429` responses caused by a temporary rate limit and `503` responses caused by temporary model overload. It does not mean that quota, billing, or other errors that require user action can be resolved by retrying.  ### Fine-tuning rate limits@@ -92,4 +92,15 @@ As a rule of thumb, once your traffic reaches 1 million input tokens per minute Enterprise customers whose pay-as-you-go traffic routinely hits ramp-rate limits can consider Scale Tier for more predictable capacity on eligible models. For GPT-5.6 and later models, see Reserved Tier. Capacity tiers don't change how you should handle a `slow_down` response: follow `Retry-After` when it's present, reduce traffic, and ramp gradually. +#### Update existing error handlers++If your application handled earlier throttling and overload responses, check both the HTTP status and `error.code`:++- On endpoints that previously returned `503` with the `slow_down` code for both conditions, rapid traffic increases now return `429` with `slow_down`. Model overload remains `503` but uses `server_is_overloaded`.+- Video requests rejected before creating a job previously returned `429` with the `invalid_request_error` type and `rate_limit_exceeded` code for these conditions. Rapid traffic increases now return `429` with `rate_limit_error` and `slow_down`; model overload returns `503` with `service_unavailable_error` and `server_is_overloaded`. Errors reported in a video job's status are a separate case.++Handle both `429` and `503` in your SDK error handlers. For example, Python, TypeScript, and Ruby use `RateLimitError` for `429` and `InternalServerError` for `503`; Java uses `RateLimitException` and `InternalServerException`. Keep support for earlier response codes while your application can still receive them. Other errors can use the same HTTP statuses, so inspect the error body before choosing a recovery action.++For streaming requests, these HTTP error responses apply before the stream starts. An error after streaming begins can arrive as a stream event; don't automatically replay a request after consuming output.+ ### What are some steps I can take to mitigate this? @@ -104,7 +115,9 @@ To protect against automated and high-volume misuse, set a usage limit for indiv When a request exceeds a temporary rate limit, the API returns a `429` error. The response can include a `Retry-After` header that tells you how many seconds to wait before trying again. Treat this value as a minimum: wait at least that long and add a small random delay so multiple clients don't retry at the same time. -Each official OpenAI SDK automatically retries eligible rate-limit errors and honors `Retry-After` when it's present. You don't need to parse the header or add another retry loop for standard API calls.+Each official OpenAI SDK automatically retries eligible `429` and `503` responses, subject to its retry settings. Handling of `Retry-After`, especially long delays, varies by SDK version and configuration. Check the retry behavior of your installed version rather than assuming every server delay is supported.++If a valid server delay exceeds the supported or configured maximum retry delay, stop retrying and defer the request rather than retrying sooner. An SDK can return the original HTTP error when it declines a delay above its limit. Continue to handle cancellation and timeout errors separately: a canceled request or expired deadline can stop retries without returning that HTTP error. A timeout for each attempt isn't necessarily a deadline for the entire operation. -If you're using your own HTTP client, follow `Retry-After` when the header is present and contains a valid value. If it's missing or invalid, fall back to exponential backoff with jitter. Limit both the number of attempts and the total time spent retrying. If you add application-level retries, account for the retries your SDK already performs. Don't retry quota, billing, or other errors that require you to take action.+If you're using your own HTTP client, follow `Retry-After` when the header is present and contains a valid value. If it's missing or invalid, fall back to exponential backoff with jitter. Limit both the number of attempts and the total time spent retrying. If you manage retries in your application, disable SDK retries or account for them in those limits so nested retry loops don't multiply requests. Don't retry quota, billing, or other errors that require you to take action.  Exponential backoff means waiting briefly after an unsuccessful request, then increasing the delay after each unsuccessful retry. This continues until the request succeeds or reaches a configured retry limit.@@ -118,5 +131,5 @@ This approach has many benefits: Note that unsuccessful requests contribute to your per-minute limit, so continuously resending a request won't work. -Below are a few example solutions for Python that use exponential backoff.+The Python examples below demonstrate fallback backoff. They don't inspect `Retry-After`: before using them, add handling for valid server hints so the wrappers don't retry sooner than requested. Disable SDK retries or account for them in your application's retry limits.  Tenacity is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything. To add exponential backoff to your requests, you can use the `tenacity.retry` decorator. The below example uses the `tenacity.wait_random_exponential` function to add random exponential backoff to a request.
← OpenAI timeline