Rate limits

The gateway limits request rates to keep the service fair and stable, and upstream providers apply limits of their own. When you exceed a limit you get 429 Too Many Requests.

Handling 429

Retry with exponential backoff and jitter: wait about one second, then two, four and so on, adding a random delay each time, and give up after a few attempts.

The official OpenAI and Anthropic SDKs already retry 429 and 5xx responses with backoff. You can raise the number of attempts:

from openai import OpenAI

client = OpenAI(base_url="https://api.cheapaimodel.ai/v1", max_retries=5)

Avoiding limits

• Run requests with bounded concurrency instead of firing them all at once.
• Use streaming for long generations so a single slow request does not hold a connection open without output.
• If your workload needs sustained high throughput, email support@cheapaimodel.ai.