Rate limits

The proxy itself does not enforce a requests-per-minute quota. Hosts cap concurrent in-flight jobs. When every slot is taken the gateway returns:

rate limit exceeded: too many concurrent requests (24/24)

That is a 429.

The queue

Every chat request waits in a per-model queue for a free slot, up to 45 seconds, and is sent in arrival order. A 429 from a full host puts that request back at the front of the line instead of returning it immediately. If the wait expires, the proxy returns 429 queue_timeout.

Persist through rate limits

In Settings turn on Persist through rate limits to keep retrying for up to two minutes. You should usually get a response; it may just take longer.

• The 45 second queue is always on. The two minute retry is off by default
• Applies to API and Chat
• Your HTTP client must wait (timeout ≥ 120s) or it will see a client-side timeout instead

What this is not

It does not add GPUs. If the whole network is saturated, you wait or you fail. It also does not bypass 402 / 401 / 404.

Client-side backoff

Even with persist off, back off on 429. A stampede of retries makes the 429 worse for everyone, including you.

Capacity on Status

Status shows per-model load_pct and in-flight vs effective max concurrency. 100% load on DeepSeek with a small effective cap is expected under burst traffic — extra escrows on our side do not add host weight.