Streaming

Set "stream": true. The proxy returns text/event-stream with OpenAI-style chunks:

data: {"id":"...","choices":[{"delta":{"content":"Hel"}}]}
data: {"id":"...","choices":[{"delta":{"content":"lo"}}]}
data: [DONE]

Clients

The official SDKs already know this format:

stream = client.chat.completions.create(
    model="MiniMaxAI/MiniMax-M2.7",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for event in stream:
    delta = event.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Billing

Usage is taken from the final upstream usage (or an estimate if a host omits it). You are billed for tokens that were actually produced, including a stream you cancel after tokens started — cancel early if you did not mean to run the job.

Timeouts

Long generations can sit near host limits. Default client timeouts of 30–60s are often too short for fat prompts. Use 2–3 minutes for agent / RAG traffic. If you enabled persist through rate limits, a 429 can sit open for up to two minutes before a slot frees.

Tool calls while streaming

Tool-call deltas follow the OpenAI stream (delta.tool_calls). If a host only emits JSON in content, the proxy may assemble tool_calls at the end. See Tool calling.