Streaming

Example

curl -N https://tokenkiln.com/api/v1/chat/completions \
  -H "Authorization: Bearer tk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [{"role": "user", "content": "Write a haiku about kilns."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Wire format

The response is text/event-stream. Each data: line is a chunk with a delta; the stream ends with data: [DONE]:

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"Fire"},"index":0}]}

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":" holds"},"index":0}]}

data: {"id":"chatcmpl-...","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":17,"total_tokens":31}}

data: [DONE]

The usage chunk (empty choices, populated usage) appears only when you send "stream_options": {"include_usage": true}. SDKs handle all of this for you; the raw format matters only if you parse the stream yourself.

Usage accounting

The gateway always asks the engine for token usage internally, whether or not you request the usage chunk, so streamed requests bill exactly like non-streamed ones: reserve first, settle to real counts after. Requesting include_usage changes what you see, not what you pay.

Disconnects

• If your client disconnects mid-stream, the gateway stops the engine and settles for the tokens generated up to that point.
• If the upstream connection drops, the stream carries a final data:event with an error object so your parser isn't left hanging, and the request settles as an upstream error.