Streaming responses

By default the API generates the entire reply before sending anything back. For a chat interface that means the user stares at a spinner for the full generation time. Streaming flips this: the server sends each piece of the reply the moment it is generated, and your UI renders text within a few hundred milliseconds of the request.

Why stream

• Perceived latency drops from full generation time to time-to-first-token.
• Users can start reading — and interrupting — before the reply is finished.
• Long replies no longer risk client-side request timeouts, because bytes flow continuously.

Enabling streaming

Add "stream": true to the request body. Everything else — endpoint, auth, messages — stays identical to a normal chat completion. The response switches from a single JSON object to a stream of server-sent events (SSE).

curl https://api.tase.app/v1/chat/completions \
  -N \
  -H "Authorization: Bearer tase_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tase-8b",
    "stream": true,
    "messages": [
      { "role": "user", "content": "Write a haiku about mornings." }
    ]
  }'

The SSE wire format

Each event is one line starting with data: followed by a JSON chunk. Instead of a message, each chunk carries a delta — just the new fragment of the reply. The stream ends with the literal line data: [DONE].

data: {"id":"chatcmpl-9f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Pale"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" light on the sill"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

• The first chunk usually carries only delta.role with no content.
• Text arrives in choices[0].delta.content — concatenate the fragments in order to rebuild the full reply.
• The final content chunk has an empty delta and a non-null finish_reason.
• data: [DONE] is a plain string, not JSON — check for it before parsing.

JavaScript: fetch and ReadableStream

This example works in the browser and in Node 18+. It reads the raw byte stream, splits it into lines, keeps any partial line in a buffer for the next read, and accumulates the deltas into the full reply.

const response = await fetch('https://api.tase.app/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer ' + process.env.TASE_API_KEY,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'tase-8b',
    stream: true,
    messages: [{ role: 'user', content: 'Write a haiku about mornings.' }],
  }),
});

if (!response.ok) {
  // Errors are returned as a normal JSON body, not a stream.
  const err = await response.json();
  throw new Error('Request failed (' + response.status + '): ' + JSON.stringify(err));
}

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
let fullText = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop(); // keep the trailing partial line for the next chunk

  for (const line of lines) {
    const trimmed = line.trim();
    if (!trimmed.startsWith('data:')) continue;

    const payload = trimmed.slice('data:'.length).trim();
    if (payload === '[DONE]') break;

    const chunk = JSON.parse(payload);
    const delta = chunk.choices?.[0]?.delta?.content;
    if (delta) {
      fullText += delta;
      process.stdout.write(delta); // or append to your UI
    }
  }
}

console.log('\n--- complete ---');
console.log(fullText);

Python

With plain requests, pass stream=True and iterate over lines — the library handles line buffering for you.

import json
import os

import requests

response = requests.post(
    "https://api.tase.app/v1/chat/completions",
    headers={
        "Authorization": "Bearer " + os.environ["TASE_API_KEY"],
        "Content-Type": "application/json",
    },
    json={
        "model": "tase-8b",
        "stream": True,
        "messages": [{"role": "user", "content": "Write a haiku about mornings."}],
    },
    stream=True,
)
response.raise_for_status()

full_text = ""
for raw_line in response.iter_lines(decode_unicode=True):
    if not raw_line or not raw_line.startswith("data:"):
        continue
    payload = raw_line[len("data:"):].strip()
    if payload == "[DONE]":
        break
    chunk = json.loads(payload)
    delta = chunk["choices"][0]["delta"].get("content")
    if delta:
        full_text += delta
        print(delta, end="", flush=True)

print()
print("--- complete ---")
print(full_text)

If you already use the OpenAI SDK, streaming is even shorter — the SDK parses the SSE protocol and hands you typed chunks.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TASE_API_KEY"],
    base_url="https://api.tase.app/v1",
)

stream = client.chat.completions.create(
    model="tase-8b",
    stream=True,
    messages=[{"role": "user", "content": "Write a haiku about mornings."}],
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Cancelling a stream mid-flight

A Stop button is table stakes in a streaming UI. Pass an AbortController signal to fetch and call abort() when the user cancels. The connection closes immediately and the read loop ends with an AbortError. Keep the text accumulated so far — showing the partial reply feels better than clearing it.

const controller = new AbortController();

// Wire this to a Stop button in your UI.
stopButton.addEventListener('click', () => controller.abort());

try {
  const response = await fetch('https://api.tase.app/v1/chat/completions', {
    method: 'POST',
    signal: controller.signal,
    headers: {
      Authorization: 'Bearer ' + process.env.TASE_API_KEY,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'tase-8b',
      stream: true,
      messages: [{ role: 'user', content: 'Write a long story.' }],
    }),
  });

  // ...read the stream exactly as shown above...
} catch (err) {
  if (err.name === 'AbortError') {
    console.log('Stream cancelled by the user; keeping partial text.');
  } else {
    throw err;
  }
}

Error handling

• Before the stream starts: a bad key, malformed body, or rate limit returns a normal non-200 JSON response. Check response.ok before you start reading the body as a stream.
• Mid-stream disconnects: if the network drops, the read loop ends without [DONE] and without a chunk carrying a finish_reason. Treat the text as incomplete and decide per feature: retry the request, or keep the partial reply and mark it as interrupted.
• Defensive parsing: skip empty lines, check for [DONE] before JSON.parse, and guard the delta access — chunks without content (the role chunk, the final chunk) are normal.