LLM Streaming Protocols in Depth: SSE, Data Streams, and Parsing

Streaming output is the lifeblood of perceived performance in AI applications, and no one disputes that anymore: a one-second time to first token plus output as it is generated feels far better than waiting ten seconds and receiving everything at once, even if the total time is longer.

But the term "streaming" has been used so much that many people remain confused about what is actually flowing underneath. Open the Network panel and you will see a text/event-stream response containing line after line of data: {...}. This is not WebSocket, but a protocol called SSE. AI SDK adds another layer on top called the "data stream protocol," packing tokens, tool calls, errors, and aborts into a type-tagged event stream. Above that sit Nginx and Cloudflare; one misconfigured buffering setting can turn your "streaming" response back into an "all-at-once" response.

This article explains each protocol layer: from persistent HTTP connections to SSE syntax, typed data stream events, the complexity of streaming tool calls, interruptible streams, and proxy-buffering pitfalls. By the end, you should be able to write a streaming client by hand that can parse responses, cancel requests, and pass through a CDN, rather than only knowing how to call useChat.

1. Why SSE: a one-way stream over a persistent HTTP connection

First, the conclusion: The de facto standard for LLM streaming is SSE (Server-Sent Events), not WebSocket.

SSE is essentially an ordinary HTTP/1.1 GET (or POST) response that the server keeps open while continuously writing data into it. Three points matter:

1. Content-Type: text/event-stream tells the browser, "This is an event stream; do not wait for the response to finish before delivering it."
2. One-way communication: only the server pushes data to the client. The client does not send data back through this connection. In a chat application, user input is sent in a separate POST request, so bidirectional communication is unnecessary.
3. Built on HTTP: it uses standard HTTP infrastructure and naturally works through CDNs, H2/H3, authorization headers, and cookies, without a protocol-upgrade handshake.

Compared with WebSocket:

In 99% of LLM use cases, the client sends one question and the server streams back a large response, which is entirely one-way. WebSocket is overkill here and also brings the burden of handshakes, proxy compatibility, and custom reconnection logic. SSE won not because it is more powerful, but because it fits the use case better.

One important detail: the browser's built-in EventSource supports only GET and does not allow custom request headers. AI applications usually need a POST body and an Authorization header, so they almost always parse SSE manually with fetch instead of using EventSource. That is why the next section covers manual parsing.

2. Raw SSE parsing: ReadableStream + TextDecoder + splitting on \n\n

First, let us clarify the SSE protocol syntax itself. An SSE stream looks like this:

data: {"v":1}

data: {"v":2}

event: ping
data: keep-alive

id: 42
data: {"v":3}

There are only a few rules: Each event consists of several lines, and events are separated by a blank line, or \n\n; each line uses the form field: value; data: contains the payload, event: specifies the event type, id: contains the event ID, and retry: suggests a reconnection interval in milliseconds. An event may contain multiple data: lines, which must be joined to reconstruct the complete payload.

After fetch receives such a stream, response.body is a ReadableStream<Uint8Array>. The core decoding and framing logic is:

async function parseSSE(
  res: Response,
  onData: (data: string, event?: string) => void,
) {
  const reader = res.body!
    .pipeThrough(new TextDecoderStream())   // bytes -> string
    .getReader();
  let buffer = "";
  let event = undefined;

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += value;

    // SSE events are separated by a blank line (\n\n)
    let sep: number;
    while ((sep = buffer.indexOf("\n\n")) !== -1) {
      const raw = buffer.slice(0, sep);
      buffer = buffer.slice(sep + 2);

      const dataLines: string[] = [];
      for (const line of raw.split("\n")) {
        if (line.startsWith("data:")) dataLines.push(line.slice(5).trimStart());
        else if (line.startsWith("event:")) event = line.slice(6).trim();
        // id: / retry: work the same way; collect them if needed
      }
      if (dataLines.length) onData(dataLines.join("\n"), event);
      event = undefined;
    }
  }
}

The two easiest pitfalls are both in this code:

• You must buffer partial chunks. A chunk returned by reader.read() may end in the middle of a data: line. Splitting it immediately can tear one event in half. The correct approach is to process only complete \n\n separators found in buffer, leaving the trailing fragment for the next read.
• Use indexOf("\n\n") rather than split. On a long stream, split scans the entire buffer every time. With indexOf, you find and process one separator at a time, which performs better.
• Be careful with line endings: the specification uses \n, but some servers send \r\n\r\n. Production code should normalize them first with replaceAll("\r\n", "\n").

This is the lowest-level capability: given any SSE stream, you can correctly reconstruct its individual events. Next, let us examine what AI SDK adds on top.

3. AI SDK's data stream protocol: type tags on top of SSE

Raw SSE tells you only that a piece of data arrived. You still need your own convention to determine whether it is a text token, a tool call, or an error. AI SDK's data stream protocol provides that convention: everything generated by the model becomes a JSON event with a type field and is placed inside an SSE data: line.

Be aware of one pitfall: this protocol has changed. AI SDK v3/v4 used a "line prefix" format such as 0:"text", 2:[{...}], and 9:{tool_call}, where the leading number represented the type. Many older blog posts and tutorials still describe that format. Starting with v5, it changed to a more standard SSE + JSON format. More importantly, the new format supports ping heartbeats, reconnection, and cache-friendly behavior. The following uses the current version.

Current data stream event types, each sent as one data: {...} line:

Notice the design: Text is not represented by a single text event. It uses three phases, start/delta/end, and every text block has a unique id. This supports multiple text blocks, reasoning, and tool calls interleaved within one message. The frontend uses each id to apply every delta to the correct block.

The current backend API for sending a stream with AI SDK uses toUIMessageStream and createUIMessageStreamResponse:

// app/api/chat/route.ts (Next.js)
import { convertToModelMessages, createUIMessageStreamResponse,
  streamText, toUIMessageStream, UIMessage } from "ai";

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();
  const result = streamText({
    model,                                  // your provider
    messages: await convertToModelMessages(messages),
  });
  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

On the frontend, useChat from @ai-sdk/react consumes the stream directly. Internally, it reconstructs the events in the table above as message.parts. If you are connecting a non-JavaScript backend, such as Python FastAPI, to useChat, you can emit the events manually according to this table. Remember to include the x-vercel-ai-ui-message-stream: v1 response header, or the frontend will parse the response using the old protocol.

(There is also a simpler text stream protocol consisting only of plain-text chunks without type tags. It is suitable when you only need to emit text and do not need tool calls. Use toTextStream / createTextStreamResponse. The rest of this article focuses on the more capable data stream protocol.)

4. Streaming tool calls: the model emits tool arguments as it generates them

Streaming tool calls are the most elegant and failure-prone part of the entire protocol.

First consider how a model calls a tool without streaming: it finishes generating a complete set of JSON arguments, which are then parsed and executed. Simple and direct. In streaming mode, however, the model outputs one token at a time, so you may see an incomplete JSON fragment such as {"city":"San. The protocol must solve this problem: How can the frontend show that getWeather is being called before all arguments have been generated, while still receiving the complete arguments once generation finishes?

The data stream protocol solves this with three phases:

data: {"type":"tool-input-start","toolCallId":"call_abc","toolName":"getWeather"}

data: {"type":"tool-input-delta","toolCallId":"call_abc","inputTextDelta":"{\"city\":\"San"}

data: {"type":"tool-input-delta","toolCallId":"call_abc","inputTextDelta":" Francisco\"}"}

data: {"type":"tool-input-available","toolCallId":"call_abc","toolName":"getWeather","input":{"city":"San Francisco"}}

• tool-input-start: the frontend can immediately render a placeholder card saying "Calling getWeather...", which feels much better than making the user wait with no feedback.
• tool-input-delta: concatenate the argument fragments to provide a live preview of the arguments being generated. Be careful: you cannot run JSON.parse on incomplete JSON. Either display the raw text or use a fault-tolerant incremental JSON parser.
• tool-input-available: This is the authoritative source of arguments. The server has already parsed the complete arguments. Use them directly instead of trusting the string you assembled yourself.

Several factors create additional complexity: parallel tool calls (the model may invoke multiple tools concurrently in one message, so the frontend must route them separately by toolCallId); changing arguments midway (the model may change the structure after emitting part of it, causing the preview to jump, which should be smoothed); and streaming tool execution itself (before tool-output-available, a long-running tool can push its own data-* progress events). This is why text blocks and tool blocks require IDs instead of being appended blindly.

A practical lesson: The frontend UI must distinguish between an argument preview and final arguments. While deltas have arrived but available has not, render the preview with lower contrast or a typing cursor. Switch to the final card as soon as available arrives. Otherwise, users are forced to watch an incomplete JSON document constantly changing, which looks poor.

5. Interruptible streams: AbortController + backend cancellation awareness

Cancellation is a standard feature of streaming. When the user clicks "Stop," generation must end immediately while preserving everything already emitted. This requires frontend and backend cooperation.

The frontend uses AbortController:

const controller = new AbortController();
fetch("/api/chat", { method: "POST", body: ..., signal: controller.signal });
// The user clicks Stop
controller.abort();   // fetch throws AbortError, and the reader closes too

That is what useChat().stop() does internally. The critical part is ensuring that the backend observes the cancellation. In a Next.js Route Handler, req.signal is the AbortSignal bound to the client connection:

export async function POST(req: Request) {
  const result = streamText({
    model,
    messages,
    abortSignal: req.signal,        // Forward the client cancellation signal to the model
  });
  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

Client abort() → TCP connection closes → server req.signal fires → streamText receives the signal → it stops pulling additional tokens from the provider. If any link in this chain is missing, the cancellation is fake: the user clicks Stop while the backend continues burning tokens and money.

Several pitfalls:

• Cancellation is not an error, so do not handle it as one. An aborted fetch throws an AbortError whose name is "AbortError". Catch it separately and preserve the generated text in the UI instead of displaying an error.
• Preserve partial output. When the client cancels, the data stream protocol allows the server to send {"type":"abort","reason":"user cancelled"}. The frontend can then mark the message as "Aborted" rather than "Completed." Do not discard the text-delta events already received.
• Close the provider stream promptly. After abortSignal fires, ensure that the upstream connection to the provider, such as OpenAI or Anthropic, is actively closed. Otherwise, the client has stopped while the upstream provider continues billing.

6. Proxy/CDN buffering: the leading cause of broken streaming

This is one of the most frustrating problems in production: The code is completely correct, but streaming still degrades into "wait a long time, then receive everything at once." In 99% of cases, an intermediate layer buffered the stream.

For SSE to work, data must be forwarded as soon as it is produced. If any layer waits before sending it onward, the experience collapses. Common sources of buffering include:

1. Nginx reverse proxies. The default proxy_buffering on buffers the response until it reaches a certain size or the upstream response ends. Disable it for the streaming location:

location /api/chat {
  proxy_pass http://upstream;
  proxy_buffering off;          # Critical
  proxy_cache off;
  proxy_http_version 1.1;       # Persistent connection
  proxy_read_timeout 600s;      # LLMs can be slow; do not use the default 60s
  add_header X-Accel-Buffering no;
}

An easier approach is to have the backend response include the X-Accel-Buffering: no header directly. Nginx will disable buffering for that response, so you do not need to change its configuration. This is especially useful in Serverless environments.

2. Compression middleware. gzip/br compression must accumulate a batch of bytes to be worthwhile, so enabling it usually breaks streaming. Compression must be skipped for text/event-stream responses.

3. CDNs such as Cloudflare. Cloudflare buffers by default, and under some conditions its free tier waits until the response ends before sending anything downstream, eliminating the streaming experience. During diagnosis, bypass Cloudflare and request the origin directly for comparison; this immediately identifies the problem.

4. HTTP/2 flow control. H2 is theoretically friendly to streaming, but some intermediate H2 implementations aggregate data. Retaining H1.1 with proxy_http_version 1.1 is generally the most reliable option.

To test it yourself, inspect when the request body appears in the Network panel. With healthy streaming, data starts arriving within one second and the response size continues growing. With buffering, nothing happens for a long time and then the entire response arrives at once. The DevTools EventStream panel can also display each event directly.

7. Error streams and reconnection

Streaming introduces two failure modes beyond a one-shot request: errors during generation and connection loss. The protocol must handle both.

Errors during generation. The model may fail halfway through because of rate limiting, content moderation, or an internal provider error. The data stream sends {"type":"error","errorText":"..."} and then usually ends. When the frontend handles this event, preserve the generated text, mark the message as "Partially generated + error," and provide a Retry button. Clearing everything and starting over is disastrous for the user experience.

Client disconnection. Distinguish between a deliberate user abort, which produces the AbortError described above, and an involuntary disconnect caused by network instability. The SSE specification includes reconnection: the server sends id: 123, and after a disconnect, browser EventSource automatically reconnects with Last-Event-ID: 123. A server can send retry: 3000 to tell the browser to reconnect after three seconds. Most AI applications use fetch rather than EventSource, however, so you must implement reconnection yourself: record the last received message position and resend the request with exponential backoff after a disconnect.

In practice, most products use lightweight retries rather than strict resumption after a midstream disconnect. Model streams are difficult to resume from the middle because the provider must support it and the context must be reconstructed. A pragmatic solution is to show "Network interrupted. Click to retry," preserve the generated fragment as an assistant message, and regenerate on retry. Strict, transparent resumption usually offers poor value for its complexity unless very long generations are a hard requirement.

Summary

The word "stream" in streaming output hides several stacked protocol layers, each with details that must be understood:

• SSE is the foundation: a one-way persistent HTTP connection using text/event-stream, with events separated by \n\n. LLM applications almost always parse it manually with fetch instead of EventSource because they need POST and custom headers.
• The data stream protocol is a convention: it puts JSON events with a type field inside SSE data: lines for text, tool input, data, errors, finish events, and more. Starting with v5, the JSON-SSE format replaced the old 0:/9: line prefixes. Non-JavaScript backends can emit the events manually according to the table, but must include the x-vercel-ai-ui-message-stream: v1 header.
• Streaming tool calls are the most elegant part: arguments use three phases, start/delta/available. The frontend must distinguish the preview state from the final state, trust only available for final arguments, and never force incomplete JSON through a parser.
• Cancellation must work end to end: AbortController → req.signal → abortSignal forwarded to the provider. If any link is missing, the cancellation is fake. Cancellation is not an error, and generated content must be preserved.
• Buffering is the leading cause of failure: use Nginx proxy_buffering off, the X-Accel-Buffering: no response header, skip gzip for text/event-stream, and investigate Cloudflare. When the code is correct but the stream does not move, the problem is usually here.

In one sentence: True streaming means that data reaches the user's screen as soon as it is produced. If any intermediate layer holds it back, the experience falls apart. Trace the entire path from the browser to the provider and ensure that every layer avoids buffering, supports cancellation, and handles reconnection. Only then does your AI application deserve to be called "real-time." This work separates a demo that merely runs from a product people can actually use.