Source Code Walkthrough: Implementing LLM Streaming on the Front End

AI applications follow a counterintuitive rule of perceived performance: a 1-second time to first character + 5 seconds of streaming output feels much better than "waiting 3 seconds for the entire response at once"—even though the former takes longer overall. The reason is that streaming shows users that the app is "working," eliminating the anxiety of waiting; time to first character has a greater impact on perceived speed than total duration.

Streaming is therefore not optional; it is a standard feature of AI applications. This article explains front-end streaming from protocol to rendering.

I. Under the Hood: Server-Sent Events

The de facto standard for LLM streaming is SSE (Server-Sent Events), through which the server continuously pushes data one-way to the browser. It is essentially a long-lived HTTP response with Content-Type: text/event-stream, divided into chunks using data: ...\n\n.

When the backend (a Next.js Route Handler / Server Action) calls the model, it uses the streaming API and writes tokens to the response as they are generated:

// app/api/chat/route.ts
import { streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: anthropic("claude-sonnet-5"),
    messages,
  });
  return result.toDataStreamResponse();  // Automatically converted into an SSE stream
}

toDataStreamResponse() wraps the model's streaming tokens in the AI SDK data stream protocol, which sits on top of SSE and includes type markers. The front end parses it using the corresponding client implementation.

II. Front End: Implementing a Stream Receiver by Hand

Before using an SDK, examine a bare implementation to understand how it works:

async function streamChat(input: string, onToken: (t: string) => void) {
  const res = await fetch("/api/chat", {
    method: "POST",
    body: JSON.stringify({ messages: [{ role: "user", content: input }] }),
  });

  const reader = res.body!.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += value;
    // Split SSE chunks on \n\n
    const parts = buffer.split("\n\n");
    buffer = parts.pop()!;  // The last segment may be incomplete; keep it for the next read
    for (const part of parts) {
      const line = part.split("\n").find(l => l.startsWith("data: "));
      if (!line) continue;
      const data = line.slice(6);
      if (data === "[DONE]") return;
      onToken(JSON.parse(data).text);  // Parse the token and invoke the callback
    }
  }
}

The core is ReadableStream + TextDecoderStream: read and decode incrementally, split at SSE chunk boundaries, parse each chunk's data:, and invoke the callback with each incremental token.

III. React Rendering: Incremental Without the Jank

Appending each received token with setState sounds simple, but there is a catch:

function Chat() {
  const [text, setText] = useState("");
  return <div>{text}</div>;
}
// streamChat(input, t => setText(prev => prev + t))

With long outputs containing thousands of tokens, one setState per token plus re-rendering the entire Markdown document causes jank. To optimize it:

1. Batch updates. Do not call setState for every token. Accumulate tokens in a small buffer and flush it periodically, such as every 30ms or every 5 tokens. This reduces the number of renders.

2. Do not recompute Markdown rendering on the main thread. Re-parsing the complete Markdown document on every frame during streaming causes jank. Either show a plain-text preview while streaming and render Markdown after completion, or use a streaming-friendly Markdown library such as markdown-to-jsx or the AI SDK's built-in support.

3. Use useRef + requestAnimationFrame. Store incremental tokens in a ref and use rAF to batch updates into state, avoiding high-frequency setState calls.

IV. AI SDK's useChat: Simplifying Everything Above

In practice, you do not need to implement all of this by hand. AI SDK's useChat handles "calling the endpoint + receiving the stream + state management + rendering optimizations" for you:

"use client";
import { useChat } from "ai/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <>
      {messages.map(m => <div key={m.id}>{m.content}</div>)}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </>
  );
}

Inside, useChat:

• Manages the messages state, including user and assistant messages.
• Streams the assistant's reply and incrementally updates the last item in messages.
• Handles loading, error, and abort states.

For 99% of use cases, using useChat directly is enough. The hand-written version helps you understand what it is doing and debug it when something goes wrong.

V. Interruptions and Errors (Easy to Miss)

Streaming introduces several states that a single complete response does not have:

1. User-initiated interruption. If users do not want to wait for a long response, they need a way to "stop generating." useChat provides stop(), which uses AbortController internally to cancel the fetch request. In a hand-written implementation, remember to pass AbortController.signal to fetch and explicitly close the reader when interrupted.

2. Connection interruption. If the network disconnects, the reader throws an error. Catch it and give users a "Retry" button instead of leaving the interface stuck in a "Loading" state.

3. Midstream errors. The model may encounter an error while streaming, such as rate limiting or content moderation. SSE then pushes an error event. The parser must recognize it, display "Generation interrupted," and preserve the content already generated.

4. Empty streams. Occasionally, the model finishes without generating a single token. Do not leave an empty message in the interface.

VI. A Few UX Details

• Time to first character matters most. Optimize the backend so the first token arrives as soon as possible through measures such as prompt caching and avoiding heavy computation before generation. A time to first character < 1s is the sweet spot.
• Typewriter pacing is optional. Some products deliberately limit output speed to simulate a typewriter because it feels more comfortable visually, but do not make it slower than actual generation—artificial waiting serves no purpose.
• Follow the scroll. Automatically scroll to the bottom during streaming, but respect users who scroll upward. Do not force them back to the bottom while they are reviewing earlier content.
• Keep it copyable. Allow users to copy the portion already generated during streaming instead of making them wait until completion.

Summary

Streaming output can make or break the perceived experience of an AI application. Technically, it combines several pieces:

• Protocol: SSE, a one-way server stream.
• Backend: Model streaming API → SSE response using AI SDK's toDataStreamResponse.
• Front end: Parse incremental data with ReadableStream, either by hand to understand the mechanics or with useChat to handle it directly.
• Rendering: Batch updates and use streaming-friendly Markdown instead of re-rendering on every token and causing jank.
• Robustness: Interruption with AbortController, reconnection after disconnection, midstream errors, and empty streams.

Remember one rule: time to first character has a greater impact on perceived speed than total duration. Reduce it to < 1s and pair it with streaming rendering, and your AI application immediately changes from "waiting" to "working"—an improvement that is often more noticeable than upgrading the model by one tier.