How to Stream AI Tokens to the Browser: SSE vs WebSockets vs Chunked Fetch
On this page
When you add an AI feature to your app, the first thing users notice is the wait. The model is “thinking” and the screen just sits there. Streaming fixes that. Tokens show up one by one, like someone typing, and the whole thing feels alive.
The question is how to get those tokens from your server to the browser. Most people reach for WebSockets out of habit. That is usually the wrong call. Let me explain why, and what I actually use.
Start with one question: is it one-way?
Token streaming is a one-way problem. The server sends tokens, the browser receives them. The browser does not need to talk back mid-stream. It just listens.
That single fact removes WebSockets from the table for most cases. WebSockets exist for two-way, real-time chat between client and server. If you only need server to client, you are paying for a feature you will never use.
So the real choice is not “SSE vs WebSockets”. It is “SSE vs chunked fetch”. Both ride on plain HTTP.
Option 1: SSE with EventSource
Server-Sent Events is the built-in browser API for this. You point an EventSource at a URL and it streams text events to you, with automatic reconnect baked in.
const es = new EventSource('/stream.php?prompt=' + encodeURIComponent(prompt));
es.onmessage = (e) => { output.textContent += e.data; };Clean, right? But here is the catch that bites everyone with AI features.
EventSource is GET only. It cannot send a request body, and it cannot set headers like Authorization. Your prompt is often long, and your auth token should never sit in a URL, because it ends up in server logs and browser history.
So for an LLM call, plain EventSource falls apart fast. That leads to option two.
Option 2: chunked fetch with a ReadableStream
This is what the OpenAI and Anthropic JavaScript clients actually do under the hood. You make a normal fetch POST, then read the response body as a stream as it arrives.
You get everything EventSource could not give you: a POST body for the prompt, real headers for auth, and clean cancellation with AbortController.
Here is the browser side. The server sends events in SSE format, and we parse them ourselves:
const controller = new AbortController();
const res = await fetch('/stream.php', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt }),
signal: controller.signal,
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
// SSE frames are separated by a blank line
const frames = buffer.split('nn');
buffer = frames.pop(); // keep the unfinished tail for next read
for (const frame of frames) {
const data = frame.replace(/^data: /, '');
if (data === '[DONE]') return;
const json = JSON.parse(data);
const token = json.choices?.[0]?.delta?.content || '';
output.textContent += token;
}
}To stop a generation, you just call controller.abort(). No special protocol needed.
The PHP server side
Your server sits in the middle. It holds the API key, calls the model with streaming on, and forwards each chunk to the browser as it arrives. The key never reaches the client.
The important part is forwarding chunks the moment they land, without PHP or your web server holding them back:
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('X-Accel-Buffering: no'); // tell nginx not to buffer
while (ob_get_level() > 0) { ob_end_flush(); }
ob_implicit_flush(true);
$body = json_decode(file_get_contents('php://input'), true);
$prompt = $body['prompt'] ?? '';
$ch = curl_init('https://openrouter.ai/api/v1/chat/completions');
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . OPENROUTER_KEY,
'Content-Type: application/json',
],
CURLOPT_POSTFIELDS => json_encode([
'model' => 'anthropic/claude-haiku-4.5',
'stream' => true,
'messages' => [['role' => 'user', 'content' => $prompt]],
]),
// Fires every time a chunk arrives from the model.
CURLOPT_WRITEFUNCTION => function ($ch, $chunk) {
echo $chunk; // forward it straight to the browser
flush();
return strlen($chunk);
},
]);
curl_exec($ch);That flush() plus the X-Accel-Buffering header is the whole trick. Skip them and your tokens pile up somewhere and arrive all at once at the end, which defeats the point.
The gotchas nobody mentions
This is where most tutorials stop and where real apps break.
Proxy buffering. Nginx buffers responses by default. Your PHP flushes perfectly, then nginx holds it all back. Turn it off with X-Accel-Buffering: no, or proxy_buffering off for that route.
PHP-FPM workers. This one matters at scale. Every open stream holds one PHP-FPM worker for the entire generation. If pm.max_children is 20 and 20 people are streaming, the 21st person waits in line. PHP was not built for thousands of long-lived connections. For low traffic it is fine. For high traffic, move streaming to something built for it, like Node, Swoole, or a small dedicated service, and keep the rest of your app in PHP.
The HTTP/1.1 connection limit. Browsers allow only 6 connections per domain on HTTP/1.1. A long-lived stream eats one of those 6 the whole time. On HTTP/2 this goes away because it multiplexes many streams over one connection. If you stream, serve over HTTP/2.
Cancellation. When the user closes the tab or hits stop, you want the upstream model call to stop too, so you are not paying for tokens nobody reads. On the browser use AbortController. On the server, check connection_aborted() and bail out of the cURL loop.
When WebSockets do make sense
WebSockets earn their cost when you genuinely need two-way, low-latency traffic. Think a voice agent where the user interrupts mid-sentence, a collaborative doc where many people type at once, or a multiplayer game.
For those, the persistent two-way channel is worth the extra infrastructure: sticky sessions, a pub/sub layer to broadcast across servers, and your own reconnect logic. For streaming text one direction, all of that is dead weight.
Quick decision guide
- Just showing tokens, and a GET with no auth is fine: SSE with EventSource. Simplest, free reconnect.
- Streaming an LLM with a prompt body, auth header, and a stop button: chunked fetch with a ReadableStream. This is the right default for AI features.
- Truly two-way and real-time, such as interrupts, multiplayer, or voice: WebSockets.
The short version: do not default to WebSockets. Token streaming is one-way, so let plain HTTP do the work. Reach for fetch with a ReadableStream, flush properly on the server, and watch out for the proxy and the worker pool. That is the whole game.
Frequently asked questions
Should I use WebSockets to stream AI responses?
Usually no. Token streaming is one way, from server to client, so plain HTTP with SSE or a chunked fetch is simpler and cheaper. Save WebSockets for genuinely two-way features like voice or multiplayer.
Why not just use EventSource for LLM streaming?
EventSource is GET only and cannot send a request body or an Authorization header. LLM calls usually need both, so a fetch with a ReadableStream is the better fit.
My tokens arrive all at once at the end. What is wrong?
Something is buffering the response. Disable proxy buffering, for nginx send X-Accel-Buffering: no or set proxy_buffering off, and make sure PHP flushes after each chunk.
Does streaming keep a PHP process busy the whole time?
Yes. Each open stream holds one PHP-FPM worker until it finishes. That is fine at low volume, but for high concurrency move streaming to a runtime built for many long-lived connections.