Server-Sent Events (SSE)
A lightweight protocol for streaming data from server to browser over HTTP.
Definition
Server-Sent Events (SSE) is an HTTP-based protocol where the server keeps a connection open and pushes data to the client as a stream of events. Unlike WebSockets, SSE is unidirectional (server → client) and works over standard HTTP/1.1 — making it ideal for streaming LLM token-by-token responses to browsers.
Why it matters for AI APIs
Users don't want to wait 10 seconds for a full LLM response. SSE lets you stream tokens as they're generated, providing a ChatGPT-like experience with dramatically better perceived latency. Implementing SSE correctly with backpressure, client disconnection handling, and token counting is non-trivial.
In FastAPI AI Kit
The kit ships an SSE-ready streaming endpoint using FastAPI's `StreamingResponse`. Token counts are tracked per chunk and accumulated for billing. Client disconnection is handled gracefully. Works with OpenAI and Anthropic streaming protocols automatically.