Problem RestatementProblem
Design the comments feature for live video. While a live stream is playing, viewers post short comments, and everyone watching sees new comments appear within about a second. A popular stream can have millions of viewers, and one viewer's comment must reach all of them.
A common variant: comments are tied to a position in the video (like a synchronized overlay), so people watching a replay later see the comments appear at the same moments.
RequirementsRequirements
1.1 Functional
- Post a comment on a live video.
- See new comments in near real time.
- New viewers see recent comments when they join.
- Moderation: filter spam and abuse, let the host delete comments and block users.
- (Variant) Replay comments in sync with video playback.
1.2 Non-Functional
- Low latency: under ~1–2 seconds from post to display.
- Huge fan-out: one message → millions of screens.
- Scalable for many streams at once, most of them small.
- Graceful: during huge events it is fine to show a sample of comments, but the system must not fall over.
1.3 Scale Estimates
- 1M concurrent live streams, most with fewer than 100 viewers.
- A top stream: 5M viewers posting 10K comments/sec. Delivering every comment to everyone would mean 50 billion deliveries per second, which is impossible. We must sample.
- Each WebSocket server holds ~100K connections, so 5M viewers need ~50 servers for that one stream.
1.4 API Design
POST /v1/videos/{id}/comments{ text, video_ts_ms? }→{ comment_id }- WebSocket
/v1/videos/{id}/live→ server pushes{ comment_id, user, text, ts } GET /v1/videos/{id}/comments?from_ts=&to_ts=(recent history and replay)
High-Level ArchitectureArchitecture
2.1 Overview
- Comment Service: validates, checks rate limits and moderation, stores the comment, and publishes it.
- Moderation: quick filters (blocked words, spam checks) inline, and heavier ML checks async.
- Pub/Sub (Redis pub/sub, Kafka or a dedicated fan-out tier): a channel per video.
- Gateway / WebSocket servers: hold viewer connections. Each server subscribes only to the video channels its viewers are watching.
- Comment store: Cassandra/DynamoDB, keyed by video and time.
2.2 Architecture Diagram
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
V1["Viewer posting"] --> CS["Comment Service - rate limit, moderation"]
CS --> DB[("Comments DB - by video, time")]
CS -->|"publish video:123"| PS[("Pub/Sub - channel per video")]
PS --> G1["WebSocket server 1"]
PS --> G2["WebSocket server 2"]
PS --> G3["WebSocket server N"]
G1 --> A["Viewers"]
G2 --> B["Viewers"]
G3 --> C["Viewers"]Data ModelData model
comments:
video_id (partition key), ts_bucket (e.g., minute), comment_id (time-ordered),
user_id, text, video_ts_ms, status (visible/removed)Partitioning by video and time bucket keeps "recent comments for this video" and "comments between minute 12 and 13" fast.
Key FlowsFlows
4.1 Posting and delivering
- A viewer posts. The service applies a rate limit (e.g., 1 comment per 2 seconds per user) and runs fast moderation.
- It stores the comment, then publishes it to channel
video:{id}. - Every WebSocket server subscribed to that channel receives it once, then pushes it to all its local viewers of that video.
This two-level fan-out is the key. Pub/sub sends to ~50 servers, and each server sends to ~100K local connections. No single machine talks to millions of viewers.
4.2 Joining a stream
The client opens a WebSocket, and its server subscribes to the video channel if it isn't already. The client also loads the last ~50 comments from the store so the screen isn't empty.
Deep Dive A — A stream with two million viewersDeep dive
Ten thousand comments a second, two million people watching. The naive fan-out is the product of those two numbers, and that product is the whole problem.
Send every comment to every viewer
A comment arrives, the fan-out layer pushes it down all two million WebSocket connections.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
C["10,000 comments/sec"] --> FAN["Fan-out"]
V["2,000,000 connections"] --> FAN
FAN --> N["20 billion messages/sec"]
N --> B1["Servers saturate on socket writes"]
N --> B2["Phones burn battery parsing"]
N --> B3["Text scrolls past faster than anyone can read"]The number is absurd before anyone asks about the network. And notice the third line: even if the infrastructure could deliver it, nobody can read ten thousand comments a second. Sending them all is not just expensive, it is pointless.
Batch what you send
Collect comments for 200–500 ms and send them as one message instead of one per comment. Fewer frames, far less per-message overhead, and the user cannot perceive a quarter-second of delay in a comment stream.
This is a genuine win on the transport — maybe an order of magnitude — and it changes nothing about the underlying volume. Two million viewers still receive ten thousand comments a second, now in tidier parcels.
Decide what each viewer actually needs
Cut the volume at the source, per viewer:
- Sample down to a readable rate. Each WebSocket server sends any one viewer at most ~10 comments/sec, chosen by rules rather than at random: the host first, then people the viewer follows, then verified accounts, then a random sample of the rest. The stream feels alive, which is the actual requirement — not completeness.
- Aggregate reactions. Hearts and likes are a counter, not messages. Send
+4,812 in the last second, one number, instead of 4,812 events. - Isolate the hot streams. Route a stream with millions of viewers to its own fan-out cluster, so the tail of small streams is not competing with it for sockets and CPU.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
ALL["10,000 comments/sec"] --> SEL["Per-viewer selection - host, follows, verified, sample"]
SEL --> BATCH["Batch every 200-500 ms"]
BATCH --> V["~10 comments/sec per viewer"]
LIKES["Reaction events"] --> CNT["Counted"]
CNT --> TOT["One total per second"]
TOT --> VThe comment author's own comment is always shown back to them immediately, whatever the sampling decides. A viewer who cannot see their own message assumes the feature is broken.
Deep Dive B — Syncing with video time (replays)Deep dive
For overlays pinned to the video timeline, store video_ts_ms (the playback position when the comment was posted).
- Live: show comments as they arrive.
- Replay: the player requests comments in windows (e.g., the next 30 seconds of video time) and shows each one when playback reaches its
video_ts_ms. Seeking just loads a different window. - Pre-compute a sampled "highlight" set for very busy videos, so replays don't fetch millions of comments.
Trade-offs & AlternativesTrade-offs
| Decision | Choice | Why | Alternative |
|---|---|---|---|
| Transport | WebSockets | Instant push | Long polling: simpler, more overhead |
| Fan-out | Pub/sub → gateway servers → viewers | Scales to millions | Comment service pushes to every viewer: can't scale |
| Big streams | Per-viewer sampling | Keeps the UI readable and servers healthy | Deliver everything: impossible at 10K/sec |
| Storage | Wide-column by video + time | Fast recent and range reads | SQL: fine for small scale |
Common Follow-up QuestionsFollow-ups
- "How do you delete a comment already sent?" Publish a "remove comment_id" event on the same channel, and clients hide it.
- "Ordering?" Order per video by server timestamp. Small reordering is acceptable for comments.
- "Multi-region?" Each region has its own WebSocket servers. Publish comments to a global bus (or replicate channels across regions), so every region's servers receive them.
Wrap-UpWrap-up
Store each comment and publish it to a per-video pub/sub channel. WebSocket servers subscribe to the channels their viewers watch and fan out locally. That two-level fan-out scales to millions of viewers. For huge streams, sample and batch comments per viewer and aggregate reactions, and for replays, store the video timestamp so comments reappear in sync with playback.