Problem RestatementProblem
Design likes for a platform like Roblox (games) or any social app. Users like and unlike items. Each item shows a like count, and each user sees whether they liked it. Popular items get huge bursts ("a viral game gets 50,000 likes per second"). Reads far outnumber writes. A user can like an item at most once. Roblox asked this twice.
RequirementsRequirements
like(user, item),unlike(user, item): idempotent.get_count(item)andhas_liked(user, item)(often for a list of 50 items at once).- Optional: list who liked an item, and "liked by 3 friends".
- Counts can be slightly delayed (seconds) but must eventually be accurate.
1.1 Scale Estimates
- 1B likes stored (~50 bytes each → 50 GB, sharded).
- Writes: 20K/sec normally, 100K/sec bursts, concentrated on a few hot items.
- Reads: 500K/sec (every page shows counts).
Data ModelData model
likes: (item_id, user_id) PRIMARY KEY, created_at -- source of truth, one row per like
user_likes: (user_id, item_id), created_at -- "did I like it?" / "my liked items"
like_counts: item_id → count -- derived, cachedThe primary key (item_id, user_id) guarantees one like per user per item. Inserting twice does nothing.
ArchitectureArchitecture
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Clients"] --> API["Likes API"]
API --> DB[("Likes DB - sharded by item_id")]
API --> K[("Like events")]
K --> AGG["Counter aggregator - batches +1/-1"]
AGG --> CNT[("Counts - Redis + DB")]
U -->|"read counts"| RD["Read API"]
RD --> CNT
RD --> UL[("user_likes cache")]Deep Dive — 50,000 likes a second on one itemDeep dive
A game goes viral. Every like is a write to the same counter, and every viewer reads it. This is the hot-row problem in its purest form.
Increment a column
UPDATE items SET like_count = like_count + 1 WHERE item_id = ?
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
L["50,000 likes/sec - one item"] --> ROW["Single row in the items table"]
ROW --> LOCK["Every write locks the same row"]
LOCK --> SER["Serialised - a few thousand/sec at best"]
LOCK --> WAIT["Transactions queue, then time out"]
WAIT --> SPREAD["Connection pool exhausted - unrelated queries fail"]Row-level locking means the writes run one at a time regardless of how many database servers exist. Throughput is capped an order of magnitude below the requirement, and the backed-up transactions take the connection pool down with them.
Count in Redis
INCR likes:{item_id}, served from memory, with the value written back to the database periodically.
In-memory increments handle tens of thousands per second comfortably, and this is the right shape. Two gaps: a single hot key still lands on one Redis node, so the very hottest items concentrate on one core; and nothing here stops a user from liking twice — a double tap or a retry increments again, and the count drifts away from reality with no way to recompute it.
Shard the counter, and keep the fact separate from the count
Split each item's counter across N keys and sum on read, and store who liked what as its own durable fact:
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Like from user 42"] --> SET{"Add to set liked:item - user 42"}
SET -->|"already there"| NOOP["No-op - at most one like per user"]
SET -->|"newly added"| SH["INCR likes:item:shard_N - N = hash(user) % 16"]
SH --> RD["Read: sum 16 shards, cache 1-5 s"]
SH --> WB["Write-behind to the database every few seconds"]
WB --> REC["Reconciliation - recount from the like facts"]- Sharded counters spread one item's writes across 16 keys and therefore across nodes. A read sums 16 small numbers and caches the result for a second or two — nobody can tell a viral counter apart from one that is two seconds stale.
- A membership set per item enforces "at most one like per user" and makes the operation idempotent: a retried like is a set add that changes nothing. It also answers "did I like this?", which the page needs anyway.
- The facts are the source of truth. The counter is a cache of them, so it can be recomputed. Reconciliation periodically recounts from the like records and corrects drift, which a bare counter can never do.
For items with millions of likers, the membership set gets expensive — keep the full set in the durable store and a per-user bitmap or a bloom-style filter in cache for the read path.
Key FlowsFlows
5.1 Like
INSERT INTO likes (item_id, user_id) ... ON CONFLICT DO NOTHING.- If a row was actually inserted (not a duplicate), publish
{ item_id, +1 }. If it was a duplicate, do nothing, which makes the API idempotent. - Update
user_likesso the user immediately sees their own like (read-your-writes).
5.2 Unlike
Delete the row. If a row was deleted, publish { item_id, -1 }.
5.3 Counting without hot-row contention
If every like did UPDATE counts SET n = n + 1 WHERE item_id = X, a viral item's single row would become a bottleneck. Instead:
- Batch in the aggregator: consume events and sum them per item for ~1 second, then apply one
INCRBY item, +4312. Thousands of writes become one. - Sharded counters for extreme items: split the count into N sub-counters (
count:item:0..15). Writers pick one at random, and readers sum them (cache the sum briefly). - Store counts in Redis for fast reads, and persist them to the DB periodically.
5.4 Reading a page of 50 items
- One multi-get for 50 counts from Redis.
- One multi-get for
has_likedof those 50 for this user (a per-user set of recently liked items in cache, falling back touser_likes).
Accuracy and Reconciliation
- Counts are eventually consistent: the aggregator may lag a second or two.
- If events are lost or double-applied, counts drift. A nightly job recounts
SELECT count(*)per item (or per changed item) from the source-of-truth table and fixes the counter. - Clients show the user's own like instantly (optimistic UI), even before the count updates.
- For display, large numbers can be approximate ("1.2M"), which also hides small lag.
Trade-offs & AlternativesTrade-offs
| Decision | Choice | Why | Alternative |
|---|---|---|---|
| Truth | One row per (item, user) | Idempotent, auditable, supports "who liked" | Only a counter: can't prevent double likes |
| Counting | Async aggregation + batching | Handles hot items | Synchronous row increment: hot-row contention |
| Hot items | Sharded counters | Spreads writes | Single key: bottleneck |
| Correctness | Periodic recount | Fixes drift | Trust counters forever: slow drift |
Wrap-UpWrap-up
Store each like as a row keyed by (item, user) so likes are idempotent, and keep a user-side index for "did I like it". Publish +1/-1 only when a row actually changes, aggregate those events in batches (with sharded counters for viral items) into Redis-backed counts, and serve pages with multi-gets. Accept a second or two of lag, show the user's own action instantly, and reconcile counts with a periodic recount.