Problem RestatementProblem
Design a digital game store like Steam or the Epic Games Store. Users browse a catalog, buy games (often on sale), and see them in their library. They download large game files (tens of GB) and get patches when games update. Refunds, regional prices, promotions and license checks when a game starts are all part of it. Launch days bring huge spikes: millions of people buying and downloading the same game at the same hour.
RequirementsRequirements
1.1 Functional
- Catalog: browse, search, game pages and prices per region.
- Cart, purchase, and gifting.
- Library and entitlements (who owns which game and DLC).
- Download and install, and update with patches.
- Refunds (e.g., within 14 days and under 2 hours played).
- Promotions and discount codes.
1.2 Non-Functional
- Money correctness: never charge twice, never grant a game without payment (or the reverse).
- Fast downloads worldwide.
- Survive launch spikes for purchases and downloads.
- High availability for game launch (license checks shouldn't stop people from playing).
1.3 Scale Estimates
- 100M users, 5M purchases/day ≈ 60/sec, with spikes of 5K/sec at big launches or sales.
- Downloads: a 60 GB launch × 2M buyers on day one = 120 PB of traffic. Only a CDN can deliver that.
1.4 API Design
/v1/games/{id}?region=IN(price in local currency)/v1/orders(Idempotency-Key) { items: [{ game_id, edition }], promo_code? }/v1/users/me/library/v1/games/{id}/builds/latest/manifest→ the list of files and chunks for download/v1/orders/{id}/refund/v1/licenses/check{ game_id, device_id } → a signed license tokenHigh-Level ArchitectureArchitecture
2.1 Overview
- Catalog service + search index + CDN-cached game pages.
- Pricing & promotions: regional prices, sale schedules and discount codes.
- Order service: the order state machine, integrated with payments.
- Entitlement service: the source of truth for ownership. It grants on successful payment and revokes on refund or chargeback.
- Content pipeline: developers upload builds, and the pipeline chunks, compresses and signs them.
- CDN: serves game chunks.
- License service: issues signed tokens that allow offline play for a period.
2.2 Architecture Diagram
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Player"] --> CAT["Catalog + Pricing"]
U --> ORD["Order Service"]
ORD --> PAY["Payment Service"]
ORD -->|"paid event"| ENT["Entitlement Service"]
ENT --> EDB[("Entitlements DB")]
U --> LIB["Library"]
LIB --> ENT
DEV["Developers"] --> PIPE["Build pipeline - chunk, sign"]
PIPE --> OS[("Build storage")]
OS --> CDN["CDN"]
U -->|"download chunks"| CDN
U --> LIC["License Service - signed tokens"]
LIC --> ENTData ModelData model
games: game_id, title, developer_id, release_at, status
prices: game_id, region, currency, amount, valid_from, valid_to
orders: order_id, user_id, items, total, currency, status (created, paid, fulfilled, refunded), idempotency_key
entitlements: user_id, game_id, source (purchase|gift|promo), order_id, status (active/revoked), granted_at
builds: game_id, build_id, version, manifest_key, status (staged, live, rolled_back)Key FlowsFlows
4.1 Purchase
- The client creates an order with an idempotency key. The price is locked at order time.
- Payment is authorized and captured (see the payment system design).
- On a successful payment event, the entitlement service grants the game. The grant is idempotent (by order_id), so a replayed event can't grant twice.
- The game appears in the library, and the download can start.
4.2 Download and patch
- The client gets the manifest: the list of files, each split into chunks identified by content hash.
- It downloads only the chunks it doesn't have, from the nearest CDN edge, in parallel, and verifies each hash.
- Patching: a new build's manifest shares most chunks with the old one, so the client only downloads the changed chunks. A 2 GB patch instead of 60 GB.
4.3 Refund
Check the policy (time since purchase, hours played), refund through payments, and revoke the entitlement. The next license check fails, so the game can't be launched.
Deep Dive A — Launch nightDeep dive
A 90 GB game unlocks at midnight and ten million people want it at once. That is roughly 900 petabytes of egress concentrated into a few hours, against a CDN that serves a fraction of that on a normal day.
Open the download at launch
The build becomes available at 00:00 and clients start pulling.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
T["00:00:00"] --> M["10M clients request 90 GB"]
M --> CDN["CDN edge - saturated in seconds"]
CDN --> ERR["Timeouts and 503s"]
ERR --> RETRY["Every client retries immediately"]
RETRY --> CDN
ERR --> SLOW["Players who got through crawl at 200 KB/s"]The demand curve is a vertical line, and no amount of capacity planning smooths a vertical line. The retry loop makes it worse: failed clients re-request instantly and in unison, so the surge sustains itself well past the initial spike.
Buy more capacity and pre-fill it
Contract several CDN providers, push the build to every edge location before launch, and add peer-assisted delivery where the network allows it.
This is necessary work and it treats the symptom. You are still buying peak capacity for one night that then sits idle, the cost scales with the size of the launch, and a popular enough release will exceed whatever you provisioned. The shape of the demand has not changed.
Move the bytes early and gate them with a key
Let buyers download the encrypted game days in advance. At launch, release a small decryption key.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
PRE["Days before launch"] --> DL["Clients download the encrypted build - spread over days"]
DL --> DISK["90 GB sitting on the player's disk, unusable"]
T0["00:00 launch"] --> KEY["Release a few-KB decryption key"]
KEY --> PLAY["Players decrypt locally and play immediately"]
T0 --> LOAD["Launch-night bandwidth: kilobytes per user"]The 900 petabytes now flow across several days at whatever rate the network has spare, and the midnight event moves a few kilobytes per player. This is the only rung that changes the shape of the curve rather than trying to absorb it — and it gives players a better experience too, because the game starts at midnight instead of downloading until morning.
Around it:
- Purchases are a separate spike. Put a queue in front of order creation, pre-scale, and cache catalogue pages hard — a game page is byte-identical for everyone in a region.
- Client back-off with jitter on CDN errors, so a partial failure does not become a synchronised retry storm.
- Stage the patches that follow: 5% of players first, watch crash rates, then the rest — with a kill switch back to the previous build manifest.
Deep Dive B — Entitlements and licensesDeep dive
- The entitlement DB is the single source of truth for ownership. Orders, gifts, promos and refunds all go through it.
- License tokens: when a game starts, it asks for a signed token (valid e.g. 30 days) with
user, game, device, expiry. The game verifies the signature offline, so short outages of the license service don't stop play. - Fraud: chargebacks revoke entitlements, and stolen cards trigger holds. Rate-limit gift purchases, which are a common fraud path.
- Staged rollouts for builds: release a new patch to 5% of players first, watch crash rates, then 100%. Keep a kill switch to roll back to the previous build manifest.
Trade-offs & AlternativesTrade-offs
| Decision | Choice | Why | Alternative |
|---|---|---|---|
| Ownership | Separate entitlement service | One truth for purchases, gifts, refunds | Derive from orders each time: slow, error-prone |
| Downloads | Content-hashed chunks + CDN | Delta patches, verification, dedup | Whole-file downloads: huge patches |
| Launch | Pre-load + key release | Spreads bandwidth over days | Everyone downloads at launch: CDN overload |
| License | Signed offline tokens | Play survives outages | Online check every launch: fragile |
Common Follow-up QuestionsFollow-ups
- "Regional pricing and taxes?" Store prices per region and currency, pick the region from the billing address (not just IP), and add tax at checkout using a tax service.
- "Flash sales?" Schedule price changes ahead of time and pre-warm caches. The price in the order is what counts, even if the sale ends during checkout.
- "Family sharing?" Model it as entitlements that point to the owner's entitlement, with limits on concurrent play.
Wrap-UpWrap-up
Handle purchases with idempotent orders and payments, and let a dedicated entitlement service grant and revoke ownership as the single source of truth. Deliver games as content-hashed chunks through CDNs so patches only download changed chunks, pre-load big launches and release keys at launch time, and issue signed offline license tokens so play doesn't depend on the store being up.