Problem RestatementProblem
Goldman Sachs asked (Superday): design Pastebin. Users paste text or code, click "create", and get a unique short link (e.g., paste.example/aB3xK9q). Anyone with the link can view it. Options: an expiry (10 minutes, 1 day, never), public or unlisted visibility, syntax highlighting, and a size limit (e.g., 10 MB). The interviewer expects the full flow: requirements, core entities, APIs, then the high-level design.
RequirementsRequirements
- Functional: create a paste (text, title, language, expiry, visibility) → short link. View a paste. Optionally delete (by the owner) and list "my pastes".
- Non-functional: very read-heavy (a paste may be viewed thousands of times), low-latency reads, high availability, durable storage, and unguessable links for unlisted pastes.
1.1 Estimates
- 10M new pastes/day ≈ 115 writes/sec. Reads 10:1 → ~1,200/sec, with spikes for viral pastes.
- Average size 10 KB → 100 GB/day → ~36 TB/year, so store content in object storage, not the DB.
Core Entities and APIs
paste: key, owner_id?, title, language, visibility (public|unlisted), size, content_ref,
created_at, expires_atPOST /v1/pastes{ content, title?, language?, expires_in?, visibility? }→{ key, url }GET /v1/pastes/{key}→ metadata + content (or a redirect to a CDN URL for the raw content)DELETE /v1/pastes/{key}(owner only)
High-Level DesignArchitecture
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["User"] --> API["Paste API"]
API --> KG["Key generator - random base62"]
API --> DB[("Metadata DB - key to paste")]
API --> OS[("Object storage - content")]
V["Viewer"] --> CDN["CDN / cache"]
CDN --> API
API --> C[("Redis - hot metadata")]
JOB["Expiry cleanup job"] --> DB
JOB --> OS3.1 Create
- Validate the size (≤ 10 MB) and rate-limit per IP or user.
- Generate a key: 8 random base62 characters (62^8 ≈ 218 trillion), so it can't be guessed. On the rare collision, retry (insert with "if not exists").
- Upload the content to object storage at
pastes/{key}, then insert the metadata row (content first, so a paste never points at missing content). - Return the URL.
3.2 View
- Request → CDN. Popular public pastes are cached at the edge (a TTL up to their expiry).
- On a miss: the API reads metadata (Redis, then DB), checks expiry, and streams the content from object storage (or redirects to a signed CDN URL).
- Syntax highlighting happens in the browser (a JS library), so the server just returns text and language.
Deep Dive — Making expiry actually expireDeep dive
A paste is set to expire in ten minutes. When that moment passes, nobody should be able to read it — and its bytes should stop costing money.
Delete expired pastes in a background job
A job runs periodically and removes rows whose expires_at has passed.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
EXP["Paste expires at 14:00"] --> JOB["Cleanup job runs at 14:05"]
JOB --> GAP["14:00 to 14:05 - still fully readable"]
JOB -->|"job is backed up or failed"| LONG["Readable for hours"]
LONG --> BREACH["A 10-minute secret lives all afternoon"]Expiry is a promise the product makes and the job is best-effort. Anything that delays it — a backlog, a failed deploy, a paused cron — silently extends the lifetime of content the user expected to be gone.
Check expiry on read
Compare expires_at on every read and return 404 when it has passed.
Now expiry is exact from the reader's point of view: the instant the timestamp passes, the paste is unreachable regardless of what any job is doing. This is the important half and it is not sufficient on its own — expired rows and their content objects accumulate forever, so storage grows without limit even though nothing is readable.
Enforce on read, reclaim in the background
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
GET["GET /p/aB3xK9q"] --> CHK{"now > expires_at?"}
CHK -->|"yes"| N404["404 immediately - authoritative"]
CHK -->|"no"| SERVE["Serve the content"]
SWEEP["Batch job over an index on expires_at"] --> DELM["Delete metadata rows"]
SWEEP --> DELO["Delete content objects"]
LIFE["Object store lifecycle rules / DB TTL"] --> DELO
BURN["Burn after reading"] --> COND["Conditional delete on first successful view"]
COND --> ONCE["Two simultaneous readers - exactly one wins"]- The read check is the guarantee; the sweeper is housekeeping. Separating them means correctness never depends on a job running on time.
- Reclaim in batches through an index on
expires_at, or hand it to the infrastructure — object store lifecycle rules and database TTLs do this natively and cannot fall behind. - Burn-after-reading needs a conditional delete, not a read followed by a delete. Two readers arriving together must resolve to exactly one winner, and only an atomic operation gives that.
One ordering detail that matters on the write side too: upload the content before inserting the metadata, so a paste never points at an object that does not exist — and on delete, remove the metadata first, so nothing can reference bytes that are about to disappear.
Abuse and Safety
- Rate limits on creation, size limits, and CAPTCHAs for anonymous heavy users.
- Scan content for malware links or leaked secrets (API keys), and support takedown requests.
- Unlisted pastes aren't indexed or listed publicly, so only the random key grants access.
Wrap-UpWrap-up
Store paste text in object storage and small metadata (key, visibility, expiry, language) in a database with a Redis cache, keyed by random 8-character base62 IDs generated with an insert-if-absent retry. Serve reads through a CDN and cache, since the load is heavily read-dominated, enforce expiry on read plus background cleanup, and protect the service with size limits, rate limiting and content scanning.