Problem RestatementProblem
Design a platform that gives developers a development machine in the cloud, like GitHub Codespaces, Replit or a hosted notebook. A user opens a workspace (a project with its files and tools), edits code in a browser IDE or connects over SSH or a desktop IDE plugin, runs commands, builds and tests in isolation, and later comes back to find everything as they left it.
This was asked at OpenAI many times, with variations: SSH-based access with scheduled build/test tasks, a multi-tenant online IDE, and hosted notebooks.
RequirementsRequirements
1.1 Functional
- Create a workspace from a repo or template.
- Start, stop, delete, and resume with files preserved.
- Browser IDE (editor, terminal), plus SSH and IDE plugin access.
- Run commands and servers, and preview web apps through forwarded ports.
- Scheduled tasks (e.g., nightly build or test in the workspace).
- Different machine sizes (CPU, RAM, GPU).
1.2 Non-Functional
- Strong isolation: users run arbitrary code, and one tenant can never access another.
- Fast start: under ~10–30 seconds.
- Durable files: nothing lost when a machine dies.
- Cost efficient: idle workspaces shouldn't burn compute.
1.3 Scale Estimates
- 1M workspaces exist, 100K running at peak.
- Each running workspace uses ~2–4 vCPU and 8 GB RAM → a fleet of thousands of hosts.
- Storage: ~10 GB per workspace → 10 PB total (most of it idle, so cheap storage tiers).
1.4 API Design
POST /v1/workspaces{ repo, template, machine_type }→{ workspace_id }POST /v1/workspaces/{id}/start/stop/DELETEGET /v1/workspaces/{id}→{ state, ide_url, ssh_host }POST /v1/workspaces/{id}/tasks{ cron, command }- The IDE connects via WebSocket to the workspace agent (terminal, file sync, language server).
High-Level ArchitectureArchitecture
2.1 Overview
- Control plane: the workspace API, the state machine (
creating → starting → running → stopping → stopped), quotas and scheduling. - Scheduler: places a workspace on a host with capacity (bin-packing), using pre-warmed slots.
- Host fleet: each host runs many workspaces as isolated microVMs (Firecracker) or hardened containers (gVisor).
- Workspace agent inside each VM: terminal sessions, file operations, heartbeats and port forwarding.
- Gateway / proxy: authenticates users and routes browser, SSH and port traffic to the right workspace.
- Storage: a persistent volume per workspace (network block storage or snapshots to object storage).
2.2 Architecture Diagram
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Browser IDE / SSH / IDE plugin"] --> GW["Gateway - auth, routing"]
GW --> AG["Workspace agent in microVM"]
API["Control plane API"] --> DB[("Workspace DB")]
API --> SCH["Scheduler"]
SCH --> H["Host fleet - microVMs"]
H --> AG
AG --> VOL[("Persistent volume")]
VOL -->|"snapshot on stop"| OS[("Object storage")]
CRON["Task scheduler"] --> APIKey FlowsFlows
3.1 Start a workspace
- The API checks quota and sets the state to
starting. - The scheduler picks a host (right size, same region as the user) and ideally a pre-warmed microVM from a pool, which saves boot time.
- Attach the workspace volume. If stopped long ago, restore it from the object storage snapshot (lazy loading: fetch blocks on first access so startup is fast).
- The agent starts and heartbeats. The state becomes
running, and the gateway registers the routeworkspace_id → host:port. - The IDE connects over WebSocket through the gateway.
3.2 Idle and stop
If there's no activity (no keystrokes, terminal output or connections) for e.g. 30 minutes, stop the VM: flush and snapshot the volume, release the host slot, and set the state to stopped. Files are preserved, and compute cost goes to zero.
3.3 Scheduled tasks
The task scheduler wakes the workspace (starts it if stopped), runs the command via the agent, stores logs, and stops it again if it was stopped before.
Deep Dive A — Isolating a workspace where the user is rootDeep dive
A development workspace is not a sandbox the user is trying to escape — it is one we hand them root inside, on purpose, because apt install and docker build have to work. Isolation has to hold against a privileged user by design.
A container per workspace
Each workspace is a container on a shared host: own file system, own process namespace, CPU and memory limits.
Root inside a plain container is root against a shared kernel. The user is not even being malicious — they are installing kernel modules, running Docker, mounting things, all of which the product promised. Any one of those is a path to the host and to every other customer's workspace on it.
Harden the container
Map container root to an unprivileged host user (user namespaces), drop capabilities, apply a seccomp profile, forbid privileged containers.
This is real protection, and it breaks the product. The hardening that makes root safe is exactly the hardening that stops a developer from running Docker, debugging with ptrace, or mounting a FUSE file system. Every exception you grant to make a workspace usable reopens the hole. Container isolation and "you are root and everything works" cannot both be true.
A microVM per workspace
Give each workspace a Firecracker microVM with its own kernel. The boundary is the hypervisor, not kernel namespaces, so root inside is genuinely root of that machine and still nothing to the host.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Developer - root inside"] --> VM["Firecracker microVM - own kernel, ~100 ms boot"]
VM --> VOL[("Persistent volume - files survive restarts")]
VM --> NET["Default-deny between workspaces, egress via NAT"]
NET --> BLOCK["Cloud metadata endpoint blocked"]
VM --> TOK["Short-lived scoped token - push to this repo only"]
VM --> QUOTA["CPU / memory / disk / PID quotas"]Boot time is the objection people expect, and Firecracker answers it at roughly 100 ms — not slower than a container in any way the user notices.
Around the VM, the things that matter as much as the boundary itself:
- Network default-deny between workspaces, egress through a controlled NAT, and the cloud metadata endpoint blocked — otherwise a workspace reads the host's IAM credentials and the VM boundary was decoration.
- Short-lived scoped tokens injected from a vault, never a long-lived secret on disk. A workspace gets a token that can push to that user's repo and nothing else.
- Quotas on CPU, memory, disk and processes, so a runaway build starves its own workspace and not its neighbours.
Deep Dive B — Reliability and fast startupDeep dive
- Host failure: the agent's heartbeats stop, and the control plane marks the workspace
failed/recoveringand restarts it on another host from the latest snapshot. Unsaved in-memory state is lost, but files on the persistent volume survive (or, with network block storage, are fully intact). - Reconnect: IDE sessions reconnect automatically. Terminals run inside
tmux-like session managers in the VM, so a network blip doesn't kill a running build. - Fast start: prebuilt images per template, dependency caches, prebuilds (run
npm installwhen the repo changes, before anyone opens a workspace), and warm VM pools per region and machine size. - Capacity: bin-pack workspaces on hosts, overcommit CPU a little (most workspaces are idle), and autoscale the host fleet by demand and time of day.
Trade-offs & AlternativesTrade-offs
| Decision | Choice | Why | Alternative |
|---|---|---|---|
| Isolation | Firecracker microVMs | Strong, fast boot | Containers + gVisor: lighter, weaker than VMs |
| Storage | Persistent volume + snapshots | Durable, cheap when idle | Local disk only: fast, lost on host failure |
| Idle cost | Auto-stop after inactivity | Big savings | Always on: simple, expensive |
| Startup | Warm pools + prebuilds | Seconds instead of minutes | Cold boot: slow |
Common Follow-up QuestionsFollow-ups
- "Hosted notebooks?" The same platform: the "workspace" runs a Jupyter kernel instead of an IDE. Kernels are stateful, so also store notebook outputs and allow kernel restarts.
- "GPU workspaces?" A separate host pool, stricter quotas, shorter idle timeouts (GPUs are costly), and queueing when no GPU is free.
- "How do you bill?" Meter running time × machine size, plus storage per GB-month.
Wrap-UpWrap-up
A control plane manages each workspace's lifecycle and schedules it onto hosts, where it runs in its own microVM with a workspace agent. A gateway routes authenticated browser, SSH and port traffic to it. Files live on persistent volumes with snapshots, so idle workspaces can be stopped cheaply and restarted anywhere. Warm pools and prebuilds make startup fast, while microVM isolation, network rules and short-lived scoped tokens keep tenants safe.