Problem RestatementProblem
Bloomberg asked (90-minute round): design an internal web application that supports:
- User onboarding (new employees or clients get accounts),
- Password reset,
- Requesting company information (reports, documents),
- IT help requests (tickets).
Requirements: parts of the system can be deployed independently, every request is authenticated and validated, and the data is highly sensitive financial information. Expect questions on SQL vs NoSQL, ACID and CAP, load balancing, caching, blue/green deployments, API gateways, and cloud vs on-premise.
ArchitectureArchitecture
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
U["Employees - browser"] --> WAF["WAF + load balancer (TLS)"]
WAF --> GW["API gateway - token validation, rate limits, request validation"]
IDP["Identity provider - SSO, MFA"] --> GW
GW --> ON["Onboarding service"]
GW --> PW["Password / account recovery service"]
GW --> INFO["Company info service"]
GW --> IT["IT help desk service"]
ON --> SQL1[("Postgres - users, roles")]
INFO --> DOCS[("Document store + object storage - encrypted")]
INFO --> CACHE[("Cache - non-sensitive metadata")]
IT --> SQL2[("Postgres - tickets")]
ON --> AUD[("Audit log")]
INFO --> AUD- Separate services per capability, each with its own database and CI/CD pipeline → independent deployments.
- API gateway: one entry point that validates tokens, enforces rate limits and request schemas, and routes traffic.
Authentication and Authorization
- SSO via a central identity provider (OIDC/SAML), with MFA required (hardware keys or an authenticator app for sensitive roles).
- Short-lived access tokens, and RBAC: roles like analyst, manager, IT agent, with least privilege. Sensitive documents are checked per request against entitlements.
- Password reset: prefer passwordless/SSO. If passwords exist: time-limited single-use reset tokens sent to a verified channel, identity verification (MFA or manager approval for privileged accounts), rate limiting, and notifications to the user.
- Onboarding: triggered from the HR system, creates accounts with default roles, and requires manager approval for extra access. Offboarding revokes everything immediately.
Deep Dive — Choosing where each kind of data livesDeep dive
The app holds users, roles, tickets, uploaded documents and reports. Treating them as one storage problem produces the wrong answer for most of them.
Put everything in one document store
One NoSQL database holds users, roles, tickets and file contents.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
DOC[("Single document store")] --> T["Assign a ticket to an agent"]
T --> RACE["Two assignments race - no transaction - ticket has two owners"]
DOC --> ROLE["Revoke a role + update its permissions"]
ROLE --> HALF["One write lands, one does not - inconsistent permissions"]
DOC --> FILE["50 MB PDFs stored as documents"]
FILE --> COST["Database bloated, backups slow, reads expensive"]The relationships here need transactions — a ticket has exactly one assignee, a role change must apply wholly or not at all — and storing large binaries in a database is expensive in every dimension.
Relational for the core, and keep files out of it
Users, roles and tickets go in PostgreSQL where ACID transactions and foreign keys enforce the invariants. Documents go to object storage with their metadata in SQL.
This is the right split and covers most of the answer: moderate data sizes, strong integrity, cheap and scalable blob storage, with the database holding only the pointer.
What is still unstated is how the app behaves when a store is slow or unavailable — and for an app gating access to internal data, that is a security question, not only an availability one.
State the consistency choice and what may be cached
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
SQL[("PostgreSQL - users, roles, tickets - ACID")] --> CP["Under partition: choose consistency"]
CP --> REJECT["Reject the request rather than serve stale permissions"]
OBJ[("Object storage - encrypted documents")] --> META["Metadata + pointer in SQL"]
META --> SRCH["Search index - built from SQL, permission-filtered at query time"]
CACHE["Cache"] --> SAFE["Only non-sensitive or permission-safe data"]
CACHE --> NEVER["Never cache authorisation decisions across users"]- Prefer consistency over availability for anything account- or permission-related. A brief error is recoverable; showing a stale permission — granting access that was revoked — is not. Saying which side of CAP you sit on, and why it differs per data type, is what the question is testing.
- Never cache an authorisation decision across users, and keep TTLs on any permission-derived cache short. This is the same mistake as an un-keyed feed cache, with worse consequences.
- Encrypt documents at rest with access mediated by short-lived signed URLs, so the object store cannot be read directly even if a URL leaks.
Choosing per data type rather than per application is the whole point: the same system can be strongly consistent for roles and eventually consistent for a search index, provided the boundary is deliberate and the search results are re-checked against permissions before they are returned.
Security in Depth
- TLS everywhere (mTLS between services), encryption at rest with managed keys (KMS/HSM), and field-level encryption for the most sensitive fields.
- Input validation at the gateway and in each service, with protection against injection attacks and CSRF.
- A complete audit log (who viewed or changed what), which is immutable and monitored for anomalies (bulk downloads).
- Network segmentation: services in private subnets, with the admin tools reachable only via VPN or a zero-trust proxy.
Reliability and Deployments
- Stateless services behind load balancers across zones, and database primaries with standby replicas and backups.
- Blue/green deployments: run the new version (green) alongside the old (blue), switch traffic after checks, and switch back instantly if problems appear. Database changes must be backward-compatible (expand/contract migrations).
- Cloud vs on-prem: cloud gives managed security services and elasticity. On-prem may be required by regulations or data residency. Hybrid is common. Discuss the compliance requirements (e.g., SOC 2 and financial regulators).
Wrap-UpWrap-up
Split the app into independently deployable services behind a WAF, load balancer and API gateway that validates every request. Authenticate with SSO + MFA, authorize with least-privilege RBAC and per-document entitlements, and handle password reset and onboarding with verified, time-limited, audited flows. Use ACID relational stores for users and tickets and encrypted object storage for documents, cache only permission-safe data, encrypt everything, audit every access, and release with blue/green deployments and backward-compatible migrations.