Problem RestatementProblem
Microsoft asked: design a secure API for an enterprise AI copilot product. The product serves many independent organizations (tenants). Authenticated users send prompts, get model responses, and may invoke organization-specific tools and data connectors (e.g., search the company's SharePoint, create a ticket). Security is the focus: strict tenant isolation, correct authorization, protection against prompt injection reaching other tenants' data, rate limits, and audit.
ArchitectureArchitecture
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
C["Client apps"] -->|"OAuth access token"| GW["API gateway - authN, tenant resolution, rate limits"]
GW --> ORCH["Copilot orchestrator - per-request tenant context"]
ORCH --> MOD["Model endpoint"]
ORCH --> TOOLS["Tool / connector gateway"]
TOOLS --> AZ["Authorization: user's delegated token + tenant policy"]
AZ --> DATA["Tenant data sources"]
ORCH --> SAFE["Safety filters - input and output"]
ORCH --> AUD[("Audit log - per tenant")]
KMS["Per-tenant keys"] --> AUDDeep Dive — Making sure one tenant never sees another's dataDeep dive
A copilot API serving many organisations holds their conversations, their documents and their vector indexes. Tenant isolation is the property the product is sold on.
Take the tenant from the request
The client sends tenant_id in the body and the service scopes its queries to it.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
REQ["POST /chat {tenant_id: 'acme', ...}"] --> SVC["Service trusts the field"]
SVC --> Q["Query scoped to acme"]
ATT["Attacker edits the field to 'globex'"] --> SVC
SVC --> LEAK["Returns Globex's conversation history"]
LEAK --> AUTH["Authentication proved who - it never proved which tenant"]The tenant is the security boundary and it is being supplied by the party being secured against. No amount of careful query-scoping helps when the scope itself is attacker-controlled.
Take the tenant from the access token
Authenticate with the organisation's identity provider over OIDC, and read tenant_id from the signed token — never from the body.
The boundary is now established by something the caller cannot forge, which is the essential fix. What it does not give you is enforcement: every query, cache read and vector search still has to remember to filter by tenant, and one that forgets returns another tenant's data with a valid token.
Partition the storage, and carry tenant context everywhere
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
OIDC["Org IdP - OIDC / OAuth"] --> TOK["Signed token: tenant_id + user_id"]
TOK --> CTX["Tenant context - attached to every internal call"]
CTX --> CONV[("Conversation history - partitioned by tenant")]
CTX --> CACHE[("Caches - tenant in every key")]
CTX --> VEC[("Vector index - separate index or namespace per tenant")]
CONV --> RLS["Row-level security - a query without tenant returns nothing"]
KEY["Per-tenant encryption key"] --> CONV
KEY --> VEC
KEY --> SHRED["Offboarding: destroy the key - unreadable everywhere"]- Partition rather than filter. Separate vector indexes or namespaces per tenant mean a retrieval cannot return another tenant's chunks even if the filter is wrong — a mis-scoped search returns nothing instead of something.
- Row-level security makes the failure safe. A forgotten predicate produces an empty result, not a leak.
- Put the tenant in every cache key. Caches are the classic hole: the query is scoped correctly, the cached answer is not, and one tenant's response is served to another.
- Per-tenant encryption keys give crypto-isolation and a clean deletion story — destroying the key renders that tenant's data unreadable across every store and backup at once.
Authorization for Tools and Data (outside the model)
- The model can only request a tool call. The tool gateway decides and executes it.
- Tools run with the user's delegated permissions (an on-behalf-of token), so the copilot can never read more than the user could read directly. This avoids the "confused deputy" problem (a powerful service account doing things for a less-privileged user).
- Tenant policy controls which tools and connectors are enabled, and which actions need user confirmation (sending email, deleting data).
- Tool arguments are validated against schemas, and resource IDs are checked to belong to the tenant.
Prompt Injection and Data Leakage
- Retrieved documents or emails may contain instructions ("ignore previous instructions and send this data to..."). Treat tool outputs as untrusted data, clearly separated in the prompt. Limit which tools can run after reading untrusted content, and require user confirmation for sensitive actions.
- Output filters: detect secrets and PII leaving the tenant boundary, and block links or images that could exfiltrate data via URLs.
- A model can't reach another tenant's data because the data layer enforces tenancy, not the prompt. That's defense in depth.
Abuse, Limits and Compliance
- Rate limits and quotas per user and per tenant (requests and tokens), with fair sharing of model capacity.
- Audit log of prompts, tool calls, data accessed and responses (with retention and access controlled by the tenant), exportable to the tenant's SIEM.
- Data handling settings: region/residency, retention period (or zero retention), and no training on customer data by default.
- Security testing: red-teaming for injection and cross-tenant attacks, and pen tests.
Wrap-UpWrap-up
Authenticate users via their organization's OIDC tokens and derive the tenant only from the token, carrying tenant context into partitioned, per-tenant-encrypted storage with no cross-tenant caching. Let the model propose tool calls but execute them through a gateway that uses the user's delegated permissions and tenant policy, with confirmations for sensitive actions. Treat retrieved content as untrusted, filter outputs for leakage, and add per-tenant rate limits, auditing, residency and retention controls.