Problem RestatementProblem
Bloomberg asked: there's an API that answers "Is date D a holiday in market X?" (or "list holidays for market X in a range", or "what's the next business day?"). Today it calls a downstream service on every request, which is slow (tens to hundreds of ms) and sometimes unavailable. Many systems (trading, settlement date calculations) call it heavily. Make it much lower latency while keeping answers correct.
Deep Dive — Serving data that is tiny and barely changesDeep dive
"Is 12 March a holiday in market X?" is asked constantly and the answer changes a few times a year. Sizing the data first is what makes the design obvious.
Call the downstream service per request
Every lookup forwards to the upstream holiday service.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
REQ["Every request"] --> DS["Downstream holiday service"]
DS --> LAT["Network latency on a lookup that is a set membership test"]
DS --> DEP["Our availability is capped by theirs"]
DS --> RL["Their rate limit becomes our throughput limit"]
DS --> SAME["The same answer, returned millions of times"]A few hundred kilobytes of near-static data is being fetched over the network on every call. It also inverts the dependency: a service answering a question about a fixed calendar is only as available as a system it does not control.
Cache with a TTL
Cache each (market, date) answer for an hour.
Load on the downstream collapses and latency improves. Two things remain awkward. The cache is sparse, so the first request for any uncached date still pays the network — and "next business day" walks forward through several dates, each potentially a miss. And a TTL means correctness is a race: a newly announced closure is wrong for up to an hour, with no way to force it.
Load the whole dataset into memory
Do the arithmetic out loud: ~100 markets × ~15 holidays a year × a few years ≈ tens of thousands of dates, a few hundred KB. That fits in memory many times over.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
SRC["Downstream holiday service"] --> LOAD["Scheduled full load - every few minutes"]
LOAD --> SNAP["In-memory snapshot, version stamped"]
SNAP --> Q1["is_holiday - hash set lookup, microseconds"]
SNAP --> Q2["next_business_day - loop in memory"]
SNAP --> Q3["holidays in range - filter in memory"]
LOAD -->|"downstream unavailable"| KEEP["Keep serving the last good snapshot"]
ADMIN["Urgent closure announced"] --> FORCE["Forced refresh - no waiting for a TTL"]- Every query becomes local.
is_holidayis a set lookup, andnext_business_dayis a loop over in-memory data instead of a sequence of network calls. - Refresh on a schedule, not on expiry. The whole dataset is reloaded periodically and swapped in atomically with a version stamp, so there is no per-key staleness to reason about and responses can say which version answered them.
- A downstream outage stops mattering. The last good snapshot keeps serving, because holiday data from ten minutes ago is still correct.
The transferable lesson, and the reason this question gets asked: size the data before designing around it. A surprising amount of reference data — currency codes, market calendars, tax rates, country lists — is small enough to hold entirely in memory, and treating it as a remote lookup adds latency, coupling and failure modes for nothing.
Design
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
CL["Callers"] --> SVC["Holiday service - in-memory calendars"]
SVC -->|"lookup: hash set per market"| MEM["Memory: market to set of dates + version"]
SRC["Downstream holiday source"] -->|"periodic full load / change events"| LOAD["Loader"]
LOAD --> MEM
LOAD --> SNAP[("Local snapshot file - last good version")]- Preload: at startup, load the full calendars (e.g., next 5 years + last 5 years) into memory:
market → set of holiday dates, plus weekend rules per market (some markets have Friday–Saturday weekends). - Lookup = a hash set check, taking microseconds. "Next business day" = step forward until a date is neither a weekend nor in the set.
- Refresh: re-fetch the full data every few minutes (it's small), or subscribe to change events from the source. Swap atomically to the new version (build the new map, then replace the reference), so readers never see a half-updated calendar.
- Fallback: persist the last good snapshot to local disk. If the downstream is down at startup or refresh, keep serving the last good version and alert.
- Clients can embed it: for the lowest latency, ship a client library that keeps its own in-memory copy (with the same refresh logic), so callers don't even need a network hop.
Correctness and Freshness
- Every response can include the
calendar_versionandas_of. - Validation before swap: sanity checks on new data (no market suddenly missing, no absurd number of holidays). Reject bad updates and alert, rather than serving wrong calendars.
- Emergency closures (e.g., a market closed for an unexpected event): a push-based update path (change event → all instances refresh within seconds).
- Consistency across regions: all instances converge on the same version quickly. Monitor version skew.
If Data Were Large (general lesson)
For larger reference data, use a layered cache: an in-process LRU (fast) → a shared cache like Redis (shared warm data) → the source. Use TTLs plus explicit invalidation on change events, and request coalescing so a cache miss for a hot key triggers only one downstream call.
Wrap-UpWrap-up
Holiday calendars are tiny and rarely change, so preload them fully into memory as per-market date sets (with weekend rules), answer lookups and next-business-day queries in microseconds, and refresh periodically or on change events with validation and atomic version swaps. Keep a local last-good snapshot for downstream outages, expose version metadata, and optionally embed the cache in a client library to remove the network hop entirely.