Snap Map Design

0Problem RestatementProblem

Design a real-time location sharing and discovery platform like Snap Map. The system should allow millions of users to broadcast their live location to friends and discover trending geographic "hotspots" or clusters of activity. Key goals include ultra-low latency updates, high scalability for 100M+ DAU, and robust privacy controls.

1RequirementsRequirements

1.1 Functional

• Live Location Updates: Users publish their GPS coordinates every few seconds.
• Friend View: See precise friend locations (pins) on a map interface.
• Hotspots/Clustering: View aggregated clusters of anonymous user activity when zoomed out.
• Privacy Controls: Support "Ghost Mode", custom visibility lists, and location rounding.
• Auto-Expiry: Locations should automatically expire after a few hours of inactivity.

1.2 Non-Functional

• Low Latency: End-to-end latency for location updates should be < 2 seconds.
• Scalability: Handle 100 million Daily Active Users (DAU).
• Battery Efficiency: Minimize mobile device power consumption through adaptive polling.
• Reliability: Ensure location sharing persists even during intermittent connectivity.

1.3 Scale Estimates

• Update Frequency: ~30 seconds when moving, adaptive when static.
• Fan-out Load: High variance; updates pushed only to active friend viewers.

1.4 API Design

The core APIs required for the service:

2High-Level ArchitectureArchitecture

2.1 Overview

• Location Ingestion Service: Validates and enriches incoming updates (e.g., adding H3/S2 cell indices).
• Hot Store (Redis): Low-latency storage for the most recent location of every active user.
• Pub/Sub (Kafka): Decouples the ingestion flow from the fan-out and analytics pipelines.
• Fan-out Service: Identifies active friend observers and pushes updates via WebSockets.
• Clustering Service: Periodically aggregates location data into geographic tiles for heatmap rendering.

2.2 Architecture Diagram

%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart TB
    U1["Sharer (Mobile)"] -->|"1. Publish Location"| AG["API Gateway"]
    AG -->|"2. Validate/Ingest"| LS["Ingestion Service"]
    LS -->|"3. Write with TTL"| Redis[(Redis - Hot Store)]
    LS -->|"4. Stream Event"| K["Kafka"]
    
    K -->|"5. Process"| FS["Fan-out Service"]
    FS -->|"6. Push Pin"| WS["WebSocket Gateway"]
    WS -->|"7. Update friend"| U2["Friend (Mobile)"]
    
    K -->|"8. Aggregate"| CS["Clustering Service"]
    CS -->|"9. Write Tiles"| TileDB[(Geo-Tile DB)]
    U2 -->|"10. Fetch Map"| CS
    
    classDef sharer fill:#f0f8ff,stroke:#333,stroke-width:1px;
    classDef core fill:#e0ffe0,stroke:#333,stroke-width:1px;
    classDef async fill:#fff0f0,stroke:#333,stroke-width:1px;
    class U1,U2 sharer;
    class AG,LS,Redis core;
    class K,FS,CS,TileDB async;

3Data ModelData model

LivePoint (Redis Hot Store)

{
  "user_id": "string",
  "lat": "float",
  "lon": "float",
  "timestamp": "long",
  "accuracy": "float",
  "cell_id": "string (H3/S2)"
}

UserShareConfig (PostgreSQL)

{
  "user_id": "string",
  "audience": "enum (all_friends, custom, ghost)",
  "precision": "enum (exact, blurred)",
  "expiry_minutes": "int"
}

4FlowsFlows

4.1 Location Publishing

1. Client sends update to Ingestion Service.
2. Service enriches with Geo-Cell ID.
3. Update is written to Redis with a TTL of ~8 hours.
4. Event is published to Kafka for downstream consumption.

4.2 Location Discovery (Fan-out)

1. Fan-out service consumes from Kafka.
2. Queries the Friend Graph to find active observers.
3. Checks privacy settings (Ghost Mode etc).
4. Pushes update to authorized friends via WebSocket Gateway.

5Scale ConsiderationsScale

• Geographic Sharding: Distribute Redis and Ingestion services based on geographic regions to minimize latency.
• Adaptive Polling: Use accelerometer data to stop GPS usage when the user is stationary.
• Fan-out Optimization: Only push updates to friends who have the app open and are currently viewing the map.
• Clustering: Pre-calculate clusters at various zoom levels to avoid real-time aggregation overhead.

6Deep Dive TopicsDeep dive

6.1 Hierarchical Geospatial Indexing (H3/S2)

• The Problem: "Which friends/users are near this point?" over raw lat/lon means scanning or range-querying two columns — distance math (haversine) is cheap, but checking millions of points is not. We need an index that turns "nearby" into a small set of keys.
• The Solution: Use Uber's H3 (Hexagons) or Google's S2 (Hilbert Curve).
• Benefit: Each point maps to a cell ID; "nearby" becomes "these few cells", so a radius query is a handful of key lookups instead of a scan. With H3, all six neighbors are the same distance from a cell's center, so a ring of cells approximates a circle evenly; computing a cell's neighbors is constant time. You still filter the candidates in those cells by exact distance.
• Clustering: By truncating the H3 index resolution, we can instantly aggregate millions of points into single "hotspot" hexes for zoomed-out views.

6.2 Privacy & Scaling

• Ghost Mode: A simple boolean check in the Fan-out Service if (user.is_ghost) return;.
• Precision Rounding: For "blurred" sharing, coordinates are snapped to the center of a coarse H3 cell (e.g., resolution 7), protecting exact house-level addresses while showing the general neighborhood.

7Tradeoffs & ExtensionsTrade-offs

7.1 Tradeoffs

• Server-side vs. Client-side Clustering: Server-side (pre-calculated tiles) is faster for mobile clients and saves battery, but client-side clustering allows for more interactive, dynamic "pin-merging" animations.
• TTL vs. Cleanup Jobs: Redis TTL is simpler but doesn't allow for "last seen" historical analytics; a hybrid approach with a cold store is needed for features like "Your 2024 Year in Review".

7.2 Extensions

• Semantic Location: Instead of showing coordinates, show "At Home", "At Starbucks", or "Driving" using reverse-geocoding and motion sensors.
• Social Heatmaps: Highlight "trending" venues where friends are congregating in real-time.

8Wrap-UpWrap-up

Snap Map is a masterclass in high-throughput geospatial engineering. By combining hierarchical indexing for clustering with a massive fan-out service for friend updates, the system maintains a "live" feel across 100M+ users without melting the database or the user's phone.