Problem RestatementProblem
LinkedIn asked: design a system that detects and resolves stale job postings in a large job marketplace. A listing is stale when it's no longer really open: it was filled, closed on the employer's own site, abandoned by the poster, is a duplicate, or is simply old. Stale jobs waste job seekers' time and hurt trust, but closing a job that's actually open hurts employers.
Deep Dive — Deciding a job listing is deadDeep dive
Filled and abandoned listings waste applicants' time and destroy trust in the board. Nothing explicitly tells you a job has closed.
Expire listings after a fixed age
Remove anything older than 30 days.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
AGE["30 days old"] --> KILL["Auto-expired"]
KILL --> GOOD["A senior role with a 90-day hiring cycle, still open, removed"]
YOUNG["Filled on day 3"] --> LIVE["Stays up for 27 more days"]
LIVE --> WASTE["Applicants apply into a closed role"]Age is weakly correlated with being closed and the correlation differs enormously by role. A fixed cutoff removes open jobs and keeps dead ones, which is both halves of the problem.
Check the source
Re-crawl the employer's career page or ATS feed. A 404, a "position filled" page, or disappearance from the feed is strong evidence.
This is the highest-quality signal available and it should drive the system. Its limit is coverage: not every listing has a crawlable source, pages change layout, and the most common death — a recruiter who has simply stopped working the role — leaves the source page up indefinitely.
Combine strong and weak signals into a score
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
S1["Source page 404 or 'filled'"] --> STRONG["Strong - close it"]
S2["Gone from the ATS feed"] --> STRONG
S3["Poster has not logged in, viewed applicants or replied for weeks"] --> WEAK["Weak signals"]
S4["Many applications, zero recruiter actions"] --> WEAK
S5["Age far beyond typical time-to-fill for this role"] --> WEAK
S6["Near-identical listing reposted by the same company"] --> WEAK
WEAK --> SCORE["Staleness score"]
SCORE --> MID["Mid confidence: demote in ranking, label 'posted 60 days ago'"]
SCORE --> HIGH["High confidence: ask the employer to confirm, then close"]
STRONG --> CLOSE["Closed"]- Strong signals close; weak signals demote. Ranking down a probably-dead listing costs nothing if the guess is wrong, while removing a live job costs the employer a hire — so the action should match the confidence.
- Recruiter behaviour is the best weak signal. A poster who has not opened their applicants in three weeks is the clearest sign of an abandoned listing, and it is available even when the source page is not.
- Normalise age by role. Typical time-to-fill differs by a factor of several between a warehouse role and a director search; comparing a listing to its own category's distribution is what makes age usable at all.
- Ask before closing. A one-click "still hiring?" to the employer converts a guess into a fact and gets engagement from exactly the recruiters who are still active.
Measure it on applicant outcomes — the share of applications that receive any recruiter response — not on how many listings were removed. It is trivial to improve the second number while making the board worse.
ArchitectureArchitecture
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","classText":"#1e1e1e"}}}%%
flowchart LR
JOBS[("Job listings")] --> SCH["Check scheduler - prioritized"]
SCH --> CR["Source checker - crawl apply URL / ATS API"]
ACT["Recruiter activity + applicant events"] --> FE["Feature builder"]
CR --> FE
REP["Seeker reports"] --> FE
FE --> SC["Staleness scorer - rules + ML"]
SC --> DEC["Decision engine"]
DEC -->|"high confidence"| CLOSE["Auto-close + notify employer"]
DEC -->|"medium"| ASK["Ask employer to confirm + demote in search"]
DEC -->|"low"| KEEP["Keep, recheck later"]
CLOSE --> JOBS
ASK --> JOBSKey Parts
- Source checks: for jobs imported from ATS feeds or career pages, re-fetch the source regularly. Prioritize popular listings (many views) and old ones, and back off for recently verified ones. Respect crawl politeness.
- Features: listing age vs typical for the role and location, days since the last recruiter action, applicant count, source status, duplicate similarity (text embeddings + company + title + location), and seeker reports.
- Scoring: start with clear rules (the source says closed → stale), plus an ML model trained on labeled outcomes (employer-confirmed closures, jobs that closed soon after) that outputs a probability.
- Actions by confidence:
- Very high (the source confirms closed) → auto-close and notify the employer (with an easy "reopen").
- Medium → ask the employer to confirm ("Is this job still open?" email or in-app), and demote it in search meanwhile.
- Low → keep, and recheck later.
- Duplicates: keep the newest and merge or redirect the older ones.
Measuring Success
- The stale rate in search results (sampled and labeled), the rate of seekers applying to closed jobs, and the report rate.
- False closures: employer reopen rate after auto-close (must stay very low), used to tune thresholds.
- An A/B test on demotion: apply rates and seeker satisfaction.
Wrap-UpWrap-up
Combine signals of staleness (source/ATS status, recruiter inactivity, listing age vs typical fill time, duplicate detection and seeker reports) through a prioritized checking pipeline into a staleness score from rules plus ML. Act by confidence: auto-close with easy reopen when the source confirms, ask employers and demote in search when uncertain, recheck otherwise. Measure stale-rate reduction against false closures.