Building the Backend for an Advanced Job Board
How seven incompatible ATS feeds become one searchable job board using local models, independent judging, evidence checks, and careful batching.
A useful job board looks simple: search, filter, open the right role. The backend is dealing with seven applicant tracking system (ATS) formats, inconsistent titles, buried salary language, vague workplace policies, duplicate records, changed descriptions, and tens of thousands of documents that need to stay searchable.
I love this class of problem because the answer is not one clever model call. It is a chain of small systems that hand work to one another, with a clear rule for what each one is allowed to decide.
A Cloudflare scheduler assigns groups of companies to each run. A Supabase Edge Function then fetches Greenhouse, Lever, Ashby, SmartRecruiters, Workable, Workday, and BambooHR through a separate importer for each vendor. Those seven formats leave the function as the same job record.
During import, Pathos validates the public link, standardizes location and employment type, preserves salary fields supplied by the employer, and records a fingerprint of the description so unchanged jobs can be skipped. A role disappears only after it is missing from repeated successful checks. One broken vendor request cannot make live jobs look closed.
What the system guarantees
- Seven ATS-specific importers turn incompatible source data into the same job record.
- Each classifier answers one narrow question: occupation, industry, level, pay, workplace policy, or listed tools.
- Qwen and Gemma must cite the posting; fixed rules verify their evidence and can reject both answers.