Entry 001 · 2025–present · live in production

A job-search platform that never fabricates

An AI platform that discovers jobs, evaluates fit, and tailors résumés and cover letters, built on the position that in a domain where fabrication has real consequences for real people, honesty is architecture, not a prompt instruction.

The problem

AI job tools have a trust problem: résumé generators invent skills, auto-apply services spray unsuitable applications, and users can’t see what was done on their behalf. The interesting engineering problem isn’t generating text, it’s building a system whose outputs can be trusted by the user, defended to an employer, and audited after the fact.

Architecture: grounded or refused

Every user has an evidence corpus, structured items mined from their real work history, each anchored to a verbatim source. Tailoring is retrieve-then-compose: a hybrid retriever (reciprocal-rank fusion over lexical overlap and dense embedding cosine) selects evidence; the writer may only make claims that cite it. Keywords a job wants but the corpus can’t support surface as acknowledged gaps. The enforcement is server-side, the model is never trusted to police itself.

The résumé becomes a per-job rendering of verified evidence, not a document the model imagined.

A second, fresh-context agent then critiques each tailored packet against the job before it’s staged, and the rendered PDF is verified as an artifact: page count, ATS text-layer extraction, contact details present, keyword coverage measured, badges the user sees before approving.

The evaluation funnel, and its calibration loop

Full LLM evaluation of every discovered job is the dominant cost, at peak it was two-thirds of total model spend. The fix is a funnel: cheap preliminary scoring (keyword rubric + a small embedding-similarity signal) gates which jobs get full evaluations, with low scorers deferred to a paced drain rather than dropped.

Because the prelim gate is a model of a model, it gets its own instrumentation: a weekly calibration loop measures preliminary-versus-full agreement and refines the rubric only when the change provably improves agreement. Measurement also caught the gate flattering itself, prelim scores ran up to +2.1 high in some lanes, which is corrected at display time while the raw score stays untouched for gating, so the calibration loop never trains on its own correction.

Guardrails and receipts

• Server-side gates on everything irreversible: legal acceptance, spend caps, application submission. Human-only transitions are enforced in the state machine, the agent cannot mark its own work as applied.
• Graduated autonomy: the agent stages applications by default; auto-submission requires per-provider owner approval, daily caps, and a kill switch, and even then a cancelable countdown window.
• Glass-box receipts: every submission writes an immutable record, what was submitted, with which résumé version, from which evidence, at what model cost. Users can export all of it.

Operations and FinOps

Every model call is recorded with tokens and cost against production pricing; an adaptive (AIMD) rate limiter shares capacity fairly across accounts and jobs; model routing is a config decision with per-role fallback chains, the writer model was swapped after live acceptance-rate data showed an 11% success rate, without touching code. Nightly backups, deploy-by-commit-SHA, and postmortems for the incidents below.

What broke in production

Two incidents shaped the system more than any feature. A liveness checker misclassified rate-limit responses (403/429) as “job expired” and discarded 1,314 live listings, the fix reclassified everything except definitive death (404/410) as blocked-not-dead, and recovered the listings. And an over-eager company-deduplication merge cascaded on a bad similarity key; the remediation added dry-run-first discipline for destructive operations and a board-identity guard on the merge key. Both postmortems are part of the system’s documentation, and both rules the incidents produced are now enforced in code.

Security

Two adversarial security sweeps (multi-agent, attacker-mindset) produced 24 findings, SSRF in the headless fetcher, CSP/HSTS gaps, PII redaction, inbound-email spoofing surface, worker RCE kill-switch, all fixed and deployed. Inbound email auto-advances application status only on DKIM/DMARC-verified senders.

The system is live with real users. Code is proprietary, I’m happy to do a live walkthrough of the running system and its decision records in any interview process.