santifer/career-ops · warning
Removed stale reservation sentinel: ${name}
Error message
Removed stale reservation sentinel: ${name} What it means
verify-pipeline.mjs check 8 garbage-collects reservation sentinels: reserve-report-num.mjs drops NNN-RESERVED.md files into reports/ as atomic claims on report slots, and if the claiming process dies before writing the real report and deleting the sentinel, the claim lingers and would skew future allocation. Sentinels older than 4 hours (SENTINEL_MAX_AGE_MS) are unlinked here with this warning; statSync races with concurrent deletion are swallowed (empty catch).
Source
Thrown at verify-pipeline.mjs:246
if (boldScores === 0) ok('No bold in scores');
// --- Check 8: Stale report-number sentinels (GC) ---
// reserve-report-num.mjs drops NNN-RESERVED.md files in reports/ when a
// number is claimed. If the process crashed before writing the real report
// and deleting the sentinel it will linger. Sentinels older than 4 h are
// stale; remove them here so they don't skew the next slot allocation.
const SENTINEL_MAX_AGE_MS = 4 * 60 * 60 * 1000;
let staleSentinels = 0;
if (existsSync(REPORTS_DIR)) {
const now = Date.now();
for (const name of readdirSync(REPORTS_DIR)) {
if (!name.endsWith('-RESERVED.md')) continue;
const full = join(REPORTS_DIR, name);
try {
const { mtimeMs } = statSync(full);
if (now - mtimeMs > SENTINEL_MAX_AGE_MS) {
unlinkSync(full);
warn(`Removed stale reservation sentinel: ${name}`);
staleSentinels++;
}
} catch {
// Already gone between readdir and stat — fine.
}
}
}
if (staleSentinels === 0) ok('No stale reservation sentinels');
// --- Check 9: Duplicate reports for the same company+role (#1425) ---
// Two concurrent evaluators can each write a report for the same role.
// merge-tracker dedups the TRACKER, but nothing watched reports/ itself.
// Warning-level, not error: duplicates can be legitimate (re-evaluation
// after a JD change).
const REPORT_FILE_RE = /^(\d+)-(.+)-\d{4}-\d{2}-\d{2}\.md$/;
// Shares normalizeTextKey with Check 2 so the two checks fold text the same
// way (#2393). That is where the guarantee ends: this check keys off the
// FILENAME slug, already ASCII by the time a report is written, while Check 2View on GitHub (pinned to 60398d6549)
Solutions
- Nothing to repair -- the GC already removed the file; this warning records that a prior run died without releasing its slot.
- If it recurs, identify which worker keeps dying and why (see test-all's slow-kill warning, OOM logs) -- the sentinel is a symptom, not the disease.
- Always release in a finally path: node reserve-report-num.mjs --release NNN-MMM when a reservation goes unused.
Example fix
// before: worker can die between reserve and write, leaking the sentinel
const nums = await reserveReportNumbers(count, opts);
await writeReports(nums); // crash here -> stale NNN-RESERVED.md
// after: release whatever was not consumed
const nums = await reserveReportNumbers(count, opts);
try { await writeReports(nums); }
finally { await releaseReportNumbers(unusedOf(nums), opts).catch(() => {}); } Defensive patterns
Strategy: try-catch
Try / catch
// Make every reservation crash-safe: release unconsumed slots in finally
const nums = await reserveReportNumbers(count, { reportsDir });
let consumed = [];
try {
consumed = await writeReports(nums);
} finally {
try { await releaseReportNumbers(nums.filter(n => !consumed.includes(n)), { reportsDir }); }
catch { /* verify-pipeline GCs anything left after 4h */ }
} Prevention
- Reserve right before spawning workers (the sentinel lifetime is only 4h) and always release in a finally block.
- Treat recurring 'Removed stale reservation sentinel' lines as a symptom of workers dying mid-run — find the killer (CI timeout, OOM), don't just let the GC mop up.
- Run node verify-pipeline.mjs as part of post-batch cleanup so crashed-run residue is collected deterministically.
When it happens
Trigger: A batch worker killed mid-run (CI timeout, OOM, Ctrl-C) after reserving but before writing; a releaseReportNumbers() failure (the #226 path); machine crash/sleep during a fan-out.
Common situations: CI ceilings killing long evaluators; laptop sleep interrupting overnight batches; any interrupted reservation whose --release was never issued.
Related errors
- Could not claim ${count} report slot(s) after ${MAX_RETRIES}
- ⚠️ Could not release report reservation: ${err.message}
- ⚠️ Could not release report reservation: ${err.message}
- ⚠️ Could not release report reservation: ${err.message}
- Sync check could not run: ${err.message}
AI-assisted analysis of santifer/career-ops@60398d6549 (2026-08-20).
Data as JSON: /api/errors/2e3174b28c44e622.
Report an issue: GitHub.