{"record":{"id":"eb81131efe9a20fb","repo":"santifer/career-ops","slug":"portal-health-lock-timeout-lockdir-held-ti","errorCode":null,"errorMessage":"portal-health lock timeout: ${lockDir} held > ${timeoutMs}ms","messagePattern":"portal-health lock timeout: (.+?) held > (.+?)ms","errorType":"exception","errorClass":"LockTimeoutError","httpStatus":null,"severity":"error","filePath":"portal-health-lock.mjs","lineNumber":160,"sourceCode":"        // otherwise disable stale recovery forever. The guard normally lives\n        // for milliseconds, so an old one is judged by the same age rule.\n        if (lockCanRecover(recoverGuardDir, staleMs)) {\n          rmSync(recoverGuardDir, { recursive: true, force: true });\n        }\n      }\n\n      if (hasRecoverGuard) {\n        try {\n          if (lockCanRecover(lockDir, staleMs)) {\n            rmSync(lockDir, { recursive: true, force: true });\n            continue; // retry acquisition immediately\n          }\n        } finally {\n          rmSync(recoverGuardDir, { recursive: true, force: true });\n        }\n      }\n\n      if (Date.now() > deadline) throw new LockTimeoutError(lockDir, timeoutMs);\n      await sleep(retryMs);\n      continue;\n    }\n\n    // Acquired. Record ownership; an owner-less lock would block every future\n    // acquirer until the age-out, so clean up if the stamp can't be written.\n    try {\n      writeFileSync(join(lockDir, 'owner.json'), JSON.stringify({\n        pid: process.pid,\n        token,\n        started_at: new Date().toISOString(),\n        file: filePath,\n      }, null, 2));\n    } catch (ownerErr) {\n      rmSync(lockDir, { recursive: true, force: true });\n      throw ownerErr;\n    }\n","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/santifer/career-ops/blob/9b17a8ac97b398a496b38e423ae24e433b43254f/portal-health-lock.mjs#L142-L178","documentation":"portal-health-lock.mjs implements a cross-process advisory lock (a directory-based mkdir lock, same idiom as the tracker lock) to serialize writes to data/portal-health.tsv between scan.mjs appenders and read-modify-write cleanups. acquire() loops retrying mkdir until it wins; if the deadline (default 8000ms, configurable via timeoutMs) expires while the lock is still held by another (live) process, it throws a LockTimeoutError. Stale locks (dead owner PID, or aged out) are reclaimed automatically before this fires — so this error specifically means a LIVE process held the lock longer than the budget.","triggerScenarios":"Two or more concurrent processes both writing data/portal-health.tsv (e.g. two `scan` runs launched in parallel, or a scan running while a cleanup/repair job holds the lock) and the first holds it past timeoutMs. Also possible if the holder crashed in a way that left owner.json but the PID is reused by an unrelated long-running process (PID-liveness false positive).","commonSituations":"Overlapping scheduled scans (cron + manual run); a long-running scan stuck on a slow network fetch while holding the lock; a previous scan killed with SIGKILL whose PID got reused by another process, making a dead lock look live so it never auto-reclaims.","solutions":["Wait and retry — if a scan is legitimately in progress, it will release the lock; the next acquire succeeds.","Check for a hung scan process: `ps aux | grep scan.mjs` and, if it is genuinely stuck (not making progress), kill it so the lock's owner PID dies and stale-reclaim kicks in.","As a last resort, remove the lock directory manually: `rm -rf data/portal-health.tsv.lock` AND `data/portal-health.tsv.lock.recover` — only when you are certain no scan is running, since manually deleting a live lock can interleave writers.","If contention is expected, raise timeoutMs at the call site (the lock API is caller-configurable for exactly this)."],"exampleFix":"# Check for a live holder first\nps aux | grep -E 'scan\\.mjs|portal-health'\n\n# If none is genuinely running, remove the stale lock dirs\nrm -rf data/portal-health.tsv.lock data/portal-health.tsv.lock.recover\n\n# Then retry the scan","handlingStrategy":"try-catch","validationCode":"// Detect a likely-stuck lock before acquiring, so you can warn rather than block.\nimport { existsSync, readFileSync, statSync } from 'fs';\nfunction lockLooksStale(lockDir, staleMs = 30000) {\n  if (!existsSync(lockDir)) return false;\n  try {\n    const owner = JSON.parse(readFileSync(`${lockDir}/owner.json`, 'utf-8'));\n    if (owner?.pid && !processExists(owner.pid)) return true; // dead owner\n    const age = Date.now() - statSync(lockDir).mtimeMs;\n    return age > staleMs;\n  } catch {\n    return Date.now() - statSync(lockDir).mtimeMs > staleMs;\n  }\n}\nfunction processExists(pid) { try { process.kill(pid, 0); return true; } catch { return false; } }","typeGuard":"/** @param {unknown} e @returns {e is import('portal-health-lock.mjs').LockTimeoutError} */\nfunction isLockTimeout(e) {\n  return e instanceof Error && e.name === 'LockTimeoutError' && typeof e.lockDir === 'string';\n}","tryCatchPattern":"import { acquire, LockTimeoutError } from './portal-health-lock.mjs';\ntry {\n  const release = await acquire(filePath, { timeoutMs: 10000 });\n  try { /* ...read-modify-write portal-health.tsv... */ }\n  finally { await release(); }\n} catch (err) {\n  if (err instanceof LockTimeoutError) {\n    console.warn(`portal-health lock busy (${err.lockDir}); skipping this write rather than blocking.`);\n  } else throw err;\n}","preventionTips":["Never hold the lock across slow network I/O — read-modify-write should be quick (read file, mutate, write file).","Serialize scan runs via a scheduler/cron guard so two scans don't contend for the same lock.","Always release in a finally block so a thrown error can't orphan the lock.","If you manually delete a lock dir, also delete the .recover guard dir, and only when no scan is running."],"tags":["lock","concurrency","portal-health","filesystem"],"backgroundTag":null,"analyzedSha":"9b17a8ac97b398a496b38e423ae24e433b43254f","analyzedAt":"2026-08-13T00:48:39.135Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}