{"record":{"id":"0cd6c9adae001ff1","repo":"santifer/career-ops","slug":"pipeline-lock-timeout-lockdir-held-timeout","errorCode":null,"errorMessage":"pipeline lock timeout: ${lockDir} held > ${timeoutMs}ms","messagePattern":"pipeline lock timeout: (.+?) held > (.+?)ms","errorType":"exception","errorClass":"LockTimeoutError","httpStatus":null,"severity":"error","filePath":"pipeline-lock.mjs","lineNumber":165,"sourceCode":"        // otherwise disable stale recovery forever. The guard normally lives\n        // for milliseconds, so an old one is judged by the same age rule.\n        if (lockCanRecover(recoverGuardDir, staleMs)) {\n          rmSync(recoverGuardDir, { recursive: true, force: true });\n        }\n      }\n\n      if (hasRecoverGuard) {\n        try {\n          if (lockCanRecover(lockDir, staleMs)) {\n            rmSync(lockDir, { recursive: true, force: true });\n            continue; // retry acquisition immediately, still holding the guard's decision\n          }\n        } finally {\n          rmSync(recoverGuardDir, { recursive: true, force: true });\n        }\n      }\n\n      if (Date.now() > deadline) throw new LockTimeoutError(lockDir, timeoutMs);\n      await sleep(retryMs);\n      continue;\n    }\n\n    // Acquired. Record ownership; an owner-less lock would block every future\n    // acquirer until the age-out, so clean up if the stamp can't be written.\n    try {\n      writeFileSync(join(lockDir, 'owner.json'), JSON.stringify({\n        pid: process.pid,\n        token,\n        started_at: new Date().toISOString(),\n        pipeline: pipelinePath,\n      }, null, 2));\n    } catch (ownerErr) {\n      rmSync(lockDir, { recursive: true, force: true });\n      throw ownerErr;\n    }\n","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/santifer/career-ops/blob/9b17a8ac97b398a496b38e423ae24e433b43254f/pipeline-lock.mjs#L147-L183","documentation":"Thrown as LockTimeoutError by acquireWithRecovery() in pipeline-lock.mjs when Date.now() exceeds the deadline while the lock directory still cannot be created (it keeps hitting EEXIST and is not eligible for stale recovery). It means another holder kept the lock for the entire configured timeoutMs window.","triggerScenarios":"A long-running pipeline process holds the lock legitimately longer than the configured timeoutMs; a crashed process left a non-stale lock (owner.json newer than staleMs) that recovery cannot reclaim; two concurrent operations contending with too-short a timeout; a hung process never releasing.","commonSituations":"Running scan and pipeline concurrently on the same tracker; a prior run was killed -9 leaving owner.json; timeoutMs set too low for a large pipeline batch; a frozen/suspended process holding the lock across a sleep/hibernate.","solutions":["Wait for the in-flight operation to finish, then retry.","If no operation is actually running, remove the stale lock directory shown in the message (or raise staleMs so lockCanRecover reclaims it).","Increase the timeoutMs passed to acquireWithRecovery for large batches.","Ensure only one pipeline-mutating process runs at a time per tracker."],"exampleFix":"// before\nconst lock = await acquireWithRecovery(lockDir, { timeoutMs: 5_000 });\n// after: give large batches enough headroom and let stale recovery work\nconst lock = await acquireWithRecovery(lockDir, { timeoutMs: 120_000, staleMs: 60_000 });","handlingStrategy":"retry","validationCode":"null","typeGuard":"/** Narrows a caught error to the lock-timeout class. */\nfunction isLockTimeout(e) {\n  return e instanceof Error && /pipeline lock timeout/.test(e.message);\n}","tryCatchPattern":"import { LockTimeoutError } from './pipeline-lock.mjs';\ntry {\n  const lock = await acquireWithRecovery(lockDir, { timeoutMs, staleMs });\n} catch (e) {\n  if (e instanceof LockTimeoutError || /lock timeout/.test(e.message)) {\n    // wait and retry, or escalate if a holder is genuinely stuck\n  } else throw e;\n}","preventionTips":["Size timeoutMs to the worst-case duration of a legitimate holder for your batch.","Ensure only one pipeline-mutating process runs per tracker.","Let staleMs reclaim crashed holders; don't manually delete owner.json unless you are sure."],"tags":["locking","concurrency","timeout","pipeline-lock"],"backgroundTag":null,"analyzedSha":"9b17a8ac97b398a496b38e423ae24e433b43254f","analyzedAt":"2026-08-13T00:48:39.135Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}