{"record":{"id":"cd2857518ff6125b","repo":"paperclipai/paperclip","slug":"run-dispatch-run-input-runid-changed-issue-con","errorCode":null,"errorMessage":"run-dispatch: run ${input.runId} changed issue context repeatedly while acquiring locks","messagePattern":"run-dispatch: run (.+?) changed issue context repeatedly while acquiring locks","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"server/src/modules/run-dispatch/adapters/postgres.ts","lineNumber":219,"sourceCode":"              and(eq(heartbeatRuns.id, input.runId), eq(heartbeatRuns.companyId, input.companyId)),\n            )\n            // Keep the run status stable through the semantic decision and any\n            // resulting mutation and synchronous dispatch handoff. Never await\n            // adapter-owned work while this transaction holds the row locks.\n            .for(\"update\")\n            .then((rows) => rows[0] ?? null);\n          if (!run) return { kind: \"missing\" as const };\n\n          const lockedIssueId = readNonEmptyString(parseObject(run.contextSnapshot).issueId);\n          if (lockedIssueId !== hintedIssueId) return { kind: \"retry\" as const };\n          return { kind: \"value\" as const, value: await operation(typedTx, run) };\n        });\n\n      if (result.kind === \"missing\") return onMissing();\n      if (result.kind === \"value\") return result.value;\n    }\n\n    throw new Error(\n      `run-dispatch: run ${input.runId} changed issue context repeatedly while acquiring locks`,\n    );\n  }\n\n  async function loadGateFacts(\n    input: LoadGateFactsInput,\n    now: Date,\n    tx?: unknown,\n  ): Promise<LoadGateFactsResult> {\n    // Semantic adapter operations pass their transaction here so the fact\n    // read and the state transition share one unit of work. This helper is\n    // deliberately not exposed through the module's public API.\n    const dbOrTx = (tx as Db | undefined) ?? db;\n    const budgetsForRead = tx ? budgetService(dbOrTx) : budgets;\n    const treeControlForRead = tx ? issueTreeControlService(dbOrTx) : treeControlSvc;\n    const issuesSvcForRead = tx ? issueService(dbOrTx) : issuesSvc;\n\n    const agent = await dbOrTx","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/server/src/modules/run-dispatch/adapters/postgres.ts#L201-L237","documentation":"Raised by withIssueThenRunLocks in the run-dispatch Postgres adapter when it repeatedly fails to acquire issue+run locks because the run's issue context (issueId) kept changing between read attempts. It is an internal retry-exhaustion guard against livelock when issue context mutates concurrently.","triggerScenarios":"Concurrent operations (cancelStaleQueuedRun, dispatchResolvedInteractionIfCurrent, transactionResult) racing with something that keeps re-pointing the run at a different issue, exhausting the internal retry loop.","commonSituations":"Two dispatch/cancel workers racing on the same run; a transaction moving the run between issues while a lock acquisition is in flight; retry storm under heavy scheduling contention.","solutions":["Retry the operation — the loop exits once issue context stops changing","Identify and serialize the writer that repeatedly mutates the run's issue context","Ensure only one dispatch/cancel worker operates per run (idempotent job claim)","Reduce contention by spacing watchdog/dispatch schedules"],"exampleFix":"// before\nawait adapter.cancelStaleQueuedRun(input); // may throw after repeated context change\n// after\ntry {\n  await adapter.cancelStaleQueuedRun(input);\n} catch (e) {\n  if (e.message.includes('changed issue context repeatedly')) await delay(200), retry(input);\n  throw e;\n}","handlingStrategy":"retry","validationCode":"// confirm issue context is stable before dispatching\nconst a = await getRunIssueContext(runId); await delay(50); const b = await getRunIssueContext(runId);\nif (a !== b) await delay(500);","typeGuard":null,"tryCatchPattern":"try { await op(input) } catch (e) { if (e.message.includes('changed issue context repeatedly')) { await backoff(); return op(input); } throw e; }","preventionTips":["Serialize per-run operations with a job lock so only one worker dispatches/cancels at a time","Avoid reassigning a run's issue while dispatch is in flight","Use idempotent dispatch jobs with claim semantics","Space out watchdog and dispatch schedules to reduce lock contention"],"tags":["concurrency","locking","postgres"],"backgroundTag":"invalid-state-transition","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}