{"record":{"id":"24c7f686d09d8583","repo":"mastra-ai/mastra","slug":"durable-agent-recover-already-in-progress","errorCode":"DURABLE_AGENT_RECOVER_ALREADY_IN_PROGRESS","errorMessage":"DurableAgent \"${this.name}\" recover(${runId}): another process is already recovering this run.","messagePattern":"DurableAgent \"(.+?)\" recover\\((.+?)\\): another process is already recovering this run\\.","errorType":"error_code","errorClass":"MastraError","httpStatus":null,"severity":"error","filePath":"packages/core/src/agent/durable/durable-agent.ts","lineNumber":573,"sourceCode":"   * through the thread stream. The thread lease cannot provide this guarantee:\n   * its owner is the logical runId, so two processes recovering the same run\n   * are indistinguishable to an idempotent lease backend.\n   */\n  async #acquireRecoveryLease(runId: string, abortController: AbortController): Promise<RecoveryLease> {\n    const pubsub = this.pubsub;\n    const unwrap = (pubsub as { getLeaseProvider?: () => LeaseProvider | undefined }).getLeaseProvider;\n    const provider =\n      typeof unwrap === 'function'\n        ? (unwrap.call(pubsub) ?? NoopLeaseProvider)\n        : isLeaseProvider(pubsub)\n          ? pubsub\n          : NoopLeaseProvider;\n    const key = `mastra:durable-agent-recovery:v1:${JSON.stringify([this.id, runId])}`;\n    const owner = crypto.randomUUID();\n\n    const localOwner = localRecoveryClaims.get(key);\n    if (localOwner) {\n      throw new MastraError({\n        id: 'DURABLE_AGENT_RECOVER_ALREADY_IN_PROGRESS',\n        domain: ErrorDomain.AGENT,\n        category: ErrorCategory.USER,\n        text: `DurableAgent \"${this.name}\" recover(${runId}): another process is already recovering this run.`,\n        details: { agentName: this.name, runId },\n      });\n    }\n    localRecoveryClaims.set(key, owner);\n\n    const leaseAcquireStartedAt = Date.now();\n    let acquired: Awaited<ReturnType<LeaseProvider['acquireLease']>>;\n    try {\n      acquired = await provider.acquireLease(key, owner, RECOVERY_LEASE_TTL_MS);\n    } catch (cause) {\n      if (localRecoveryClaims.get(key) === owner) localRecoveryClaims.delete(key);\n      throw new MastraError(\n        {\n          id: 'DURABLE_AGENT_RECOVER_LEASE_ACQUIRE_FAILED',","sourceCodeStart":555,"sourceCodeEnd":591,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/agent/durable/durable-agent.ts#L555-L591","documentation":"DurableAgent#recoveryLease calls #acquireRecoveryLease, which first checks a local in-process claim map (localRecoveryClaims). If this process already holds a claim for the lease key `mastra:durable-agent-recovery:v1:[agentId, runId]`, it throws immediately, treating it as another recovery already in progress. It prevents concurrent recoveries of the same run within the same process.","triggerScenarios":"Calling agent.recover(runId) twice concurrently in the same process for the same run — e.g. overlapping recover() invocations before the first finishes and releases its claim, or a retry loop firing while a previous attempt is still running.","commonSituations":"A retry wrapper re-invoking recover() on timeout while the original call is still in flight; a job scheduler dispatching the same recovery to two workers in one process; event handlers firing recover() multiple times for one run.","solutions":["Await the in-flight recover() promise instead of starting a second one; serialize recoveries per runId.","Add an in-flight guard (map of runId -> Promise) so concurrent calls share one recovery.","If a previous recover() crashed without releasing its local claim, restart the process or wait for the lease TTL to expire, then retry."],"exampleFix":"// before\nruns.forEach(runId => agent.recover(runId)); // may overlap\n// after\nconst inflight = new Map<string, Promise<void>>();\nconst recoverOnce = (runId: string) => {\n  if (!inflight.has(runId)) {\n    inflight.set(runId, agent.recover(runId).finally(() => inflight.delete(runId)));\n  }\n  return inflight.get(runId);\n};\nawait Promise.all(runs.map(recoverOnce));","handlingStrategy":"retry","validationCode":"if (inflightRecoveries.has(runId)) return inflightRecoveries.get(runId); // dedupe before calling","typeGuard":null,"tryCatchPattern":"try {\n  await agent.recover(runId);\n} catch (e) {\n  if (String(e?.id) === 'DURABLE_AGENT_RECOVER_ALREADY_IN_PROGRESS') {\n    await waitForInflightRecovery(runId); // await the existing attempt, don't start another\n  } else throw e;\n}","preventionTips":["Keep one in-flight recover() per runId per process (promise map).","Avoid retry loops that re-enter recover() while the previous attempt is still running.","Ensure job schedulers don't dispatch the same runId twice within one process."],"tags":["durable-agent","recovery","concurrency","lease","duplicate-work"],"backgroundTag":"recovery-already-in-progress","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}