coleam00/Archon · error

Resumable run snapshot changed during reset (expected ${Stri

Error message

Resumable run snapshot changed during reset (expected ${String(resumable.length)}, cancelled ${String(result.rowCount)})

What it means

cancelResumableRunsForConversation first SELECTs the paused/failed runs it intends to cancel, then cancels them with an UPDATE and verifies the affected row count matches the snapshot. This error is thrown when rowCount differs, meaning another writer inserted, deleted, or transitioned a resumable run between the read and the write. It is a deliberate optimistic-concurrency guard, not a driver failure.

Source

Thrown at packages/core/src/db/workflows.ts:618

        `SELECT * FROM remote_agent_workflow_runs
         WHERE conversation_id = $1 OR parent_conversation_id = $2
         ORDER BY started_at DESC${rowLockClause()}`,
        [conversationId, conversationId]
      );
      const resumable = snapshot.rows.filter(
        run => run.status === 'paused' || run.status === 'failed'
      );
      if (resumable.length === 0) return [];

      const result = await query(
        `UPDATE remote_agent_workflow_runs
         SET status = 'cancelled', completed_at = ${dialect.now()}
         WHERE (conversation_id = $1 OR parent_conversation_id = $2)
           AND status IN ('paused', 'failed')`,
        [conversationId, conversationId]
      );
      if (result.rowCount !== resumable.length) {
        throw new Error(
          `Resumable run snapshot changed during reset (expected ${String(resumable.length)}, cancelled ${String(result.rowCount)})`
        );
      }
      for (const run of resumable) {
        await insertWorkflowEvent(query, {
          workflow_run_id: run.id,
          event_type: 'workflow_cancelled',
        });
      }
      return resumable.map(run => normalizeWorkflowRun(run));
    });
  } catch (error) {
    const err = error as Error;
    getLog().error({ err, conversationId }, 'db.workflow_run_cancel_resumable_for_conv_failed');
    throw new Error(`Failed to cancel resumable runs for conversation: ${err.message}`);
  }
}

View on GitHub (pinned to 0773b97458)

Solutions

1. Retry the whole cancelResumableRunsForConversation call: the next execution takes a fresh snapshot and usually succeeds.
2. Identify the concurrent writer (resume path, admin action) and serialize the reset against it.
3. If it recurs, wrap the SELECT+UPDATE in a single transaction with row locks (SELECT ... FOR UPDATE) so the snapshot cannot drift.
4. Inspect workflow_events for the run that changed to confirm who mutated it.

Example fix

// before
try {
  await cancelResumableRunsForConversation(conversationId);
} catch (err) {
  if (String(err).includes('snapshot changed')) throw err;
}
// after
for (let attempt = 0; attempt < 3; attempt++) {
  try {
    await cancelResumableRunsForConversation(conversationId);
    break;
  } catch (err) {
    if (!String(err).includes('snapshot changed') || attempt === 2) throw err;
  }
}

// Pre-check: snapshot the resumable runs and refuse if state is visibly churning
const before = await pool.query(
  `SELECT id FROM workflow_runs WHERE (conversation_id = $1 OR parent_conversation_id = $1) AND status IN ('paused','failed')`,
  [conversationId]
);
if (before.rows.length === 0) return; // nothing to cancel, skip the racy path

function isSnapshotMismatch(err: unknown): boolean {
  return err instanceof Error && err.message.includes('Resumable run snapshot changed during reset');
}

for (let attempt = 0; attempt < 3; attempt++) {
  try {
    await cancelResumableRunsForConversation(conversationId);
    break;
  } catch (err) {
    if (!isSnapshotMismatch(err) || attempt === 2) throw err;
    await sleep(50 * (attempt + 1)); // back off so the concurrent writer settles
  }
}

When it happens

Trigger: Calling cancelResumableRunsForConversation while a concurrent actor (another process, a resume call, a manual DB edit) pauses/fails/cancels/deletes a run in the same conversation between the SELECT and the UPDATE, so the UPDATE affects fewer (or more) rows than were snapshotted.

Common situations: Two chat handlers racing on the same conversation reset; a user approving/resuming a paused run at the same moment an operator cancels it; a background sweeper flipping run status concurrently.

Related errors

• Failed to get workflow run status: ${err.message}
• Failed to get active workflow run: ${err.message}
• Failed to cancel resumable runs for conversation: ${err.mess
• Workflow run not found or not in running state (id: ${id})
• Failed to claim write-back apply: ${err.message}

AI-assisted analysis of coleam00/Archon@0773b97458 (2026-09-01). Data as JSON: /api/errors/cd40d5aa998d0296. Report an issue: GitHub.