coleam00/Archon · error

Failed to cancel fan-out run: ${err.message}

Error message

Failed to cancel fan-out run: ${err.message}

What it means

This error is thrown by the fan-out run cancel function in the database layer after the underlying cancellation transaction fails. The original database error is logged (db.workflow_run_fan_out_cancel_failed) and re-thrown wrapped in a message that includes the inner error message, so the caller sees one combined failure explaining that cancelling a fan-out workflow run could not be persisted.

Source

Thrown at packages/core/src/db/workflows.ts:1380

         SET status = 'cancelled',
             completed_at = ${dialect.now()},
             metadata = ${dialect.jsonMerge('metadata', 2)}
         WHERE id = $1 AND status NOT IN ('completed', 'cancelled')`,
        [id, JSON.stringify({ cancelled_reason: reason })]
      );
      if ((update.rowCount ?? 0) > 0) {
        await insertWorkflowEvent(query, {
          workflow_run_id: id,
          event_type: 'workflow_cancelled',
          data: { reason },
        });
      }
      return update;
    });
  } catch (error) {
    const err = error as Error;
    getLog().error({ err, workflowRunId: id, reason }, 'db.workflow_run_fan_out_cancel_failed');
    throw new Error(`Failed to cancel fan-out run: ${err.message}`);
  }
  const cancelled = (result.rowCount ?? 0) > 0;
  if (!cancelled) {
    getLog().info({ workflowRunId: id, reason }, 'db.workflow_run_fan_out_cancel_noop');
  }
  return { cancelled };
}

/**
 * Pause a running workflow run for human approval.
 * Sets status to 'paused' and stores approval context in metadata.
 * Does NOT set completed_at — the run is not finished.
 *
 * The stored `metadata.approval` is REPLACED with `approvalContext` wholesale
 * (writeApprovalMetadata), never merged into. So a fresh pause stores exactly
 * the keys the caller set — at every depth — and nothing a prior gate of the
 * same run left behind can survive, on either dialect (#2673). Readers treat an
 * absent key exactly like a JSON null (`!= null`, `=== true`, `?? ''`), and

View on GitHub (pinned to 0773b97458)

Solutions

1. Read the inner err.message in the thrown message and the db.workflow_run_fan_out_cancel_failed log entry to identify the underlying database error
2. Verify the database is reachable and the connection pool is healthy, then retry the cancel
3. Check for concurrent operations on the same workflow_run row (deadlock/serialization failure) and retry with backoff
4. Confirm the workflowRunId exists and is a fan-out run in a state that permits cancellation
5. If the run is already terminal (completed/failed/cancelled), treat the cancel as a no-op instead of retrying

Example fix

// before
await cancelWorkflowRunFanOut(id, reason); // unhandled -> crashes caller
// after
try {
  await cancelWorkflowRunFanOut(id, reason);
} catch (err) {
  logger.error({ id, err }, 'cancel failed; checking run state before retry');
  const run = await getWorkflowRun(id);
  if (run?.status === 'running') await cancelWorkflowRunFanOut(id, reason);
}

const run = await getWorkflowRun(id);
if (!run) throw new Error(`run ${id} not found before cancel`);
if (run.status !== 'running') return { cancelled: false };

function isErrorWithMessage(e: unknown): e is Error {
  return e instanceof Error && typeof e.message === 'string';
}

try {
  await cancelWorkflowRunFanOut(id, reason);
} catch (e) {
  const msg = e instanceof Error ? e.message : String(e);
  logger.error({ id, msg }, 'fan-out cancel failed');
  if (/connect|ECONNREFUSED|timeout/i.test(msg)) scheduleRetry(id, reason);
}

When it happens

Trigger: Calling cancelWorkflowRunFanOut (the fan-out cancel path, packages/core/src/db/workflows.ts:1380) when the underlying SQL UPDATE/transaction throws: database connection failure, constraint violation, deadlock, or the transaction aborted mid-cancel.

Common situations: Postgres or SQLite temporarily unreachable during a cancel request; concurrent cancellation or completion racing the UPDATE inside the transaction; malformed workflowRunId type; the DB pool exhausted under load.

Related errors

• Failed to cancel workflow run: ${err.message}
• Failed to pause workflow run: ${err.message}
• Failed to pause workflow run for wait: ${err.message}
• Failed to abandon workflow run ${run.id}: ${err.message}
• Failed to get workflow run: ${err.message}

AI-assisted analysis of coleam00/Archon@0773b97458 (2026-09-01). Data as JSON: /api/errors/aaa06e3c872af2fd. Report an issue: GitHub.