coleam00/Archon · warning

run lookup failed (NOT reaped): ${(err as Error).message}

Error message

run lookup failed (NOT reaped): ${(err as Error).message}

What it means

A per-environment error recorded during container environment cleanup (cleanupContainerEnvironments) when the lookup of a live run owning the environment (getLiveRunOwningEnv) throws. The environment is deliberately NOT reaped and the loop continues to the next row, because reaping under uncertainty could destroy an environment a live run owns; the failure is logged as container_env_reap_lookup_failed.

Source

Thrown at packages/core/src/services/cleanup-service.ts:190

  const backend = new ContainerBackend({
    store: isolationEnvDb.createIsolationStore(),
    config: CLEANUP_PLACEHOLDER_CONTAINER_CONFIG,
  });

  for (const row of rows) {
    // FAIL CLOSED on an ambiguous lookup (H3): a DB error is NOT "no run" — treating
    // it as an orphan would destroy a claimable run's container on a transient blip
    // (violating No-Autonomous-Lifecycle-Mutation). Report + skip, never destroy.
    //
    // Same lock the worktree sweeps use. getRunByIsolationEnvId, which this replaced,
    // took the newest run row BEFORE filtering status, so a newer terminal run could
    // shadow an older claimable one and the container would be reaped underneath it.
    let liveRun: Awaited<ReturnType<typeof isolationEnvDb.getLiveRunOwningEnv>>;
    try {
      liveRun = await isolationEnvDb.getLiveRunOwningEnv(row.id);
    } catch (err) {
      report.errors.push({
        id: row.id,
        error: `run lookup failed (NOT reaped): ${(err as Error).message}`,
      });
      getLog().warn({ err, envId: row.id }, 'container_env_reap_lookup_failed');
      continue;
    }
    if (liveRun) {
      report.skipped.push({
        id: row.id,
        reason: `run ${liveRun.id.slice(0, 8)} is ${liveRun.status}`,
      });
      continue;
    }
    if (row.days_since_created < daysStale) {
      report.skipped.push({
        id: row.id,
        reason: `${Math.floor(row.days_since_created)}d old (< ${daysStale}d threshold)`,
      });
      continue;

View on GitHub (pinned to 0773b97458)

Solutions

1. Re-run the cleanup report/sweep — the environment was skipped (NOT reaped) and will be reconsidered.
2. Check the container_env_reap_lookup_failed log entries for the underlying DB error.
3. Fix database health (connectivity, pool limits, locks) before the next cleanup cycle.
4. Verify the environment's owning run state manually if it remains uncleaned across multiple sweeps.

Example fix

// before
cleanup report   # 'run lookup failed (NOT reaped)' for env 7
// after
pg_isready -h localhost   # restore DB, then
cleanup report   # env 7 reaped or claimed correctly

// Probe DB reachability before starting a cleanup sweep:
try { await db.execute(sql`select 1`); } catch { console.error('DB unreachable; postponing cleanup sweep'); return; }

try {
  liveRun = await isolationEnvDb.getLiveRunOwningEnv(envId);
} catch (err) {
  getLog().warn({ err, envId }, 'container_env_reap_lookup_failed');
  // skip: never reap when run ownership is unknown
  continue;
}

When it happens

Trigger: During a cleanup sweep over claimable isolation environments, the per-row DB query getLiveRunOwningEnv throws — database unreachable mid-sweep, query timeout, or a transient connection error — for a specific environment id.

Common situations: Cleanup service running while the database is being restarted or is under heavy load; connection pool exhausted by concurrent workflows; network blip between the cleanup worker and the DB.

Related errors

• (err as Error).message
• Cannot create worktree: database lookup failed. Error: ${res
• Cannot create worktree: repository registration failed. Erro
• Failed to access database: ${err.message} Hint: Check that D
• Failed to create the workflow run: ${(error as Error).messag

AI-assisted analysis of coleam00/Archon@0773b97458 (2026-09-01). Data as JSON: /api/errors/da87bae027ccade7. Report an issue: GitHub.