What causes "Killing container past absolute ceiling" in nanocoai/nanoclaw?

Check why the container was unresponsive: docker events, OOM kills, agent-runner logs before exit

Killing container past absolute ceiling

Error message

Killing container past absolute ceiling

What it means

The 60s host sweep kills a session's container because its heartbeat file age exceeded the absolute ceiling even though the session is considered running. This is a last-resort SLA enforcement: the agent container is unresponsive past the hard limit, so it is killed and stuck processing rows are reset for retry by a fresh container.

Source

Thrown at src/host-sweep.ts:276

function enforceRunningContainerSla(
  inDb: InboundMailbox,
  outDb: OutboundMailbox,
  session: Session,
  agentGroupId: string,
): void {
  const decision = decideStuckAction({
    now: Date.now(),
    heartbeatMtimeMs: heartbeatMtimeMs(agentGroupId, session.id),
    containerStartedAtMs: getContainerStartedAtMs(session.id),
    containerState: outDb.getContainerState(),
    claims: outDb.getProcessingClaims(),
  });

  if (decision.action === 'ok') return;

  if (decision.action === 'kill-ceiling') {
    log.warn('Killing container past absolute ceiling', {
      sessionId: session.id,
      heartbeatAgeMs: decision.heartbeatAgeMs,
      ceilingMs: decision.ceilingMs,
    });
    killContainer(session.id, 'absolute-ceiling');
    resetStuckProcessingRows(inDb, outDb, session, 'absolute-ceiling');
    return;
  }

  log.warn('Killing container — message claimed then silent', {
    sessionId: session.id,
    messageId: decision.messageId,
    claimAgeMs: decision.claimAgeMs,
    toleranceMs: decision.toleranceMs,
  });
  killContainer(session.id, 'claim-stuck');
  resetStuckProcessingRows(inDb, outDb, session, 'claim-stuck');
}

View on GitHub (pinned to 294ef2aee8)

Solutions

Check why the container was unresponsive: docker events, OOM kills, agent-runner logs before exit
If workloads legitimately exceed the ceiling, raise the SLA ceiling config for that group
Verify the heartbeat mount is writable inside the container (mount correctness)
If it recurs, capture docker inspect of the dying container before the sweep kills it to post-mortem

Defensive patterns

Strategy: retry

Prevention

Monitor heartbeat age and alert before the ceiling is reached
Keep long-running agent turns under the configured SLA or raise the ceiling deliberately
Verify the heartbeat mount is writable in the image (touch /workspace/.heartbeat at startup as a smoke test)

When it happens

Trigger: maintainSessionMailbox computes a kill-ceiling decision (heartbeatAgeMs > ceilingMs) from the heartbeat file at /workspace/.heartbeat; killContainer(session.id, 'absolute-ceiling') then resets processing_ack rows.

Common situations: Agent container hung on a long LLM call or deadlocked; host-gateway/network stall so the container can't touch the heartbeat; overloaded Docker host; container paused or OOM-frozen while host still lists it running.

Related errors

AI-assisted analysis of nanocoai/nanoclaw@294ef2aee8 (2026-08-28). Data as JSON: /api/errors/2295e3ab8e710de4. Report an issue: GitHub.