What causes "shutdown stage gateway.process_reap: PID {} is still alive" in astrid-runtime/astrid?

Check what the PID actually is (ps -p ) — if it's the real gateway, use the normal stop path instead of marker cleanup

shutdown stage gateway.process_reap: PID {} is still alive

Error message

shutdown stage gateway.process_reap: PID {} is still alive

What it means

remove_dead_gateway_markers is meant to clean up markers for a gateway believed dead, but the PID from the ready record is still alive, so cleanup is refused to avoid clobbering a functioning gateway's state.

Source

Thrown at crates/astrid-cli/src/commands/mcp/lifecycle.rs:710

    R: tokio::io::AsyncRead + Unpin,
{
    let mut line = Vec::new();
    for _ in 0..=MAX_CONTROL_BYTES {
        let byte = reader
            .read_u8()
            .await
            .context("failed to read control frame")?;
        if byte == b'\n' {
            return Ok(line);
        }
        line.push(byte);
    }
    anyhow::bail!("MCP gateway control frame is missing or too large")
}

async fn remove_dead_gateway_markers(record: &GatewayReady) -> Result<()> {
    if crate::commands::daemon_control::is_process_alive(record.pid) {
        anyhow::bail!(
            "shutdown stage gateway.process_reap: PID {} is still alive",
            record.pid
        );
    }
    let lifecycle = try_acquire_gateway_lifecycle()?;
    let Some(lifecycle) = lifecycle else {
        anyhow::bail!(
            "shutdown stage gateway.lifecycle_fence: a successor gateway lifecycle remains active"
        );
    };
    let socket = gateway_socket_path()?;
    match astrid_core::local_transport::connect_outcome(&socket)
        .await
        .context("shutdown stage gateway.listener_probe")?
    {
        astrid_core::local_transport::ConnectOutcome::Connected(_) => {
            anyhow::bail!(
                "shutdown stage gateway.listener_absence: a gateway is still accepting connections"

View on GitHub (pinned to affd8760f4)

Solutions

Check what the PID actually is (ps -p <pid>) — if it's the real gateway, use the normal stop path instead of marker cleanup
If it's an unrelated process (PID reuse), remove the stale ready/marker files manually once verified
Re-run the operation; transient races between liveness checks often resolve on retry
Ensure only one client manages the gateway to avoid racing shutdowns

Example fix

// before
remove_dead_gateway_markers(&record).await?;
// after: verify the PID isn't the live gateway
if is_process_alive(record.pid) && process_is_gateway(record.pid) {
    stop_ready_gateway(&record).await?; // proper shutdown path
} else {
    remove_dead_gateway_markers(&record).await?;
}

Defensive patterns

Strategy: validation

Validate before calling

if daemon_control::is_process_alive(record.pid) {
    // verify it really is the gateway before any cleanup
    anyhow::ensure!(process_exe(record.pid)? == record.exe, "PID alive but not the gateway");
}

Type guard

fn marker_removal_is_safe(record: &GatewayReady) -> bool {
    !crate::commands::daemon_control::is_process_alive(record.pid)
}

Try / catch

match remove_dead_gateway_markers(&record).await {
    Err(e) if e.to_string().contains("still alive") => {
        // process alive: take the owned stop path instead of cleanup
        stop_ready_gateway(&record).await?;
    },
    other => other?,
}

Prevention

Don't forcibly kill gateway processes outside the library
Re-check PID identity after any suspected PID reuse
Run a single supervisor for gateway lifecycle

When it happens

Trigger: wait_for_gateway or stop_ready_gateway determines the ready record is stale and calls remove_dead_gateway_markers, yet daemon_control::is_process_alive(record.pid) returns true — the process outlived its recorded state.

Common situations: PID reuse: an unrelated process now owns the recorded PID; the gateway is actually fine but its ready file was corrupted/replaced; shutdown raced process exit between liveness checks.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of astrid-runtime/astrid@affd8760f4 (2026-09-09). Data as JSON: /api/errors/61c60333f9cbf4fc. Report an issue: GitHub.