influxdata/influxdb · error · CatalogError

invalid stop ack for node

Error message

invalid stop ack for node '{node_id}' (current state: {current_state})

What it means

Raised when a node sends a stop acknowledgement that does not match its current lifecycle state in the catalog. The node state machine expects stop acks only from nodes in an eligible state (e.g. stopping); an ack from a node already stopped, running, or in another state is rejected as invalid.

Solutions

1. Check the node's current state in the catalog and only send the stop ack when the state machine expects it.
2. Treat repeated 'idempotent no-op' vs invalid-state errors: if the transition already happened, skip resending the ack.
3. Ensure only one orchestrator instance drives node stop operations to avoid duplicate acks.
4. Restart the node lifecycle flow from a clean state if state tracking has drifted.

Example fix

// before
client.post_stop_ack(node_id).await?;
// after
let state = catalog.node_state(node_id)?.current_state;
if state == "stopping" {
    client.post_stop_ack(node_id).await?;
}

let state = catalog.node_state(node_id)?.current_state;
if state != "stopping" {
    return Err(format!("stop ack invalid in state {state}"));
}

match result {
    Err(CatalogError::InvalidStopAck { current_state, .. }) => {
        log::warn!("stale stop ack; node state={current_state}");
    }
    other => other?,
}

When it happens

Trigger: Posting a stop-ack (heartbeat/stop acknowledgment) for a node_id whose catalog state is not one that accepts stop acks; duplicated or stale stop acks delivered after the transition already completed.

Common situations: Retried HTTP stop-ack requests after the first succeeded; split-brain or stale node processes acking after manual intervention; orchestration tooling replaying old requests.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• invalid unregister for node
• node ' ' has compact mode and cannot be removed
• idempotent no-op
• metric should be in progress
• no metric in progress

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/f5f1481c2e6ea849. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_catalog/src/error.rs:311

    #[error("cannot modify internal db")]
    CannotModifyInternalDatabase,

    #[error("tried to stop a node ({node_id}) that is already stopped")]
    NodeAlreadyStopped { node_id: Arc<str> },

    #[error(
        "node '{node_id}' is not fully stopped (current state: {current_state}); run \"stop node\" first"
    )]
    NodeNotFullyStopped {
        node_id: Arc<str>,
        current_state: &'static str,
    },

    #[error("node '{node_id}' has compact mode and cannot be removed")]
    NodeModeNotRemovable { node_id: Arc<str> },

    #[error("invalid stop ack for node '{node_id}' (current state: {current_state})")]
    InvalidStopAck {
        node_id: Arc<str>,
        current_state: &'static str,
    },

    #[error("invalid unregister for node '{node_id}' (current state: {current_state})")]
    InvalidUnregister {
        node_id: Arc<str>,
        current_state: &'static str,
    },

    #[error("idempotent no-op")]
    IdempotentNoOp,

    #[error("cannot delete operator token")]
    CannotDeleteOperatorToken,

    #[error(

View on GitHub (pinned to 06200ef96b)