zeroclaw-labs/zeroclaw · error · anyhow::Error

Node request failed: {} {}

Error message

Node request failed: {} {}

What it means

NodeTransport::send signs a JSON payload and POSTs it to https://{node_address}/api/node-control/{endpoint}. Network-level failures (DNS, connect, TLS) propagate as reqwest errors through `?`; this bail fires only after a response arrives with a non-2xx status, and embeds both the HTTP status and the peer's response body. It tells you the peer answered and refused.

Source

Thrown at crates/zeroclaw-runtime/src/nodes/transport.rs:108

        let body = serde_json::to_vec(&payload)?;
        let timestamp = Utc::now().timestamp();
        let nonce = uuid::Uuid::new_v4().to_string();
        let signature = sign_request(&self.shared_secret, &body, timestamp, &nonce)?;

        let url = format!("https://{node_address}/api/node-control/{endpoint}");
        let resp = self
            .http
            .post(&url)
            .header("X-ZeroClaw-Timestamp", timestamp.to_string())
            .header("X-ZeroClaw-Nonce", &nonce)
            .header("X-ZeroClaw-Signature", &signature)
            .header("Content-Type", "application/json")
            .body(body)
            .send()
            .await?;

        if !resp.status().is_success() {
            bail!(
                "Node request failed: {} {}",
                resp.status(),
                resp.text().await.unwrap_or_default()
            );
        }

        Ok(resp.json().await?)
    }

    /// Verify an incoming request from a peer node.
    pub fn verify_incoming(
        &self,
        payload: &[u8],
        timestamp_header: &str,
        nonce_header: &str,
        signature_header: &str,
    ) -> Result<bool> {
        let timestamp: i64 = timestamp_header.parse().map_err(|_| {

View on GitHub (pinned to 88bb9c8533)

Solutions

1. Read the embedded status: 401/403 → compare and re-distribute the shared secret on both nodes; 404 → verify node_address and endpoint name; 5xx → inspect the peer node's logs
2. Confirm both nodes run compatible versions exposing /api/node-control/{endpoint}
3. If a proxy fronts the peer, check it passes the X-ZeroClaw-* headers and path unchanged
4. Retry transient 5xx with backoff; do not retry 4xx — those are deterministic rejections

Example fix

// before: fire and forget
let v = transport.send(&addr, "exec", payload).await?;

// after: classify the failure
match transport.send(&addr, "exec", payload).await {
    Ok(v) => v,
    Err(e) if e.to_string().contains("Node request failed: 40") => {
        log::warn!("auth/path problem: {e}");
        return Err(e);
    }
    Err(e) => { retry_with_backoff(...).await? }
}

// optional preflight: cheap health probe before the signed call
let resp = reqwest::get(format!("https://{node_address}/api/node-control/health")).await;
if !resp.map(|r| r.status().is_success()).unwrap_or(false) {
    // defer the signed request; peer is unhealthy or unreachable
}

match transport.send(node_address, endpoint, payload).await {
    Ok(v) => v,
    Err(e) => {
        let msg = e.to_string();
        if msg.contains("Node request failed: 401") || msg.contains(": 403") {
            // deterministic auth failure: fix/rotate shared secret; do not retry
        } else if msg.contains("Node request failed: 5") {
            // transient: retry with bounded exponential backoff
        } else {
            return Err(e); // network-level reqwest error or unknown status
        }
    }
}

When it happens

Trigger: 401/403: signature/secret verification rejected on the peer (shared secrets differ) or the timestamp window (error 867) fired; 404: wrong node_address or unknown endpoint segment; 500/502/503: peer-side handler crashed, upstream gateway error, peer under maintenance; the peer returned an HTML error page, which appears verbatim in the message.

Common situations: Nodes provisioned with different shared secrets (rotated on one side only); reverse proxy in front of the node rewriting paths; peer process half-dead so its gateway returns 502; typos in the endpoint string.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• Embedding API error {status}: {text}
• Request timestamp too old or too far in future
• URL must include a host
• Composio v3 API error: {err}
• QQ channel requires the `channel-qq` feature

AI-assisted analysis of zeroclaw-labs/zeroclaw@88bb9c8533 (2026-08-23). Data as JSON: /api/errors/cecc6fd6c5a1a78e. Report an issue: GitHub.