nautechsystems/nautilus_trader · error · anyhow::Error

All {} requests failed: {errors:?}

Error message

All {} requests failed: {errors:?}

What it means

Every parallel BitMEX cancel request issued by the broadcast canceller failed, so `process_cancel_results` returns an aggregate error listing all individual failures (`{errors:?}`). The canceller fans each cancel out to all healthy transport clients; when none succeeds, the operation cannot be completed.

Source

Thrown at crates/adapters/bitmex/src/broadcast/canceller.rs:605

                            "{operation} request failed [{client_id}]: {error_msg} {params}"
                        );
                        errors.push(error_msg);
                    }
                }
                Err(e) => {
                    log::warn!("{operation} task join error: {e:?}");
                    errors.push(format!("Task panicked: {e:?}"));
                }
            }
        }

        // All tasks failed
        self.failed_cancels.fetch_add(1, Ordering::Relaxed);
        log::error!(
            "All {} requests failed: {errors:?} {params}",
            operation.to_lowercase(),
        );
        Err(anyhow::anyhow!(
            "All {} requests failed: {errors:?}",
            operation.to_lowercase(),
        ))
    }

    /// Broadcasts a single cancel request to all healthy clients in parallel.
    ///
    /// # Returns
    ///
    /// - `Ok(Some(report))` if successfully cancelled with a report.
    /// - `Ok(None)` if the order was already cancelled (idempotent success).
    /// - `Err` if all requests failed.
    ///
    /// # Errors
    ///
    /// Returns an error if all cancel requests fail or no healthy clients are available.
    pub async fn broadcast_cancel(
        &self,

View on GitHub (pinned to 18893faf8b)

Solutions

1. Read the `{errors:?}` list in the log — the per-request errors identify the shared root cause (401 auth, 429 rate limit, 400 bad order id).
2. Verify BitMEX API key/secret and that the key has order permissions for the target network (mainnet vs testnet).
3. Check the order IDs / symbol exist and are still open; already-filled or canceled orders return errors.
4. Check network/proxy connectivity from all transport clients and whether BitMEX API is up; retry after backoff.

Example fix

// before: blanket cancel of possibly-closed orders
broadcast_cancel(instrument_id, client_order_id)?;
// after: check order state first or tolerate already-closed
if let Some(order) = cache.order(&client_order_id) { if order.is_open() { broadcast_cancel(...)?; } }

if !bitmex_client.api_key_valid() || !bitmex_client.key_has_order_permission() {
    return Err(anyhow::anyhow!("bitmex credentials lack order-cancel permission"));
}
// also confirm the order is still open before cancelling

match process_cancel_results(results).await {
    Ok(()) => Ok(()),
    Err(e) if all_errors_are_auth(&e) => {
        log::error!("auth failure on all cancel attempts; not retrying");
        Err(e)
    }
    Err(e) if all_errors_are_rate_limit(&e) => {
        tokio::time::sleep(backoff).await;
        retry_cancel().await
    }
    Err(e) => Err(e),
}

When it happens

Trigger: `broadcast_cancel`, `broadcast_batch_cancel`, or `broadcast_cancel_all` dispatch requests to all transports and every response is an error — e.g. auth signature rejected, API key lacks order permission, invalid symbol/order IDs, network outage, or IP rate-limited/banned.

Common situations: Expired or wrong API credentials; BitMEX geo-block or API maintenance; cancelling order IDs that already closed; sending to testnet with mainnet keys (or vice versa); all transports unhealthy after a network partition.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• Either client_order_id or venue_order_id must be provided
• rate limiter decision lock poisoned
• Invalid `ConnectionMode` value: {value}
• Binance Futures data teardown failed: {}
• Binance Spot data teardown failed: {}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/4214f094d404e60c. Report an issue: GitHub.