xai-org/grok-build · error

failed to build shared HTTP client

Error message

failed to build shared HTTP client

What it means

shared_client lazily builds the process-wide reqwest blocking Client (keepalive, pool tuning) inside a cached initializer and .expect()s the build. A panic here means reqwest could not construct the client - nearly always TLS backend initialization or provider/certificate setup failing, not a transient condition.

Solutions

1. Verify xai-grok-extra-ca and rustls/aws-lc-rs features are consistent workspace-wide and rebuild (cargo clean if needed)
2. Check for a conflicting rustls::crypto::CryptoProvider::install_default earlier in the process
3. Pre-flight build the client once at startup and surface a clear error instead of relying on the expect
4. If this reproduces only in one environment, diff its TLS/crypto library availability against a working one

Example fix

// before
let resp = send_with_retry_escaping_pool(&req); // panics if shared_client build fails
// after
let client = xai_grok_http::shared_client(); // pre-warm at startup with clear panic context
tracing::info!("shared HTTP client ready");
let resp = send_with_retry_escaping_pool(&req);

// pre-warm and convert the panic into a checkable failure
let client = std::panic::catch_unwind(xai_grok_http::shared_client)
    .map_err(|_| anyhow!("shared HTTP client build failed (TLS init?)"))?;

null

std::panic::catch_unwind(|| send_with_retry_escaping_pool(&req))
    .map_err(|_| anyhow!("HTTP stack unavailable: shared client build failed"))?

When it happens

Trigger: First call to shared_client() (e.g. via send_with_retry_escaping_pool) when the rustls/aws-lc-rs provider fails to init, the extra root CA material cannot be loaded, or the reqwest builder rejects the configuration.

Common situations: Inconsistent rustls/crypto-provider feature flags; deployment image missing crypto prerequisites; a previously installed global CryptoProvider conflicting with this build; static/musl builds with a broken TLS backend.

Related errors

• default reqwest client builds
• failed to build shared upload HTTP client
• hook HTTP client config is valid
• aws-lc-rs supports the default protocol versions
• Failed to parse part

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/2c435377c9782eea. Report an issue: GitHub.

Appendix: source

Thrown at crates/codegen/xai-grok-http/src/lib.rs:300

/// Without the health checks reqwest reuses it for new streams, so every retry fails identically and a reachable server looks unreachable.
/// Idle and TCP eviction drops connections before the upstream idle window (~60-100s; 30s is a conservative default) closes them.
/// The HTTP/2 keepalive ping detects a dead connection so the pool stops handing it out.
pub fn shared_client() -> reqwest::Client {
    static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
    CLIENT
        .get_or_init(|| {
            let _timer = startup_timer!("startup.http_client_build");
            xai_grok_extra_ca::build_reqwest_client(|builder| {
                builder
                    .connect_timeout(std::time::Duration::from_secs(30))
                    .user_agent(process_user_agent_string())
                    .pool_idle_timeout(std::time::Duration::from_secs(30))
                    .http2_keep_alive_interval(std::time::Duration::from_secs(20))
                    .http2_keep_alive_timeout(std::time::Duration::from_secs(10))
                    .http2_keep_alive_while_idle(true)
                    .tcp_keepalive(std::time::Duration::from_secs(30))
            })
            .expect("failed to build shared HTTP client")
        })
        .clone()
}

/// Wrap a raw client with [`AuthRetryMiddleware`] for automatic 401 retry.
pub fn with_auth_retry(
    client: reqwest::Client,
    credentials: std::sync::Arc<dyn xai_grok_auth::AuthCredentialProvider>,
) -> reqwest_middleware::ClientWithMiddleware {
    reqwest_middleware::ClientBuilder::new(client)
        .with(xai_grok_auth::AuthRetryMiddleware::new(credentials, 1))
        .build()
}

/// Returns a shared [`reqwest::Client`] for GCS uploads, creating it on first call.
///
/// Unlike `shared_client()`, this client has aggressive connection pool eviction to avoid reusing stale or poisoned connections during retry loops.
/// When uploads fail and trigger exponential backoff (1s, 2s, 4s...), idle connections may be closed by the server, Cloudflare, or load balancers.

View on GitHub (pinned to bc7f02eddd)