xai-org/grok-build · error

failed to build shared HTTP client

Error message

failed to build shared HTTP client

What it means

shared_client lazily builds the process-wide reqwest blocking Client (keepalive, pool tuning) inside a cached initializer and .expect()s the build. A panic here means reqwest could not construct the client - nearly always TLS backend initialization or provider/certificate setup failing, not a transient condition.

Source

Thrown at crates/codegen/xai-grok-http/src/lib.rs:300

/// Without the health checks reqwest reuses it for new streams, so every retry fails identically and a reachable server looks unreachable.
/// Idle and TCP eviction drops connections before the upstream idle window (~60-100s; 30s is a conservative default) closes them.
/// The HTTP/2 keepalive ping detects a dead connection so the pool stops handing it out.
pub fn shared_client() -> reqwest::Client {
    static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
    CLIENT
        .get_or_init(|| {
            let _timer = startup_timer!("startup.http_client_build");
            xai_grok_extra_ca::build_reqwest_client(|builder| {
                builder
                    .connect_timeout(std::time::Duration::from_secs(30))
                    .user_agent(process_user_agent_string())
                    .pool_idle_timeout(std::time::Duration::from_secs(30))
                    .http2_keep_alive_interval(std::time::Duration::from_secs(20))
                    .http2_keep_alive_timeout(std::time::Duration::from_secs(10))
                    .http2_keep_alive_while_idle(true)
                    .tcp_keepalive(std::time::Duration::from_secs(30))
            })
            .expect("failed to build shared HTTP client")
        })
        .clone()
}

/// Wrap a raw client with [`AuthRetryMiddleware`] for automatic 401 retry.
pub fn with_auth_retry(
    client: reqwest::Client,
    credentials: std::sync::Arc<dyn xai_grok_auth::AuthCredentialProvider>,
) -> reqwest_middleware::ClientWithMiddleware {
    reqwest_middleware::ClientBuilder::new(client)
        .with(xai_grok_auth::AuthRetryMiddleware::new(credentials, 1))
        .build()
}

/// Returns a shared [`reqwest::Client`] for GCS uploads, creating it on first call.
///
/// Unlike `shared_client()`, this client has aggressive connection pool eviction to avoid reusing stale or poisoned connections during retry loops.
/// When uploads fail and trigger exponential backoff (1s, 2s, 4s...), idle connections may be closed by the server, Cloudflare, or load balancers.

View on GitHub (pinned to bc7f02eddd)

Solutions

1. Verify xai-grok-extra-ca and rustls/aws-lc-rs features are consistent workspace-wide and rebuild (cargo clean if needed)
2. Check for a conflicting rustls::crypto::CryptoProvider::install_default earlier in the process
3. Pre-flight build the client once at startup and surface a clear error instead of relying on the expect
4. If this reproduces only in one environment, diff its TLS/crypto library availability against a working one

Example fix

// before
let resp = send_with_retry_escaping_pool(&req); // panics if shared_client build fails
// after
let client = xai_grok_http::shared_client(); // pre-warm at startup with clear panic context
tracing::info!("shared HTTP client ready");
let resp = send_with_retry_escaping_pool(&req);

// pre-warm and convert the panic into a checkable failure
let client = std::panic::catch_unwind(xai_grok_http::shared_client)
    .map_err(|_| anyhow!("shared HTTP client build failed (TLS init?)"))?;

null

std::panic::catch_unwind(|| send_with_retry_escaping_pool(&req))
    .map_err(|_| anyhow!("HTTP stack unavailable: shared client build failed"))?

When it happens

Trigger: First call to shared_client() (e.g. via send_with_retry_escaping_pool) when the rustls/aws-lc-rs provider fails to init, the extra root CA material cannot be loaded, or the reqwest builder rejects the configuration.

Common situations: Inconsistent rustls/crypto-provider feature flags; deployment image missing crypto prerequisites; a previously installed global CryptoProvider conflicting with this build; static/musl builds with a broken TLS backend.

Related errors

• default reqwest client builds
• failed to build shared upload HTTP client
• hook HTTP client config is valid
• Failed to parse upload response: {}
• Failed to parse part {} response: {}

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/2c435377c9782eea. Report an issue: GitHub.