windmill-labs/windmill · error

Error requesting oidc token from {url}: {e:#?}

Error message

Error requesting oidc token from {url}: {e:#?}

What it means

AuthedClient::get_id_token POSTs to the workspace's OIDC token endpoint (/api/w/{ws}/oidc/token/{audience}) to mint a short-lived token for a given audience. This error is thrown when the HTTP request itself fails at the transport level — connection refused, DNS failure, TLS error, timeout — before any status code is returned. The reqwest error is formatted with {e:#?} so the full error chain is embedded in the message.

Source

Thrown at backend/windmill-common/src/client.rs:91

    pub async fn get_id_token(&self, audience: &str) -> anyhow::Result<String> {
        let url = format!(
            "{}/api/w/{}/oidc/token/{}",
            self.base_internal_url, self.workspace, audience
        );
        let response = self
            .force_client
            .as_ref()
            .unwrap_or(&HTTP_CLIENT)
            .post(&url)
            .header(
                reqwest::header::AUTHORIZATION,
                reqwest::header::HeaderValue::from_str(&format!("Bearer {}", self.token))?,
            )
            .send()
            .await
            .map_err(|e| {
                tracing::error!("Error requesting oidc token from {url}: {e:#?}");
                anyhow::anyhow!("Error requesting oidc token from {url}: {e:#?}")
            })?;

        match response.status().as_u16() {
            200u16 => Ok(response.text().await.context("reading oidc token body")?),
            status => {
                let body = response.text().await.unwrap_or_default();
                Err(anyhow::anyhow!(
                    "oidc token request to {url} failed with status {status}: {body}"
                ))
            }
        }
    }

    pub async fn get_resource_value<T: DeserializeOwned>(&self, path: &str) -> anyhow::Result<T> {
        let url = format!(
            "{}/api/w/{}/resources/get_value/{}",
            self.base_internal_url, self.workspace, path
        );

View on GitHub (pinned to e474e8803c)

Solutions

1. Verify base_internal_url is reachable from the process calling get_id_token: curl -v <base_internal_url>/api/health
2. Check the backend is running and listening on the port in base_internal_url (ss -ltnp / docker ps)
3. Fix DNS/hostname: use the in-cluster service name for workers, not an external URL
4. If TLS is involved, confirm the CA is trusted by the worker container or use the proper scheme (http vs https)
5. For transient outages, wrap the call in a bounded retry with backoff

Example fix

// before: guessing the URL
let client = AuthedClient::new("http://windmill.internal:9000".into(), ws, token, None);
// after: build from an env-configured, health-checked base URL
let base = std::env::var("WM_BASE_INTERNAL_URL").expect("WM_BASE_INTERNAL_URL set");
reqwest::get(format!("{base}/api/health")).await.expect("backend reachable before auth");
let client = AuthedClient::new(base, ws, token, None);

let health = reqwest::get(format!("{base_internal_url}/api/health")).await;
assert!(health.is_ok(), "backend unreachable at {base_internal_url} before calling get_id_token");

match client.get_id_token(aud).await {
    Ok(tok) => tok,
    Err(e) if e.to_string().contains("Error requesting oidc token") => {
        // transport-level failure: log and retry with backoff
        tokio::time::sleep(Duration::from_secs(2)).await;
        client.get_id_token(aud).await.context("oidc token after retry")?
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling get_id_token (e.g. from a worker or script needing a service token) when base_internal_url points to an unreachable or wrong host/port, the backend is down or restarting, DNS cannot resolve the hostname, or a proxy/firewall blocks the connection. Also on TLS certificate failures and reqwest total-timeout expiry on HTTP_CLIENT.

Common situations: Self-hosted workers with a misconfigured BASE_URL/internal URL, Kubernetes/Docker networking where the worker cannot reach the API service name, backend temporarily unavailable during rolling deploys, split-horizon DNS where the internal URL resolves only inside the cluster.

Related errors

• HTTP agent request POST {} failed {}
• HTTP agent request GET {} failed {}
• HTTP agent request PUT {} failed {}
• ApiError with mapped HTTP status message (e.g. "Not Found",
• Generic Error: status: ${errorStatus}; status text: ${errorS

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/fe5b917afe89f76f. Report an issue: GitHub.