openai/openai-python · critical · SubjectTokenProviderError

Failed to fetch GCP subject token from metadata server: HTTP

Error message

Failed to fetch GCP subject token from metadata server: HTTP {response.status_code}

What it means

The GCP identity-token provider requests a token from the Compute Engine metadata server (metadata.google.internal) with the Metadata-Flavor: Google header. An HTTP error status triggers this SubjectTokenProviderError with the status code attached. Typical causes: 404 when the metadata endpoint path or query is wrong, 403/401 from GCE firewall blocks on metadata requests, or 503 during metadata server restarts.

Source

Thrown at src/openai/auth/_workload.py:193

        audience: the unique URI agreed upon by both the instance and the system verifying
            the instance's identity. Defaults to `https://api.openai.com/v1`.
        timeout: the request timeout in seconds. Defaults to 10.0.
        http_client: optional httpx2.Client instance to use for requests. If not provided, a new client will be created for each request.
    """

    def get_token() -> str:
        try:
            url = "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity"
            params = {"audience": audience}

            if http_client is not None:
                response = http_client.get(url, params=params, headers={"Metadata-Flavor": "Google"}, timeout=timeout)
            else:
                with httpx2.Client() as client:
                    response = client.get(url, params=params, headers={"Metadata-Flavor": "Google"}, timeout=timeout)

            if response.is_error:
                raise SubjectTokenProviderError(
                    f"Failed to fetch GCP subject token from metadata server: HTTP {response.status_code}",
                    response=response,
                )
            token = response.text.strip()
            if not token:
                raise SubjectTokenProviderError("GCP metadata server returned an empty token", response=response)
            return token
        except Exception as e:
            raise SubjectTokenProviderError(f"Failed to fetch GCP subject token from metadata server: {e}") from e

    return {"token_type": "id", "get_token": get_token}


class _WorkloadIdentityAuth(Generic[_WorkloadIdentityT]):
    def __init__(
        self,
        *,
        workload_identity: _WorkloadIdentityT,

View on GitHub (pinned to 9917c6e28e)

Solutions

  1. Retry with backoff on 5xx/503 — metadata server restarts are transient
  2. Verify the request from the same host: curl -H 'Metadata-Flavor: Google' 'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=...'
  3. Check the audience matches the API you're authenticating to
  4. Ensure the service account has the roles/permissions needed for the target API

Example fix

# before
provider = gcp_id_token_provider(audience="https://wrong.api/")

# after
provider = gcp_id_token_provider(audience="https://api.openai.com/v1")
Defensive patterns

Strategy: retry

Validate before calling

import httpx
r = httpx.get(META_URL, headers={"Metadata-Flavor":"Google"})
assert not r.is_error, r.status_code

Try / catch

for attempt in range(3):
    try:
        return provider_get_token()
    except SubjectTokenProviderError as e:
        if "HTTP 5" in str(e): time.sleep(2**attempt); continue
        raise

Prevention

When it happens

Trigger: Running with gcp_id_token_provider on GCE/Cloud Run/GKE where the metadata server responds with an error: wrong audience/resource parameters, metadata server blocked by organization policy, or the instance's metadata server restarting.

Common situations: Query-string parameters (audience) misconfigured for the target API; GKE Workload Identity sidecar not ready at startup; org policies blocking legacy metadata endpoints; startup races before the metadata agent is available.

Related errors

AI-assisted analysis of openai/openai-python@9917c6e28e (2026-08-28). Data as JSON: /api/errors/f8a4242f09e92555. Report an issue: GitHub.