pola-rs/polars · error · ValueError

unsupported: cannot combine token with {unhandled_key} in st

Error message

unsupported: cannot combine token with {unhandled_key} in storage_options

What it means

In the GCP branch of the auto credential-provider builder (py-polars/src/polars/io/cloud/credential_provider/_builder.py:502-526), storage_options keys 'token'/'bearer_token' select a user-provided OAuth bearer token. Any other unrecognized key means raw credentials dispatched to the Rust side. Specifying a token together with any other credential key is contradictory and raises ValueError naming the conflicting key; an unrecognized key WITHOUT a token would simply fall through to native handling.

Source

Thrown at py-polars/src/polars/io/cloud/credential_provider/_builder.py:526

                    k = k.lower()

                    # https://docs.rs/object_store/latest/object_store/gcp/enum.GoogleConfigKey.html
                    if k in {"token", "bearer_token"}:
                        token = v
                    elif k in AUTOINIT_IGNORED_KEYS:
                        continue
                    else:
                        # We assume some sort of access key was given, so we
                        # just dispatch to the rust side.
                        unhandled_key = k

            if unhandled_key is not None:
                if token is not None:
                    msg = (
                        "unsupported: cannot combine token with "
                        f"{unhandled_key} in storage_options"
                    )
                    raise ValueError(msg)

                return None

            if token is not None:
                return CredentialProviderBuilder(
                    InitializedCredentialProvider(UserProvidedGCPToken(token))
                )

            if (default := get_default_credential_provider()) is not None:
                return default

            return CredentialProviderBuilder(AutoInit(CredentialProviderGCP))

        return None

    credential_provider_init = f()

    if verbose():

View on GitHub (pinned to df599052da)

Solutions

1. Remove either the token or the other credential key so exactly one auth source remains
2. For service-account/file-based auth, drop 'token' and let the native path handle the remaining keys
3. Regenerate the token per run instead of mixing it into a static config dict

Example fix

# before
opts = {"token": TOKEN, "service_account": "/secrets/sa.json"}
lf = pl.scan_parquet("gs://bucket/f.parquet", storage_options=opts)
# after
lf = pl.scan_parquet("gs://bucket/f.parquet", storage_options={"service_account": "/secrets/sa.json"})

def validate_gcs_storage_options(opts: dict) -> None:
    has_token = any(k.lower() in {"token", "bearer_token"} for k in opts)
    others = [k for k in opts if k.lower() not in {"token", "bearer_token"}]
    if has_token and others:
        raise ValueError(f"cannot combine token with {others}; keep exactly one auth source")

try:
    lf = pl.scan_parquet(path, storage_options=opts)
except ValueError as e:
    if "cannot combine token" in str(e):
        opts = {k: v for k, v in opts.items() if k.lower() not in {"token", "bearer_token"}}
        lf = pl.scan_parquet(path, storage_options=opts)
    else:
        raise

When it happens

Trigger: pl.scan_parquet('gs://bucket/f.parquet', storage_options={'token': 'ya29....', 'service_account': '/secrets/sa.json'}) - token plus any non-ignored key raises, with the offending key name in the message.

Common situations: Short-lived access tokens injected by a scheduler combined with a base config carrying service-account settings; merging a shared GCS options dict with a per-run token.

Related errors

• unsupported: cannot combine aws_profile with {unhandled_key}
• the provided `credential` object {credential!r} does not hav
• google-auth must be installed to use `CredentialProviderGCP`
• index positions should be smaller than 2^32
• index positions should be greater than or equal to -2^32

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/b5aa70beeeba9589. Report an issue: GitHub.