pola-rs/polars · error · ImportError

google-auth must be installed to use `CredentialProviderGCP`

Error message

google-auth must be installed to use `CredentialProviderGCP`

What it means

Polars' `CredentialProviderGCP` requires the `google-auth` package to obtain GCP credentials. At construction time, `_ensure_module_availability` checks whether `google.auth` is importable and raises this ImportError immediately if it is not. This is a deliberate fail-fast so users learn the optional dependency is missing rather than getting an obscure failure later during I/O.

Solutions

Install the missing dependency: `pip install google-auth` (or `pip install 'polars[gcp]'` if the extra is available).
If credentials are supplied another way (service-account key JSON in storage_options, GOOGLE_APPLICATION_CREDENTIALS with google-auth installed), install google-auth anyway or use a different credential mechanism such as explicit `storage_options` tokens.
Verify the active interpreter/venv has the package: `python -c "import google.auth"`.
If GCP is not actually being used, remove the `CredentialProviderGCP` usage from the scan/read call.

Example fix

// before
provider = CredentialProviderGCP()
df = pl.read_csv("gs://bucket/file.csv", credential_provider=provider)
# after
# pip install google-auth
provider = CredentialProviderGCP()
df = pl.read_csv("gs://bucket/file.csv", credential_provider=provider)

Defensive patterns

Strategy: try-catch

Validate before calling

import importlib.util
if importlib.util.find_spec("google.auth") is None:
    raise SystemExit("Install google-auth: pip install google-auth")

Type guard

def has_google_auth() -> bool:
    return importlib.util.find_spec("google.auth") is not None

Try / catch

try:
    provider = CredentialProviderGCP()
except ImportError as e:
    if "google-auth" in str(e):
        provider = None  # fall back to storage_options/env credentials
    else:
        raise

Prevention

Pin optional cloud extras in requirements (e.g. polars[gcp] or google-auth) for any environment reading gs:// paths.
Run `python -c "import google.auth"` as a CI smoke test before I/O jobs.
Keep one canonical venv per project so installs don't land in the wrong interpreter.

When it happens

Trigger: Instantiating `pl.credential_providers.CredentialProviderGCP()` (or passing it to scan/read functions lazily when credentials are fetched) in an environment where the `google-auth` package is not installed.

Common situations: Running on machines/CI images without GCP libraries installed while reading GCS paths (gs://) with cloud auth enabled; using a slim polars install since polars does not bundle google-auth as a hard dependency; virtualenv not activated so the package installed elsewhere isn't visible.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/25fbe23314092eb4. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/cloud/credential_provider/_providers.py:588

        creds.refresh(google.auth.transport.requests.Request())  # type: ignore[no-untyped-call, unused-ignore]

        return {"bearer_token": creds.token}, (  # type: ignore[dict-item]
            int(
                (
                    expiry.replace(tzinfo=zoneinfo.ZoneInfo("UTC"))
                    if expiry.tzinfo is None
                    else expiry
                ).timestamp()
            )
            if (expiry := creds.expiry) is not None
            else None
        )

    @classmethod
    def _ensure_module_availability(cls) -> None:
        if importlib.util.find_spec("google.auth") is None:
            msg = "google-auth must be installed to use `CredentialProviderGCP`"
            raise ImportError(msg)


class UserProvidedGCPToken(CredentialProvider):
    """User-provided GCP token in storage_options."""

    def __init__(self, token: str) -> None:
        self.token = token

    def __call__(self) -> CredentialProviderFunctionReturn:
        return {"bearer_token": self.token}, None


def _get_credentials_from_provider_expiry_aware(
    credential_provider: CredentialProviderFunction,
) -> dict[str, str] | None:
    if (
        isinstance(credential_provider, CredentialProviderAWS)
        and not credential_provider._can_use_as_provider()

View on GitHub (pinned to fe841f959e)