pola-rs/polars · error · ImportError

boto3 must be installed to use `CredentialProviderAWS`

Error message

boto3 must be installed to use `CredentialProviderAWS`

What it means

CredentialProviderAWS._ensure_module_availability (py-polars/src/polars/io/cloud/credential_provider/_providers.py:305-309) checks importlib.util.find_spec('boto3') and raises ImportError when the Python-side AWS credential provider is required but boto3 is absent. boto3 is optional: polars' native Rust cloud stack works without it, but aws_profile resolution and assume_role support route through Python/boto3.

Source

Thrown at py-polars/src/polars/io/cloud/credential_provider/_providers.py:310

            return False

        return True

    def _session(self) -> boto3.Session:
        # Note: boto3 automatically sources the AWS_PROFILE env var
        import boto3

        return boto3.Session(
            profile_name=self.profile_name,
            region_name=self.region_name,
        )

    @classmethod
    def _ensure_module_availability(cls) -> None:
        if importlib.util.find_spec("boto3") is None:
            msg = "boto3 must be installed to use `CredentialProviderAWS`"
            raise ImportError(msg)

    class EmptyCredentialError(Exception):
        """
        Raised when boto3 returns empty credentials.

        This generally indicates that no credentials could be found in the
        environment.
        """


class CredentialProviderAzure(CachingCredentialProvider):
    """
    Azure Credential Provider.

    Using this requires the `azure-identity` Python package to be installed.

    .. warning::
        This functionality is considered **unstable**. It may be changed

View on GitHub (pinned to df599052da)

Solutions

1. pip install boto3 in any environment that uses aws_profile or CredentialProviderAWS
2. Rely on the native credential path instead: standard AWS env vars (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY) or instance roles, with no aws_profile key
3. Keep static key-based storage_options without a profile, which dispatch to the Rust side directly

Example fix

# before - ImportError: boto3 must be installed
lf = pl.scan_parquet("s3://bucket/f.parquet", storage_options={"aws_profile": "prod"})
# after
# $ pip install boto3
lf = pl.scan_parquet("s3://bucket/f.parquet", storage_options={"aws_profile": "prod"})

import importlib.util

def require_boto3() -> None:
    if importlib.util.find_spec("boto3") is None:
        raise ImportError("pip install boto3 to use aws_profile / CredentialProviderAWS")

try:
    lf = pl.scan_parquet(url, storage_options=opts)
except ImportError as e:
    if "boto3" in str(e):
        raise SystemExit("this job uses aws_profile: pip install boto3") from e
    raise

When it happens

Trigger: pl.scan_parquet('s3://...', storage_options={'aws_profile': 'x'}) or credential_provider=pl.CredentialProviderAWS(...) in an environment where boto3 is not installed.

Common situations: Minimal Docker/CI images; adding an aws_profile to storage_options in a project that previously relied on anonymous access or env-var credentials (which need no boto3).

Related errors

• azure-identity must be installed to use `CredentialProviderA
• google-auth must be installed to use `CredentialProviderGCP`
• `fsspec` is required for `storage_options` argument
• unsupported: cannot combine aws_profile with {unhandled_key}
• expiration time in STS response did not contain timezone inf

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/f4a0c0402fe0c2d7. Report an issue: GitHub.