pola-rs/polars · error · ComputeError

cannot load requested aws_profile

Error message

cannot load requested aws_profile '{self.profile_name}': {type(e).__name__}: {e}

What it means

When the AWS credential provider cannot import boto3 and an explicit aws_profile was requested, polars wraps the ImportError in a ComputeError naming the profile, because the requested profile simply cannot be loaded without the AWS SDK.

Solutions

pip install boto3 (and botocore) in the environment.
Install polars with the relevant extra, e.g. pip install 'polars[aws]'.
If no profile is actually needed, drop aws_profile so the provider can fall back gracefully.

Example fix

// before
 lf = pl.scan_parquet("s3://b/f", credential_provider="aws_profile:prod")
// after
 # pip install boto3
 lf = pl.scan_parquet("s3://b/f", credential_provider="aws_profile:prod")

Defensive patterns

Strategy: try-catch

Validate before calling

import importlib.util
if use_aws_profile and importlib.util.find_spec('boto3') is None:
    raise RuntimeError('boto3 required for aws_profile credential provider')

Type guard

def boto3_available() -> bool:
    return importlib.util.find_spec('boto3') is not None

Try / catch

try:
    provider = pl.CredentialProviderAWS(profile_name='prod')
except polars.exceptions.ComputeError as e:
    if 'cannot load requested aws_profile' in str(e):
        provider = None  # fall back to env-var credentials
    else:
        raise

Prevention

Add boto3 to install requirements wherever aws_profile auth is configured.
Smoke-test profile loading at service startup, not on first query.
Keep dependency extras ('polars[aws]') in sync with configured credential providers.

When it happens

Trigger: Passing credential_provider='aws' (or storage_options with aws_profile='<name>') while boto3/botocore is not installed, so profile loading fails at import time.

Common situations: Deployments that installed polars without the AWS extra; docker images stripped of boto3; specifying a profile in config but forgetting the SDK dependency in the new environment.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/690d95be3e5afbbe. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/cloud/credential_provider/_providers.py:285

        if self._auto_init_unhandled_key:
            if verbose():
                eprint(
                    "[CredentialProviderAWS]: Will not be used as a provider: "
                    f"unhandled key in storage_options: '{self._auto_init_unhandled_key}'"
                )

            return False

        try:
            self()

        except ImportError as e:
            if self.profile_name:
                msg = (
                    "cannot load requested aws_profile "
                    f"'{self.profile_name}': {type(e).__name__}: {e}"
                )
                raise polars.exceptions.ComputeError(msg) from e

            return False

        except self.EmptyCredentialError:
            if verbose():
                eprint("[CredentialProviderAWS]: Did not find any credentials")

            return False

        return True

    def _session(self) -> boto3.Session:
        # Note: boto3 automatically sources the AWS_PROFILE env var
        import boto3

        return boto3.Session(
            profile_name=self.profile_name,
            region_name=self.region_name,

View on GitHub (pinned to fe841f959e)