{"record":{"id":"6d4db93f18ab2dfe","repo":"pola-rs/polars","slug":"unsupported-cannot-combine-aws-profile-with-unha","errorCode":null,"errorMessage":"unsupported: cannot combine aws_profile with {unhandled_key} in storage_options","messagePattern":"unsupported: cannot combine aws_profile with (.+?) in storage_options","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/cloud/credential_provider/_builder.py","lineNumber":484,"sourceCode":"                        \"aws_endpoint\",\n                        \"aws_endpoint_url\",\n                        \"endpoint\",\n                        \"endpoint_url\",\n                    }:\n                        has_endpoint_url = True\n                    elif k in AUTOINIT_IGNORED_KEYS:\n                        continue\n                    else:\n                        # We assume this is some sort of access key\n                        unhandled_key = k\n\n            if unhandled_key is not None:\n                if profile is not None:\n                    msg = (\n                        \"unsupported: cannot combine aws_profile with \"\n                        f\"{unhandled_key} in storage_options\"\n                    )\n                    raise ValueError(msg)\n\n            if (\n                unhandled_key is None\n                and (default := get_default_credential_provider()) is not None\n            ):\n                return default\n\n            return CredentialProviderBuilder(\n                AutoInit(\n                    CredentialProviderAWS,\n                    profile_name=profile,\n                    region_name=region or default_region,\n                    _auto_init_unhandled_key=unhandled_key,\n                    _storage_options_has_endpoint_url=has_endpoint_url,\n                )\n            )\n\n        elif _is_gcp_cloud(scheme):","sourceCodeStart":466,"sourceCodeEnd":502,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/io/cloud/credential_provider/_builder.py#L466-L502","documentation":"When a cloud path is scanned and credentials resolve via the 'auto' builder, the AWS branch (py-polars/src/polars/io/cloud/credential_provider/_builder.py:447-484) inspects storage_options: aws_region/region, aws_default_region/default_region, aws_profile/profile, and endpoint keys are recognized, an internal allow-list is ignored, and every OTHER key is treated as a raw access-key credential. Supplying a profile together with any such unrecognized key is contradictory (profile-based vs key-based auth) and raises ValueError naming the conflicting key.","triggerScenarios":"pl.scan_parquet('s3://bucket/f.parquet', storage_options={'aws_profile': 'prod', 'aws_access_key_id': 'AKIA...'}) - profile plus any non-recognized key raises; region/endpoint keys alongside a profile are fine.","commonSituations":"Templates that merge a shared credentials dict with a per-environment profile; copy-pasted storage_options from other tools; CI that injects key-based credentials while the config also sets a profile.","solutions":["Pick one auth mode: keep aws_profile (plus optional region/endpoint) and delete access-key entries, or drop the profile and keep the keys","Move credentials to the standard chain (env vars, ~/.aws/credentials, instance role) and pass no credential keys at all","For explicit control pass credential_provider=pl.CredentialProviderAWS(...) or a custom callable"],"exampleFix":"# before\nlf = pl.scan_parquet(\n    \"s3://bucket/f.parquet\",\n    storage_options={\"aws_profile\": \"prod\", \"aws_access_key_id\": \"AKIA...\", \"aws_secret_access_key\": \"...\"},\n)\n# after\nlf = pl.scan_parquet(\n    \"s3://bucket/f.parquet\",\n    storage_options={\"aws_profile\": \"prod\", \"region\": \"eu-west-1\"},\n)","handlingStrategy":"validation","validationCode":"AWS_PROFILE_COMPATIBLE = {\n    \"aws_region\", \"region\", \"aws_default_region\", \"default_region\",\n    \"aws_profile\", \"profile\",\n    \"aws_endpoint\", \"aws_endpoint_url\", \"endpoint\", \"endpoint_url\",\n}\n\ndef validate_s3_storage_options(opts: dict) -> None:\n    has_profile = any(k.lower() in {\"aws_profile\", \"profile\"} for k in opts)\n    unknown = [k for k in opts if k.lower() not in AWS_PROFILE_COMPATIBLE]\n    if has_profile and unknown:\n        raise ValueError(f\"cannot combine aws_profile with {unknown}; pick one auth mode\")","typeGuard":null,"tryCatchPattern":"try:\n    lf = pl.scan_parquet(path, storage_options=opts)\nexcept ValueError as e:\n    if \"cannot combine aws_profile\" in str(e):\n        opts = {k: v for k, v in opts.items() if k.lower() not in {\"aws_profile\", \"profile\"}}\n        lf = pl.scan_parquet(path, storage_options=opts)\n    else:\n        raise","preventionTips":["Never merge profile-based and key-based dicts when building storage_options","Keep one canonical credentials strategy per environment (profile, keys, or env vars)","Log storage_options key NAMES only (never values) at startup to catch accidental merges"],"tags":["polars","aws","s3","credentials","storage-options","valueerror"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}