pola-rs/polars · error · ValueError

cannot use credential_provider when passing a DeltaTable obj

Error message

cannot use credential_provider when passing a DeltaTable object

What it means

Raised by scan_delta (and the related read path) when source is an existing DeltaTable object and credential_provider is also passed (anything other than the default 'auto'). A DeltaTable already carries its own authenticated handles; applying an extra credential provider on top of it is contradictory, so polars rejects it.

Source

Thrown at py-polars/src/polars/io/delta/functions.py:309

    from polars.io.cloud.credential_provider._builder import (
        _init_credential_provider_builder,
    )

    table: DeltaTable | None = None

    if importlib.util.find_spec("deltalake") is not None:
        from deltalake import DeltaTable

        if isinstance(source, DeltaTable):
            table = source

    if table is None:
        credential_provider_builder = _init_credential_provider_builder(
            credential_provider, source, storage_options, "scan_delta"
        )
    elif credential_provider is not None and credential_provider != "auto":
        msg = "cannot use credential_provider when passing a DeltaTable object"
        raise ValueError(msg)
    else:
        credential_provider_builder = None

    del credential_provider

    if table is not None and (
        table._storage_options is not None or storage_options is not None
    ):
        storage_options = {
            **(table._storage_options or {}),
            **(storage_options or {}),
        }

    dataset = DeltaDataset(
        table_=NoPickleOption(table),
        table_uri_=str(source) if table is None else None,
        version=version,
        storage_options=storage_options,

View on GitHub (pinned to df599052da)

Solutions

1. Drop the credential_provider argument when passing a DeltaTable — build the DeltaTable with its own storage_options/credentials instead
2. Or pass the table location string (not the object) and keep credential_provider, letting polars construct the table
3. In shared wrappers, only forward credential_provider when source is a str/path

Example fix

# before
pl.scan_delta(delta_table, credential_provider=credential_provider_func)

# after
pl.scan_delta(delta_table)
# or
pl.scan_delta("s3://bucket/table", credential_provider=credential_provider_func)

from deltalake import DeltaTable

if isinstance(source, DeltaTable):
    assert credential_provider in (None, "auto"), "drop credential_provider for DeltaTable input"

from deltalake import DeltaTable

def is_delta_table(source: object) -> TypeGuard[DeltaTable]:
    return isinstance(source, DeltaTable)

When it happens

Trigger: pl.scan_delta(delta_table_obj, credential_provider=my_provider); code that unconditionally threads a credential provider through every call, including ones given a pre-built DeltaTable.

Common situations: Cloud credential helpers (GCP/ADLS/AWS factories) wired into generic wrappers; refactoring where source changed from a path to a shared DeltaTable but the credential plumbing was left in place.

Related errors

• DataTypeGroup items must be dtypes; found {qualified_type_na
• cannot use credential_provider when passing a DeltaTable obj
• the given column-schema names do not match the data dictiona
• Pandas dataframe contains non-unique indices and/or column n
• Pandas indices and column names must not overlap.

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/560ea031f923272d. Report an issue: GitHub.