pola-rs/polars · error · ImportError

could not get Databricks token: databricks-sdk is not instal

Error message

could not get Databricks token: databricks-sdk is not installed

What it means

ImportError from Catalog._get_databricks_token (py-polars/src/polars/catalog/unity/client.py:699-709). When bearer_token='databricks-sdk' (or the 'auto' path resolves to SDK-based auth inside a Databricks runtime), polars must read the workspace credentials via the databricks-sdk package; importlib.util.find_spec('databricks.sdk') returning None means the SDK is not installed in the environment, and the token cannot be retrieved. polars does not depend on databricks-sdk, so it must be installed separately.

Source

Thrown at py-polars/src/polars/catalog/unity/client.py:704

            ), storage_options

        # This should generally not happen, but if using the temporary
        # credentials API fails for whatever reason, we fallback to our built-in
        # credential provider resolution.

        from polars.io.cloud.credential_provider._builder import (
            _init_credential_provider_builder,
        )

        return _init_credential_provider_builder(
            "auto", table_info.storage_location, storage_options, caller_name
        ), storage_options

    @classmethod
    def _get_databricks_token(cls) -> str:
        if importlib.util.find_spec("databricks.sdk") is None:
            msg = "could not get Databricks token: databricks-sdk is not installed"
            raise ImportError(msg)

        # We code like this to bypass linting
        m = importlib.import_module("databricks.sdk.core").__dict__

        return m["DefaultCredentials"]()(m["Config"]())()["Authorization"][7:]


class CatalogCredentialProvider:
    """Retrieves credentials from the Unity catalog temporary credentials API."""

    catalog: Catalog
    table_id: str
    write: bool

    def __init__(self, catalog: Catalog, table_id: str, *, write: bool) -> None:
        self.catalog = catalog
        self.table_id = table_id
        self.write = write

View on GitHub (pinned to df599052da)

Solutions

1. pip install databricks-sdk (add to requirements/devcontainer)
2. Alternatively pass an explicit bearer_token string so no SDK lookup is needed
3. Verify visibility before the call: importlib.util.find_spec('databricks.sdk') is not None

Example fix

# before
Catalog(workspace_url=url, bearer_token='databricks-sdk')  # ImportError

# after
# shell: pip install databricks-sdk
Catalog(workspace_url=url, bearer_token='databricks-sdk')

import importlib.util

def databricks_sdk_available() -> bool:
    return importlib.util.find_spec('databricks.sdk') is not None

When it happens

Trigger: Catalog(workspace_url=..., bearer_token='databricks-sdk') without `pip install databricks-sdk`; CI images missing the optional dependency; 'auto' mode inside DATABRICKS_RUNTIME_VERSION environments that lack the SDK.

Common situations: New deployment environments where only polars was installed; slim Docker images; local dev mimicking Databricks auth without the SDK installed.

Related errors

• the graphviz `dot` binary should be on your PATH.(If not ins
• a non-HTTPS workspace_url was given ({workspace_url}). To al
• cannot apply delta_table_version for table of type {data_sou
• cannot apply delta_table_options for table of type {data_sou
• write_table: table format of {catalog_name}.{namespace}.{tab

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/302d155a65548a03. Report an issue: GitHub.