pola-rs/polars · error · ImportError

could not get Databricks token: databricks-sdk is not…

Error message

could not get Databricks token: databricks-sdk is not installed

What it means

`_get_databricks_token` raises ImportError when it needs to fetch a Databricks access token (bearer_token set to "databricks-sdk" or "auto" inside a Databricks environment) but the `databricks-sdk` package is not installed. Polars intentionally does not bundle the SDK and delegates authentication to its DefaultCredentials chain.

Solutions

1. Install the SDK: `pip install databricks-sdk` (or add it to your dependencies)
2. Alternatively pass an explicit bearer_token string so the SDK lookup is skipped entirely
3. Verify the environment running Polars is the one where the SDK is installed (check `python -c "import databricks.sdk"`)

Example fix

// before (shell)
pip install polars  # missing databricks-sdk
client = CatalogClient(workspace_url=url, bearer_token="databricks-sdk")
// after
pip install "databricks-sdk"
client = CatalogClient(workspace_url=url, bearer_token="databricks-sdk")

import importlib.util
if bearer_token in {'databricks-sdk','auto'} and importlib.util.find_spec('databricks.sdk') is None:
    raise ImportError('pip install databricks-sdk')

def databricks_sdk_installed() -> bool:
    import importlib.util
    return importlib.util.find_spec('databricks.sdk') is not None

try:
    client = CatalogClient(workspace_url=url, bearer_token='databricks-sdk')
except ImportError as e:
    if 'databricks-sdk is not installed' in str(e):
        client = CatalogClient(workspace_url=url, bearer_token=os.environ['DATABRICKS_TOKEN'])
    else:
        raise

When it happens

Trigger: Creating the Unity Catalog client with bearer_token="databricks-sdk", or bearer_token="auto" while running inside Databricks (notebook/compute), in an environment where `databricks.sdk` cannot be imported.

Common situations: Installing polars in a slim Docker image or serverless environment without databricks-sdk; forgetting to add the dependency to requirements; runtime Databricks cluster whose SDK isn't visible to the current Python env (wrong interpreter/venv).

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• a non-HTTPS workspace_url was given
• cannot apply delta_table_options for table of type
• cannot apply delta_table_version for table of type
• google-auth must be installed to use `CredentialProviderGCP`
• 'numpy' is required to convert numpy dtype

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/302d155a65548a03. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/catalog/unity/client.py:700

            ), storage_options

        # This should generally not happen, but if using the temporary
        # credentials API fails for whatever reason, we fallback to our built-in
        # credential provider resolution.

        from polars.io.cloud.credential_provider._builder import (
            _init_credential_provider_builder,
        )

        return _init_credential_provider_builder(
            "auto", table_info.storage_location, storage_options, caller_name
        ), storage_options

    @classmethod
    def _get_databricks_token(cls) -> str:
        if importlib.util.find_spec("databricks.sdk") is None:
            msg = "could not get Databricks token: databricks-sdk is not installed"
            raise ImportError(msg)

        # We code like this to bypass linting
        m = importlib.import_module("databricks.sdk.core").__dict__

        return m["DefaultCredentials"]()(m["Config"]())()["Authorization"][7:]


class CatalogCredentialProvider:
    """Retrieves credentials from the Unity catalog temporary credentials API."""

    catalog: Catalog
    table_id: str
    write: bool

    def __init__(self, catalog: Catalog, table_id: str, *, write: bool) -> None:
        self.catalog = catalog
        self.table_id = table_id
        self.write = write

View on GitHub (pinned to fe841f959e)