pola-rs/polars · error · TypeError

dataframe contains unsupported data types: {overlap!r}

Error message

dataframe contains unsupported data types: {overlap!r}

What it means

Raised by _check_for_unsupported_types when writing a DataFrame whose schema (including nested fields, via unpack_dtypes) contains Time or Null dtypes, which the Delta format cannot represent. The check runs before any I/O so the write fails fast with the offending set of types.

Source

Thrown at py-polars/src/polars/io/delta/_utils.py:107

    return dl_tbl


def _check_if_delta_available() -> None:
    if not _DELTALAKE_AVAILABLE:
        msg = "deltalake is not installed\n\nPlease run: pip install deltalake"
        raise ModuleNotFoundError(msg)


def _check_for_unsupported_types(dtypes: list[DataType]) -> None:
    schema_dtypes = unpack_dtypes(*dtypes)
    unsupported_types = {Time, Null}
    # Note that this overlap check does NOT work correctly for Categorical, so
    # if Categorical is added back to unsupported_types a different check will
    # need to be used.

    if overlap := schema_dtypes & unsupported_types:
        msg = f"dataframe contains unsupported data types: {overlap!r}"
        raise TypeError(msg)


def _extract_table_statistics_from_delta_add_actions(
    add_actions_df: DataFrame,
    *,
    filter_columns: list[str],
    schema: SchemaDict,
    verbose: bool,
) -> DataFrame | None:
    import polars as pl

    if "num_records" not in add_actions_df:
        if verbose:
            eprint(
                "scan_delta: statistics load failed: 'num_records' column not present"
            )

        return None

View on GitHub (pinned to df599052da)

Solutions

1. Cast Time columns to a supported representation: pl.Datetime (with a reference date), pl.String ('%H:%M:%S'), or pl.Int64 (microseconds since midnight)
2. Drop all-Null columns (df.drop(...)) or give them a concrete dtype, e.g. pl.lit(None, dtype=pl.String)
3. Re-run after fixing each dtype named in the error set

Example fix

# before
df = df.with_columns(pl.lit(None).alias("note"), pl.col("shift_start").cast(pl.Time))
df.write_delta("./tbl")

# after
df = df.with_columns(
    pl.lit(None, dtype=pl.String).alias("note"),
    shift_start_us=pl.col("shift_start").cast(pl.Time).dt.total_microseconds(),
).drop("shift_start")
df.write_delta("./tbl")

import polars as pl
from polars.datatypes import unpack_dtypes

bad = unpack_dtypes(*df.schema.dtypes()) & {pl.Time, pl.Null}
if bad:
    raise TypeError(f"fix these dtypes before write_delta: {bad}")

def is_delta_writable(df: pl.DataFrame) -> bool:
    from polars.datatypes import unpack_dtypes
    return not (unpack_dtypes(*df.schema.dtypes()) & {pl.Time, pl.Null})

When it happens

Trigger: pl.write_delta() with a frame containing a pl.Time column or a column typed pl.Null (e.g. created by pl.lit(None) without a dtype or select(pl.lit(None))); nested Time/Null inside structs/lists is also caught by unpack_dtypes.

Common situations: ETL jobs adding an all-NULL marker column; time-of-day data ingested from CSV/JDBC as pl.Time; schema-on-write validation catching leftover placeholder columns.

Related errors

• passing Expr objects to the DataFrame constructor is not sup
• can't convert {pyseries.dtype()} to Decimal
• cannot select elements using key of type {qualified_type_nam
• cannot select columns using Series of type {dtype}
• cannot select columns using NumPy array of type {key.dtype}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/68656507e37536e1. Report an issue: GitHub.