pola-rs/polars · error · NotImplementedError

sink to Iceberg table with '{transform}' partition transform

Error message

sink to Iceberg table with '{transform}' partition transform

What it means

This is the fallback branch of the same transform-dispatch in `_partition_key_exprs`: the partition field's transform is not one of any supported transform classes, so the sink cannot build a partition key expression at all. It raises NotImplementedError without even a matching source type, meaning the transform itself is entirely unsupported by the sink.

Source

Thrown at py-polars/src/polars/io/iceberg/_sink.py:330

                expr = expr.cast(pl.Date).cast(pl.Int32)
            else:
                expr = expr.dt.epoch("us") // 3_600_000_000
        elif isinstance(transform, TruncateTransform):
            if isinstance(source_type, (IntegerType, LongType)):
                expr = expr - expr % transform.width
            elif isinstance(source_type, StringType):
                expr = expr.str.slice(0, transform.width)
            elif isinstance(source_type, BinaryType):
                expr = expr.bin.slice(0, transform.width)
            else:
                msg = (
                    "sink to Iceberg table with "
                    f"'{transform}' partition transform on '{source_type}'"
                )
                raise NotImplementedError(msg)
        else:
            msg = f"sink to Iceberg table with '{transform}' partition transform"
            raise NotImplementedError(msg)

        key_name = f"__POLARS_ICEBERG_PARTITION_{field.field_id}"
        while key_name in reserved_names:
            key_name += "_"
        reserved_names.add(key_name)
        exprs.append(expr.alias(key_name))

    return exprs


@dataclass(kw_only=True)
class IcebergSinkState:
    py_catalog_class_module: str
    py_catalog_class_qualname: str

    catalog_name: str
    catalog_properties: dict[str, str]

View on GitHub (pinned to fc24390824)

Solutions

Check which transform the field uses (`[f.transform for f in table.spec().fields]`) and compare against polars' supported set (identity, bucket, truncate, temporal)
Rebuild the table's partition spec using only supported transforms (identity/bucket/truncate/year/month/day/hour)
Upgrade polars (and pyiceberg) — support for additional transforms is added over time
Materialize the transform yourself into a top-level column and partition with identity transform on it

Example fix

// before: partition spec uses an unsupported transform
# partition: void('status')
// after: rewrite spec with a supported transform
# partition: identity('status')
lf.sink_iceberg(table)

Defensive patterns

Strategy: validation

Validate before calling

SUPPORTED = {'identity', 'bucket', 'truncate', 'year', 'month', 'day', 'hour'}
for field in table.spec().fields:
    if str(field.transform).split('[')[0].strip() not in SUPPORTED:
        raise ValueError(f"unsupported partition transform: {field.transform}")

Try / catch

try:
    lf.sink_iceberg(table)
except NotImplementedError as e:
    if str(e) == "sink to Iceberg table with '{}' partition transform".format(e.args[0]) or 'partition transform' in str(e):
        raise RuntimeError("table uses a partition transform unsupported by polars; rebuild spec or upgrade") from e
    raise

Prevention

List all partition transforms before sinking and reject unknown ones
Keep polars and pyiceberg up to date to pick up newly supported transforms
Avoid tables with void/custom transforms for polars sink workflows

When it happens

Trigger: `sink_iceberg` on a table whose partition spec contains a transform the sink has no code path for — e.g. `void` transform, custom/future transform types from newer Iceberg specs, or unknown transform names in the table metadata.

Common situations: Tables created by newer Iceberg/Spark versions using transforms polars doesn't support yet; metadata with `void` partition transforms (columns dropped from partitioning); hand-written or migrated table metadata referencing unknown transforms.

Related errors

AI-assisted analysis of pola-rs/polars@fc24390824 (2026-09-02). Data as JSON: /api/errors/afcab0822e55bcfe. Report an issue: GitHub.