pola-rs/polars · error · NotImplementedError

Iceberg sort transform

Error message

Iceberg sort transform '{transform}'

What it means

The final branch of _sort_key_exprs raises when a sort order field uses a transform that Polars' Iceberg sink does not implement. Supported transforms are Identity, Year/Month/Day/Hour, Truncate, Bucket, and Void; anything else (e.g. a custom or newer Iceberg transform) triggers this NotImplementedError when sort_order is supplied to sink_iceberg.

Solutions

1. Remove or replace the unsupported transform in the sort order with a supported one (identity, temporal, truncate, bucket, or void).
2. Sink without a sort_order argument and sort the data before/at write time with polars .sort(), then apply a sort-order metadata-only update afterwards.
3. Align pyiceberg and polars versions so no transform types outside the sink's supported set appear in the table metadata.
4. Add the missing transform handling to polars or file an upstream issue.

Example fix

// before
SortField(source_id=col, transform=UnknownNewTransform(), direction=ASC)
// after
SortField(source_id=col, transform=IdentityTransform(), direction=ASC)

from pyiceberg.transforms import (BucketTransform, DayTransform, HourTransform,
    IdentityTransform, MonthTransform, TruncateTransform, VoidTransform, YearTransform)
SUPPORTED = (IdentityTransform, YearTransform, MonthTransform, DayTransform,
             HourTransform, TruncateTransform, BucketTransform, VoidTransform)
if sort_order is not None:
    for f in sort_order.fields:
        if not isinstance(f.transform, SUPPORTED):
            raise ValueError(f"sort transform {f.transform} unsupported for polars sink")

def sinkable_sort_transform(transform) -> bool:
    from pyiceberg.transforms import (BucketTransform, DayTransform, HourTransform,
        IdentityTransform, MonthTransform, TruncateTransform, VoidTransform, YearTransform)
    return isinstance(transform, (IdentityTransform, YearTransform, MonthTransform,
        DayTransform, HourTransform, TruncateTransform, BucketTransform, VoidTransform))

try:
    lf.sink_iceberg(target, sort_order=order)
except NotImplementedError as e:
    if "sort transform" in str(e):
        lf.sink_iceberg(target.sort(lf.collect_schema().names()))  # pre-sorted, no sort_order
    else:
        raise

When it happens

Trigger: sink_iceberg with sort_order containing a field whose transform is not one of the recognized pyiceberg transform types — typically a transform type added in newer pyiceberg versions but not yet handled here.

Common situations: Tables whose sort order uses transform types introduced after polars' sink was written; version skew between pyiceberg (new transforms) and polars (older handler); programmatically constructed SortOrder with unsupported transforms.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Iceberg sort transform
• sink to Iceberg table with
• schema_mode='overwrite' is not supported for Iceberg tables…
• sink to Iceberg table with partition field
• sink to Iceberg table with sort order

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/5ae464ca3166357f. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/iceberg/_sink.py:472

                expr = expr.to_physical() // transform.width
            elif isinstance(source_type, StringType):
                expr = expr.str.slice(0, transform.width)
            elif isinstance(source_type, BinaryType):
                expr = expr.bin.slice(0, transform.width)
            else:
                msg = f"Iceberg sort transform '{transform}' on '{source_type}'"
                raise NotImplementedError(msg)
        elif isinstance(transform, BucketTransform):
            expr = expr.map_batches(
                _SortTransform(transform, source_type),
                return_dtype=pl.Int32,
                is_elementwise=True,
            )
        elif isinstance(transform, VoidTransform):
            continue
        else:
            msg = f"Iceberg sort transform '{transform}'"
            raise NotImplementedError(msg)

        exprs.append(expr)
        descending.append(field.direction is SortDirection.DESC)
        nulls_last.append(field.null_order is NullOrder.NULLS_LAST)

    return exprs, descending, nulls_last


@dataclass(kw_only=True)
class IcebergSinkState:
    py_catalog_class_module: str
    py_catalog_class_qualname: str

    catalog_name: str
    catalog_properties: dict[str, str]

    table_name: str
    mode: Literal["append", "overwrite"]

View on GitHub (pinned to fe841f959e)