pola-rs/polars · error · NotImplementedError

Iceberg sort transform

Error message

Iceberg sort transform '{transform}' on '{source_type}'

What it means

_sort_key_exprs supports TruncateTransform for sort keys only on integer, long, decimal, string, and binary source columns. When the table's sort order truncates a column of another type, Polars cannot produce a sort key expression and raises NotImplementedError during sink_iceberg. This only triggers when a sort_order is configured for the sink.

Solutions

1. Adjust the sort order to use truncate only on supported types, or drop the truncate transform and sort by identity on that column.
2. Cast the sort source column to a supported type and redefine the sort order accordingly.
3. Sink without a sort order (omit sort_order) and apply data sorting/clustering afterwards with your catalog's rewrite tools.
4. Implement the missing transform via a feature request or upstream patch in polars.

Example fix

// before
SortField(source_id=ts_col, transform=TruncateTransform(10), direction=ASC)
// after
SortField(source_id=ts_col, transform=IdentityTransform(), direction=ASC)

from pyiceberg.transforms import TruncateTransform
from pyiceberg.types import (IntegerType, LongType, DecimalType, StringType, BinaryType)
if sort_order is not None:
    sort_order.check_compatible(schema)
    for f in sort_order.fields:
        src = schema.find_field(f.source_id)
        if isinstance(f.transform, TruncateTransform) and not isinstance(
            src.field_type, (IntegerType, LongType, DecimalType, StringType, BinaryType)
        ):
            raise ValueError(f"truncate sort transform unsupported on {src.field_type}")

def truncate_sort_supported(field_type) -> bool:
    from pyiceberg.types import (IntegerType, LongType, DecimalType, StringType, BinaryType)
    return isinstance(field_type, (IntegerType, LongType, DecimalType, StringType, BinaryType))

try:
    lf.sink_iceberg(target, sort_order=order)
except NotImplementedError as e:
    if "sort transform" in str(e):
        lf.sink_iceberg(target)  # sink without sort order, cluster later
    else:
        raise

When it happens

Trigger: sink_iceberg with sort_order (via _attach_resolved_sink) where a sort field has TruncateTransform on a source type outside {Integer, Long, Decimal, String, Binary} — e.g. truncate-sort on a float/date column.

Common situations: Reusing a table's existing sort order written by another engine that allows more type/transform combos; hand-rolled SortOrder with truncate on a double column; catalog-created tables with unusual sort specs.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Iceberg sort transform
• sink to Iceberg table with
• schema_mode='overwrite' is not supported for Iceberg tables…
• sink to Iceberg table with partition field
• sink to Iceberg table with sort order

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/7b40813b764a829a. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/iceberg/_sink.py:461

                    )
                    .otherwise(bits)
                )
                expr = bits ^ pl.when(bits < 0).then((1 << 63) - 1).otherwise(0)
        elif isinstance(
            transform, (YearTransform, MonthTransform, DayTransform, HourTransform)
        ):
            expr = _temporal_transform_expr(expr, source_type, transform)
        elif isinstance(transform, TruncateTransform):
            if isinstance(source_type, (IntegerType, LongType, DecimalType)):
                # The quotient preserves ordering and ties without truncation overflow.
                expr = expr.to_physical() // transform.width
            elif isinstance(source_type, StringType):
                expr = expr.str.slice(0, transform.width)
            elif isinstance(source_type, BinaryType):
                expr = expr.bin.slice(0, transform.width)
            else:
                msg = f"Iceberg sort transform '{transform}' on '{source_type}'"
                raise NotImplementedError(msg)
        elif isinstance(transform, BucketTransform):
            expr = expr.map_batches(
                _SortTransform(transform, source_type),
                return_dtype=pl.Int32,
                is_elementwise=True,
            )
        elif isinstance(transform, VoidTransform):
            continue
        else:
            msg = f"Iceberg sort transform '{transform}'"
            raise NotImplementedError(msg)

        exprs.append(expr)
        descending.append(field.direction is SortDirection.DESC)
        nulls_last.append(field.null_order is NullOrder.NULLS_LAST)

    return exprs, descending, nulls_last

View on GitHub (pinned to fe841f959e)