{"record":{"id":"7b40813b764a829a","repo":"pola-rs/polars","slug":"iceberg-sort-transform-transform-on-source-type","errorCode":null,"errorMessage":"Iceberg sort transform '{transform}' on '{source_type}'","messagePattern":"Iceberg sort transform '(.+?)' on '(.+?)'","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/iceberg/_sink.py","lineNumber":461,"sourceCode":"                    )\n                    .otherwise(bits)\n                )\n                expr = bits ^ pl.when(bits < 0).then((1 << 63) - 1).otherwise(0)\n        elif isinstance(\n            transform, (YearTransform, MonthTransform, DayTransform, HourTransform)\n        ):\n            expr = _temporal_transform_expr(expr, source_type, transform)\n        elif isinstance(transform, TruncateTransform):\n            if isinstance(source_type, (IntegerType, LongType, DecimalType)):\n                # The quotient preserves ordering and ties without truncation overflow.\n                expr = expr.to_physical() // transform.width\n            elif isinstance(source_type, StringType):\n                expr = expr.str.slice(0, transform.width)\n            elif isinstance(source_type, BinaryType):\n                expr = expr.bin.slice(0, transform.width)\n            else:\n                msg = f\"Iceberg sort transform '{transform}' on '{source_type}'\"\n                raise NotImplementedError(msg)\n        elif isinstance(transform, BucketTransform):\n            expr = expr.map_batches(\n                _SortTransform(transform, source_type),\n                return_dtype=pl.Int32,\n                is_elementwise=True,\n            )\n        elif isinstance(transform, VoidTransform):\n            continue\n        else:\n            msg = f\"Iceberg sort transform '{transform}'\"\n            raise NotImplementedError(msg)\n\n        exprs.append(expr)\n        descending.append(field.direction is SortDirection.DESC)\n        nulls_last.append(field.null_order is NullOrder.NULLS_LAST)\n\n    return exprs, descending, nulls_last\n","sourceCodeStart":443,"sourceCodeEnd":479,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/io/iceberg/_sink.py#L443-L479","documentation":"_sort_key_exprs supports TruncateTransform for sort keys only on integer, long, decimal, string, and binary source columns. When the table's sort order truncates a column of another type, Polars cannot produce a sort key expression and raises NotImplementedError during sink_iceberg. This only triggers when a sort_order is configured for the sink.","triggerScenarios":"sink_iceberg with sort_order (via _attach_resolved_sink) where a sort field has TruncateTransform on a source type outside {Integer, Long, Decimal, String, Binary} — e.g. truncate-sort on a float/date column.","commonSituations":"Reusing a table's existing sort order written by another engine that allows more type/transform combos; hand-rolled SortOrder with truncate on a double column; catalog-created tables with unusual sort specs.","solutions":["Adjust the sort order to use truncate only on supported types, or drop the truncate transform and sort by identity on that column.","Cast the sort source column to a supported type and redefine the sort order accordingly.","Sink without a sort order (omit sort_order) and apply data sorting/clustering afterwards with your catalog's rewrite tools.","Implement the missing transform via a feature request or upstream patch in polars."],"exampleFix":"// before\nSortField(source_id=ts_col, transform=TruncateTransform(10), direction=ASC)\n// after\nSortField(source_id=ts_col, transform=IdentityTransform(), direction=ASC)","handlingStrategy":"validation","validationCode":"from pyiceberg.transforms import TruncateTransform\nfrom pyiceberg.types import (IntegerType, LongType, DecimalType, StringType, BinaryType)\nif sort_order is not None:\n    sort_order.check_compatible(schema)\n    for f in sort_order.fields:\n        src = schema.find_field(f.source_id)\n        if isinstance(f.transform, TruncateTransform) and not isinstance(\n            src.field_type, (IntegerType, LongType, DecimalType, StringType, BinaryType)\n        ):\n            raise ValueError(f\"truncate sort transform unsupported on {src.field_type}\")","typeGuard":"def truncate_sort_supported(field_type) -> bool:\n    from pyiceberg.types import (IntegerType, LongType, DecimalType, StringType, BinaryType)\n    return isinstance(field_type, (IntegerType, LongType, DecimalType, StringType, BinaryType))","tryCatchPattern":"try:\n    lf.sink_iceberg(target, sort_order=order)\nexcept NotImplementedError as e:\n    if \"sort transform\" in str(e):\n        lf.sink_iceberg(target)  # sink without sort order, cluster later\n    else:\n        raise","preventionTips":["Validate the table's default sort order before passing it to sink_iceberg","Use truncate sort transforms only on int/long/decimal/string/binary columns","Sort data explicitly with polars .sort() and sink without sort_order when unsure"],"tags":["iceberg","sort-order","not-implemented","transform"],"backgroundTag":"unsupported-operation","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}