{"record":{"id":"5ae464ca3166357f","repo":"pola-rs/polars","slug":"iceberg-sort-transform-transform","errorCode":null,"errorMessage":"Iceberg sort transform '{transform}'","messagePattern":"Iceberg sort transform '(.+?)'","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/iceberg/_sink.py","lineNumber":472,"sourceCode":"                expr = expr.to_physical() // transform.width\n            elif isinstance(source_type, StringType):\n                expr = expr.str.slice(0, transform.width)\n            elif isinstance(source_type, BinaryType):\n                expr = expr.bin.slice(0, transform.width)\n            else:\n                msg = f\"Iceberg sort transform '{transform}' on '{source_type}'\"\n                raise NotImplementedError(msg)\n        elif isinstance(transform, BucketTransform):\n            expr = expr.map_batches(\n                _SortTransform(transform, source_type),\n                return_dtype=pl.Int32,\n                is_elementwise=True,\n            )\n        elif isinstance(transform, VoidTransform):\n            continue\n        else:\n            msg = f\"Iceberg sort transform '{transform}'\"\n            raise NotImplementedError(msg)\n\n        exprs.append(expr)\n        descending.append(field.direction is SortDirection.DESC)\n        nulls_last.append(field.null_order is NullOrder.NULLS_LAST)\n\n    return exprs, descending, nulls_last\n\n\n@dataclass(kw_only=True)\nclass IcebergSinkState:\n    py_catalog_class_module: str\n    py_catalog_class_qualname: str\n\n    catalog_name: str\n    catalog_properties: dict[str, str]\n\n    table_name: str\n    mode: Literal[\"append\", \"overwrite\"]","sourceCodeStart":454,"sourceCodeEnd":490,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/io/iceberg/_sink.py#L454-L490","documentation":"The final branch of _sort_key_exprs raises when a sort order field uses a transform that Polars' Iceberg sink does not implement. Supported transforms are Identity, Year/Month/Day/Hour, Truncate, Bucket, and Void; anything else (e.g. a custom or newer Iceberg transform) triggers this NotImplementedError when sort_order is supplied to sink_iceberg.","triggerScenarios":"sink_iceberg with sort_order containing a field whose transform is not one of the recognized pyiceberg transform types — typically a transform type added in newer pyiceberg versions but not yet handled here.","commonSituations":"Tables whose sort order uses transform types introduced after polars' sink was written; version skew between pyiceberg (new transforms) and polars (older handler); programmatically constructed SortOrder with unsupported transforms.","solutions":["Remove or replace the unsupported transform in the sort order with a supported one (identity, temporal, truncate, bucket, or void).","Sink without a sort_order argument and sort the data before/at write time with polars .sort(), then apply a sort-order metadata-only update afterwards.","Align pyiceberg and polars versions so no transform types outside the sink's supported set appear in the table metadata.","Add the missing transform handling to polars or file an upstream issue."],"exampleFix":"// before\nSortField(source_id=col, transform=UnknownNewTransform(), direction=ASC)\n// after\nSortField(source_id=col, transform=IdentityTransform(), direction=ASC)","handlingStrategy":"validation","validationCode":"from pyiceberg.transforms import (BucketTransform, DayTransform, HourTransform,\n    IdentityTransform, MonthTransform, TruncateTransform, VoidTransform, YearTransform)\nSUPPORTED = (IdentityTransform, YearTransform, MonthTransform, DayTransform,\n             HourTransform, TruncateTransform, BucketTransform, VoidTransform)\nif sort_order is not None:\n    for f in sort_order.fields:\n        if not isinstance(f.transform, SUPPORTED):\n            raise ValueError(f\"sort transform {f.transform} unsupported for polars sink\")","typeGuard":"def sinkable_sort_transform(transform) -> bool:\n    from pyiceberg.transforms import (BucketTransform, DayTransform, HourTransform,\n        IdentityTransform, MonthTransform, TruncateTransform, VoidTransform, YearTransform)\n    return isinstance(transform, (IdentityTransform, YearTransform, MonthTransform,\n        DayTransform, HourTransform, TruncateTransform, BucketTransform, VoidTransform))","tryCatchPattern":"try:\n    lf.sink_iceberg(target, sort_order=order)\nexcept NotImplementedError as e:\n    if \"sort transform\" in str(e):\n        lf.sink_iceberg(target.sort(lf.collect_schema().names()))  # pre-sorted, no sort_order\n    else:\n        raise","preventionTips":["Keep pyiceberg and polars versions aligned; new pyiceberg transforms may not be handled by the sink","Inspect sort_order.fields' transforms before sinking","Apply unsupported sort transforms as pre-sorting with polars .sort() instead of sort_order metadata"],"tags":["iceberg","sort-order","not-implemented","transform"],"backgroundTag":"unsupported-operation","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}