{"record":{"id":"fc604705e8a30aa7","repo":"pola-rs/polars","slug":"partition-source-field-source-id-has-non-struct","errorCode":null,"errorMessage":"partition source field {source_id} has non-struct parent","messagePattern":"partition source field (.+?) has non-struct parent","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/iceberg/_sink.py","lineNumber":72,"sourceCode":"        if schema.accessor_for_field(field.source_id).inner is not None\n    }\n\n\ndef _partition_source_expr(schema: Schema, source_id: int) -> pl.Expr:\n    from pyiceberg.types import StructType\n\n    import polars as pl\n\n    accessor = schema.accessor_for_field(source_id)\n    source_field = schema.fields[accessor.position]\n    expr = pl.col(source_field.name)\n\n    while accessor.inner is not None:\n        accessor = accessor.inner\n        source_type = source_field.field_type\n        if not isinstance(source_type, StructType):\n            msg = f\"partition source field {source_id} has non-struct parent\"\n            raise TypeError(msg)\n        source_field = source_type.fields[accessor.position]\n        expr = expr.struct.field(source_field.name)\n\n    return expr\n\n\ndef _infer_partition_from_statistics(\n    statistics: DataFileStatistics, spec: PartitionSpec, schema: Schema\n) -> Record:\n    from pyiceberg.partitioning import partition_record_value\n    from pyiceberg.typedef import Record\n\n    partition_values: list[Any] = []\n    for field in spec.fields:\n        aggregate = statistics.column_aggregates.get(field.source_id)\n        if aggregate is None:\n            partition_values.append(None)\n            continue","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/pola-rs/polars/blob/fc24390824009e8fc5b7870e256273c1b140a019/py-polars/src/polars/io/iceberg/_sink.py#L54-L90","documentation":"Polars' Iceberg sink resolves the source expression for a partition field by walking nested accessors down from the top-level table column. This error means an intermediate level of that path is not an Iceberg StructType, so the sink cannot descend into it with `.struct.field(...)` to build the partition column expression. It is a schema-shape limitation of nested partitioning support, not a data error.","triggerScenarios":"Sinking a LazyFrame to an Iceberg table whose partition spec references a source_id whose parent accessor chain passes through a non-struct type (e.g. a partition defined on a field nested inside a list or map rather than a struct). Raised inside `_partition_source_expr` while iterating `accessor.inner`.","commonSituations":"Tables partitioned on fields nested in list/map columns (Iceberg allows this, polars does not); metadata drift where the table's partition spec targets a nested column of unexpected type; hand-edited or migrated Iceberg metadata where source_id/position mapping no longer lines up with a struct.","solutions":["Recreate or alter the table's partition spec so partition fields sit at the top level or inside plain struct columns only","Flatten the struct column in the DataFrame (`.unnest(...)` or select the nested field) and sink to a table partitioned on the flattened top-level column","Check `table.spec().fields` and `schema.accessor_for_field(source_id)` with pyiceberg to confirm what the partition path actually traverses before sinking","Upgrade polars; nested partition support may have expanded in newer versions"],"exampleFix":"// before: partition spec on a field nested inside a list column\n# partition field source_id -> ListType element\n// after: repartition the table on a top-level column\n# iceberg partition spec: identity transform on top-level column 'category'\nlf.sink_iceberg(table)","handlingStrategy":"validation","validationCode":"from pyiceberg.types import StructType\nschema = table.metadata.current_schema\nfor field in table.spec().fields:\n    accessor = schema.accessor_for_field(field.source_id)\n    node, ft = accessor, schema.fields[accessor.position]\n    while node.inner is not None:\n        node = node.inner\n        if not isinstance(ft.field_type, StructType):\n            raise ValueError(f\"partition field {field.source_id} has non-struct parent\")\n        ft = ft.field_type.fields[node.position]","typeGuard":"def is_struct_parent_path(schema, source_id) -> bool:\n    from pyiceberg.types import StructType\n    accessor = schema.accessor_for_field(source_id)\n    f = schema.fields[accessor.position]\n    while accessor.inner is not None:\n        accessor = accessor.inner\n        if not isinstance(f.field_type, StructType):\n            return False\n        f = f.field_type.fields[accessor.position]\n    return True","tryCatchPattern":null,"preventionTips":["Keep partition fields on top-level columns or inside plain structs; avoid list/map nested partitioning for polars sinks","Inspect table.spec().fields and the schema accessor chain before sinking","Pin table metadata creation to engines/specs known to work with polars' sink"],"tags":["iceberg","partitioning","nested-schema","polars"],"backgroundTag":"unsupported-nested-partition-schema","analyzedSha":"fc24390824009e8fc5b7870e256273c1b140a019","analyzedAt":"2026-09-02T20:08:03.494Z","contentChangedAt":"2026-09-02T20:08:03.494Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}