{"record":{"id":"dd1d3ed07aaea306","repo":"pola-rs/polars","slug":"iceberg-source-field-source-id-has-non-struct-parent","errorCode":null,"errorMessage":"Iceberg source field {source_id} has non-struct parent","messagePattern":"Iceberg source field (.+?) has non-struct parent","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/iceberg/_sink.py","lineNumber":75,"sourceCode":"        if schema.accessor_for_field(field.source_id).inner is not None\n    }\n\n\ndef _iceberg_source_expr(schema: Schema, source_id: int) -> pl.Expr:\n    from pyiceberg.types import StructType\n\n    import polars as pl\n\n    accessor = schema.accessor_for_field(source_id)\n    source_field = schema.fields[accessor.position]\n    expr = pl.col(source_field.name)\n\n    while accessor.inner is not None:\n        accessor = accessor.inner\n        source_type = source_field.field_type\n        if not isinstance(source_type, StructType):\n            msg = f\"Iceberg source field {source_id} has non-struct parent\"\n            raise TypeError(msg)\n        source_field = source_type.fields[accessor.position]\n        expr = expr.struct.field(source_field.name)\n\n    return expr\n\n\ndef _infer_partition_from_statistics(\n    statistics: DataFileStatistics, spec: PartitionSpec, schema: Schema\n) -> Record:\n    from pyiceberg.partitioning import partition_record_value\n    from pyiceberg.typedef import Record\n\n    partition_values: list[Any] = []\n    for field in spec.fields:\n        aggregate = statistics.column_aggregates.get(field.source_id)\n        if aggregate is None:\n            partition_values.append(None)\n            continue","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/io/iceberg/_sink.py#L57-L93","documentation":"While building the source expression for an Iceberg partition/sort key, _iceberg_source_expr walks a nested accessor chain; every intermediate field must be a StructType to keep descending. A non-struct parent field was encountered mid-path, so polars raises TypeError.","triggerScenarios":"Defining an Iceberg sink with partition_by or sort keys referencing nested columns (e.g. 'a.b.c') where an intermediate field is a list, map, or primitive instead of a struct.","commonSituations":"Typos in nested key paths, schema evolution turning a struct field into a list/primitive, or partition specs copied from a table with a different schema shape.","solutions":["Fix the nested key path so every parent along it is a struct: verify with the table schema before writing.","Flatten needed values into top-level columns in the DataFrame and partition on those instead.","If the schema changed, update the sink's partition/sort specification to match the new types."],"exampleFix":"// before\npartition_by=[\"address.city\"]  # 'address' is a list, not a struct\n\n// after\ndf = df.with_columns(pl.col(\"address\").struct.field(\"city\").alias(\"city\"))\n# partition_by=[\"city\"]","handlingStrategy":"validation","validationCode":"schema = tbl.schema()\nfor key in partition_by:\n    f = schema\n    for part in key.split('.'):\n        f = f[part]\n        if not isinstance(f.field_type, StructType) and part != key.split('.')[-1]:\n            raise ValueError(f\"{part} is not a struct; fix nested key '{key}'\")","typeGuard":"def is_struct(field) -> bool:\n    return isinstance(field.field_type, StructType)","tryCatchPattern":"try:\n    df.sink_iceberg(target, partition_by=keys)\nexcept TypeError as e:\n    if \"non-struct parent\" in str(e):\n        df.sink_iceberg(target, partition_by=[keys[0].split('.')[0]])\n    else:\n        raise","preventionTips":["Validate nested partition/sort key paths against the Iceberg schema before sinking.","Flatten nested values you partition on into top-level columns.","Re-check partition specs after any schema evolution."],"tags":["iceberg","nested-schema","type-mismatch","sink"],"backgroundTag":"type-mismatch","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}