{"record":{"id":"eb7ff3d763b39c5b","repo":"pola-rs/polars","slug":"schema-mode-overwrite-is-not-supported-for-parti","errorCode":null,"errorMessage":"schema_mode='overwrite' is not supported for partitioned Iceberg tables","messagePattern":"schema_mode='overwrite' is not supported for partitioned Iceberg tables","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/iceberg/_sink.py","lineNumber":499,"sourceCode":"        evolved_schema = transaction.table_metadata.schema()\n        source_schema = pyarrow_to_schema(\n            self._get_source_schema(), name_mapping=evolved_schema.name_mapping\n        )\n        return schema_to_pyarrow(source_schema)\n\n    def _attach_resolved_sink(self, plf: PyLazyFrame) -> PyLazyFrame:\n        from pyiceberg.table import TableProperties\n        from pyiceberg.utils.properties import property_as_bool, property_as_int\n\n        import polars as pl\n\n        table = self.table()\n        table_metadata = table.metadata\n        table_properties = table_metadata.properties\n\n        if self.schema_mode == \"overwrite\" and table.spec().fields:\n            msg = \"schema_mode='overwrite' is not supported for partitioned Iceberg tables\"\n            raise NotImplementedError(msg)\n\n        partition_key_exprs = _partition_key_exprs(table, self.source_schema)\n\n        if table.sort_order().fields:\n            msg = \"sink to Iceberg table with sort order\"\n            raise NotImplementedError(msg)\n\n        if location_provider_impl := table_properties.get(\n            TableProperties.WRITE_PY_LOCATION_PROVIDER_IMPL\n        ):\n            msg = (\n                \"sink to Iceberg table with custom location provider\"\n                f\" '{location_provider_impl}'\"\n            )\n            raise NotImplementedError(msg)\n\n        object_storage_enabled = property_as_bool(\n            table_properties,","sourceCodeStart":481,"sourceCodeEnd":517,"githubUrl":"https://github.com/pola-rs/polars/blob/fc24390824009e8fc5b7870e256273c1b140a019/py-polars/src/polars/io/iceberg/_sink.py#L481-L517","documentation":"`schema_mode='overwrite'` in `sink_iceberg` replaces the table's schema, which conflicts with an existing partition spec derived from the old schema's fields. Because partition fields depend on source column ids/types, polars refuses this combination with NotImplementedError rather than producing an inconsistent table.","triggerScenarios":"Calling `lf.sink_iceberg(table, schema_mode='overwrite')` where the target Iceberg table has a non-empty partition spec (`table.spec().fields` is non-empty).","commonSituations":"Evolving a partitioned table's schema by overwrite instead of append; copying a sink configuration written for an unpartitioned table and reusing it against a partitioned one; migrating tables where the spec wasn't dropped first.","solutions":["Use `schema_mode='append'` (the default) if the data conforms to the existing schema","Drop or recreate the table: create a new table with the desired schema and partition spec, then sink into it","If you truly need a new schema, create a fresh table with matching partition spec fields that reference the new schema, then sink with overwrite on the new (unpartitioned-target) workflow","Keep schema evolution within compatible types so append mode works instead of overwrite"],"exampleFix":"// before: overwrite against a partitioned table\nlf.sink_iceberg(table, schema_mode='overwrite')\n// after: append, or recreate the table with the new schema/spec first\nlf.sink_iceberg(table, schema_mode='append')  # or create new table, then sink","handlingStrategy":"validation","validationCode":"if table.spec().fields and schema_mode == 'overwrite':\n    raise ValueError(\"schema_mode='overwrite' cannot be used on a partitioned Iceberg table\")","typeGuard":null,"tryCatchPattern":"try:\n    lf.sink_iceberg(table, schema_mode=schema_mode)\nexcept NotImplementedError as e:\n    if \"schema_mode='overwrite'\" in str(e):\n        raise RuntimeError(\"use append mode, or recreate the table with the desired schema\") from e\n    raise","preventionTips":["Default to schema_mode='append' when sinking into existing partitioned tables","If schema changes are needed, create a new table with the new schema and partition spec instead of overwriting","Check `len(table.spec().fields)` before choosing overwrite"],"tags":["iceberg","schema-mode","partitioning","unsupported","polars"],"backgroundTag":"schema-mode-unsupported-for-partitioned-table","analyzedSha":"fc24390824009e8fc5b7870e256273c1b140a019","analyzedAt":"2026-09-02T20:08:03.494Z","contentChangedAt":"2026-09-02T20:08:03.494Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}