pola-rs/polars · error · ValueError

schema_mode='overwrite' requires mode='overwrite'

Error message

schema_mode='overwrite' requires mode='overwrite'

What it means

Raised by polars' Iceberg sink when sink_iceberg is configured with schema_mode='overwrite' while mode is left at its default 'append' (the guard runs first thing in IcebergSinkState.new, _sink.py:65-67). Overwriting a table's schema is only coherent when the write also replaces the table contents — the same coupling pyiceberg and Spark enforce between data mode and schema mode — so append + schema-overwrite is rejected as contradictory. The check is eager: it fires while the sink state is built, before any data is read or written.

Source

Thrown at py-polars/src/polars/io/iceberg/_sink.py:67

    sink_uuid_str: str

    table_: NoPickleOption[pyiceberg.table.Table]
    source_schema: pa.Schema | None
    commit_result_df: NoPickleOption[pl.DataFrame]

    @staticmethod
    def new(
        target: str | pyiceberg.table.Table,
        *,
        mode: Literal["append", "overwrite"] = "append",
        schema_mode: Literal["merge", "overwrite"] | None = None,
        snapshot_properties: dict[str, str] | None = None,
        catalog: pyiceberg.catalog.Catalog | IcebergCatalogConfig | None = None,
        storage_options: StorageOptionsDict | None = None,
    ) -> IcebergSinkState:
        if schema_mode == "overwrite" and mode != "overwrite":
            msg = "schema_mode='overwrite' requires mode='overwrite'"
            raise ValueError(msg)

        catalog_config = (
            (
                IcebergCatalogConfig._from_api_parameter_or_environment_default(
                    catalog,
                    fn_name="sink_iceberg",
                )
            )
            if isinstance(target, str)
            else (
                IcebergCatalogConfig(
                    class_=type(target.catalog),
                    name=target.catalog.name,
                    properties=target.catalog.properties,
                )
            )
        )

View on GitHub (pinned to 4db92c12c0)

Solutions

1. Add mode='overwrite' alongside schema_mode='overwrite' when you intend to replace both table contents and schema.
2. If the goal is appending rows while evolving the schema, use schema_mode='merge' with the default mode='append'.
3. If no schema handling is needed, omit schema_mode entirely.

Example fix

# before
lf.sink_iceberg("catalog.db.events", schema_mode="overwrite")

# after
lf.sink_iceberg("catalog.db.events", mode="overwrite", schema_mode="overwrite")

from typing import Literal

Mode = Literal["append", "overwrite"]
SchemaMode = Literal["merge", "overwrite"] | None

def validate_sink_iceberg_options(mode: Mode, schema_mode: SchemaMode) -> None:
    if schema_mode == "overwrite" and mode != "overwrite":
        msg = "schema_mode='overwrite' requires mode='overwrite'"
        raise ValueError(msg)

from typing import Literal, TypeGuard

Mode = Literal["append", "overwrite"]
SchemaMode = Literal["merge", "overwrite"] | None

def sink_options_consistent(mode: Mode, schema_mode: SchemaMode) -> TypeGuard[bool]:
    return schema_mode != "overwrite" or mode == "overwrite"

try:
    df.sink_iceberg("catalog.db.tbl", schema_mode="overwrite")
except ValueError as exc:
    if "schema_mode" in str(exc):
        # option pairing bug: fix mode/schema_mode and retry
        ...
    raise

When it happens

Trigger: df.sink_iceberg(target, schema_mode='overwrite') with mode omitted (defaults to 'append'); any call pairing schema_mode='overwrite' with mode='append'. Valid pairings: mode='overwrite' accepts schema_mode 'merge'/'overwrite'/None; mode='append' accepts only 'merge'/None.

Common situations: Porting Spark writeTo(...).option('mergeSchema'/'overwriteSchema', ...) or pyiceberg writer configs where schema mode is set independently of save mode; pipeline templates that always set schema_mode='overwrite' to force schema sync; assuming schema_mode is pure DDL that leaves append behavior untouched.

Related errors

• list.to_struct() got a str instead of a list. hint: pass ['{
• `{name}` was removed in version {version}
• {objname} object has no attribute {name!r}
• {func_name!r} received both {param.name!r} and {param.new_na
• the argument {param.name!r} for {func_name!r}{was_deprecated

AI-assisted analysis of pola-rs/polars@4db92c12c0 (2026-08-23). Data as JSON: /api/errors/e36767d5116d23c9. Report an issue: GitHub.