pola-rs/polars · error · ValueError

you need to pass delta_merge_options with at least a given p

Error message

you need to pass delta_merge_options with at least a given predicate for `MERGE` to work.

What it means

sink_delta(mode='merge') performs a Delta Lake MERGE (upsert), which requires merge semantics — at minimum a predicate — supplied via delta_merge_options. Without them polars raises ValueError because it cannot guess how to match source rows to target rows.

Source

Thrown at py-polars/src/polars/lazyframe/frame.py:3340

        # We aren't calling into polars-native write functions so we just update
        # the storage_options here.
        storage_options = (
            {**(storage_options or {}), **credential_provider_creds}
            if storage_options is not None or credential_provider_builder is not None
            else None
        )
        stream = self.collect_batches(
            engine=engine,
            maintain_order=False,
            chunk_size=None,
            lazy=True,
            optimizations=optimizations,
        )

        if mode == "merge":
            if delta_merge_options is None:
                msg = "you need to pass delta_merge_options with at least a given predicate for `MERGE` to work."
                raise ValueError(msg)
            if isinstance(target, str):
                dt = DeltaTable(table_uri=target, storage_options=storage_options)
            else:
                dt = target

            return dt.merge(stream, **delta_merge_options)  # type: ignore[arg-type]

        else:
            if delta_write_options is None:
                delta_write_options = {}

            write_deltalake(  # pyrefly: ignore[no-matching-overload]
                table_or_uri=target,
                data=stream,  # type: ignore[call-overload]
                mode=mode,
                storage_options=storage_options,
                **delta_write_options,
            )

View on GitHub (pinned to df599052da)

Solutions

1. Pass delta_merge_options with predicate, e.g. {'predicate': 'target.id = source.id', 'source_alias': 'source', 'target_alias': 'target'} plus insert/update/delete actions as needed
2. If you only want to replace data, use mode='overwrite' with delta_write_options instead
3. Verify the predicate references the correct source/target aliases

Example fix

# before
lf.sink_delta('delta:/tbl', mode='merge')

# after
lf.sink_delta(
    'delta:/tbl', mode='merge',
    delta_merge_options={
        'predicate': 'target.id = source.id',
        'source_alias': 'source',
        'target_alias': 'target',
        'when_matched_update_all': True,
    },
)

if mode == 'merge' and not delta_merge_options:
    raise ValueError('mode=merge requires delta_merge_options with a predicate')
lf.sink_delta(target, mode=mode, delta_merge_options=delta_merge_options)

def is_valid_merge_options(opts: dict | None) -> bool:
    return bool(opts) and 'predicate' in opts

When it happens

Trigger: lf.sink_delta(target, mode='merge') with no delta_merge_options; passing delta_merge_options=None; confusing mode='merge' with mode='overwrite'/'append' semantics.

Common situations: Building incremental upsert pipelines into Delta tables; switching a write job from append to merge without adding merge predicates.

Related errors

• cannot use credential_provider when passing a DeltaTable obj
• cannot {operation}: no data_source_format found
• cannot select columns using key of type {qualified_type_name
• expected {df.width} values when selecting columns by boolean
• index {key} is out of bounds for DataFrame of height {num_ro

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/e41505354b915900. Report an issue: GitHub.