pathwaycom/pathway · error · ValueError

primary_key must be specified for the snapshot table type

Error message

primary_key must be specified for the snapshot table type

What it means

Raised by pw.io.duckdb.write when output_table_type="snapshot" is used without a primary_key. Snapshot mode maintains the current table state via keyed upserts/deletes, so it needs a key to identify rows; without one the connector cannot apply retractions correctly.

Source

Thrown at python/pathway/io/duckdb/__init__.py:256

    Afterwards the embeddings can be searched with plain DuckDB SQL:

    .. code-block:: sql

        SELECT text, list_cosine_similarity(embedding, [1.0, 0.0, 0.0]) AS score
        FROM documents
        WHERE diff = 1
        ORDER BY score DESC
        LIMIT 5;
    """
    _check_entitlements("duckdb")

    is_snapshot_mode = output_table_type == SNAPSHOT_OUTPUT_TABLE_TYPE
    if not is_snapshot_mode and primary_key is not None:
        raise ValueError(
            "primary_key can only be specified for the snapshot table type"
        )
    if is_snapshot_mode and not primary_key:
        raise ValueError("primary_key must be specified for the snapshot table type")
    if is_snapshot_mode and sort_by is not None:
        # In snapshot mode an update is a deletion (-1) followed by an insertion
        # (+1) of the same key. sort_by reorders the changes within a minibatch
        # and can place the insertion before the deletion, so the upsert is
        # immediately wiped by the delete — silently losing rows. sort_by has no
        # effect on a snapshot's final (unordered, keyed) state anyway, so reject
        # the combination instead of corrupting the output.
        raise ValueError(
            "sort_by cannot be used with the snapshot table type: a snapshot "
            "reflects the current state of the table regardless of the order "
            "changes are applied within a minibatch, and reordering would corrupt "
            "the upsert/delete sequence. Remove sort_by, or use "
            'output_table_type="stream_of_changes".'
        )

    database_str = fspath(database)
    _reject_directory_path(database_str)

View on GitHub (pinned to fa2f74a464)

Solutions

Supply a non-nullable column (or columns) as primary_key, e.g. pw.io.duckdb.write(t, table_name="t", output_table_type="snapshot", primary_key=t.id).
If no natural key exists, keep the default output_table_type="stream_of_changes" which needs no key.
Ensure the key column is not declared Optional and is not a list/array/tuple/JSON column (those raise separate errors).

Example fix

# before
pw.io.duckdb.write(t, table_name="t", output_table_type="snapshot")

# after
pw.io.duckdb.write(t, table_name="t", output_table_type="snapshot", primary_key=t.id)

Defensive patterns

Strategy: validation

Validate before calling

if output_table_type == "snapshot" and not primary_key:
    raise ValueError("snapshot output requires primary_key")

Prevention

Always define a natural non-nullable key column before adopting snapshot mode.
Guard conditional primary_key assignment: pk = pk if pk is not None else default_key.

When it happens

Trigger: pw.io.duckdb.write(table, table_name="t", output_table_type="snapshot") with no primary_key, or with primary_key=None (e.g. a conditional that evaluates to None).

Common situations: Switching a pipeline from stream_of_changes to snapshot and forgetting the key; primary_key passed conditionally (primary_key=pk if cond else None) where the None branch is hit.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/5ca87b0efca87e9b. Report an issue: GitHub.