pathwaycom/pathway · error · ValueError

Column(s) {collisions} collide with the 'time' and 'diff' me

Error message

Column(s) {collisions} collide with the 'time' and 'diff' metadata columns appended in stream_of_changes mode. Rename these columns in the Pathway table, or use output_table_type="snapshot".

What it means

Raised by pw.io.duckdb.write in stream_of_changes mode when the table's own schema already contains a column named 'time' or 'diff' (case-insensitive). The stream mode appends time/diff metadata columns to the destination table, so the generated CREATE TABLE would declare that column twice; the connector rejects it with a clear message instead of an opaque driver error at startup.

Source

Thrown at python/pathway/io/duckdb/__init__.py:317

            f"({case_collisions}). DuckDB treats identifiers case-insensitively, "
            "so CREATE TABLE would reject them as duplicates. Rename these columns "
            "in the Pathway table so every column name is unique case-insensitively."
        )

    if not is_snapshot_mode:
        # Stream-of-changes mode appends `time` / `diff` metadata columns to the
        # destination table. If the user's own schema already has a column with
        # one of those names, the generated CREATE TABLE would declare that column
        # twice. Catch it here with a clear message instead of an opaque driver
        # error at start-up.
        reserved_metadata_columns = {"time", "diff"}
        collisions = sorted(
            field.name
            for field in value_fields
            if field.name.lower() in reserved_metadata_columns
        )
        if collisions:
            raise ValueError(
                f"Column(s) {collisions} collide with the 'time' and 'diff' "
                "metadata columns appended in stream_of_changes mode. Rename "
                "these columns in the Pathway table, or use "
                'output_table_type="snapshot".'
            )

    key_field_names: list[str] | None = None
    if primary_key is not None:
        # Duplicate entries in `primary_key` produce a nonsensical SQL template
        # and reorder DELETE bindings using the duplicated index, so retractions
        # would silently match no rows. Reject here with a clear message.
        names_seen: set[str] = set()
        duplicates: list[str] = []
        for pkey in primary_key:
            if pkey.name in names_seen and pkey.name not in duplicates:
                duplicates.append(pkey.name)
            names_seen.add(pkey.name)
        if duplicates:

View on GitHub (pinned to fa2f74a464)

Solutions

Rename the offending columns before writing, e.g. t.rename(time="event_time", diff="delta").
Or use output_table_type="snapshot", which does not append time/diff metadata columns (requires primary_key).

Example fix

# before
# t has columns: time, value
pw.io.duckdb.write(t, table_name="t")

# after
t = t.rename(time="event_time")
pw.io.duckdb.write(t, table_name="t")

Defensive patterns

Strategy: validation

Validate before calling

if output_table_type != "snapshot":
    reserved = {"time", "diff"}
    collisions = [c for c in table.column_names() if c.lower() in reserved]
    assert not collisions, f"columns colliding with time/diff: {collisions}"

Prevention

Avoid naming source columns exactly 'time' or 'diff' when writing to connectors; prefer event_time/delta.
Rename defensively before writing when the source schema is external and unstable.

When it happens

Trigger: pw.io.duckdb.write(t, table_name="t") (default stream_of_changes mode) where t has a column named time, diff, TIME, Diff, etc.

Common situations: Writing event/log tables that naturally carry a 'time' timestamp column or a 'diff' delta column; migrating data whose schema was designed around pathway's own output format.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/db4ac456444a528e. Report an issue: GitHub.