pathwaycom/pathway · error · ValueError

Pathway schema column(s) {offending} collide with the 'time'

Error message

Pathway schema column(s) {offending} collide with the 'time' and 'diff' metadata columns appended by `pw.io.postgres.write_snapshot`. Rename the column(s) in your schema or migrate to `pw.io.postgres.write` with output_table_type='snapshot' (no metadata columns appended).

What it means

Raised by the deprecated pw.io.postgres.write_snapshot when the Pathway schema contains a column named time or diff (case-insensitive). write_snapshot runs in legacy mode and appends its own time/diff metadata columns to every INSERT, so a user column with either name makes PostgreSQL reject the statement with 'column time specified more than once'. Pathway catches this before the run to avoid an opaque engine-worker db error.

Source

Thrown at python/pathway/io/postgres/__init__.py:1084

    # ``write_snapshot`` runs in ``legacy_mode=True`` which keeps the
    # ``time``/``diff`` metadata columns on the writer's INSERT — and
    # the generated SQL becomes ``INSERT INTO foo ("time", "value",
    # "time", "diff") VALUES (...)`` if the user schema also carries a
    # ``time`` or ``diff`` column, which PostgreSQL rejects with
    # ``column "time" specified more than once``. The non-legacy
    # sibling ``write`` runs the same check (and is already pinned by
    # ``test_psql_write_stream_mode_rejects_reserved_column_name``);
    # mirror it here so the deprecated path doesn't surface an opaque
    # engine-worker ``db error``.
    offending = sorted(
        {
            cname
            for cname in table.schema.column_names()
            if cname.lower() in ("time", "diff")
        }
    )
    if offending:
        raise ValueError(
            f"Pathway schema column(s) {offending} collide with "
            "the 'time' and 'diff' metadata columns appended by "
            "`pw.io.postgres.write_snapshot`. Rename the column(s) "
            "in your schema or migrate to `pw.io.postgres.write` "
            "with output_table_type='snapshot' (no metadata "
            "columns appended)."
        )

    postgres_settings = _augment_postgres_settings(postgres_settings, name)
    data_storage = api.DataStorage(
        storage_type="postgres",
        connection_string=_connection_string_from_settings(postgres_settings),
        max_batch_size=max_batch_size,
        snapshot_maintenance_on_output=True,
        table_name=table_name,
        schema_name="public",
        table_writer_init_mode=init_mode_from_str(init_mode),
        legacy_mode=True,

View on GitHub (pinned to fa2f74a464)

Solutions

Rename the offending column in the schema (e.g. time -> event_time, diff -> delta) and re-run.
Rename upstream of the writer with table operations so the written schema no longer collides.
Best: migrate to pw.io.postgres.write with output_table_type='snapshot', which appends no metadata columns and accepts time/diff names.

Example fix

# before
class EventSchema(pw.Schema):
    time: float
    value: int
pw.io.postgres.write_snapshot(t, conn, "events", primary_key=["time"])

# after
class EventSchema(pw.Schema):
    event_time: float
    value: int
pw.io.postgres.write(t, conn, "events", primary_key=[t.event_time], output_table_type="snapshot")

Defensive patterns

Strategy: validation

Validate before calling

reserved = {"time", "diff"}
colliding = {c for c in t.schema.column_names() if c.lower() in reserved}
assert not colliding, f"columns {colliding} collide with write_snapshot metadata"
pw.io.postgres.write_snapshot(t, conn, "t", primary_key=["id"])

Type guard

def snapshot_safe_schema(schema) -> bool:
    return not ({c.lower() for c in schema.column_names()} & {"time", "diff"})

Prevention

Avoid naming columns time or diff in schemas destined for legacy Postgres writes.
Prefer pw.io.postgres.write(output_table_type='snapshot'), which appends no metadata columns.
Add a startup assert when writing with the deprecated API.

When it happens

Trigger: pw.io.postgres.write_snapshot with a schema declaring 'class time: ...' or 'class diff: ...' (any casing, e.g. Time, DIFF), or reading a source whose columns include time/diff and writing it straight to Postgres.

Common situations: Event/telemetry schemas that naturally carry a time column; changelog-derived tables with a diff column; CSV/JSON ingestion that keeps upstream column names verbatim.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/986955ae2a4ec2fb. Report an issue: GitHub.