pathwaycom/pathway · error · ValueError

pw.Schema has column names that differ only in case ({case_c

Error message

pw.Schema has column names that differ only in case ({case_collisions}). DuckDB treats identifiers case-insensitively, so CREATE TABLE would reject them as duplicates. Rename these columns in the Pathway table so every column name is unique case-insensitively.

What it means

Raised by pw.io.duckdb.write when the Pathway table's schema contains two or more column names that differ only in letter case (e.g. 'Name' and 'name'). DuckDB matches identifiers case-insensitively, so the generated CREATE TABLE would fail with a raw 'Column with name ... already exists' catalog error; the connector surfaces it up front with a clear message.

Source

Thrown at python/pathway/io/duckdb/__init__.py:297

            'detach_between_batches=True cannot be used with database=":memory:": '
            "an in-memory DuckDB database is dropped when its last connection "
            "closes, so all data would be lost after every batch. Use an on-disk "
            "database file instead."
        )

    value_fields = _format_output_value_fields(table)

    # DuckDB matches identifiers case-insensitively, so two schema columns whose
    # names differ only in case would make CREATE TABLE fail with a raw
    # "Column with name ... already exists" catalog error. Surface it here.
    case_groups: dict[str, list[str]] = {}
    for field in value_fields:
        case_groups.setdefault(field.name.lower(), []).append(field.name)
    case_collisions = [
        sorted(names) for names in case_groups.values() if len(names) > 1
    ]
    if case_collisions:
        raise ValueError(
            f"pw.Schema has column names that differ only in case "
            f"({case_collisions}). DuckDB treats identifiers case-insensitively, "
            "so CREATE TABLE would reject them as duplicates. Rename these columns "
            "in the Pathway table so every column name is unique case-insensitively."
        )

    if not is_snapshot_mode:
        # Stream-of-changes mode appends `time` / `diff` metadata columns to the
        # destination table. If the user's own schema already has a column with
        # one of those names, the generated CREATE TABLE would declare that column
        # twice. Catch it here with a clear message instead of an opaque driver
        # error at start-up.
        reserved_metadata_columns = {"time", "diff"}
        collisions = sorted(
            field.name
            for field in value_fields
            if field.name.lower() in reserved_metadata_columns
        )

View on GitHub (pinned to fa2f74a464)

Solutions

Rename the colliding columns in the Pathway table before writing: t.select(**{c.lower(): t[c] for c in t.column_names()}) or targeted t.rename(...).
Fix the source schema / headers so every column name is unique case-insensitively.
Drop one of the duplicate-cased columns with t.without() if it is redundant.

Example fix

# before
# schema has columns: Name, name
pw.io.duckdb.write(t, table_name="t", database="out.duckdb")

# after
t = t.rename(**{"Name": "full_name"})
pw.io.duckdb.write(t, table_name="t", database="out.duckdb")

Defensive patterns

Strategy: validation

Validate before calling

names = [c for c in table.column_names()]
lowered = {}
for n in names:
    lowered.setdefault(n.lower(), []).append(n)
collisions = {k: v for k, v in lowered.items() if len(v) > 1}
assert not collisions, f"case-insensitive duplicate columns: {collisions}"

Prevention

Normalize column casing (e.g. all lower_snake_case) right after ingest from CSV/JSON.
Add a unit test asserting len(set(n.lower() for n in table.column_names())) == len(table.column_names()) before writes.

When it happens

Trigger: pw.io.duckdb.write(t, table_name="t") where t's schema declares columns like userId and UserID, or where a rename/with_columns produced case-variant names.

Common situations: Ingesting CSV/JSON sources with inconsistent header casing; joining or unioning tables from different producers that each normalized casing differently; applying .rename() or selecting prefixed columns that collide case-insensitively.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/a1a9bf1310ee7122. Report an issue: GitHub.