pathwaycom/pathway · error · ValueError

pw.Schema has column names that differ only in case ({case_c

Error message

pw.Schema has column names that differ only in case ({case_collisions}). SQLite treats identifiers case-insensitively, so CREATE TABLE would reject them as duplicates. Rename these columns in the Pathway table so every column name is unique case-insensitively.

What it means

SQLite compares identifiers case-insensitively (ASCII), so columns named e.g. `ID` and `id` in a Pathway schema would collapse into one destination column and make the generated CREATE TABLE fail with a raw 'duplicate column name' driver error. Pathway's sqlite write() pre-validates the schema at call time and raises this clearer ValueError listing the colliding case-groups, e.g. [['ID', 'id']].

Source

Thrown at python/pathway/io/sqlite/__init__.py:324

    _reject_directory_path(path_str)

    value_fields = _format_output_value_fields(table)

    # SQLite identifier matching is case-insensitive (`ID` and `id` are
    # the same column), so any pair of schema columns whose names
    # differ only in case would make ``CREATE TABLE`` fail with a raw
    # ``duplicate column name`` driver error at pipeline-start. Surface
    # the collision here with a clear, Pathway-authored message
    # instead, matching how the ``time`` / ``diff`` reserved-name check
    # below works.
    case_groups: dict[str, list[str]] = {}
    for field in value_fields:
        case_groups.setdefault(field.name.lower(), []).append(field.name)
    case_collisions = [
        sorted(names) for names in case_groups.values() if len(names) > 1
    ]
    if case_collisions:
        raise ValueError(
            f"pw.Schema has column names that differ only in case "
            f"({case_collisions}). SQLite treats identifiers "
            "case-insensitively, so CREATE TABLE would reject them as "
            "duplicates. Rename these columns in the Pathway table so "
            "every column name is unique case-insensitively."
        )

    if not is_snapshot_mode:
        # Stream-of-changes mode appends `time` / `diff` metadata columns
        # to the destination table so the output can be replayed as a
        # change log. If the user's own schema already has a column with
        # one of those names, the generated CREATE TABLE would declare
        # that column twice and SQLite would reject it. Catch this at
        # write() time with a clear message instead of letting the user
        # hit an opaque "duplicate column name" error at start-up.
        # Matching is case-insensitive, consistent with SQLite's identifier
        # comparison rules.
        reserved_metadata_columns = {"time", "diff"}

View on GitHub (pinned to fa2f74a464)

Solutions

Rename the colliding columns in the Pathway table before writing, e.g. with table.select(**{...}) or table.rename_columns(), so every name is unique case-insensitively.
Drop one of the duplicate-cased columns with table.without() if it is a redundant duplicate.
Fix the upstream schema definition (pw.Schema class or input connector format) that produced the near-duplicate names.

Example fix

# before
t = pw.debug.table_from_markdown('''
ID | Id
1  | 2
''')
pw.io.sqlite.write(t, "db.sqlite", "t", init_mode="replace")

# after
t = t.select(pw.this.ID, id2=pw.this.Id)
pw.io.sqlite.write(t, "db.sqlite", "t", init_mode="replace")

Defensive patterns

Strategy: validation

Validate before calling

def check_case_unique_columns(table):
    groups = {}
    for name in table.schema.column_names():
        groups.setdefault(name.lower(), []).append(name)
    collisions = [v for v in groups.values() if len(v) > 1]
    if collisions:
        raise ValueError(f"case-insensitive column collisions: {collisions}")
    return True

Prevention

Standardize on one casing convention (e.g. snake_case) for all schema columns at ingest time.
When loading schemas from external files/APIs, normalize or assert case-unique column names before building the pipeline.

When it happens

Trigger: Calling pw.io.sqlite.write() on a table whose schema contains two or more columns whose names differ only in case (owner/OWNER, pet/Pet), with any init_mode that creates the table (or even before that, since the check is unconditional).

Common situations: Schema defined from external sources (CSV headers, JSON keys, REST payloads) where casing varies between fields; renaming one column by capitalizing it while the original still exists; cross-team schemas combining snake_case and CamelCase variants of the same field.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/be1900cf494202bc. Report an issue: GitHub.