{"record":{"id":"c9a11f754a7c3d16","repo":"pathwaycom/pathway","slug":"primary-key-contains-duplicate-column-s-duplicat","errorCode":null,"errorMessage":"primary_key contains duplicate column(s) {duplicates}","messagePattern":"primary_key contains duplicate column\\(s\\) (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/postgres/__init__.py","lineNumber":932,"sourceCode":"        # flush. Reject up-front with a clear message.\n        table_columns = set(table.schema.column_names())\n        foreign = sorted(n for n in key_field_names if n not in table_columns)\n        if foreign:\n            raise ValueError(\n                f\"primary_key references column(s) {foreign} that are \"\n                \"not present in the written table; pass ColumnReferences \"\n                \"from the table being written, e.g. primary_key=[table.k].\"\n            )\n        # A duplicate reference like `primary_key=[t.k, t.k]` would slip\n        # through to `SqlQueryTemplate` and either yield malformed SQL\n        # (``PRIMARY KEY (\"k\", \"k\")``) on CREATE TABLE or corrupt the\n        # ``primary_key_fields`` index-swap used on DELETE. Reject it\n        # here with a clear message instead.\n        duplicates = sorted(\n            {name for name in key_field_names if key_field_names.count(name) > 1}\n        )\n        if duplicates:\n            raise ValueError(f\"primary_key contains duplicate column(s) {duplicates}\")\n        # A nullable primary-key column in snapshot mode is silently\n        # broken: either the `NOT NULL` PRIMARY KEY we emit on\n        # create_if_not_exists / replace rejects the NULL row at\n        # insert time, or (against a pre-existing table that allows\n        # NULL in the PK) the retraction ``DELETE ... WHERE pkey=$1``\n        # never matches anything because SQL ``= NULL`` is always\n        # false. Both are data-loss footguns, so we refuse the setup\n        # here with an actionable message.\n        if is_snapshot_mode:\n            for pkey_field in primary_key:\n                if isinstance(pkey_field._column.dtype, dtype.Optional):\n                    raise ValueError(\n                        f\"primary_key column '{pkey_field.name}' is \"\n                        \"declared nullable; primary_key columns must be \"\n                        \"non-nullable in snapshot mode. Either remove \"\n                        \"the Optional wrapper in the schema or filter \"\n                        \"out nulls upstream via \"\n                        \".filter(t.pkey.is_not_none()).\"","sourceCodeStart":914,"sourceCodeEnd":950,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/postgres/__init__.py#L914-L950","documentation":"Raised by pw.io.postgres.write when the primary_key list contains the same column name more than once (e.g. primary_key=[t.k, t.k]). Pathway rejects this eagerly because a duplicate would otherwise flow into the SQL template and produce malformed DDL (PRIMARY KEY (k, k) with the column listed twice) on CREATE TABLE or corrupt the primary_key_fields index swap used for DELETE retractions.","triggerScenarios":"Calling pw.io.postgres.write(table, ..., primary_key=[t.k, t.k]) or building the primary_key list dynamically in a way that appends the same ColumnReference twice (e.g. primary_key=keys + keys, or a loop that adds a column per sort key plus the explicit key again).","commonSituations":"Composite primary keys assembled programmatically from several sources where deduplication was forgotten; copy-paste of a key column into a key list; passing sorted(columns) where a column appears in two roles.","solutions":["Inspect the primary_key argument and remove the duplicate ColumnReference so each column appears exactly once.","If the key list is built dynamically, deduplicate while preserving order, e.g. list(dict.fromkeys(key_cols)) before passing it to write().","If you intended a composite key, verify each element is a distinct column from the table being written (table.k, table.name, ...)."],"exampleFix":"# before\npw.io.postgres.write(t, conn, \"t\", primary_key=[t.k, t.k])\n\n# after\npw.io.postgres.write(t, conn, \"t\", primary_key=[t.k])","handlingStrategy":"validation","validationCode":"key_cols = [t.k, t.name]\nnames = [c.name() for c in key_cols]\nassert len(set(names)) == len(names), f\"duplicate primary_key columns: {names}\"\npw.io.postgres.write(t, conn, \"t\", primary_key=key_cols)","typeGuard":"def unique_key_columns(cols: list) -> bool:\n    names = [c.name() for c in cols]\n    return len(set(names)) == len(names)","tryCatchPattern":"try:\n    pw.io.postgres.write(t, conn, \"t\", primary_key=keys)\nexcept ValueError as e:\n    if \"duplicate column\" in str(e):\n        keys = list(dict.fromkeys(keys))\n        pw.io.postgres.write(t, conn, \"t\", primary_key=keys)\n    else:\n        raise","preventionTips":["Build primary_key lists from a set/dict of column references, never by concatenating lists.","Deduplicate with list(dict.fromkeys(cols)) before passing key lists assembled from multiple sources.","Write a unit test asserting the key list has no duplicate names."],"tags":["pathway","postgres","primary-key","validation"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}