{"record":{"id":"b4e15080bcab3cd9","repo":"pathwaycom/pathway","slug":"primary-key-contains-duplicate-column-s-sorted-d","errorCode":null,"errorMessage":"primary_key contains duplicate column(s) {sorted(duplicates)}. Each column may appear at most once.","messagePattern":"primary_key contains duplicate column\\(s\\) (.+?)\\. Each column may appear at most once\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/duckdb/__init__.py","lineNumber":336,"sourceCode":"                f\"Column(s) {collisions} collide with the 'time' and 'diff' \"\n                \"metadata columns appended in stream_of_changes mode. Rename \"\n                \"these columns in the Pathway table, or use \"\n                'output_table_type=\"snapshot\".'\n            )\n\n    key_field_names: list[str] | None = None\n    if primary_key is not None:\n        # Duplicate entries in `primary_key` produce a nonsensical SQL template\n        # and reorder DELETE bindings using the duplicated index, so retractions\n        # would silently match no rows. Reject here with a clear message.\n        names_seen: set[str] = set()\n        duplicates: list[str] = []\n        for pkey in primary_key:\n            if pkey.name in names_seen and pkey.name not in duplicates:\n                duplicates.append(pkey.name)\n            names_seen.add(pkey.name)\n        if duplicates:\n            raise ValueError(\n                f\"primary_key contains duplicate column(s) {sorted(duplicates)}. \"\n                \"Each column may appear at most once.\"\n            )\n        key_field_names = []\n        for pkey in primary_key:\n            # Raises ValueError when `pkey` belongs to a different table or does\n            # not name a column of `table`, so users get a clear message at\n            # write() time instead of an opaque runtime error.\n            get_column_index(table, pkey)\n            # A nullable primary key makes DELETE ... WHERE pk = NULL never match\n            # on retractions, so the destination would keep stale rows.\n            if isinstance(pkey._column.dtype, dt.Optional):\n                raise ValueError(\n                    f\"primary_key column {pkey.name!r} is declared nullable \"\n                    f\"({pkey._column.dtype}); primary-key columns must be \"\n                    \"non-nullable in snapshot mode.\"\n                )\n            # DuckDB cannot build a PRIMARY KEY / index on a list, array, tuple or","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/duckdb/__init__.py#L318-L354","documentation":"Raised by pw.io.duckdb.write when the primary_key sequence contains the same column more than once. Duplicate key entries produce a nonsensical SQL template and reorder DELETE bindings using the duplicated index, so retractions would silently match no rows; the connector rejects duplicates up front.","triggerScenarios":"pw.io.duckdb.write(t, table_name=\"t\", output_table_type=\"snapshot\", primary_key=[t.id, t.id]) — typically when a key list is built dynamically (e.g. [*default_keys, *extra_keys]) and the same column ends up in both.","commonSituations":"Programmatically composing composite keys from config plus hardcoded columns; concatenating key lists without deduplication after a refactor.","solutions":["Deduplicate the key list while preserving order: primary_key=list(dict.fromkeys([t.id, t.tenant_id, t.id])).","Remove the repeated column from the primary_key list manually."],"exampleFix":"# before\nkeys = [*config_keys, *default_keys]  # t.id appears twice\npw.io.duckdb.write(t, table_name=\"t\", output_table_type=\"snapshot\", primary_key=keys)\n\n# after\nkeys = list(dict.fromkeys([*config_keys, *default_keys]))\npw.io.duckdb.write(t, table_name=\"t\", output_table_type=\"snapshot\", primary_key=keys)","handlingStrategy":"validation","validationCode":"primary_key = list(dict.fromkeys(primary_key))  # order-preserving dedup","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never concatenate key lists without deduplicating.","Prefer a single tuple of key columns defined in one place instead of composed lists."],"tags":["duckdb","primary-key","connector","api-misuse"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}