{"record":{"id":"e407e4b6e8e4be32","repo":"pathwaycom/pathway","slug":"defining-a-primary-key-in-the-schema-is-not-suppor","errorCode":null,"errorMessage":"Defining a primary key in the schema is not supported for pw.io.elasticsearch.read. The connector keys the resulting table by id_column. If you need to reindex the table by a different column, use pw.Table.with_id_from() after reading.","messagePattern":"Defining a primary key in the schema is not supported for pw\\.io\\.elasticsearch\\.read\\. The connector keys the resulting table by id_column\\. If you need to reindex the table by a different column, use pw\\.Table\\.with_id_from\\(\\) after reading\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/elasticsearch/__init__.py","lineNumber":318,"sourceCode":"    ...     index_name=\"logs\",\n    ...     schema=LogSchema,\n    ...     timestamp_column=\"ts\",\n    ...     id_column=\"doc_id\",\n    ...     max_transaction_duration=datetime.timedelta(minutes=5),\n    ... )\n    \"\"\"\n\n    _check_entitlements(\"elasticsearch\")\n\n    column_names = schema.column_names()\n    if timestamp_column not in column_names:\n        raise ValueError(\n            f\"timestamp_column {timestamp_column!r} is not present in the schema\"\n        )\n    if id_column not in column_names:\n        raise ValueError(f\"id_column {id_column!r} is not present in the schema\")\n    if schema.primary_key_columns():\n        raise ValueError(\n            \"Defining a primary key in the schema is not supported for \"\n            \"pw.io.elasticsearch.read. The connector keys the resulting table by \"\n            \"id_column. If you need to reindex the table by a different column, use \"\n            \"pw.Table.with_id_from() after reading.\"\n        )\n\n    max_transaction_duration_ms = round(\n        as_duration_seconds(max_transaction_duration, \"max_transaction_duration\") * 1000\n    )\n    poll_interval_ms = round(as_duration_seconds(poll_interval, \"poll_interval\") * 1000)\n    if read_batch_size <= 0:\n        raise ValueError(\"read_batch_size must be positive\")\n\n    data_storage = api.DataStorage(\n        storage_type=\"elasticsearch\",\n        mode=internal_connector_mode(mode),\n        elasticsearch_params=api.ElasticSearchParams(\n            host=host,","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/elasticsearch/__init__.py#L300-L336","documentation":"Raised by pw.io.elasticsearch.read when the provided pw.Schema defines a primary key. The connector itself keys the resulting table by id_column; a schema-level primary key conflicts with that and is explicitly unsupported. To re-key the table, use pw.Table.with_id_from() after reading.","triggerScenarios":"pw.io.elasticsearch.read(..., schema=class S(pw.Schema): doc_id: str = pw.column_definition(primary_key=True)) — any schema passed with primary_key=True on one or more columns.","commonSituations":"Reusing a schema class from pw.io.csv.read or pw.io.json.read where primary_key=True is idiomatic; habitually marking id columns as primary keys in every schema.","solutions":["Remove primary_key=True from the schema class passed to pw.io.elasticsearch.read; let the connector key the table by id_column.","If you need a different key, read first and then re-key: table = table.with_id_from(table.some_col)."],"exampleFix":"# before\nclass DocSchema(pw.Schema):\n    doc_id: str = pw.column_definition(primary_key=True)\n    text: str\npw.io.elasticsearch.read(..., schema=DocSchema, id_column=\"doc_id\")\n\n# after\nclass DocSchema(pw.Schema):\n    doc_id: str\n    text: str\ntable = pw.io.elasticsearch.read(..., schema=DocSchema, id_column=\"doc_id\")","handlingStrategy":"validation","validationCode":"assert not schema.primary_key_columns(), (\n    \"elasticsearch schemas must not define primary keys; use with_id_from() after read\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Maintain a dedicated ES schema class without primary_key instead of reusing schemas from file connectors.","Re-key with pw.Table.with_id_from() rather than schema-level keys for ES sources."],"tags":["elasticsearch","schema","primary-key","connector"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}