{"record":{"id":"51132910fb06551d","repo":"pathwaycom/pathway","slug":"id-column-id-column-r-is-not-present-in-the-sche","errorCode":null,"errorMessage":"id_column {id_column!r} is not present in the schema","messagePattern":"id_column (.+?) is not present in the schema","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/elasticsearch/__init__.py","lineNumber":316,"sourceCode":"    ...     host=\"http://localhost:9200\",\n    ...     auth=pw.io.elasticsearch.ElasticSearchAuth.basic(\"admin\", \"admin\"),\n    ...     index_name=\"logs\",\n    ...     schema=LogSchema,\n    ...     timestamp_column=\"ts\",\n    ...     id_column=\"doc_id\",\n    ...     max_transaction_duration=datetime.timedelta(minutes=5),\n    ... )\n    \"\"\"\n\n    _check_entitlements(\"elasticsearch\")\n\n    column_names = schema.column_names()\n    if timestamp_column not in column_names:\n        raise ValueError(\n            f\"timestamp_column {timestamp_column!r} is not present in the schema\"\n        )\n    if id_column not in column_names:\n        raise ValueError(f\"id_column {id_column!r} is not present in the schema\")\n    if schema.primary_key_columns():\n        raise ValueError(\n            \"Defining a primary key in the schema is not supported for \"\n            \"pw.io.elasticsearch.read. The connector keys the resulting table by \"\n            \"id_column. If you need to reindex the table by a different column, use \"\n            \"pw.Table.with_id_from() after reading.\"\n        )\n\n    max_transaction_duration_ms = round(\n        as_duration_seconds(max_transaction_duration, \"max_transaction_duration\") * 1000\n    )\n    poll_interval_ms = round(as_duration_seconds(poll_interval, \"poll_interval\") * 1000)\n    if read_batch_size <= 0:\n        raise ValueError(\"read_batch_size must be positive\")\n\n    data_storage = api.DataStorage(\n        storage_type=\"elasticsearch\",\n        mode=internal_connector_mode(mode),","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/elasticsearch/__init__.py#L298-L334","documentation":"Raised by pw.io.elasticsearch.read when the id_column argument does not name a column of the provided schema. The connector uses that column as the document id to key the resulting Pathway table, so it must exist in schema.","triggerScenarios":"pw.io.elasticsearch.read(..., schema=DocSchema, id_column=\"doc_id\") where DocSchema declares no 'doc_id' field — typo, renamed field, or a schema written for a different index.","commonSituations":"ES documents keyed by '_id' with no materialized id field in the mapping; copy-pasted schema classes; case or spelling mismatches between the mapping and the schema class.","solutions":["Fix id_column to name an existing schema column and ensure that field is declared in the schema class.","Check DocSchema.column_names() for the exact spelling/case of the id field.","If the index only has ES's internal _id, materialize an explicit id field in the documents or pick a unique existing field."],"exampleFix":"# before\nclass DocSchema(pw.Schema):\n    _id: str\n    text: str\npw.io.elasticsearch.read(..., schema=DocSchema, id_column=\"doc_id\")\n\n# after\nclass DocSchema(pw.Schema):\n    doc_id: str\n    text: str\npw.io.elasticsearch.read(..., schema=DocSchema, id_column=\"doc_id\")","handlingStrategy":"validation","validationCode":"column_names = schema.column_names()\nassert id_column in column_names, (\n    f\"id_column {id_column!r} not in schema columns {column_names}\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep the ES schema class and the read() call arguments in the same module so renames stay in sync.","Use schema.column_names() in assertions or tests to catch mapping drift."],"tags":["elasticsearch","schema","connector","api-misuse"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}