{"record":{"id":"7bdc47a1283d8854","repo":"apache/beam","slug":"the-specified-event-time-timestamp-column-s-does-not-exist","errorCode":null,"errorMessage":"The specified 'event-time.timestamp-column' ('%s') does not exist in the table schema.","messagePattern":"The specified 'event-time\\.timestamp-column' \\('(.+?)'\\) does not exist in the table schema\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java","lineNumber":72,"sourceCode":"          \"A 'datagen' table requires either 'rows-per-second' (for unbounded) or \"\n              + \"'number-of-rows' (for bounded) in TBLPROPERTIES.\");\n    }\n\n    String behavior = properties.path(\"timestamp.behavior\").asText(\"processing-time\");\n    @Nullable String eventTimeColumn = null;\n\n    if (\"event-time\".equalsIgnoreCase(behavior)) {\n      JsonNode columnNode = properties.path(\"event-time.timestamp-column\");\n\n      if (columnNode.isMissingNode() || columnNode.isNull()) {\n        throw new IllegalArgumentException(\n            \"For 'event-time' behavior, 'event-time.timestamp-column' must be specified.\");\n      }\n      eventTimeColumn = columnNode.asText();\n\n      // Validate that the specified column exists and is of type TIMESTAMP.\n      if (!schema.hasField(eventTimeColumn)) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"The specified 'event-time.timestamp-column' ('%s') does not exist in the table schema.\",\n                eventTimeColumn));\n      }\n\n      Schema.Field eventTimeField = schema.getField(eventTimeColumn);\n      if (!Schema.TypeName.DATETIME.equals(eventTimeField.getType().getTypeName())) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"The specified 'event-time.timestamp-column' ('%s') must be of type TIMESTAMP, but was '%s'.\",\n                eventTimeColumn, eventTimeField.getType()));\n      }\n\n      long maxOutOfOrdernessMs = properties.path(\"event_time.max-out-of-orderness\").asLong(0L);\n      generator = generator.withTimestampFn(new AdvancingTimestampFn(maxOutOfOrdernessMs));\n    }\n\n    return input","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java#L54-L90","documentation":"In event-time mode the datagen transform validates that the column named by 'event-time.timestamp-column' actually exists in the table schema (and, next, that it is TIMESTAMP). expand throws IllegalArgumentException when the named column is absent from the schema.","triggerScenarios":"TBLPROPERTIES has \"event-time.timestamp-column\": \"ts\" but the CREATE TABLE schema has no field named 'ts' — due to a typo, case mismatch, or renaming the column without updating TBLPROPERTIES.","commonSituations":"Renaming the timestamp column in the schema but leaving the old name in TBLPROPERTIES; referencing a column from another table; case-sensitive mismatch like 'EventTime' vs 'eventtime'.","solutions":["Set 'event-time.timestamp-column' to the exact (case-sensitive) name of a field present in the schema.","Add a TIMESTAMP column to the schema if none exists, e.g. (event_ts TIMESTAMP).","Verify with the subsequent check too: the referenced column must be of TIMESTAMP type or the next validation will fail."],"exampleFix":"-- before\nCREATE EXTERNAL TABLE dg (id BIGINT) TYPE 'datagen'\nTBLPROPERTIES '{\"timestamp.behavior\": \"event-time\", \"event-time.timestamp-column\": \"ts\"}';\n\n-- after\nCREATE EXTERNAL TABLE dg (id BIGINT, ts TIMESTAMP) TYPE 'datagen'\nTBLPROPERTIES '{\"timestamp.behavior\": \"event-time\", \"event-time.timestamp-column\": \"ts\"}';","handlingStrategy":"validation","validationCode":"String col = properties.path(\"event-time.timestamp-column\").asText(null);\nif (col == null || !schema.hasField(col)) {\n  throw new IllegalArgumentException(\"'event-time.timestamp-column' must name an existing schema field\");\n}\nif (schema.getField(col).getType().getTypeName() != Schema.TypeName.TIMESTAMP) {\n  throw new IllegalArgumentException(\"'event-time.timestamp-column' must be a TIMESTAMP field\");\n}","typeGuard":"boolean hasTimestampField(Schema s, String name) {\n  Schema.Field f = s.getField(name);\n  return f != null && Schema.TypeName.TIMESTAMP == f.getType().getTypeName();\n}","tryCatchPattern":"try {\n  PCollection<Row> rows = datagenTable.expand(input);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"does not exist in the table schema\")) {\n    // fix the column name in TBLPROPERTIES to match the schema exactly\n  }\n}","preventionTips":["Use the exact schema field name (case-sensitive) in event-time.timestamp-column.","Update TBLPROPERTIES whenever the schema column is renamed.","Ensure the referenced field is TIMESTAMP, not DATE or BIGINT, to pass the follow-up type check."],"tags":["java","beam-sql","datagen","configuration","schema"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}