{"record":{"id":"7d89afab5649557b","repo":"apache/beam","slug":"a-datagen-table-requires-either-rows-per-second-for","errorCode":null,"errorMessage":"A 'datagen' table requires either 'rows-per-second' (for unbounded) or 'number-of-rows' (for bounded) in TBLPROPERTIES.","messagePattern":"A 'datagen' table requires either 'rows-per-second' \\(for unbounded\\) or 'number-of-rows' \\(for bounded\\) in TBLPROPERTIES\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java","lineNumber":53,"sourceCode":"  private final ObjectNode properties;\n\n  public DataGeneratorPTransform(Schema schema, ObjectNode properties) {\n    this.schema = schema;\n    this.properties = properties;\n  }\n\n  @Override\n  public PCollection<Row> expand(PBegin input) {\n    GenerateSequence generator;\n    JsonNode rpsNode = properties.path(\"rows-per-second\");\n    JsonNode numRowsNode = properties.path(\"number-of-rows\");\n\n    if (!rpsNode.isMissingNode()) {\n      generator = GenerateSequence.from(0).withRate(rpsNode.asLong(), Duration.standardSeconds(1));\n    } else if (!numRowsNode.isMissingNode()) {\n      generator = GenerateSequence.from(0).to(numRowsNode.asLong());\n    } else {\n      throw new IllegalArgumentException(\n          \"A 'datagen' table requires either 'rows-per-second' (for unbounded) or \"\n              + \"'number-of-rows' (for bounded) in TBLPROPERTIES.\");\n    }\n\n    String behavior = properties.path(\"timestamp.behavior\").asText(\"processing-time\");\n    @Nullable String eventTimeColumn = null;\n\n    if (\"event-time\".equalsIgnoreCase(behavior)) {\n      JsonNode columnNode = properties.path(\"event-time.timestamp-column\");\n\n      if (columnNode.isMissingNode() || columnNode.isNull()) {\n        throw new IllegalArgumentException(\n            \"For 'event-time' behavior, 'event-time.timestamp-column' must be specified.\");\n      }\n      eventTimeColumn = columnNode.asText();\n\n      // Validate that the specified column exists and is of type TIMESTAMP.\n      if (!schema.hasField(eventTimeColumn)) {","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java#L35-L71","documentation":"A 'datagen' test-data table in Beam SQL needs to know how to generate rows: 'rows-per-second' for an unbounded source or 'number-of-rows' for a bounded source, both given in TBLPROPERTIES. expand throws IllegalArgumentException when neither property is present, since there is no default generation rate or row count.","triggerScenarios":"CREATE EXTERNAL TABLE ... TYPE 'datagen' (or provider 'datagen') whose TBLPROPERTIES JSON lacks both 'rows-per-second' and 'number-of-rows', then running a query that expands the table's PTransform.","commonSituations":"Copy-pasting datagen DDL examples and dropping the TBLPROPERTIES; misspelling the keys (e.g. 'rowsPerSecond' — property lookup is via exact JSON path); providing the property at query time instead of in TBLPROPERTIES.","solutions":["Add 'rows-per-second': <n> to TBLPROPERTIES for an unbounded streaming source.","Add 'number-of-rows': <n> to TBLPROPERTIES for a bounded batch source.","Verify exact key spelling: 'rows-per-second' and 'number-of-rows' (hyphenated, lowercase) in the TBLPROPERTIES JSON."],"exampleFix":"-- before\nCREATE EXTERNAL TABLE dg (id BIGINT) TYPE 'datagen' TBLPROPERTIES '{}';\n\n-- after\nCREATE EXTERNAL TABLE dg (id BIGINT) TYPE 'datagen' TBLPROPERTIES '{\"rows-per-second\": 10}';","handlingStrategy":"validation","validationCode":"JsonNode props = properties;\nif (props.path(\"rows-per-second\").isMissingNode() && props.path(\"number-of-rows\").isMissingNode()) {\n  throw new IllegalArgumentException(\"datagen TBLPROPERTIES must set 'rows-per-second' or 'number-of-rows'\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  PCollection<Row> rows = datagenTable.expand(input);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"requires either 'rows-per-second'\")) {\n    // add one of the two properties to TBLPROPERTIES\n  }\n}","preventionTips":["Decide bounded vs unbounded up front and set 'number-of-rows' or 'rows-per-second' accordingly.","Use the exact hyphenated lowercase property names; camelCase will be silently ignored (missing node).","Keep a canonical datagen DDL template in your test utilities."],"tags":["java","beam-sql","datagen","configuration"],"backgroundTag":"missing-required-config-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}