apache/beam · error · IllegalArgumentException

For 'event-time' behavior, 'event-time.timestamp-column' mus

Error message

For 'event-time' behavior, 'event-time.timestamp-column' must be specified.

What it means

When a datagen table's TBLPROPERTIES set 'timestamp.behavior' to 'event-time', the transform needs a column to source event timestamps from. expand throws IllegalArgumentException when 'event-time.timestamp-column' is missing or null, because processing-time-based generation cannot be used in event-time mode.

Source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java:65

    if (!rpsNode.isMissingNode()) {
      generator = GenerateSequence.from(0).withRate(rpsNode.asLong(), Duration.standardSeconds(1));
    } else if (!numRowsNode.isMissingNode()) {
      generator = GenerateSequence.from(0).to(numRowsNode.asLong());
    } else {
      throw new IllegalArgumentException(
          "A 'datagen' table requires either 'rows-per-second' (for unbounded) or "
              + "'number-of-rows' (for bounded) in TBLPROPERTIES.");
    }

    String behavior = properties.path("timestamp.behavior").asText("processing-time");
    @Nullable String eventTimeColumn = null;

    if ("event-time".equalsIgnoreCase(behavior)) {
      JsonNode columnNode = properties.path("event-time.timestamp-column");

      if (columnNode.isMissingNode() || columnNode.isNull()) {
        throw new IllegalArgumentException(
            "For 'event-time' behavior, 'event-time.timestamp-column' must be specified.");
      }
      eventTimeColumn = columnNode.asText();

      // Validate that the specified column exists and is of type TIMESTAMP.
      if (!schema.hasField(eventTimeColumn)) {
        throw new IllegalArgumentException(
            String.format(
                "The specified 'event-time.timestamp-column' ('%s') does not exist in the table schema.",
                eventTimeColumn));
      }

      Schema.Field eventTimeField = schema.getField(eventTimeColumn);
      if (!Schema.TypeName.DATETIME.equals(eventTimeField.getType().getTypeName())) {
        throw new IllegalArgumentException(
            String.format(
                "The specified 'event-time.timestamp-column' ('%s') must be of type TIMESTAMP, but was '%s'.",
                eventTimeColumn, eventTimeField.getType()));

View on GitHub (pinned to 12126d8942)

Solutions

1. Add "event-time.timestamp-column": "<column>" to TBLPROPERTIES naming an existing TIMESTAMP field.
2. Ensure the JSON value is a non-null string column name.
3. Switch 'timestamp.behavior' to 'processing-time' if you do not actually need event-time semantics.

Example fix

-- before
TBLPROPERTIES '{"rows-per-second": 10, "timestamp.behavior": "event-time"}'

-- after
TBLPROPERTIES '{"rows-per-second": 10, "timestamp.behavior": "event-time", "event-time.timestamp-column": "event_ts"}'

if ("event-time".equalsIgnoreCase(properties.path("timestamp.behavior").asText("processing-time"))) {
  JsonNode col = properties.path("event-time.timestamp-column");
  if (col.isMissingNode() || col.isNull()) {
    throw new IllegalArgumentException("event-time behavior requires 'event-time.timestamp-column' in TBLPROPERTIES");
  }
}

try {
  PCollection<Row> rows = datagenTable.expand(input);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("'event-time.timestamp-column' must be specified")) {
    // add the column property or fall back to processing-time behavior
  }
}

When it happens

Trigger: TBLPROPERTIES contains "timestamp.behavior": "event-time" but no 'event-time.timestamp-column' key, or its JSON value is null.

Common situations: Configuring event-time watermarking tests but forgetting the column property; nesting the property under the wrong JSON path; setting the behavior value in quotes incorrectly so only 'event-time' matches but the column key is absent.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• A 'datagen' table requires either 'rows-per-second' (for unb
• The specified 'event-time.timestamp-column' ('%s') does not
• LOCATION is required
• Schema fields count: '%s' does not fit columnsMapping count:
• columnsMapping '%s' does not fit to schema field names '%s'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/58f83ba7436afbd4. Report an issue: GitHub.