apache/beam · error · IllegalArgumentException

The specified 'event-time.timestamp-column

Error message

The specified 'event-time.timestamp-column' ('%s') must be of type TIMESTAMP, but was '%s'.

What it means

The datagen table provider validates that the column named in the 'event-time.timestamp-column' property has Beam type DATETIME (TIMESTAMP in SQL terms). This column is used to assign event-time watermarks for the generated rows, so a non-timestamp column cannot fulfill that role. The check runs during PTransform expansion, before the pipeline runs.

Solutions

Change the referenced column's type in the table schema to TIMESTAMP (Schema.TypeName.DATETIME).
Point 'event-time.timestamp-column' at an existing TIMESTAMP-typed column in the schema.
Add a TIMESTAMP column to the schema if none exists and use it as the timestamp column.

Example fix

// before
CREATE EXTERNAL TABLE orders (id INT, event_time VARCHAR)
TYPE 'datagen' TBLPROPERTIES '{"event-time.timestamp-column":"event_time"}'
// after
CREATE EXTERNAL TABLE orders (id INT, event_time TIMESTAMP)
TYPE 'datagen' TBLPROPERTIES '{"event-time.timestamp-column":"event_time"}'

Defensive patterns

Strategy: validation

Validate before calling

Schema schema = table.schema();
String col = props.path("event-time.timestamp-column").asText();
Schema.Field f = schema.getField(col);
if (f == null || !Schema.TypeName.DATETIME.equals(f.getType().getTypeName())) {
  throw new IllegalArgumentException(col + " must be a TIMESTAMP column");
}

Type guard

boolean isTimestampColumn(Schema schema, String name) {
  Schema.Field f = schema.getField(name);
  return f != null && Schema.TypeName.DATETIME.equals(f.getType().getTypeName());
}

Try / catch

try { table.expand(...); } catch (IllegalArgumentException e) { /* fix schema/property, reconfigure DDL */ }

Prevention

Declare the event-time column as TIMESTAMP in the DDL.
Add a preflight schema check when generating table DDL programmatically.

When it happens

Trigger: Declaring a datagen table with 'event-time.timestamp-column' pointing at a schema field whose type is VARCHAR, INT, DATE, or any type other than TIMESTAMP/DATETIME; the IllegalArgumentException is thrown in DataGeneratorPTransform.expand.

Common situations: Copy-pasting table DDL where the timestamp column was changed to a string; pointing the column at a logical-type field that maps to something other than DATETIME; typos resolved to an existing but wrong-typed column.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fe0ace8c8beddfb7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorPTransform.java:80

      JsonNode columnNode = properties.path("event-time.timestamp-column");

      if (columnNode.isMissingNode() || columnNode.isNull()) {
        throw new IllegalArgumentException(
            "For 'event-time' behavior, 'event-time.timestamp-column' must be specified.");
      }
      eventTimeColumn = columnNode.asText();

      // Validate that the specified column exists and is of type TIMESTAMP.
      if (!schema.hasField(eventTimeColumn)) {
        throw new IllegalArgumentException(
            String.format(
                "The specified 'event-time.timestamp-column' ('%s') does not exist in the table schema.",
                eventTimeColumn));
      }

      Schema.Field eventTimeField = schema.getField(eventTimeColumn);
      if (!Schema.TypeName.DATETIME.equals(eventTimeField.getType().getTypeName())) {
        throw new IllegalArgumentException(
            String.format(
                "The specified 'event-time.timestamp-column' ('%s') must be of type TIMESTAMP, but was '%s'.",
                eventTimeColumn, eventTimeField.getType()));
      }

      long maxOutOfOrdernessMs = properties.path("event_time.max-out-of-orderness").asLong(0L);
      generator = generator.withTimestampFn(new AdvancingTimestampFn(maxOutOfOrdernessMs));
    }

    return input
        .getPipeline()
        .apply("GenerateSequence", generator)
        .apply(
            "GenerateRows", ParDo.of(new DataGeneratorRowFn(schema, properties, eventTimeColumn)))
        .setRowSchema(schema);
  }
}

View on GitHub (pinned to 12126d8942)