apache/beam · error · IllegalStateException

No generator found for field:

Error message

No generator found for field: 

What it means

During element generation, DataGeneratorRowFn iterates every schema field and looks up a prebuilt FieldGenerator by name. If a field has no entry in the fieldGenerators map, the internal invariant that every schema field got a generator has been broken, so an IllegalStateException is thrown.

Solutions

1. Ensure every field in the table schema maps to a supported generator (check field types).
2. Configure per-field generation via 'fields.<name>.kind' properties so the builder creates a generator for the field.
3. Check for a library bug: the field-creation loop and the generation loop disagree; report/upgrade Beam version.

Set<String> schemaFields = schema.getFieldNames().stream().collect(Collectors.toSet());
if (!schemaFields.equals(fieldGenerators.keySet())) {
  throw new IllegalStateException("Missing generators for: " + Sets.difference(schemaFields, fieldGenerators.keySet()));
}

try { pipeline.run().waitUntilFinish(); } catch (IllegalStateException e) { /* inspect schema vs configured generators */ }

When it happens

Trigger: A schema field that is neither the primary timestamp field nor covered by the generators built in the constructor; typically caused by generator construction skipping certain field types or configuration shapes so the map has fewer entries than schema fields.

Common situations: Running a datagen table whose schema contains a field type that valueGenerator could not map, while the earlier creation path silently skipped it; schema mutated after the row function was built.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• A 'datagen' table requires either 'rows-per-second' (for…
• Data generator requires a defined SQL type. Beam type
• For a cycling sequence generator, both 'start' and 'end'…
• For 'event-time' behavior, 'event-time.timestamp-column'…
• For sequence generator, 'start

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8974e78dbe97bb00. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datagen/DataGeneratorRowFn.java:73

    this.fieldGenerators = new HashMap<>();

    for (Schema.Field field : schema.getFields()) {
      fieldGenerators.put(field.getName(), createGeneratorForField(field));
    }
  }

  @ProcessElement
  public void processElement(
      @Element Long index, @Timestamp Instant timestamp, OutputReceiver<Row> out) {
    Row.Builder rowBuilder = Row.withSchema(schema);
    for (Schema.Field field : schema.getFields()) {
      Object value;
      if (field.getName().equals(this.primaryTimestampField)) {
        value = timestamp.toDateTime();
      } else {
        FieldGenerator generator = fieldGenerators.get(field.getName());
        if (generator == null) {
          throw new IllegalStateException("No generator found for field: " + field.getName());
        }
        value = generator.generate(index);
      }
      rowBuilder.addValue(value);
    }
    out.output(rowBuilder.build());
  }

  @FunctionalInterface
  private interface FieldGenerator extends Serializable {
    @Nullable
    Object generate(long index);
  }

  private FieldGenerator createGeneratorForField(Schema.Field field) {
    String fieldName = field.getName();
    FieldGenerator valueGenerator = createValueGeneratorForField(field);
    double nullRate = properties.path("fields." + fieldName + ".null-rate").asDouble(0.0);

View on GitHub (pinned to 12126d8942)