apache/beam · error · IllegalStateException

createRowWriter called when schemaFactory is null; forgot…

Error message

createRowWriter called when schemaFactory is null; forgot to call prepare()?

What it means

IllegalStateException thrown by RowWriterFactory's anonymous Avro implementation when createRowWriter is invoked but the schemaFactory field is null. schemaFactory is initialized in prepare(); a null value means the factory's setup step was skipped, so the TableSchema-to-Avro-Schema conversion function is unavailable.

Solutions

Call prepare(...) with a non-null GenericRecordWriterFactory before creating row writers
If calling prepare manually, verify both arguments (writer factory and dynamic destinations) are non-null
Prefer using BigQueryIO.write() so Beam initializes the factory through its normal DoFn lifecycle

Example fix

// before
factory.prepare(null, dynamicDestinations); // schemaFactory stays null
factory.createRowWriter(prefix, destination); // IllegalStateException
// after
factory.prepare(new GenericRecordWriterFactory(tempFilePrefix, useCdcWrites), dynamicDestinations);

Defensive patterns

Strategy: validation

Validate before calling

if (schemaFactory == null) {
  throw new IllegalStateException("schemaFactory not initialized; call prepare(writerFactory, dynamicDestinations) first");
}

Type guard

static <E,D> boolean hasSchemaFactory(RowWriterFactory<E,D> f) {
  try { return f.getClass().getDeclaredField("schemaFactory").get(f) != null; }
  catch (Exception e) { return false; }
}

Try / catch

try {
  writer = factory.createRowWriter(prefix, destination);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("schemaFactory is null")) {
    factory.prepare(recordWriterFactory, dynamicDestinations);
    writer = factory.createRowWriter(prefix, destination);
  } else { throw e; }
}

Prevention

Invoke prepare() with a non-null GenericRecordWriterFactory before writing
Never pass null schema factory arguments to prepare()
Drive writes through the public BigQueryIO API rather than internal factories

When it happens

Trigger: createRowWriter called on a RowWriterFactory instance that never had prepare(GenericRecordWriterFactory, DynamicDestinations) invoked, leaving schemaFactory uninitialized even though dynamicDestinations may be set.

Common situations: Unit tests invoking createRowWriter directly without prepare; custom pipelines constructing the internal factory by hand; partial refactors where prepare was called with a null schema factory argument.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/65cd52482c8a7af6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/RowWriterFactory.java:138

    SerializableFunction<AvroWriteRequest<ElementT>, AvroT> getToAvroFn() {
      return toAvro;
    }

    @Override
    OutputType getOutputType() {
      return OutputType.AvroGenericRecord;
    }

    @Override
    BigQueryRowWriter<ElementT> createRowWriter(String tempFilePrefix, DestinationT destination)
        throws Exception {
      if (dynamicDestinations == null) {
        throw new IllegalStateException(
            "createRowWriter called when dynamicDestinations is null; forgot to call prepare()?");
      }
      if (schemaFactory == null) {
        throw new IllegalStateException(
            "createRowWriter called when schemaFactory is null; forgot to call prepare()?");
      }

      TableSchema tableSchema = dynamicDestinations.getSchema(destination);
      Schema avroSchema = schemaFactory.apply(tableSchema);
      return new AvroRowWriter<>(tempFilePrefix, avroSchema, toAvro, writerFactory);
    }

    @Override
    String getSourceFormat() {
      return "AVRO";
    }
  }
}

View on GitHub (pinned to 12126d8942)