apache/beam · error · IllegalStateException

createRowWriter called when dynamicDestinations is null…

Error message

createRowWriter called when dynamicDestinations is null; forgot to call prepare()?

What it means

IllegalStateException thrown by RowWriterFactory's anonymous Avro implementation when createRowWriter is invoked but the dynamicDestinations field is null. The factory's prepare() lifecycle method is what initializes dynamicDestinations, so a null field indicates the DoFn was used without its setup being run (or it was constructed incorrectly).

Solutions

1. Ensure prepare(...) is called on the factory before any createRowWriter call
2. If driving the DoFn manually (tests), simulate the full lifecycle: construct, call prepare with the writer factory and dynamic destinations, then createRowWriter
3. Avoid constructing RowWriterFactory subclasses directly; build the write via BigQueryIO.write() so Beam manages the lifecycle

Example fix

// before
RowWriterFactory<String, TableDestination> factory = new AvroFactory<>(toAvro, writerFactory);
factory.createRowWriter(prefix, destination); // IllegalStateException
// after
factory.prepare(writerFactory, dynamicDestinations);
factory.createRowWriter(prefix, destination);

if (factory == null || factory.hasDynamicDestinations() == false /* or assert initialized */) {
  throw new IllegalStateException("Call prepare() before createRowWriter()");
}

static <E,D> boolean isPrepared(RowWriterFactory<E,D> f) {
  try { return f.getClass().getDeclaredField("dynamicDestinations").get(f) != null; }
  catch (Exception e) { return false; }
}

try {
  writer = factory.createRowWriter(prefix, destination);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("dynamicDestinations is null")) {
    factory.prepare(writerFactory, dynamicDestinations);
    writer = factory.createRowWriter(prefix, destination);
  } else { throw e; }
}

When it happens

Trigger: Calling createRowWriter (directly or via a BigQueryIO write) on a RowWriterFactory whose prepare(GenericRecordWriterFactory, DynamicDestinations) was never invoked, e.g. by manually instantiating the factory and skipping pipeline setup.

Common situations: Custom DoFns or unit tests that construct RowWriterFactory directly and call createRowWriter without first calling prepare(); refactors that moved writer creation outside the normal @Setup/@StartBundle lifecycle; misuse of the BigQueryIO internal API.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• createRowWriter called when schemaFactory is null; forgot…
• Append to stream failed with invalid offset of
• Append to stream failed with Status Code . The stream may…
• At least one owner must be registered.
• BigQuery source must be split before being read

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/71736b1b1bd9be41. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/RowWriterFactory.java:134

        DynamicDestinations<?, DestinationT> dynamicDestinations,
        SerializableFunction<@Nullable TableSchema, Schema> schemaFactory) {
      return new AvroRowWriterFactory<>(toAvro, writerFactory, schemaFactory, dynamicDestinations);
    }

    SerializableFunction<AvroWriteRequest<ElementT>, AvroT> getToAvroFn() {
      return toAvro;
    }

    @Override
    OutputType getOutputType() {
      return OutputType.AvroGenericRecord;
    }

    @Override
    BigQueryRowWriter<ElementT> createRowWriter(String tempFilePrefix, DestinationT destination)
        throws Exception {
      if (dynamicDestinations == null) {
        throw new IllegalStateException(
            "createRowWriter called when dynamicDestinations is null; forgot to call prepare()?");
      }
      if (schemaFactory == null) {
        throw new IllegalStateException(
            "createRowWriter called when schemaFactory is null; forgot to call prepare()?");
      }

      TableSchema tableSchema = dynamicDestinations.getSchema(destination);
      Schema avroSchema = schemaFactory.apply(tableSchema);
      return new AvroRowWriter<>(tempFilePrefix, avroSchema, toAvro, writerFactory);
    }

    @Override
    String getSourceFormat() {
      return "AVRO";
    }
  }
}

View on GitHub (pinned to 12126d8942)