apache/beam · error · IllegalStateException

RecordWriter is null

Error message

RecordWriter is null

What it means

In KinesisIO.Write, the Writer instance used to publish records is created lazily (per-bundle). The writer() accessor throws IllegalStateException if a record is written before the writer was initialized or after it was closed and reset to null. This indicates a lifecycle bug: write() invoked outside the valid writer window.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIO.java:865

                  writer().write(record);
                }

                @FinishBundle
                public void finishBundle() throws Throwable {
                  writer().finishBundle();
                }

                @Teardown
                public void teardown() throws Exception {
                  if (writer != null) {
                    writer.close();
                    writer = null;
                  }
                }

                private Writer<T> writer() {
                  if (writer == null) {
                    throw new IllegalStateException("RecordWriter is null");
                  }
                  return writer;
                }
              }));
      return new Result(input.getPipeline());
    }

    /** Result of {@link KinesisIO#write()}. */
    public static class Result implements POutput {
      private final Pipeline pipeline;

      private Result(Pipeline pipeline) {
        this.pipeline = pipeline;
      }

      @Override
      public Pipeline getPipeline() {
        return pipeline;

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the transform is expanded and applied normally (KinesisIO.write().expandTo(input)) rather than invoking internal Write internals directly
2. Check that the Writer supplier/factory always returns a non-null Writer
3. Verify no custom pipeline patching skips @StartBundle initialization
4. Upgrade the Beam SDK if a known bundle-lifecycle bug is suspected

Example fix

// before
Supplier<Writer<T>> supplier = () -> null; // or writer created outside bundle
// after
KinesisIO.<T>write().withBatchMaxCount(500).withWriterSupplier(() -> new KinesisWriter<>(...)); // non-null writer per bundle

// assert writer lifecycle: writer must be created in @StartBundle and only used in @ProcessElement

if (writer == null) throw new IllegalStateException("writer not initialized for this bundle");

When it happens

Trigger: Writing a record in a DoFn lifecycle phase where the bundle writer hasn't been started (before @StartBundle) or was already torn down (@FinishBundle set writer = null), or a custom Writer supplier returned null.

Common situations: Custom DoFn code calling writer() in @ProcessElement with a misconfigured transform; pipeline reuse across bundles with a Writer factory that fails silently; subclassing the sink incorrectly.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Cannot be called outside of a DoFn's process method.
• State stream is closed.
• Not currently processing a bundle.
• Illegal access to pipeline after visitor traversal was compl
• One or more ErrorHandlers aren't closed, and this pipeline c

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/327cac133bcf9e1b. Report an issue: GitHub.