apache/beam · error · IllegalStateException

ErrorHandler must be finalized before the output can be retu

Error message

ErrorHandler must be finalized before the output can be returned

What it means

ErrorHandler.getOutput throws IllegalStateException if the handler has not been finalized with close(). The sink output only exists after close() flattens the registered error collections and writes them, so Beam refuses to return a null/uninitialized output and forces the caller to close first.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/ErrorHandler.java:136

    @Override
    public void addErrorCollection(PCollection<ErrorT> errorCollection) {
      if (isClosed()) {
        throw new IllegalStateException(
            "Error collections cannot be added after Error Handler is closed");
      }
      errorCollections.add(errorCollection);
    }

    @Override
    public boolean isClosed() {
      return closed;
    }

    @Override
    public @Nullable OutputT getOutput() {
      if (!this.isClosed()) {
        throw new IllegalStateException(
            "ErrorHandler must be finalized before the output can be returned");
      }
      return sinkOutput;
    }

    @Override
    public void close() {
      if (closed) {
        throw new IllegalStateException(
            "Error handler is already closed, and may not be closed twice");
      }
      closed = true;
      PCollection<ErrorT> flattened;
      if (errorCollections.isEmpty()) {
        LOG.info("Empty list of error pcollections passed to ErrorHandler.");
        flattened = pipeline.apply(Create.empty(coder));
      } else {
        flattened = PCollectionList.of(errorCollections).apply(Flatten.pCollections());

View on GitHub (pinned to 12126d8942)

Solutions

Call errorHandler.close() before requesting getOutput().
Use the handler within try-with-resources so close() is guaranteed to run before output use.
If you are an IO author, ensure the translation finalizes the handler before exposing its output.

Example fix

// before
PCollection<BadRecord> out = handler.getOutput(); // not closed
// after
handler.close();
PCollection<BadRecord> out = handler.getOutput();

Defensive patterns

Strategy: type-guard

Validate before calling

if (errorHandler.isClosed()) { PCollection<OutputT> out = errorHandler.getOutput(); }

Type guard

boolean outputAvailable(ErrorHandler<?> h) {
  try { return h.isClosed(); } catch (IllegalArgumentException e) { return false; }
}

Try / catch

try {
  output = errorHandler.getOutput();
} catch (IllegalStateException e) {
  errorHandler.close();
  output = errorHandler.getOutput();
}

Prevention

Always close the handler before reading its output
Use try-with-resources to guarantee finalization
In IO translations, finalize the handler before exposing output

When it happens

Trigger: Calling errorHandler.getOutput() before errorHandler.close() — e.g. returning the error output PCollection from a translate()/expand() method before the handler's lifecycle completes, or forgetting to close the handler at all.

Common situations: IO connectors (BigQuery sink translation) calling getOutput during pipeline construction when the user never finalized the handler; users forgetting close() when managing the handler manually instead of try-with-resources.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/661df87b474f5777. Report an issue: GitHub.