InboundObserver for BeamFnStatusClient completed with…

Error message

InboundObserver for BeamFnStatusClient completed with exception.

What it means

BeamFnStatusClient.close() waits up to 1 minute for the worker status inbound observer (used for worker status/harness health reporting) to complete; if the completion latch is not COMPLETED after the wait, this warning is logged before shutting the channel down. It means the status stream ended abnormally (exception, timeout, or never finished) but close() proceeds regardless.

Solutions

Check whether the status stream errored earlier in the logs (grpc error before shutdown).
Ensure the runner closes the worker status stream during job teardown.
If benign at process exit, ignore — channel.shutdown() still proceeds.
Upgrade Beam if this occurs on every clean shutdown; stream completion handling has been fixed in later versions.

Defensive patterns

Strategy: try-catch

Try / catch

Object completion = inboundObserverCompletion.get(1, TimeUnit.MINUTES);
if (completion != COMPLETED) {
  LOG.warn("InboundObserver for BeamFnStatusClient completed with exception.");
}
// finally: channel.shutdown(); awaitTermination(10s); shutdownNow() if needed;

Prevention

Ensure the runner closes the worker status stream at job teardown
Look for earlier gRPC errors on the status channel
Expect benign occurrences at abrupt process exit
Upgrade Beam if it occurs on every clean shutdown

When it happens

Trigger: Closing the status client while the inbound observer (WorkerStatusListener stream) has not completed within 1 minute — e.g. runner never closed the status stream, the stream errored, or the thread waiting timed out.

Common situations: Harness shutdown while the runner keeps the status channel open; job cancellation not closing the status stream; network partition leaving the status stream half-open.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0bafa5f8397b0278. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/harness/src/main/java/org/apache/beam/fn/harness/status/BeamFnStatusClient.java:92

    this.inboundObserverCompletion = new CompletableFuture<>();
    Thread thread = new Thread(memoryMonitor);
    thread.setDaemon(true);
    thread.setPriority(Thread.MIN_PRIORITY);
    thread.setName("MemoryMonitor");
    thread.start();

    // Start the rpc after all the initialization is complete as the InboundObserver
    // may be called any time after this.
    this.outboundObserver =
        BeamFnWorkerStatusGrpc.newStub(channel).workerStatus(new InboundObserver());
  }

  @Override
  public void close() throws Exception {
    try {
      Object completion = inboundObserverCompletion.get(1, TimeUnit.MINUTES);
      if (completion != COMPLETED) {
        LOG.warn("InboundObserver for BeamFnStatusClient completed with exception.");
      }
    } finally {
      // Shut the channel down
      channel.shutdown();
      if (!channel.awaitTermination(10, TimeUnit.SECONDS)) {
        channel.shutdownNow();
      }
    }
  }

  /**
   * Class representing the execution state of a thread.
   *
   * <p>Can be used in hash maps.
   */
  static class Stack {
    final StackTraceElement[] elements;
    final Thread.State state;

View on GitHub (pinned to 12126d8942)