What causes "Failed to close multiplexer" in apache/beam?

Check the logged cause; if it is an interrupted/cancelled channel it is usually benign at shutdown.

Failed to close multiplexer

Error message

Failed to close multiplexer

What it means

BeamFnDataGrpcClient.close() iterates over all cached BeamFnDataGrpcMultiplexer instances and closes them; if any multiplexer fails to close, the exception is logged with this warning and the loop continues, so one bad channel does not prevent closing the others. The cache is cleared regardless.

Solutions

Check the logged cause; if it is an interrupted/cancelled channel it is usually benign at shutdown.
Avoid canceling the job while data streams are actively transferring; drain work first.
Inspect gRPC channel shutdown behavior and timeout configuration.
If it recurs at every startup/shutdown, update Beam version — several multiplexer close fixes landed over releases.

Example fix

// before
// nothing to change in user code; harness-level warning at shutdown
// after
// ensure graceful shutdown: stop producing data before closing the harness client
stream.close();
client.close(); // close data streams before the multiplexer

Defensive patterns

Strategy: try-catch

Try / catch

try {
  multiplexer.close();
} catch (Exception e) {
  LOG.warn("Failed to close multiplexer", e); // benign during shutdown
}

Prevention

Close data streams before shutting down the harness client
Avoid cancelling jobs mid-stream when possible
Inspect the logged cause to distinguish benign shutdown races from real bugs
Keep Beam SDK up to date for multiplexer close fixes

When it happens

Trigger: A gRPC channel/multiplexer's close() throws (e.g. already-shutdown channel with pending streams, interrupted awaitTermination, transport error) while the harness data client is shutting down.

Common situations: Harness shutdown with in-flight data streams to the runner; job cancellation racing with gRPC calls; network problems at teardown.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ee34b269de05aaa6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/harness/src/main/java/org/apache/beam/fn/harness/data/BeamFnDataGrpcClient.java:127

      client.unregisterConsumer(instructionId);
    }
  }

  @Override
  public void poisonInstructionId(String instructionId) {
    LOG.debug("Poisoning instruction {}", instructionId);
    for (BeamFnDataGrpcMultiplexer client : multiplexerCache.values()) {
      client.poisonInstructionId(instructionId);
    }
  }

  @Override
  public void close() {
    for (BeamFnDataGrpcMultiplexer client : multiplexerCache.values()) {
      try {
        client.close();
      } catch (Exception e) {
        LOG.warn("Failed to close multiplexer", e);
      }
    }
    multiplexerCache.clear();
  }

  @Override
  public StreamObserver<Elements> getOutboundObserver(
      ApiServiceDescriptor apiServiceDescriptor, String dataStreamId) {
    return getMultiplexer(apiServiceDescriptor, dataStreamId).getOutboundObserver();
  }

  private BeamFnDataGrpcMultiplexer getMultiplexer(
      Endpoints.ApiServiceDescriptor apiServiceDescriptor, String dataStreamId) {
    MultiplexerKey key = new MultiplexerKey(apiServiceDescriptor, dataStreamId);
    return multiplexerCache.computeIfAbsent(
        key,
        k -> {
          OutboundObserverFactory.BasicFactory<Elements, Elements> baseOutboundObserverFactory =

View on GitHub (pinned to 12126d8942)