apache/beam · warning

Exception destroying pooled object.

Error message

Exception destroying pooled object.

What it means

ObjectPool (shared AWS client pooling for Beam's aws2 IO) logs this warning when the finalizer/destroyer accept(...) for a pooled shared object (e.g. an AWS SDK client) throws while releasing a reference whose refcount dropped to zero. The pool removes the entry regardless; only the cleanup of the underlying resource failed, potentially leaking connections or threads.

Solutions

1. Inspect the logged cause; if it is an SDK close failure, ensure no writes are still in flight before the pool is released (flush/await completion first).
2. If a custom finalizer was registered, wrap its cleanup logic so exceptions during close are contained or idempotent.
3. This is a cleanup-time warning: the pool already removed the object; verify no resource leak grows over the job (thread/connection counts) and ignore if benign.

Example fix

// before: finalizer that can throw
finalizer = client -> client.close();
// after: idempotent, exception-safe cleanup
finalizer = client -> {
  try { client.close(); } catch (RuntimeException e) { LOG.debug("client close ignored", e); }
};

// contain cleanup exceptions in finalizers registered with ObjectPool
pool.register(key, client, shared -> {
  try { shared.close(); } catch (Exception e) { LOG.debug("ignore close failure", e); }
});

When it happens

Trigger: releaseByKey drops the last reference and invokes the registered finalizer (typically client.close()); the close throws — e.g. an SDK client's close failing on an in-flight request, an IOException shutting down a connection pool, or a custom finalizer that throws.

Common situations: Shutting down pipelines while async clients still have pending requests; custom ObjectPool finalizers with buggy cleanup logic; AWS SDK close-time failures under network faults.

Related errors

• Error when writing batch.
• Was unable to reset bundle processor safely. Bundle…
• A function must be provided to convert the input type into…
• A PValue contained in
• A schema was provided without a data format (or viceversa)…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b2cdcde3fca77498. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/common/ObjectPool.java:89

  /**
   * Release a reference to a shared object instance using {@link KeyT}. If that instance is not
   * used anymore, it will be removed and destroyed.
   */
  public void releaseByKey(KeyT key) {
    RefCounted ref;
    synchronized (pool) {
      ref = pool.get(key);
      if (ref == null || --ref.count > 0) {
        return;
      }
      pool.remove(key);
    }
    if (finalizer != null) {
      try {
        finalizer.accept(ref.shared);
      } catch (Exception e) {
        LoggerFactory.getLogger(ObjectPool.class).warn("Exception destroying pooled object.", e);
      }
    }
  }

  /**
   * Release a reference to a shared client instance. If that instance is not used anymore, it will
   * be removed and destroyed.
   */
  public void release(ObjectT object) {
    KeyT key = pool.inverse().get(new RefCounted(object));
    if (key != null) {
      releaseByKey(key);
    }
  }

  public static <ClientT extends SdkClient, BuilderT extends AwsClientBuilder<BuilderT, ClientT>>
      ClientPool<ClientT> pooledClientFactory(BuilderT builder) {
    return new ClientPool<>(c -> buildClient(c.getLeft(), builder, c.getRight()));

View on GitHub (pinned to 12126d8942)