apache/kafka · error · KafkaException

failed closing plugin

Error message

failed closing plugin

What it means

Thrown by Plugin.close() when shutting down a wrapped plugin instance (or its associated pluginMetrics) raised an exception during AutoCloseable.close(). The Kafka Plugin wrapper closes the user-supplied instance and its metrics in sequence, captures the first failure into an AtomicReference, and rethrows it wrapped as a KafkaException so resource cleanup failures are not swallowed silently. The original cause is always attached as the exception's cause, so the real failure (e.g. a connector/serializer refusing close) is inspectable via getCause().

Source

Thrown at clients/src/main/java/org/apache/kafka/common/internals/Plugin.java:130

            plugins.add(wrapInstance(instance, metrics, key));
        }
        return plugins;
    }

    @Override
    public T get() {
        return instance;
    }

    @Override
    public void close() throws Exception {
        AtomicReference<Throwable> firstException = new AtomicReference<>();
        if (instance instanceof AutoCloseable) {
            Utils.closeQuietly((AutoCloseable) instance, instance.getClass().getSimpleName(), firstException);
        }
        pluginMetrics.ifPresent(metrics -> Utils.closeQuietly(metrics, "pluginMetrics", firstException));
        Throwable throwable = firstException.get();
        if (throwable != null) throw new KafkaException("failed closing plugin", throwable);
    }
}

View on GitHub (pinned to c31c9215e1)

Solutions

1. Inspect the attached Throwable cause (KafkaException#getCause) — the stack trace identifies the exact class and line in the plugin that failed to close.
2. Fix the offending plugin's close() to be idempotent and swallow/label expected shutdown errors (e.g. AlreadyClosedException) rather than propagating them.
3. Ensure any resources the plugin opens (sockets, executors, files) are tracked and released in close() in reverse acquisition order, inside their own try/finally.
4. If using Connect, verify the connector/plugin version is compatible with the broker/client runtime and update to a matching release.
5. If the failure is non-fatal to your shutdown path, catch KafkaException at the call site, log the cause, and continue teardown rather than letting it abort the whole shutdown.

Example fix

// before — plugin close() propagates failure
class MySerializer implements Serializer<byte[]>, AutoCloseable {
    public void close() {
        httpStream.close(); // throws if peer already reset the connection
    }
}

// after — close() is idempotent and never throws on shutdown
private final AtomicBoolean closed = new AtomicBoolean();
public void close() {
    if (!closed.compareAndSet(false, true)) return;
    try { httpStream.close(); }
    catch (java.io.IOException e) { log.warn("Error closing http stream on shutdown", e); }
}

// Before closing, confirm the wrapped instance is in a closeable state.
Plugin<?> plugin = ...;
Object inner = plugin.get();
if (inner instanceof AutoCloseable) {
    // surface any pre-close state (e.g. stop producers/consumers first)
    // so close() is unlikely to throw.
}

// Ensure the wrapped instance advertises AutoCloseable before relying on close().
static boolean isSafelyCloseable(Plugin<?> p) {
    return p.get() instanceof AutoCloseable;
}

try {
    plugin.close();
} catch (KafkaException ke) {
    // Plugin.close() bundles the underlying cause; log and continue shutdown.
    log.warn("Plugin {} did not close cleanly: {}", plugin.get().getClass().getSimpleName(), ke.getCause());
} catch (Exception e) {
    // close() declares 'throws Exception'; swallow non-Kafka exceptions during shutdown.
    log.warn("Unexpected error closing plugin", e);
}

When it happens

Trigger: Calling close() (directly or via try-with-resources) on a Plugin<T> whose wrapped instance is an AutoCloseable that throws from its close() method, or whose pluginMetrics Metrics object fails to close. Most commonly hit when the producer/consumer/connect runtime tears down a configured plugin such as a custom Serializer, Partitioner, ProducerInterceptor, ConsumerInterceptor, or Connect Converter/Transformation that misbehaves on shutdown.

Common situations: A custom plugin holds an open network connection, file handle, or thread pool and throws on close; a Connect connector worker is stopped while a task is mid-flight; a serializer backed by an external service fails to close its client; running with a buggy third-party plugin whose close() is not idempotent; version mismatch where a plugin compiled against an older client API throws a NoSuchMethodError during close.

Related errors

• provided null buffer
• Error unregistering mbean
• NetworkClient is no longer active, state is {state}
• Client was shutdown before response was read
• The timeout cannot be negative.

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/386aa9f0a03dbd90.json. Report an issue: GitHub.